Peter Denys married Lady Charlotte Fermor, daughter of George, 2nd Earl of Pomfret, on August 1, 1787. Their eldest daughter, Anna Maria, married Sir Francis Shuckburgh, 8th Bt. in 1825. Their only son, George William, was born in 1788 and was created a baronet on November 23, 1813