James Hamilton was the son of Lord Anne Hamilton (1709-1748) and Anna Charlotte Maria Powell. He was a grandson of James Hamilton, 4th Duke of Hamilton (1658-1712). He married Lucy Lloyd (1741-1790), daughter of Sir Richard Lloyd of Hintlesham, Suffolk and widow of Sir John Fytche-Barker, 7th Bt (1741-1766).
A miniature of Captain The Hon. James Hamilton signed and dated 1784 by John Smart was sold Sotheby’s, London, 7-8 July 2011, lot 295 and lot 294 was a miniature of his wife, Lucy Lloyd, also by John Smart, dated 1781. A further miniature of James Hamilton by John Smart, signed and dated 1784, was sold Phillips, London, 15 July 1991, lot 32.