The Misses Rigby were daughters of Dr. Edward Rigby (1747-1821) and his first wife, Miss Dyball (d. 1803). Their half sister by Edward Rigby's second wife, Anne Palgrave (1775?-1872) was Elizabeth (1810-1895), later Lady Eastlake, wife of Sir Charles Lock Eastlake, artist, President of the Royal Academy (1850) and in 1855, the first Director of the National Gallery.
John Downman was apprenticed to Benjamin West and studied at the Royal Academy Schools. He later travelled to Italy, and was in Rome with Joseph Wright of Derby from 1773-74. By 1777 he was living in Cambridge, but in 1778, the year the present portrait was painted, his annual contribution to the Academy exhibition was sent from Bedford Street, Covent Garden. Among the exhibits were two sets of 'Six portraits', in which the present work may have been included; his very popular small portraits were often shown thus in groups of six or nine.
There is a preparatory drawing in black, red and white chalk for the present portrait, also dated 1778, in the Fitzwilliam Museum, Cambridge, which was in the artist's personal collection until his death.