This panel once belonged to Lord Cowper, a collector best known for once owning the eponymous Small Cowper Madonna (1505) and the Niccolini-Cowper Madonna (1508) by Raphael, now in the National Gallery, Washington. In 1757 Lord Cowper, at the time still Lord Fordwich, embarked on his Grand Tour of Italy, a journey that was later recorded in the diary of his travel companion Colonel John Chastellain. Despite his election as an M.P. and the inheritance of his earldom in Hertfordshire, Cowper remained in Italy until his death in 1789, residing in the Villa Palmieri and the Villa Cipresso, a short distance from Florence. Acknowledged as an important member of Florentine society, he was nominated an honorary member of the Florentine Academy in 1766 and of the Academia della Crusca in 1768.
After becoming involved in the Florentine art community, Cowper commissioned several landscapes from Francesco Zuccarelli and soon began collecting paintings - an enterprise on which he was advised by Johann Zoffany, who was working in Florence from 1772-79 and in whose painting, the Tribuna of the Uffizi (about 1477-79; Royal Collection, Windsor), the artist and Cowper are depicted among the latter's neighbors in the left corner of the composition, admiring the Niccolini-Cowper Madonna, which was purchased by Cowper from Zoffany along with three or four other Old Masters. An inventory of about 1779 reflects the superb quality of Cowper's collection, which included Fra Bartolomeo's Rest on the Flight to Egypt (J. Paul Getty Museum, Los Angeles) and Pontormo's four scenes from the story of Joseph, originally commissioned as decorations for the bedchamber of Pierfrancesco Borgherini in celebration of his marriage in 1515 (now in the National Gallery, London).
The twenty-eight paintings in the Cowper inventory later formed the collection at the Panshanger picture gallery in Hertfordshire, rebuilt by the 5th Earl Cowper at the beginning of the nineteenth century. The collection was dispersed after the death of Lady Desborough in 1952.