The father of these three children, Vespasian Emerson Flye, was a Boston furniture store owner and a fervent democrat who engaged the services of William Matthew Prior to paint a portrait of his family in patriotic costume. In 1854, Prior painted this portrait of Hattie Elizabeth, aged four, her three-year-old brother, Ellis, and their baby sister, Eva.

A surviving document written by Hattie's daughter describes the sitting for this portrait: "When he [Vespasian] talked with the artist about having the portrait made, he insisted on having the childrens' costumes painted in red, white, and blue. My mother, [Hattie E.], said that the costumes were not red, white and blue in reality---hers being white muslin with a sprig of green scattered through it, as was favored in pre-Civil War days. But V.E.F. insisted on the patriotic colors, and moreover purchased a flag for 3-year-old Ellis to hold in his hand."