Goat in a Bayswater Landscape depicts a vegetable garden which was situated near to potter Neil Douglas's studio. The goat, a symbol of stubborn determination, appears to be humourously eyeing off the cabbage patch, prior to causing destruction. The work was painted in 1956, the year in which Perceval was arguably at his creative best, following a five year hiatus during which he seldom painted. It was in this same year that Australian Galleries opened in Collingwood, with a solo show by Perceval forming the inaugural exhibition.
In contrast with the religious and literary references which defined Perceval's work throughout the 1940s, the artist turned his attention to land and seascapes in the mid to late 1950s. Inspiration for his landscapes was found in the outlying Melbourne suburbs of Oakleigh and Bayswater and, in the company of Arthur Boyd, in explorations of the gold-mining town of Gaffney's Creek near to Eildon. In 1956 Perceval also commenced work on the original williamstown series of seascapes, which were to become iconic both within Perceval's individual body of work and within the history of Australian art.