Isleworth has been the site of a house since 1431, when it was a convent, and aspects of Syon date back to then, but the main part is 19th Century. In 1546 the convent site was given to the Duke of Somerset and after his execution, to John Dudley, Earl of Northumberland. In 1604 the house passed to the Percys, Earls of Northumberland, and from thence, in the 18th Century, to Sir Hugh Smithson, later 1st Duke of Northumberland, who commissioned Robert Adam the architect, to re-work aspects of the interior and exterior in the Antique style, in 1761.