John was born on 10 October 1680, the son of Archibald, 1st Duke of Argyll, and his wife Elizabeth, daughter of Sir Lionel Tollemache, Bt., of Helmingham. He entered the army and became Colonel successively of the 14th, 1st and 3rd Horse Guards, and on the death of his father as Duke of Argyll, he was made Colonel of the Scots Horse Guards. He fought with Marlborough in Flanders, distinguishing himself at the Battle of Malplaquet, but later fell out with him, and a bitter rivalry ensued. He became Ambassador Extraordinary to Spain in 1711 and Governor of Minorca, between 1712 and 1716, as well as Commander-in-Chief of the forces in Scotland and Governor of Edinburgh Castle. On the accession of King George I he was made Groom of the Stole and having been nominated one of the members of the regency, he undertook in this and his military capacity the difficult task of the crushing of the Jacobile rising. He was installed a Knight of the Garter on 20 December 1710, was Master-General of Ordnance 1725-30, Governor of Portsmouth 1735-6; during this time he also took an active part in politics. He died on 4 October 1743.

He married firstly on 30 December 1701 Mary, daughter of Thomas Browne; they had no children. He married secondly on 6 June 1717 Jane, daughter of Thomas Warburton of Winnington, Cheshire; they had five daughters. On the Duke's death his English titles of Duke and Earl of Greenwich and Viscount Chatham became extinct, while his Scottish titles devolved on his brother Archibald, who succeeded as 3rd Duke of Argyll.

The Duke was Aikman's greatest patron, and many portraits by the artist of him exist (see for example Scottish National Portrait Gallery, no. PG692).