John Rennie was born in Phantassie, Haddingtonshire, the son of a farmer. In 1779 set up on his own as a millwright under the guidance of Andrew Meikle (1719–1811), inventor of the threshing machine. He studied at Edinburgh University between 1780 and 1783.

In 1783 he undertook a study tour into England where he met James Watt and was employed by Boulton and Watt to work on the machinery for Watts' techincally renowned flour mill, the Albion Mill, in London, completing this work in 1786 and then continued to produce machinery for factories and other industrial concerns.

He married Martha Ann Mackintosh in 1790, who bore him children including George Rennie and Sir John Rennie, who could carry on Rennie senior's work and become eminent engineers in their own right.

In 1790 he begun civil engineering work, acting as the surveyor for the Kennet & Avon Canal. Other civil engineering projects he worked on were the London docks between 1800-1805, the mile-long protective breakwater at Plymouth Sound, started in 1811 and completed in 1848 and the original Waterloo, Southwark and London Bridges. Between 1807 and 1810 he collaborated with Robert Stevenson on the Bell Rock lighthouse off Arbroath. Between 1812 and 1815 he conducted a survey for a canal or railway to link the Durham coalfields with Stockton-on-Tees and Darlington. Rennie was elected fellow of the Royal Society in 1798.

Rennie passed away in 1821 and was buried in St. Paul's Catherdral, he is commemorated by a plaque under Waterloo Bridge in London and a memorial stands to him stands in the vicnity of his birthplace near East Linton in East Lothian.