The Rochforts were descended from an old Irish family with political connections. John Downes Rochfort's father was Col. John Staunton Rochfort of Clogrenane in Ireland and his mother was his father's second wife, Mary Burgh of Co. Kildare Downes. John Downes Rochfort inherited two family estates in Ireland and owned a villa in Monte Carlo and houses in the Boltons, Kensington, and Eaton Place.

John Downes Rochfort was a keen traveller, as his albums in the John Downes Rochfort Albums collection attest. He was also a collector and a painter of watercolours and ceramics.