Born Alexander John Sharpe in Hoxton, London, on the 14th June 1814, Ellis attended private boarding school at Walthamstow, London. While there he was offered the opportunity of a life of study and research on condition he adopted his mother’s maiden name, Ellis. In return, a relative of his mother and a schoolmaster, William Ellis, financed Alexander Ellis’s schooling and gave him opportunities to travel abroad and to publish work as an adult.

After Walthamstow, Ellis attended Shrewsbury School before going on to Eton College. In 1833 he entered Trinity College, Cambridge to study mathematics. He also learnt classical and modern languages. He was elected to a college scholarship in 1835 and graduated BA in 1837.

After leaving Cambridge Ellis travelled to study in Italy and Sicily, where he took a large amount of daguerreotypes. These daguerreotypes, now in the National Science and Media Museum collection, are the earliest topographic photographs of Italy. Whilst in Italy, Ellis became interested in pronunciation leading him to devise a phonetic notation to reflect the variation in pronunciation heard across Italian dialects.

On 31 August 1840, in Naples, he married Ann Chaytor (1814-1888) of Spennithorne Hall near Leyburn, North Yorkshire. Their first son, Edwin John Ellis (1841-1895), became known as a painter. They also had twins in 1844, named Tristram James and Miriam Anne. Following the birth of their twins the family spent some time in Dresden, Germany for their mother Ann’s health. They moved to Bath in November 1847 and to Bristol in 1853. From 1869 they lived in Kensington, London.