The undated note titled "Walworth" is presumably written by Henry Pease and it mentions the Hansard family as owners and that 'In 1466 the manor is fully deserted'. Other names mentioned are Cardinal Wolsey and 'Elisabeth heiress of W. Hansard', Sir W[?] Ayscough, the son of Elisabeth and another Elisabeth in addition to a visit from King James in 1603. The note also mentions 'The North Tower seems of earlier architecture than E[?]'.

Henry Pease purchased Pierremont house in 1845, moving in 1846. Walworth Castle is close to Pierremont, outside of Darlington. Owner of the Walworth castle during Henry's lifetime was John Harrison Aylmer, who inherited the castle in 1831 and died in the Abergale rail disaster along with his wife and eldest son in 1868, leaving the castle to Vivian and Edmund Aylmer. Vivian was later the High Sheriff of Durham, Africa explorer and traveller. After their death, the castle was sold to relatives Charles and Neville Eade. See letter from Charles Eade PEAS/1/14.