We believe we've identified the right Viscount Milton.
As the son of William Fitzwilliam, 4th Earl Fitzwilliam, Charles would have held the title Viscount Milton until the death of his father when he became known as Earl Fitzwilliam (analogous to the title 'Prince of Wales' being overridden by the title King). His father died in 1833 so in 1830, when the plaque was erected, Charles was still the current Viscount Milton.
His father had inherited great wealth from his uncle, Charles, The Marquess of Rockingham, and Charles was an only child so we have to assume he also was very rich, and we can guess who he was named for.
The family's massive Palladian ancestral home, Wentworth House (said to have 365 rooms), was built on the site of a Tudor mansion, home of Thomas Wentworth.
2025: Catherine Bailey’s history book 'Black Diamonds' (2007) tells the story of Charles's descendants, Wentworth, and the local coalmining on which their wealth was founded. London is barely mentioned but it's still a good read. Over the course of the next few generations the title died out, through sons dying without male issue, despite many sons being born. Charles had four, two called William; the 6th Earl had eight, all called William. The 8th Earl had a passionate, doomed, affair with Kathleen 'Kick' Kennedy, the future president's sister. The family's London base was and, after 1931, 10 Grosvenor Street.
One quote we particularly enjoyed, about the 1926 coal strike: "From the outset, the coal owners .. were determined not to give ground. 'It would be possible to say without exaggeration of the miners' leaders that they were the stupidest men in England, if we had not frequent occasion to meet the owners,' Lord Birkenhead .. remarked.."

Comments are provided by Facebook, please ensure you are signed in to see them