Neighbours living near to Captain's Tom's daughter claim they want to take a "sledgehammer" to the home's illegal spa after discovering it was built using his name.
Hannah Ingram-Moore has been accused of using her late-father's name to build the pool and health facility at her £1.2million home in Bedfordshire.
The luxury outbuilding, which houses a pool and changing facilities, was branded a "monstrosity" by nearby residents.
Mrs Ingram-Moore, 52, and her husband have been ordered to tear down the block after it emerged on Tuesday it had been built without planning permission.
The family used the name of the Captain Tom Foundation to put up the building back in 2021, which was erected "in connection with the Captain Tom Foundation and its charitable objectives".
But after building the pad and departing from original plans, the couple submitted a retrospective planning request which was rejected.
A spokesperson for Central Bedfordshire Council, the planning authority for the area, said: "An enforcement notice requiring the demolition of the now-unauthorised building was issued and this is now subject to an appeal to the Planning Inspectorate."
Neighbours in the village of Marston Moretaine described being appalled at the revelation, describing the C-shaped block as an eyesore.
"It's a horrible monstrosity," one elderly resident told The Sun.
"It has just blocked our view from the kitchen. Give me a sledgehammer and I'll knock the place down myself."
On Tuesday, the foundation put out a statement saying it would not seek donations, and was closing all payment channels, while the Charity Commission carried out an inquiry.
The foundation said: "At this moment in time, the sole focus of the Captain Tom Foundation is to ensure that it co-operates fully with the ongoing statutory inquiry by the Charity Commission.
"As a result, the Captain Tom Foundation is not presently actively seeking any funding from donors. Accordingly, we have also taken the decision to close all payment channels while the statutory inquiry remains open.
"Once the findings of the statutory inquiry have been communicated, the Captain Tom Foundation will be in a better position to make a decision in relation to its future, but for now, our main priority is to assist the Charity Commission with its inquiry.
"In the meantime, on behalf of the trustees of the Captain Tom Foundation, we wish to extend a warm thank you to all our supporters who have enabled us to help charities that were close to Captain Sir Tom's heart."
Sir Captain Tom Moore became a household name during the first lockdown in spring 2020 when he raised millions for NHS charities by walking 100 laps of his garden ahead of his hundredth birthday.
The veteran and centenarian, who was later knighted for his efforts, died in 2021.
