Lord of the Rings-style house under threat of demolition