Residents have lost their High Court battle to prevent surface-to-air missiles being stationed on the roof of their tower block during the Olympics.
They had said having them on Fred Wigg Tower in Leytonstone, east London, would make them a terrorist target.
The Ministry of Defence (MoD) has said the missile deployment is legitimate and proportionate.
A judge said residents were “under something of a misapprehension” regarding the equipment and risks.
Lawyer David Enright, who represents the residents, said the ruling had wide-ranging implications.
Justice Haddon-Cave said the MoD had no duty to consult, had not promised to and no “conspicuous unfairness” was caused by not consulting.
He added: “The law and the facts militate against the claim for judicial review.
“In my judgment the MoD’s voluntary engagement with the community and residents in this matter were immaculate”.
On Monday, Mr Willers, said their block could become the “focus of a terrorist attack” to make an “evil statement”.
The defence secretary was accused by the local residents’ association of breaching Article 8 and Article 1 of Protocol 1 of the European Convention on Human Rights.
These protect an individual’s right to a private life and peaceful enjoyment of their home.
The MoD was also accused of failing to carry out an equality impact assessment to take into account the needs of disabled residents.
Mr Forsdick said the decision over where to position the missiles was taken at the “highest levels after rigorous scrutiny”.
He said it was signed off by the prime minister, deputy prime minister, home secretary and defence secretary in “defence of the realm”.
The 15-storey tower block is one of six locations selected for missiles to be stationed.