The 84-year-old was taken to hospital in handcuffs, where he died while still in restraints, inspectors found.
Doctors said the Canadian man was unfit for detention or deportation after diagnosing him with Alzheimer's disease, but he was not released and no referral was made to social services.
Immigration minister Mark Harper said the use of restraint in the case of the dementia sufferer seemed "completely unjustified and must not be repeated".
Chief Inspector of Prisons Nick Hardwick said: "These were truly shocking cases, and they weren't isolated, and they reflected a culture where too often the individual human needs of the people who were being held were simply being forgotten.
