I don't remember actual numbers decreasing, but you heard a lot about "white flight" to the suburbs. There was a famous headline concerning the city's financial woes and bankruptcy:
(President) Ford To New York: Drop Dead