None of the 'big four' programs in Manhattan - Columbia, Cornell, NYU, Sinai - are exactly in the 'heart' of NYC (whatever that means).
First of all NYC doesn't really have a 'heart.' You could say that 42nd street/Times Square is about in the middle, but nobody would ever want to live there. It's really more about which neighborhoods you prefer (clean/quiet UES, academic/bourgeois UWS, gritty Hell's Kitchen, central Chelsea/Clinton, funky E Vil, quaint W Vil, ethnic Chinatown, boutiquey SoHo, etc).
But even given that, none of C/C/N/MS are in great neighborhoods. Columbia's is sort of scuzzy and downtrodden-looking and there are very few good cafes/restaurants/shops. Subway access is great but it's a thirty-minute train ride to anywhere interesting. Cornell is very pretty and in an attractive area but sort of out of the way and it's difficult to access the subway. Sinai is right next to Central Park, which is appealing; the neighborhood is just okay and subway access is difficult. NYU's main campus is in a great spot but the medical campus is in a not-very-interesting area that is too far east and again, not well situated wrt the subway.
There are some smaller hospitals with their own residency programs as well (e.g., St Luke's, Roosevelt Hospital) that I won't go into here.
Anyways a lot of the residents don't actually live right next to the hospitals. Proximity is important but so is price, neighborhood, subway access, luck, and where your friends live.