Just wrote this up for a PM and thought it would be helpful for everyone:
If you look hard you can find good places that aren't crazy expensive. The public health school is located on the medical campus, which is in a neighborhood called Washington Heights. Here is a map of Manhattan's neighborhoods:
http://nycinsiderguide.com/wp-content/uploads/2013/01/manhattan-neighborhood-map.jpg
As you can see, Washington Heights is almost all the way at the top. Though this used to be one of the most dangerous neighborhoods in all of New York City, it is actually really nice now! Lot's of families have moved North since rents are ridiculous further south.
The most convenient places to live are Washington Heights, Hudson Heights, Fort George, and Inwood (all trying to stay on the West side). These neighborhoods are especially nice because they are right near Fort Tryon Park, which is a beautiful park that is a nice retreat from the city. Even Harlem can be really nice (Bill Clinton's offices are there!). Rents in these neighborhoods are also the most reasonable in all of Manhattan, and if you live near a subway line it is super easy to get downtown.
You can also look at living in outer boroughs, or even in Westchester (a county North of NYC). The closest outer borough to live in is the Bronx. It has a bad rap because of the South Bronx area, but there are some very nice neighborhoods. For instance, Riverdale is towards the very top of the Bronx, and is about 30 minutes by train to get to the medical campus. Queens is another great option, and a much more lively place to live than the Bronx. I know a lot of people who live in Astoria (a neighborhood in Queens, has the best Greek food you will find anywhere). Astoria is 1 subway stop from Manhattan, and would only require 1 transfer to get to Columbia. Lot's of people mention Brooklyn, but that commute is a nightmare to the medical center, so I wouldn't recommend it unless you get a great deal.
Its also worth checking out neighborhoods on the West Side if you can find a deal(Upper West Side, Morningside Heights, Hell's Kitchen, Chelsea), as all of the neighborhoods are on the same subway line and lead straight to Columbia. That being said, rents in these neighborhoods will on average be a lot more. Manhattan is notorious for being difficult to traverse East-West (there are only 2 or 3 places where the subway crosses, otherwise you have to take a bus), so I wouldn't recommend the East Side, unless you are near a cross-town subway line.
All of the places I've mentioned are very safe. New York is a big city, so the most important aspect of safety is just being aware of whats going on around you.
The absolute best place to look for apartments is craigslist. You can also check out the new york times website, and Columbia has an off-campus housing posting service.
Personally, I plan on searching in the Washington Heights area, and also in Southern Westchester because of my wife's work.
I hope this helps, let me know if you have any questions about any particular locations!