I've lived in Baltimore for 5 years going now, and I really don't find it to be as bad as the rumors seem to make it. I live in the downtown area close to the University of Maryland hospital and the Inner Harbor, and go to both JHH and Homewood campuses on a regular basis. The area around JHH is not great, but as has been mentioned above, the campus itself is very safe. The undergraduate campus feels pretty safe too... it is surrounded by mostly nice neighborhoods, particularly the white collar Guilford (which is very expensive) and Roland Park, and artsy and blue collar Hampden and Charles Village. You are (most likely) not going to get shot at walking around either place!
I agree with the above post that there are certain, general, precautions that one needs to take when living in a city of any size, and the main thing about Baltimore is to get to know the city because neighborhoods transition rapidly and you can walk from a "safe" neighborhood into a "dangerous" one rather quickly. But it's totally feasible for someone going to JHH to live downtown in a very safe apartment complex and either use the Metro or the JHMI shuttle or drive to get to campus. Federal Hill, Mount Vernon, Fells Point, and Canton are all nice neighborhoods to live in that are relatively (compared with other parts of Baltimore) safe. Of course, if you REALLY want to be safe, one could always move out to the suburbs in Owings Mills, Towson, or Columbia, and commute the 20-25 minutes by car to the hospital. Those areas have houses with backyards, parks, schools, and other niceties for families.
Bottom line is that Baltimore gets a bad rap for being a city/war-zone/STD capital, but I don't think it's THAT bad. It's not New York, but for a small city it has a symphony, theaters, art museums, and other cultural events; cost of living is much lower than NYC/Boston/Philly/DC; and traffic is not nearly as bad.