I too had the same concerns. I stayed with a student host who lived just east of downtown Richmond and about a 10-15 minute walk to VCU School of Medicine. It looked to me (and I've read elsewhere) that this area was once pretty rough, but in recent years, I think it's become very gentrified, and the crime rate has consequently dropped. Many old/abandoned buildings where crime might have "brewed" before have been turned into nice apartments or retail areas. And the truth is, there is crime in every city, and no matter where you are, you have to play it smart by not walking alone at night, not going into sketchy looking areas, etc... If you follow these rules, you'll be safe in most cities, most parts of Richmond included.
For what it's worth, my host lived in a recently renovated tobacco warehouse...his place was awesome, and it was affordable to boot. At no point during my visit to Richmond did I feel I was in danger or in a bad neighborhood. This included my 10-15 minute walk to the medical campus. In fact, I was pleasantly surprised by how much I liked the area.