Sorry, but I think you are taking a rather narrow viewpoint. I personally know several people accepted at Harvard Med with somewhat below average numbers (MCAT ~ 28-30, GPA ~ 3.4-3.6), who DID NOT come from disadvantaged backgrounds, ARE NOT underrepresented minorities, and DID NOT have an amazing volunteer or research background. They managed to stand out through their essays, their interviews, or other extracurriculars. Ave. scores are a guide, not an absolute rule. That being said, if you are applying to a school with scores well below their ave. numbers, don't delude yourself and be realistic about your chances. Also keep in mind that a lot of the admissions process is about fitting into a certain niche, and each school is different about what exactly they are looking for. Numbers are not the only part of this equation.
As far as the reputation of a school is concerned, it does play a part in residency match, but only a small part. Your actual performance in med school is far more important. In conversations with numerous doctors, admissions counselors, etc., I've always heard that it looks much better to be top of your class at a "second-tier or third-tier" school, than middle of your class at say, Harvard. Just something to keep in mind, but I'm sure there are lots of people out there who will disagree with this.