Critique of USNEWS rankings by academic journal- for those who use them...

[yunfat]

Since lots of ppl are still interested in this, check out this paper in Academic Medicine:

McGaughie, WC and Thompson, JA, America's Best medical Schools: A Critique of the U.S. News & World Report Rankings. Acad. Med. 2001;76:985-992

Here's the abstract for those without online access:

America's Best Medical Schools
A Critique of the U.S. News & World Report Rankings

Rankings of American medical schools published annually by the news magazine U.S. News & World Report are widely used to judge the quality of the schools and their programs. The authors describe and then critique the rankings on methodologic and conceptual grounds, arguing that the annual U.S. News medical school evaluation falls short in both areas. Three categories of program quality indicators different from those used by U.S. News are presented as alternative ways to judge medical schools. The authors conclude that the annual U.S. News & World Report rankings of American medical schools are ill-conceived; are unscientific; are conducted poorly; ignore medical school accreditation; judge medical school quality from a narrow, elitist perspective; and do not consider social and professional outcomes in program quality calculations. The medical school rankings have no practical value and fail to meet standards of journalistic ethics.

----------

bon appetit! :wink:

-yf

---

Members don't see this ad.

 

[Jalby]

I agree that these rankings fall short. But one thing it does do is provide you with information. As long take the information with a grain of salt, the rankings are great. Where else are you going to get an opinion of how residency directors feel about different schools. But the USnews does have major flaws. Two huge ones that I have noticed:
one big factor in the clinical rankings is what percentage of graduates go into a clinical field. How does that make a difference in weather a school trains good clinicians?

Also this year, they said that the surveys they recieved back from the residency directors of clinical schools was way different from every one they have recieved in the past. because of this, they used last years residency directors score.

I also have a problem with NIH rankings being as big a factor as they are. While it is ranking a school based on research, I don't think that NIH funds is that great a way to rank schools (or any, for that matter). Take Mayo. They only have 38 students each year. But they get a decent amount of funding, prolly one of the highest per student. But another school with 200 students and double the NIH funds would be ranked higher than mayo because it's a total number. Maybe they should use NIH dollars per student or something like that.

 

[megaman]

anybody has access to the full-text?

just curious to know what alternate methods they came up with...

 

[megaman]

anybody?!?

I guess USNews is not a hot topic anymore? <img border="0" alt="[Wowie]" title="" src="graemlins/wowie.gif" />

come on! someone's gotta have read this?!?

 

[m2006]

I read it-- the authors' main criticisms of the rankings were both methodologic ie, response rates &lt;50%, measurement error, narrow focus of measures, etc and conceptual ie, accreditation standards exist, so why measure them again and the relative stability of the top schools on the list over a 25 year period.

Their alternative ideas include:
1. % of minorities included in class along with attempts to recruit and keep them enrolled

2. Assessment of clinical skills of med students-- some schools like Sinai have instituted standardized patients that test the students' abilities to perform in representative clinical settings which is arguably more important to educating great physicians than step 1 scores and MCAT scores

3. Depth of service to underserved, underprivileged and the like. To what extent does a school involve itself in the health and well being of the community or area which is purports to serve?

Although their ideas are discussion starters and I think they are excellent yard sticks, it remains that gold standards are much more difficult to develop with qualitative ideas such as these. I hope schools will begin to invest their efforts in measuring and monitoring such benchmarks.

 

[psyche]

Thanks M2006. Sounds like a good article, and I feel better about the unranked school I'll be attending since it would score very well on the alternative measures you mentioned.

What really steams me about these bogus rankings is when the schools themselves embrace them by touting the rankings in promotional literature, posting the U.S News logos on their websites, etc.
Seems a little corrupt to me.

 

[m2006]

You're welcome, Psyche. Although correlation coefficients don't tell the whole story, the correlation between the rankings in 2000 and 1996 were 0.91 which tells you that not much changes in the list from year to year. It's not hard to imagine how good schools can't "break through" onto the list when 40% of the total score is perceived reputation. Hello? Perception does not necessarily equal reality. And perceived reputation will rarely be directly correlated with satisfaction of students and quality of interpersonal care. Anyway, enough ranting...

 

[RT]

Thanks Yunfat. Great article. Very informative. I appreciate the reference.

 

[shamus1]

The US News rankings are crap. They should be taken seriously only by your Grandma' Lil. US News came up with that garbage just to keep from going out of business; nobody would buy that rag if it were not for that sort of bogus stuff.

Follow your heart and go to the school that is best fits you rather than turning it down to go to a school that has a higher rank in some jerkoff poll. You probably wouldn't pick your life partner based on what a bunch of bozos you do not know think. While med school is not quite that important, you shouldn't let popular opinion sway you in this decision.

 

[Ashman52]

I'm not on here very often, but I was about to start a new thread and I found an old one addressing the same topic. The article cited above is still relevant since US News still uses the same methodology. Just thought it might be useful for some people to put rankings into perspective since it seems like more and more people have been relying on them lately. Hope the interview season is going well, and good luck in making decisions!

 

[MrBurns10]

I don't think anyone has argued that higher ranked schools produce better doctors than lower ranked ones...and if someone did, he was probably just trying to make himself feel special. I think rankings matter insofar as the top schools may give you some connections that you otherwise may not get, and name may still mean something to some residency directors. But how much all that means to an applicant is an individual decision, and no way does your school make or break the potential quality of the physician one can become.

Before anyone asks, yes I realize this was a bumped 6 year post.

 

[NTF]

I think we all sort of suspect that US N&WR Rankings are of dubious value, but let's face it, everyone loves a list. Every university hospital I've ever been to always has some sort of banner or sign extoling their recent high ranking in US N&WR. And you always hear the rankings tossed around in discussions between premeds, med students, and faculty.

ahhhhh, the power of perception.

 

[amph119]

Wait you mean I shouldn't have based where I applied on the rankings of the schools? CRAP!

 

[PremedIowa]

That article was almost entirely fluff. I mean, seriously, the one thing that was worth anything quantitative was the critique on the response rates.

I realize this is an administrative journal rather than a medical or scientific one, but I call shenanigans.

 

[PremedIowa]

Then again, US N&WR is entirely fluff as well.

 

[n3opologist]

Why are the response rates so terrible? Residency director opinion is one of the criteria I find reasonably helpful, yet when you find out that the response rate is so low that it becomes statistically questionable, I feel confused.

Do only top ranked medical programs fill out the USNews questionnaire while unranked schools just throw them away?