NY Times article about rankings (and, of course, US News)

Ischemia

Member
10+ Year Member
15+ Year Member
Jun 15, 2002
80
0
Visit site
Status
Hey all,

I don't know if this was already posted and I just didn't see it, but a professor of economics from the University of Chicago, Austan Goolsbee, wrote a nice op-ed piece in the Times about rankings that centers around the US News rankings. Given this board's obsession with US News, it's worth taking a look...

http://www.nytimes.com/2004/04/12/opinion/12GOOL.html

You might have to log in, but it is free.
 

evines

peek-a-boo
10+ Year Member
15+ Year Member
Sep 23, 2003
980
1
40
sinking sand
Visit site
Status
Nice editorial. Anyone have any ideas about how I could telepathically zap this article into all pre-meds' minds???
I guess SDN is a good start.

ZZZZZAP!!!
 
About the Ads

umass rower

Insert clever phrase here
10+ Year Member
15+ Year Member
Jul 24, 2003
8,959
142
Status
Fellow [Any Field]
Ischemia said:
You might have to log in, but it is free.
Since I don't like giving my e-mail address out unless it's really necessary, do you think you could cut and paste the story on to here? Thanks if you can.
 

LP1CW

Senior Member
7+ Year Member
15+ Year Member
Aug 30, 2003
651
1
49
Visit site
Status
HERE IS THE COMPLETE ARTICLE

From best-selling book lists to Zagat dinner recommendations to David Letterman's top 10, it's clear that Americans love rankings. Little surprise, then, that the release this month of U.S. News & World Report's annual ratings of the nation's graduate schools has prospective students swarming newsstands for advice on where to go. The list has an undeniably huge impact. Applications typically soar at universities that get top marks; some state governments even use the rankings to help them decide how much money to give to their universities.

Critics argue that rankings like these try to quantify the unquantifiable ? that variables used to measure quality are often flawed and incomplete. But the often overlooked, and perhaps more troubling, problem is that the average person who uses these rankings may not understand basic statistics. If he did, he would know that even if all the right variables were included, once he got past the top spots, the distinctions between schools (or restaurants or places to retire) are often meaningless.

Take a look at the sales ranking data at Amazon.com. I once needed a publication, a World Bank technical paper on the regulatory environment in Bulgaria. The demand for technical working papers on Bulgaria being what it is, the publication's Amazon sales rank was extremely low (about 2.5 million down on the list). Yet after I bought it, a most amazing thing happened. My one purchase moved this working paper past almost one million other books. This happened because once buyers get out of the best sellers ? where the difference in sales can be enormous ? almost everything else is basically tied. The differences in rank are statistically meaningless, and small blips cause big changes.

This same is true for the rankings of all sorts of things. In the New York City Zagat guide, for example, a restaurant that raised its food rating by three points would pass 50 restaurants if it went from 25 to 28 (from "excellent" to "extraordinary"), but 439 establishments if it went from 19 to 22 (both considered "very good").

Then there are the education rankings. For the large mass of schools that rate in the middle of lists like U.S. News & World Report's, rankings are extremely sensitive to small blips. Any criteria included in a survey, like undergraduate grade point average or percentage of applicants accepted, will have random variations over time.

This is perfectly normal. These fluctuations, though, cause the rankings of schools in the middle to move all over the place. It's why schools at the top seldom move more than a place or two, but the Tippie College of Business at the University of Iowa can move up five spots one year and down 18 the next (which is what just happened).

This sensitivity to small changes also tends to encourage bad behavior among university administrators. Some, for example, no longer report foreign students' verbal scores on standardized admissions tests to get an overall better result, or have wait-listed candidates that are likely to turn their schools down so their university looks more selective. These things may raise measured performance a tiny bit, but tiny bits can mean a lot when everyone is clustered in the middle.

The lessons are pretty obvious: outside the very top, be careful about basing decisions on rankings data. The differences are often statistically meaningless. If possible, whether you're interested in schools or restaurants, look at the last several surveys, instead of just the most recent one, since random blips tend to average out over time. Certainly don't pick a Chinese restaurant tonight because its rating is 20 versus another's 19.

For those who really can't get enough, it's now possible to find rankings of the rankings. U.S. News & World Report may get lots of attention, but as of last week its "Ultimate College Directory" for 2004 had an Amazon sales rank of 22,305, worse than the Intercollegiate Studies Institute's "Choosing the Right College 2004" at 19,752.

How is a rankings addict to choose? Next time, perhaps, you would do well to consider an alternative: the informative (and surprisingly popular) "Cartoon Guide to Statistics." Its sales rank is 8,600.
 

BerkeleyPremed

Membership Revoked
Removed
15+ Year Member
Jul 15, 2003
946
1
Visit site
Status
That professor was the former national champion of domestic extemporaneous speaking (any of you who did speech/debate in high school know what I'm talking about). I saw him in an instructional video sponsored by the National Forensic League. Just thought I'm chime in about the professor's background...lol.. :)
 
About the Ads