• Set Yourself Up For Success Webinar

    October 6, 2021 at 2 PM Eastern/11 AM Pacific
    SDN and Osmosis are teaming up to help you get set up for success this school year! We'll be covering study tips, healthy habits, and meeting mentors.

    Register Now!

  • Funniest Story on the Job Contest Starts Now!

    Contest starts now and ends September 27th. Winner will receive a special user banner and $10 Amazon Gift card!

  • Site Updates Coming Next Week

    Site updates are coming next week on Monday, Wednesday, and Friday. Click the button below to learn more!


NY Times article about rankings (and, of course, US News)


10+ Year Member
15+ Year Member
Jun 16, 2002
Hey all,

I don't know if this was already posted and I just didn't see it, but a professor of economics from the University of Chicago, Austan Goolsbee, wrote a nice op-ed piece in the Times about rankings that centers around the US News rankings. Given this board's obsession with US News, it's worth taking a look...


You might have to log in, but it is free.
About the Ads


Senior Member
7+ Year Member
15+ Year Member
Aug 30, 2003

From best-selling book lists to Zagat dinner recommendations to David Letterman's top 10, it's clear that Americans love rankings. Little surprise, then, that the release this month of U.S. News & World Report's annual ratings of the nation's graduate schools has prospective students swarming newsstands for advice on where to go. The list has an undeniably huge impact. Applications typically soar at universities that get top marks; some state governments even use the rankings to help them decide how much money to give to their universities.

Critics argue that rankings like these try to quantify the unquantifiable ? that variables used to measure quality are often flawed and incomplete. But the often overlooked, and perhaps more troubling, problem is that the average person who uses these rankings may not understand basic statistics. If he did, he would know that even if all the right variables were included, once he got past the top spots, the distinctions between schools (or restaurants or places to retire) are often meaningless.

Take a look at the sales ranking data at Amazon.com. I once needed a publication, a World Bank technical paper on the regulatory environment in Bulgaria. The demand for technical working papers on Bulgaria being what it is, the publication's Amazon sales rank was extremely low (about 2.5 million down on the list). Yet after I bought it, a most amazing thing happened. My one purchase moved this working paper past almost one million other books. This happened because once buyers get out of the best sellers ? where the difference in sales can be enormous ? almost everything else is basically tied. The differences in rank are statistically meaningless, and small blips cause big changes.

This same is true for the rankings of all sorts of things. In the New York City Zagat guide, for example, a restaurant that raised its food rating by three points would pass 50 restaurants if it went from 25 to 28 (from "excellent" to "extraordinary"), but 439 establishments if it went from 19 to 22 (both considered "very good").

Then there are the education rankings. For the large mass of schools that rate in the middle of lists like U.S. News & World Report's, rankings are extremely sensitive to small blips. Any criteria included in a survey, like undergraduate grade point average or percentage of applicants accepted, will have random variations over time.

This is perfectly normal. These fluctuations, though, cause the rankings of schools in the middle to move all over the place. It's why schools at the top seldom move more than a place or two, but the Tippie College of Business at the University of Iowa can move up five spots one year and down 18 the next (which is what just happened).

This sensitivity to small changes also tends to encourage bad behavior among university administrators. Some, for example, no longer report foreign students' verbal scores on standardized admissions tests to get an overall better result, or have wait-listed candidates that are likely to turn their schools down so their university looks more selective. These things may raise measured performance a tiny bit, but tiny bits can mean a lot when everyone is clustered in the middle.

The lessons are pretty obvious: outside the very top, be careful about basing decisions on rankings data. The differences are often statistically meaningless. If possible, whether you're interested in schools or restaurants, look at the last several surveys, instead of just the most recent one, since random blips tend to average out over time. Certainly don't pick a Chinese restaurant tonight because its rating is 20 versus another's 19.

For those who really can't get enough, it's now possible to find rankings of the rankings. U.S. News & World Report may get lots of attention, but as of last week its "Ultimate College Directory" for 2004 had an Amazon sales rank of 22,305, worse than the Intercollegiate Studies Institute's "Choosing the Right College 2004" at 19,752.

How is a rankings addict to choose? Next time, perhaps, you would do well to consider an alternative: the informative (and surprisingly popular) "Cartoon Guide to Statistics." Its sales rank is 8,600.


Membership Revoked
15+ Year Member
Jul 15, 2003
That professor was the former national champion of domestic extemporaneous speaking (any of you who did speech/debate in high school know what I'm talking about). I saw him in an instructional video sponsored by the National Forensic League. Just thought I'm chime in about the professor's background...lol.. :)
About the Ads
This thread is more than 17 years old.

Your message may be considered spam for the following reasons:

  1. Your new thread title is very short, and likely is unhelpful.
  2. Your reply is very short and likely does not add anything to the thread.
  3. Your reply is very long and likely does not add anything to the thread.
  4. It is very likely that it does not need any further discussion and thus bumping it serves no purpose.
  5. Your message is mostly quotes or spoilers.
  6. Your reply has occurred very quickly after a previous reply and likely does not add anything to the thread.
  7. This thread is locked.