Relooking Research Ranking Methodology

I've never been a big fan of the US News rankings, but I do understand that they often play an important role in the decision making process for alot of people. As I've visited more and more schools, I've come to realize that those rankings don't really reflect the quality of reasearch or the access of medical students to take part in studies.

US News weights the total NIH research money recieved 20%. I think that this overly favors the largely institutions and those with a network of affiliated hospitals. Granted, I understand that this is important, but I'm not sure it is as important as the funding received by each professor, which allows for a more equal comparison among schools. US News gives a 10% weighting to this, but it divides the total NIH funding by the total number of faculty...still not perfect.

I've done some other calculations which I think are more telling, although no single number can accurately reflect the spirit of any institution. I found the results to be interesting if nothing else. I hope this helps some of you as well. Take it all with a grain of salt...please. I don't want to start some showdown here.

Here's what I found for the schools to which I applied. If you want me to run the numbers on other schools, let me know. Some schools, however, did not supply all the required information. All numbers were from US News.

Research Funding Per NIH Grant

Stanford: $1,315,000 (I'm suspicious of this number)
UCSF: $469,000
Wake Forest: $452,000 (shocked)
Duke: $433,000
Harvard: $415,000
Columbia: $404,000
UPenn: $365,000
UNC: $357,000
Yale: $355,000
Brown: $337,000
Vanderbilt: $307,000
Dartmouth: $278,000

Research Funding Per Total Faculty - As Used by US News

Stanford: $332,000
UCSF: $262,000
UPenn: $239,000
Duke: $191,000
Yale: $190,000
Harvard: $171,000
UNC: $167,000
Vanderbilt: $159,00
Brown: $147,000
Wake Forest: $141,000
Columbia: $123,000
Dartmouth: $117,000

Research Funding Per Total Faculty Involved in NIH Research

Ok this one only counts the faculty that are doing NIH research rather than just the total faculty...a little more telling in my opinion. Not all schools report this.

Dartmouth: $522,000
UNC: $472,000
UPenn: $464,000
Columbia: $379,000
Brown: $342,000
Harvard: $338,000
Vanderbilt: $292,000
Yale: $288,000

 

My intention is not replace the ranking system, but just provide another means of looking at schools. That's why I only did it for the schools to which I applied. I've added UChicago. I will continue to update upon requests.

Research Funding Per NIH Grant

Stanford: $1,315,000 (I'm suspicious of this number)
UCSF: $469,000
Wake Forest: $452,000
Duke: $433,000
Harvard: $415,000
Columbia: $404,000
UPenn: $365,000
UNC: $357,000
Yale: $355,000
UChicago: $352,000
Brown: $337,000
Vanderbilt: $307,000
Dartmouth: $278,000

Research Funding Per Total Faculty - As Used by US News

Stanford: $332,000
UChicago: $263,000
UCSF: $262,000
UPenn: $239,000
Duke: $191,000
Yale: $190,000
Harvard: $171,000
UNC: $167,000
Vanderbilt: $159,00
Brown: $147,000
Wake Forest: $141,000
Columbia: $123,000
Dartmouth: $117,000

Research Funding Per Total Faculty Involved in NIH Research

Ok this one only counts the faculty that are doing NIH research rather than just the total faculty...a little more telling in my opinion. Not all schools report this.

Dartmouth: $522,000
UChicago: $486,000
UNC: $472,000
UPenn: $464,000
Columbia: $379,000
Brown: $342,000
Harvard: $338,000
Vanderbilt: $292,000
Yale: $288,000

 

You have some valid points. I would agree that there is something to be said for the research "Disney Land" that is Harvard and all of its associated hospitals. There is probably more opportunity there to find a professor working on something unique or of special interest that anywhere else in the world. In addition, you might get alot if interdisciplinary collaboration as well. However, to make this the sole indicator of research quality is misleading. It's like saying GM has the best production team because they sell the most cars. It's a matter of scale. Look at how many more primary investigators they have than any other school in the country.

Thus, I think you have to look at other factors as well. I noted above that I'm not trying replicate any sort of ranking system. I just wanted to provide another window through which I could evaluate the schools to which I applied. Plus, every time I went to a school, they would talk about how they were "Top 5" in some category for research based on some quantifiable factor. Out of that, I sat down with the numbers (I'm currently unemployed) and cranked a few things out.

Now, to address your concerns about a bias towards small schools, I think that is the exception not the rule. Yes, there are some suprises (Wake and Dartmouth), but if you look at all three of the categories that I provided, you'll see that those schools do not repeat as top performers. So while they may sneak into one of the groupings, they don't top the list all the time. Thus, if you look at all three of the groups I presented, plus the total NIH Funding received (the more traditional rankings) I think you see a general trend of which schools are consistently in the top 10 or whatever. All I did was to provide another means of looking at the schools. Also, if it had a general bias towards smaller schools, you would see Dartmouth and Brown topping each category. This is not the case.

Additionally, since I have to go by the numbers provided by US News, without having access to the pure data, I can't assess the validity of what each school is reporting. So maybe Wake is just a glitch, just like I have a sneaking feeling that Stanford may be as well.

So, as I said before, take it all with a grain of salt. This is not me establishing a definitive ranking system. Who am I to judge. It's a reference. Make of it what you want. Take from it what you want.

R