IM match list analysis

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

pyrois

Full Member
10+ Year Member
15+ Year Member
Joined
Nov 22, 2006
Messages
673
Reaction score
2
I'm testing a little "match list analyzer" program algorithm that essentially breaks down matches (IM only, including other specialties at this time would take a lot more research) into 4 "tiers" and normalizes all ratings to a 4.0 scale.

A super crude analysis of the IM matches of various universities from 2007/2006.

1. Harvard - 3.85
2. Yale - 3.67
3. Hopkins - 3.61
3. Mayo - 3.60
3. UPenn - 3.56
4. UCSF - 3.34
5. WashU - 3.25
5. Stanford - 3.23
6. Pritzker - 3.13
7. Baylor - 3.04
7. Vanderbilt - 3.00
8. Northwestern - 2.85
8. UCLA - 2.84
8. U Mich - 2.82
8. U Pitt - 2.80
9. AECOM - 2.57
10. RWJ - 2.45
10. UCI - 2.44
10. USC Keck - 2.43
11. UMDNJ-NJMS - 2.28
11. SUNY D - 2.24
12. NYMC - 2.13

These are the only schools I've checked, but does it match up with your personal assessments of their relative IM matches?

Any schools you might be particularly interested in seeing entered into the program?

*update* fine tuning algorithm, updated numbers that changed

*update* since a couple people have asked "how much of a difference each point makes" I did a couple (non-statistically rigorous) manual tests, and there's a base error of around +/- .05

Translation, if a school is within .05 of another school (technically up to .10, but that's highly unlikely) their match lists can be considered "about the same."

Members don't see this ad.
 
good god man
 
I'm testing a little "match list analyzer" program algorithm that essentially breaks down matches (IM only, including other specialties at this time would take a lot more research) into 4 "tiers" and normalizes all ratings to a 4.0 scale.

That's more of just an average, not an algorithm. So you take Harvard and look at the "tier" of the IM hospital match. You sum the "tiers" and divide by the total matches. Maybe a percentage would be more appropriate for this.

So for Harvard (3.79) on your scale, you'd have about 95% chance of matching into a top tier (tier 4) school?
 
Members don't see this ad :)
Sigh

Does your tier system take into account that IM programs in good locations (eg. CA) may be more competitive than their reputation suggests?
 
do you have UCSF's IM matchlist? Please post it if you do, thanks.
 
Btw, how exactly did you divide up the "4 tiers" of IM programs?
 
do you have UCSF's IM matchlist? Please post it if you do, thanks.

Sorry, I had to use UCSF's 2006 match list in the program. It doesn't matter much though, since the standard deviation between 2002-2006 for UCSF in the algorithm is 0.04 so I doubt 2007 could produce a much different result.
 
Hey Pyrosis, thanks for doing this> some will call you a heretic for even trying to quantify the matchlists but I call you a revolutionary.

If you already haven't, look at these threads to get a better understanding of where different programs stand:
http://forums.studentdoctor.net/showpost.php?p=2611888&postcount=3

And while you're at it, can we get a score for UCSD, UMich, WashU, and vanderbilt?
 
Btw, how exactly did you divide up the "4 tiers" of IM programs?

More or less a composite of US News research and Hospital rankings with a greater emphasis on Hospital ranings.

The tiers run from 1 to 4 and for each category certain requirements must be met:

1. "Elite" match: Both officially and commonly recognized as one of the best places to match. (hospitals associated with Harvard, Hopkins, Mayo, UCSF, U Penn)

2. "1st tier" match: High level match. Associated with an academic institution. Tertiary hospital services available. (hospitals associated with places like Stanford, UCLA, UCSD, Columbia, Baylor, Cornell)

3. "2nd tier" match: Medium level match. Most associated with an academic institution. Tertiary hospital services available. Lower spectrum in terms of location and prestige. (hospitals associated with places like UC Davis, UC Irvine, Georgetown, NYMC)

4. "3rd tier" match: Basic level match. Most not associated with an academic institution. Tertiary hospital services mostly unavailable. Lower spectrum in terms of location and prestige. (I could list some, but most likely unless it's a hospital you go to, you wouldn't recognize the names)
 
Members don't see this ad :)
Hey Pyrosis, thanks for doing this> some will call you a heretic for even trying to quantify the matchlists but I call you a revolutionary.

If you already haven't, look at these threads to get a better understanding of where different programs stand:
http://forums.studentdoctor.net/showpost.php?p=2611888&postcount=3

And while you're at it, can we get a score for UCSD, UMich, WashU, and vanderbilt?

I wouldn't argue with them. I'd call myself a heretic for attempting to quantitatively rank match lists. The only reason why I dare do it, is that I believe the best way to rank any school is by measuring the performance of its graduates, and thus somebody should at least try.

I don't care if a school is ranked 50. If 100% of their students match into elite residency programs, I won't think twice about matriculating.

Also, thanks for the link! When I have more time on my hands, I'll read through that (and many other sources) and refine the tier categories.

Also, if you have a request for a school to be ranked (like UCSD, Umich, Vandy etc.) please send me a link to the most recent match list (I know where to find them, but it's time consuming enough to enter in all the information to be processed let alone stumbling through that match list thread looking for the lists themselves).
 
Well that explains why Mayo does so well in your algorithm. I don't know if it should be Tier 1 when many if not most IM applicants avoid it on the basis of location alone.
 
Vandy scored a 2.95, not bad:)
 
Well that explains why Mayo does so well in your algorithm. I don't know if it should be Tier 1 when many if not most IM applicants avoid it on the basis of location alone.

Mayo has always been a statistical outlier.

If you want to know their IM match, they only had 5 people in IM.

1 to Harvard
1 to Hopkins
2 to Mayo
1 to Cal Pacific in SF

Since it all goes by percentages, as the values are normalized, they look quite good. Consider that Hopkins matched 34% back into Hopkins and 13% into Harvard. Mayo matched 20% into Hopkins and 20% into Harvard. Even ignoring their Mayo self-matches, that's not bad.

In any case, Mayo and Hopkins are both in pretty lousy locations, but I'd still consider both to be very good residency matches. Location self-selection is still an important factor, which is why I only have 4 tiers. If I attempted to achieve a greater resolution, there would be far more "flip flops" due to location. In a 4 tier system, a hospital has to be significantly better or worse to change their tier. In the end, everything "averages out."
 
The IM message board has an ongoing thread about program rankings. I posted my own opinion of the tiers...here's a modified version:
Tier 1: JHU, MGH, Brigham, UCSF (most people will agree on this tier; the other tiers are constantly debated)
Tier 2: Penn, Duke, UW, Columbia, Yale, Stanford
Tier 2a: UChicago, Wash U, Michigan, UTSW
Tier 3: UCLA, Vanderbilt, Mayo, Beth Israel Deaconess
Tier 4: Cornell, Northwestern, UCSD, Emory, UAB, UVA, UNC, Mt. Sinai, Baylor, URochester, Pitt

This is mainly based on program reputation and how well graduates match into fellowships.
Competitiveness of getting in is different, because a lot of people choose based on location instead of pure reputation. Here's what I think are the top 10 most difficult IM programs to get into:
1. MGH
2. Brigham
3. UCSF
4. Stanford
5. Hopkins
6. Penn
7. Columbia
8. UCLA
9. Duke
10. UW

More or less a composite of US News research and Hospital rankings with a greater emphasis on Hospital ranings.

The tiers run from 1 to 4 and for each category certain requirements must be met:

1. "Elite" match: Both officially and commonly recognized as one of the best places to match. (hospitals associated with Harvard, Hopkins, Mayo, UCSF, U Penn)

2. "1st tier" match: High level match. Associated with an academic institution. Tertiary hospital services available. (hospitals associated with places like Stanford, UCLA, UCSD, Columbia, Baylor, Cornell)

3. "2nd tier" match: Medium level match. Most associated with an academic institution. Tertiary hospital services available. Lower spectrum in terms of location and prestige. (hospitals associated with places like UC Davis, UC Irvine, Georgetown, NYMC)

4. "3rd tier" match: Basic level match. Most not associated with an academic institution. Tertiary hospital services mostly unavailable. Lower spectrum in terms of location and prestige. (I could list some, but most likely unless it's a hospital you go to, you wouldn't recognize the names)
 
Is the "Elite" group you've listed comprehensive?

Stanford is a tougher match than Mayo, though some may question the quality of clinical training at Stanford (I think it's fine myself). In fact, Stanford is among the toughest IM matches due to its location and overall prestige.

Another program missing from the "Elite" group is Duke. Duke IM is easily at the level of Penn.

Columbia is not shabby either, definitely stronger than the Baylors and the UCSDs out there. Again, overall prestige + NYC location.

Finally, not all Harvards are created equal. Beth Israel Deaconess is not an "Elite" program, though it easily falls into your "1st tier" group.
 
The IM message board has an ongoing thread about program rankings. Here's the opinion I posted:
Tier 1: JHU, MGH, BWH, UCSF (most people will agree on this tier; the other tiers are constantly debated)
Tier 2: Penn, Duke, UW, Columbia, Yale, Stanford
Tier 3: UChicago, Wash U, Michigan, UTSW, UCLA, Vanderbilt, Cornell, Mayo
Tier 4: Northwestern, UCSD, Emory, UAB, UVA, UNC, Mt. Sinai, Baylor, URochester, Pitt

Yes, a pyramidal list, such as I requested! :D

You know, I've concluded that not even all "Tier 2" programs are contested. It seems like every list out there, USNews or individual SDN posters, has Penn and Duke immediately after the big 4. Opinions on the other tier 2s are more variable, as you've seen from the IM thread and others preceding it.

Of course, now we are just infiltrating the preallo thread...
 
Is the "Elite" group you've listed comprehensive?

Stanford is a tougher match than Mayo, though some may question the quality of clinical training at Stanford (I think it's fine myself). In fact, Stanford is among the toughest IM matches due to its location and overall prestige.

Another program missing from the "Elite" group is Duke. Duke IM is easily at the level of Penn.

Columbia is not shabby either, definitely stronger than the Baylors and the UCSDs out there. Again, overall prestige + NYC location.

Finally, not all Harvards are created equal. Beth Israel Deaconess is not an "Elite" program, though it easily falls into your "1st tier" group.

No, the lists are not comprehensive. In the calculations above, Duke has been ranked as a "1" (for elite, not tier 1). Stanford was originally ranked as a "1" as well, but after doing a little research and talking to some IM folk, most people prefer UCSF or Harvard. Beth Israel was also ranked as a "2" in the above calculations (separate from other Harvard programs).

If you don't believe me still, then one predictive sign is a low percentage of self-matches. All of the elites have a minority majority of people match into their own hospitals (see Harvard, Hopkins, Mayo, U Penn, Duke, UCSF). The elite medical schools (who produce students who can go anywhere they choose) send more students elsewhere (Both Yale and Stanford have a tendency to send their students to Harvard over their own programs).
 
In response to an earlier request, U Mich has been calculated at 2.82
 
The IM message board has an ongoing thread about program rankings. I posted my own opinion of the tiers...here's a modified version:
Tier 1: JHU, MGH, Brigham, UCSF (most people will agree on this tier; the other tiers are constantly debated)
Tier 2: Penn, Duke, UW, Columbia, Yale, Stanford
Tier 2a: UChicago, Wash U, Michigan, UTSW
Tier 3: UCLA, Vanderbilt, Mayo, Beth Israel Deaconess
Tier 4: Cornell, Northwestern, UCSD, Emory, UAB, UVA, UNC, Mt. Sinai, Baylor, URochester, Pitt

I actually agree with you on your tier ranking, but consider that in the 4 tiers that I'm using, all schools must fall into one of them. If I ranked Cornell, NW, UCSD, etc. as tier 4, then where do I put all the others that are significantly worse?

So if you try to squash tier 2 and tier 1 together, you may find that Columbia and Yale almost immediately drop out. Stanford and UW's are iffy, and Duke and Penn just baaarely slide in to the composite tier 1.
 
I actually agree with you on your tier ranking, but consider that in the 4 tiers that I'm using, all schools must fall into one of them. If I ranked Cornell, NW, UCSD, etc. as tier 4, then where do I put all the others that are significantly worse?

So if you try to squash tier 2 and tier 1 together, you may find that Columbia and Yale almost immediately drop out. Stanford and UW's are iffy, and Duke and Penn just baaarely slide in to the composite tier 1.

How about a 4.0, 3.5, 3.0, 2.5, 2.0 ... ranking system?
 
How about a 4.0, 3.5, 3.0, 2.5, 2.0 ... ranking system?

Or 1-8:p

When I have more time on my hands, I'll go back through all the schools and retier them on a 1-10 scale, but one of the reasons I prefer a "more vague" 1-4 scale is that if I only have 4 groups, it smooths out difference in location preference and other sorts of self-filtering.

There's also less arguing (although still much arguing) about where things go since there are fewer "histogram edges" so to speak.
 
For Hednej, WashU got a 3.25
 
Is this algorithm mostly just averaging the values you assigned to various residency program? Are you weighing the scale in somehow?

I'd also be interested in Downstate, since it geta so much press for its matchlist.

http://sls.downstate.edu/student_affairs/residency/res_place_2007.html

Thanks

The algorithm does a couple things. Although the main factor is the average program rating, it also takes into account inherent qualities of a match list, including self-matching, matching above/below "home" school, location modifiers, and in-state selection biases.

I'll run downstate in a sec.
 
I'd also be interested in Downstate, since it geta so much press for its matchlist.

SUNY Downstate's list calculated to be 2.24

Hard to tell since the only other unranked school up there is NYMC, but still not bad.
 
would be interesting to see if the ratings change around when comparing surgery or surgical subspecialties compared to brainiac IM stuff

edit// and do keck for fun

double edit// please. :)
 
would be interesting to see if the ratings change around when comparing surgery or surgical subspecialties compared to brainiac IM stuff

edit// and do keck for fun

double edit// please. :)

got a link to their list?

As for subspecialties, I personally don't know enough, nor do I know enough people who know enough for me to make any sort of judgment call on "rare" specialty rankings. Maybe in 4 years I'll know enough to rate my own specialty, but I'd need a lot of help undertaking something "specialty-wide."

IM is a good indicator of other specialty matching though. That's what I've noticed in the time I've spent looking over match lists.
 
UPenn at 3.56, monster list btw
 
Can you do Pitt? They seem to be ranked so well, but the matches don't seem to be as good....

Aannd, never mind. I guess I need the match list. Sorry

Found it: U Pitt got 2.60

*Update*
Typo, 2.80:p
 
Pritzker got a 3.13, yowza!

(fixed number)
 
Any chance you could run Stony Brook (and Buffalo too if you're bored)? I was going to ask for Downstate but you already ran it (yay!).

At least the 2.24 is sort of proof that they have a better match list than NYMC (was arguing about this in some thread lol).
 
would be interesting to see if the ratings change around when comparing surgery or surgical subspecialties compared to brainiac IM stuff

Well I think his basic premise was based on some notion that IM was the closest you were going to get to a barometer of how "good" a match list was, because the notion of what is "good" jibes the closest to the US News type ranking of med schools. If you change the specialty you get into the problem that many of the best programs no longer are ones that meet most people's radar. Schools the OP would peg as second and third tier are, in fact, top shelf in certain specialties, and the Harvard, Hopkins, Penn crowd may be best in IM, but not in everything,, so the analysis would fall apart if you move to any other specialty. This system depends on knowing which programs are best and the premise here is that you can use the US News list as a gauge in IM (but not necessarilly anything else).

Even in IM, the assumption that XYZ program is better than ABC is going to be off here and there, which is why you sit down with a mentor in your desired specialty and find out which programs are good versus malignant. To some extent there is subjectivity involved and I wouldn't try to supplant clinician subjectivity with research grant rankings, so bear in mind that this system can only be a very rough gauge. This is a very word of mouth business, driven to some extent by the personalities of individuals running the show at various departments, and not particularly influenced by US News.
 
Well I think his basic premise was based on some notion that IM was the closest you were going to get to a barometer of how "good" a match list was, because the notion of what is "good" jibes the closest to the US News type ranking of med schools. If you change the specialty you get into the problem that many of the best programs no longer are ones that meet most people's radar. Schools the OP would peg as second and third tier are, in fact, top shelf in certain specialties, and the Harvard, Hopkins, Penn crowd may be best in IM, but not in everything,, so the analysis would fall apart if you move to any other specialty. This system depends on knowing which programs are best and the premise here is that you can use the US News list as a gauge in IM (but not necessarilly anything else).

Even in IM, the assumption that XYZ program is better than ABC is going to be off here and there, which is why you sit down with a mentor in your desired specialty and find out which programs are good versus malignant. To some extent there is subjectivity involved and I wouldn't try to supplant clinician subjectivity with research grant rankings, so bear in mind that this system can only be a very rough gauge. This is a very word of mouth business, driven to some extent by the personalities of individuals running the show at various departments, and not particularly influenced by US News.

What Law2Doc said is pretty much dead on.

Since non-IM residencies differ a great deal from US News rankings, it would be impossible for me to accurately rank those specialties without performing a comprehensive review of all 100+ teaching hospitals for EACH specialty. This would mean nearly 2000 reviews for me to perform, which I'm not about to do.

On the other hand, I have a good number of friends who recently went through the match process for IM, and who have sat down with mentors from their respective schools to talk about "good/bad" programs.

So I essentially take US News hospital/research rankings, and correct them appropriately using information that seems to be common amongst the people I know (IM turns out to be pretty straight forward) and that's how I get the final relative ratings.

This, of course, makes the assumption that a good IM match list means good matches in other specialties. Although not always correct, it is highly correlative. You can take any school with a good IM match list, and look at any particular specialty (with a small exception for Columbia which matches 2 people into BNI each year, essentially bringing it to be number 1 for neurosurgery matches) and find that the school more or less compares relatively well with all other schools for that specialty.

If you don't believe me, try it yourself (and don't purposefully go and find outliers like Columbia + Neurosurgery) and you'll find it's a pretty good estimate.
 
pyrois, you are amazing!! I'm interested in IM, so all of this stuff is particularly relevant for me. Can you do Columbia? Here's the match list:

http://forums.studentdoctor.net/showpost.php?p=4929919&postcount=72

EDIT: I know you've got a lot of requests, but could you also run Cornell? Thanks!

Cornell Match List - Internal Medicine:
Cornell (7)
Columbia (3)
Vanderbilt (2)
Harvard/BI Deaconess (2)
UCLA
UCSD
NYU
UNC
National Naval Med Center
St. Vincents
N Shore U-Manhasset
 
Please run Duke.

I want to see the algorithm! (its not just an average based upon your tier rankings?)

-------------
DUKE MATCH LIST 2007


Internal Medicine University of Chicago Hospitals, Chicago, IL
Internal Medicine Duke University Medical Center, Durham, NC
Internal Medicine Duke University Medical Center, Durham, NC
Internal Medicine Johns Hopkins Hospital, Baltimore, MD
Internal Medicine Barnes and Jewish Hospital, Washington University, St. Louis, MO
Internal Medicine University of Washington Affiliated Hospitals, Seattle, WA
Internal Medicine Mount Sinai Hospital, New York, NY
Internal Medicine Duke University Medical Center, Durham, NC
Internal Medicine Stanford University Hospitals, Palo Alto, CA
Internal Medicine Johns Hopkins Hospital, Baltimore, MD
Internal Medicine Vanderbilt Medical Center, Nashville, TN
Internal Medicine Johns Hopkins Hospital, Baltimore, MD
Internal Medicine University of Washington Affiliated Hospitals, Seattle, WA
Internal Medicine Hospital of the University of Pennsylvania, Philadelphia, PA
Internal Medicine Hospital of the University of Pennsylvania, Philadelphia, PA
Internal Medicine University of Utah Health Sciences Center, Salt Lake City, UT
Internal Medicine Duke University Medical Center, Durham, NC
Internal Medicine Hospital of the University of Pennsylvania, Philadelphia, PA
Internal Medicine Vanderbilt Medical Center, Nashville, TN
Internal Medicine Hospital of the University of Pennsylvania, Philadelphia, PA
 
Damn Downstate and NYMC are significantly lower than the others! Ofcourse I expected the other places to match better, but to have such a big gap was kinda shocking especially since at a glance of their IM matches it looks like both do well. Guess L2D was right in saying just because there's a big name hospital doesn't mean the program is above average.. (e.g. I see Yale, Sinai, UCLA, Einstein, Mayo, Umich, Emory along with a buncha less-known NY/CT hospitals for NYMC)
 
Damn Downstate and NYMC are significantly lower than the others! Ofcourse I expected the other places to match better, but to have such a big gap was kinda shocking especially since at a glance of their IM matches it looks like both do well. Guess L2D was right in saying just because there's a big name hospital doesn't mean the program is above average.. (e.g. I see Yale, Sinai, UCLA, Einstein, Mayo, Umich, Emory along with a buncha less-known NY/CT hospitals for NYMC)

2.24 isn't that terrible for Downstate...not fantastic, but it's not exactly horrifying.

P.S. pyrois, run Stony Brook (in case you didn't see it in my other post). You're probably pretty tired of running this though lol.
 
2.24 isn't that terrible for Downstate...not fantastic, but it's not exactly horrifying.

P.S. pyrois, run Stony Brook (in case you didn't see it in my other post). You're probably pretty tired of running this though lol.

Haha never said it was a terrible number. I doubt any of us actually know what a terrible number is, if one even exists. I'm sure 2.13 and 2.24 aren't "bad," but it's hard to think otherwise when NYMC and DS are up against medical powerhouses on that list, and NYMC fell dead last with Downstate as a close second. Maybe we need some more lower-tiered schools to gauge better. I'll request it of Pyrois after the dust settles :D.
 
Man, I tried making a list to see if I could make some bootleg numbers myself (yes I'm kinda bored, it's either that or finish doing my taxes), and...wow...the sheer difficulty of what pyrois did is pretty insane lol.
 
OK, I sort of have some bootleg system going now, involving excel and this program me and this other guy wrote in high school, lol.

Unfortunately it would seem that my number is totally worthless right now since the way I'm ranking the schools is probably not at all like how pyrois is ranking them. Will have to retune these #'s, but it gave me 1.97 for Stony Brook. I think the problem is that I'm assigning a value of 1.0 for way too many hospitals =(

Lol, just looked at the data and realized that the stony brook matchlist abbreviated U Penn as UPMC so I gave it a value of 1.0...doh...

OK, after looking up a lot of these hospitals and checking their affiliations and services (ugh), it's spitting out a much improved score...this is gonna take a while lol.

Well, if anybody cares, my bootleg score for Stony Brook is a 2.58 (I based my tiers more or less on NIH funding, and pyrois earlier outline of how he tiered it). When I have more time I'll check how my version compares with his on stuff he's already run, lol. Speaking of which, it seems that a LOT of friggin' hospitals in NYC are affiliated with one med school or another, go figure.
 
Thanks Pyrosis.
It seems like UMich isn't doing as well as they claim to be doing(at least in IM). I mean come on, only 1 Mass General/Harvard?
Edit: Oh and no Hopkins or UCSF?
 
Please run Duke.

I want to see the algorithm! (its not just an average based upon your tier rankings?)

Hey, I'll run a couple more when I get back from class, but once I'm "done" (I'm still tuning things a bit, and bug checking) I'll release the algorithm.

It probably won't be very helpful to anybody except statisticians who know how to code (since it's all done in PHP) but I'll try to explain it as best as possible.

It's actually turning out rather nice. I use a lot of natural, and thus objective/intrinsic trends from the lists themselves to generate location/financial modifiers.
 
Hey pyrois, are you considering prelim/primary matches at all, and if so, how are you considering them? What about people who don't match into an internal and only get a prelim match?
 
Top