Matchitis Survey 2008 Results Summary

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

RxnMan

Who, me? A doctor?
Moderator Emeritus
10+ Year Member
15+ Year Member
Joined
Jan 13, 2006
Messages
4,110
Reaction score
31
To all of those who participated in the survey, thank you. I hope to do this again next year. Here are the results:

50 people responded with their USMLE I scores, the number of applications they submitted, and the number of interview offers they received. Of those, we partial data, including USMLE II from 36, and class rank from 46. 44 provided final ROL position. 6 participants were DOs.

The average USMLE I in this survey was 228 (±17), higher than both the nat'l average (215) and the EM-matched US senior average (219) for 2005. This reflects the standard SDN selection bias of BKN’s earlier survey.

Of those who reported their final position on their ROL, 67% respondents got their #1, 19% got their #2, and 7% got their third choice. (93% matched in their top 3) Only one person had to scramble. Class rank did not differ significantly between those who got their 1st, 2nd or 3rd choice.

Mean number of applications sent out per person was 29 (±10), which garnered an average of 20 (±8) invites. The number of applications each senior sent out declined with increasing score as expected. The number of interviews offered did not correlate with either USMLE score. I interpret this as evidence of generally good mentoring, as excess applications were avoided.

This survey was intended to create a tool to help future US seniors applying in EM (per poster suggestion). To that end, for each responder, I divided the number of interviews offered by divided by the number of applications sent out for each participant. This ratio gives the “effectiveness” of a senior’s applications. E.g., an applicant with an effectiveness of 0.66 had to send out 3 applications to get 2 invites, whereas a more competitive applicant may have had an effectiveness of 0.75. Working backwards, future applicants can use their USMLE I score and class rank to gauge the effectiveness of their application, and project how many applications they need to send out to get a desired number of interviews*.

Effectiveness of applications ranged from 0.15 to 1.00, with a mean of 0.72. Effectiveness correlated positively with both USMLE I score (r^2 = 0.367, df = 48, p-value = 0.0001) and class rank (r^2 = 0.214, df = 44, p-value = 0.0012). This makes sense, as highly competitive candidates have generally high scores and class rank. USMLE II score was not correlated because not at all programs require applicants to report their exact score when interview offers are made.

Interestingly, only 58% of the difference between highly effective (or competitive) applications and less effective applications was accounted for by USMLE I score and class rank. Other factors not covered in this survey (letters of recommendation, extracurricular activities, or presence of graduate degrees) may explain the remaining variance.

*I can re-post the correlation equations if it is too difficult to see from the attachments. I had to re-size the graphs to post them.

Members don't see this ad.
 

Attachments

  • USMLE I Score vs. # Apps.JPG
    USMLE I Score vs. # Apps.JPG
    15.6 KB · Views: 915
  • USMLE I Score vs. # of Offers.JPG
    USMLE I Score vs. # of Offers.JPG
    16.1 KB · Views: 760
  • USMLE II Score vs. # of Offers.JPG
    USMLE II Score vs. # of Offers.JPG
    14.8 KB · Views: 698
  • USMLE I Score vs. Effectiveness.JPG
    USMLE I Score vs. Effectiveness.JPG
    14.9 KB · Views: 772
  • Class Rank vs. Effectiveness.JPG
    Class Rank vs. Effectiveness.JPG
    16.4 KB · Views: 740
Strong work! I'm about halfway through the multivariable and logistic analyses. I should be able to post tomorrow.
 
I'll say it this way. I still believe that sLORs play the biggest role after you get an interview other than the interview itself, but scores play the biggest role in actually getting the interview. So to all the people who try to rank what is most important, it is different for each step of the way.
Just my two cents.
 
Members don't see this ad :)
wow i'm a big fat outlier. apparently having a mentor is the way to go...
 
wow @ 40+ interview offers!
 
Of those who reported their final position on their ROL, 67% respondents got their #1, 19% got their #2, and 7% got their third choice. (93% matched in their top 3)

This pretty much matches up with the numbers the NRMP usually reports and what I've always heard as well. 66% match at #1 with just about over 90% matching at 1 of their top 3.
 
Strong work! I'm about halfway through the multivariable and logistic analyses. I should be able to post tomorrow.
Thank you. I put in a lot of effort.

I'll say it this way. I still believe that sLORs play the biggest role after you get an interview other than the interview itself, but scores play the biggest role in actually getting the interview. So to all the people who try to rank what is most important, it is different for each step of the way.
Just my two cents.
I don't know when the different components of applications are processed, and you could be right. I just mentioned sLORs because they are a component of applications that we didn't ask about (I don't know how you could empirically compare their effects anyways). I don't know when PDs actually read them.

The point of the correlations is just to get future applicants from point A (choosing # of programs) to point B (target # of interviews). Once we get this year's edition of "Charting Match Outcomes," students will be able to determine next year's "magic number" of interviews needed to garauntee a match, work backwards using these correlations, and get a good idea of how many applications they need to submit.

This pretty much matches up with the numbers the NRMP usually reports and what I've always heard as well. 66% match at #1 with just about over 90% matching at 1 of their top 3.
Yes. I quoted the USMLE and Match stats so we could compare between the SDN crowd and the rest of the world (and to put everything in one place for the next Matchitis crowd!).
 
Wow. Thanks, RxnMan, I know this must have taken an enormous amount of time. This is really interesting information. I will definitely contribute to your efforts this year.
 
Wow. Thanks, RxnMan, I know this must have taken an enormous amount of time. This is really interesting information. I will definitely contribute to your efforts this year.
Thanks. It was fun and informative. I'll probably start up again in Nov.

For this year's applicants, here's the NRMP 2007 results.
 
Thank you. I put in a lot of effort.

I don't know when the different components of applications are processed, and you could be right. I just mentioned sLORs because they are a component of applications that we didn't ask about (I don't know how you could empirically compare their effects anyways). I don't know when PDs actually read them.

The point of the correlations is just to get future applicants from point A (choosing # of programs) to point B (target # of interviews). Once we get this year's edition of "Charting Match Outcomes," students will be able to determine next year's "magic number" of interviews needed to garauntee a match, work backwards using these correlations, and get a good idea of how many applications they need to submit.

Yes. I quoted the USMLE and Match stats so we could compare between the SDN crowd and the rest of the world (and to put everything in one place for the next Matchitis crowd!).

Interesting. Are you saying there's no correlation between number of interviews received and USMLE score? I guess that makes sense if people with high USMLE scores applied to fewer places.
 
there probably is, but like you say, more competitive folks probably apply to fewer places (see the FAQ). that's why I came up with the effectiveness measure, because it takes out the effect of how many places that person applied to.
 
there probably is, but like you say, more competitive folks probably apply to fewer places (see the FAQ). that's why I came up with the effectiveness measure, because it takes out the effect of how many places that person applied to.

It's been a long time since I matched, but I feel kind of good that I had a relatively high "effectiveness measure" at about 0.95.

There was a paper that came out about the time I matched that showed that grades in your EM rotations were awfully important (more so than LORs and USMLE scores) but I suppose that is all included in the sLOR huh. You might want to look at that with your next survey.
 
there probably is, but like you say, more competitive folks probably apply to fewer places (see the FAQ). that's why I came up with the effectiveness measure, because it takes out the effect of how many places that person applied to.

You could look at the percentage of interviews offered based on the number requested to help eliminate the variability in the number of places people actually applied to. This would help eliminate the confounding caused by people with lower scores applying to more places and ending up with the same number of interviews, but a lower percentage.
 
You could look at the percentage of interviews offered based on the number requested to help eliminate the variability in the number of places people actually applied to. This would help eliminate the confounding caused by people with lower scores applying to more places and ending up with the same number of interviews, but a lower percentage.
I'm afriad I don't understand. The number of interviews divided by the number of applications is the effectiveness value that is shown in the original post.
 
Are we doing this again?
 
Top