- Joined
- Jan 13, 2006
- Messages
- 4,110
- Reaction score
- 31
Introduction
Every year, thousands of medical school seniors make decisions about what specialty they want to practice. As they do so, they create their specialty application via the Electronic Residency Application Service (ERAS). This process is time-consuming and expensive, and applying students have little free time or money.
In the literature, there are several publications discussing how programs directors favor certain applicant attributes (AOA membership, USMLE scores, research), but there is no information on how many applications are required to gain enough interviews to successfully match in emergency medicine (EM). Similarly, there is no information on how different attributes affect where applicants will fall on their final rank order lists (ROL). This information would allow students to better tailor their applications such that they do not over apply, thus overspending, or under-apply, and risk not gaining a residency position.
The purpose of this survey was to obtain this information from fourth-year medical students applying to EM residency programs.
Methods
Through the online professional forum StudentDoctor.Net, fourth-year medical students applying to EM residencies were contacted anonymously. This year's survey is an extension of a similar one conducted last year, the results of which can be found here.
This year's participants hey were asked to provide the following information: USMLE I and II scores, class rank, the number of applications they sent out and the interview offers they received, their grade in their EM rotations at their home and away programs (if available). They were also asked to state if and how many research publications, presentations they had put on their applications, whether they were a member of AOA, or if they had a graduate degree. After the residency match, participants were contacted anonymously via the forum to provide where on their ROL they matched.
While this survey did not seek institutional review board approval, volunteers submitted this information to the author under pseudonyms; these submissions were subsequently de-identified prior to analysis.
Analysis of the data was completed using Microsoft Excel. Application effectiveness (AE) is defined as the number of offers received divided by the number of applications sent out. AE was calculated for each applicant, and correlated to USMLE scores and class rank. Correlations based on AE were converted into t-scores and then p-values to determine significance. The AE of sub-populations with dichotomous attributes (e.g., membership in AOA) was grouped and compared to the rest of the volunteers using 2-tailed t-tests with unequal variances. The total number of abstracts and presentations were treated as one value for each subject (referred in the paper as just "presentations"). The effect of publications and presentations were compared by subdividing the study population into groups who had 0,1-4, or 5+ publications or presentations. This approach was used by the NRMP in Charting Outcomes 2007.
All t-tests were conducted using an a priori alpha of 0.05 and were Bonferronni-corrected as appropriate for multiple observations.
Results
2009 Survey:
This year 56 people volunteered USMLE I scores, the number of applications they submitted, and the number of interview offers they received. Of those, I received have USMLE II scores from 51, and class rank from 54. 52 provided final ROL position.
The average USMLE I in this survey was 226 (±16), higher than both the nat'l average (215) and the EM-matched US senior average (219) for 2007 (Link here).
The study population sent out a total of 1727 applications and received 1056 interview offers. Mean number of applications sent out per person was 30 (±15), which garnered an average of 18 (±7) invites. Of those who reported their final position on their ROL, 54% respondents got their top choice, 23% got their second, 12% their third, and 12% matched to their fourth or higher place on their ROL (88% matched in their top 3).
As in the previous survey, the number of interviews offered per person correlated poorly with both USMLE scores and class rank (r^2 for all <0.01). The number of applications each senior sent out declined with increasing score.
The AE of applications correlated positively with USMLE Step I (r^2 = 0.12, df = 56, p-value = 0.0085), Step II (r^2 = 0.29, df = 51, p-value = 0.0004), and class rank (r^2 = 0.07, df = 54, p-value = 0.051). No significant differences between USMLE scores or rank were found amongst those who got their 1st, 2nd, 3rd, or fourth+ choice on their ROL.
The AE of subjects who were AOA members was significantly higher than those who were not members (p-value < 0.0001). All other attributes – presence of a grad degree, amount of research as evidenced by papers and presentations and EM rotation grades did not show significant differences in AE.
Grouped 2008 and 2009 Data:
This year's survey was more extensive than last year's, yet several sets of the results can be pooled between both years. The pooled data is presented here in this section. Both surveys collected a total of 106 submissions. 104 volunteered their Step I score, 85 their Step II, 99 their class rank, and 89 their final ROL position.
The average Step I was 228 (±16) and Step II was 237 (±17). The group sent out a total of 3137 applications and received 2054 interview offers, for a per-person average of 29 (±13) and 19 (±7), respectively. 59% respondents got their top choice, 22% got their second, 10% their third, and 9% matched to their fourth or higher place on their ROL (91% matched in their top 3).
The number of interviews offered per person correlated poorly with both USMLE scores and class rank (r^2 for all <0.01). The number of applications seniors sends out declines with increasing score.
Volunteer's AE correlated positively with USMLE Step I (r^2 = 0.19, df = 102, p-value < 0.0001), Step II (r^2 = 0.31, df = 83, p-value < 0.0001), and class rank (r^2 = 0.12, df = 97, p-value = 0.0005). No significant differences between USMLE scores or rank were found amongst those who got their 1st, 2nd, 3rd, or fourth+ choice on their ROL.
Discussion
As before, this survey was intended to create a tool to help future US seniors applying in EM. To that end, future applicants can use these results, their USMLE scores, and class rank, to gauge the AE of their application. From there, they should be able to predict how many applications they need to send out to get a desired number of interviews. Using Charting Outcomes data furnished by the NRMP, an applicant should be able to determine how many interviews they need to have a given likelihood of matching in EM. With the results of this survey, an applicant can determine, based on their medical school performance, how many applications they need to send out to get any desired chance of matching.
2009 Survey Discussion:
The rate of matching to one's top three choices was lower this year than in the grouped data (88% vs 91%). This may reflect a more competitive match, but it may also be a result in a variability in responses, or more trust on the part of volunteers as the survey enters it's second iteration. While the author has specifically asked for more ‘marginal' applicants with lower scores to participate, USMLE scores reported by volunteers was higher than the national average for all test-takers and all EM applicants. SDN itself may be a significant barrier to getting more realistic results, as only higher performing students may know about the website and are more willing to submit their medical school performance to an anonymous forum.
AOA status was the only attribute studied that had significantly higher application effetiveness. This was attributed to the small number of participants who had AOA status (n = 4) who all had relatively high AE values. Sub-groups divided on presence of research and graduate degrees, etc, did not have higher AE. It is likely that these attributes actually do make an applicant appear more promising to PDs, but they are simply a small part of the overall picture.
This survey had a much better rate of data completeness compared to the 2007 survey. This was attributed to an improved data submission form and instructions.
Grouped Data Discussion:
Interestingly, AE correlated the most with Step II score. USMLE II score was not correlated in the earlier survey because not at all programs require applicants to report this value when interview offers are made. I decided to calculate it this time given rumors of PDs placing increasing values on Step II. This result may be an artifact because out of the 85 subjects who submitted their Step II, 70 improved their scores an average of 16 points (±10). The old wisdom was to only report Step II scores when it benefits the applicant – those with high Step I's only needed to report their first score. Those who only reported Step I scores in these surveys had higher Step I scores (237, ±15). In post-hoc analysis, taking the applicant's highest score scores correlated with AE as well as Step II score alone (r^2 = 0.32 vs. r^2 = 0.31). This may show that PDs simply use the higher of the two scores when deciding who to interview.
Interestingly, only 62% of the difference between highly effective (or competitive) applications and less effective applications was accounted for by USMLE scores and class rank. Other factors not covered in this survey (letters of recommendation, extracurricular activities, and interview performance) may explain the remaining variance.
Conclusion
Students with high USMLE scores and class rank have higher AE when applying to EM. These values have been correlated for the convenience of readers who may be applying in the future and wish to predict their chances of a successful residency match. Step II score correlates the most strongly with AE, followed by Step I and class rank. These values have changed slightly between the two survey years, but more data is required to determine if these are real trends or simply random yearly variation. The survey has been expanded and improved greatly, and this will aid in determining the amount attributes such as research, AOA status, and graduate degrees affect AE.
To all of those who participated in the survey, thank you for your participation. Feel free to comment or ask questions.
Every year, thousands of medical school seniors make decisions about what specialty they want to practice. As they do so, they create their specialty application via the Electronic Residency Application Service (ERAS). This process is time-consuming and expensive, and applying students have little free time or money.
In the literature, there are several publications discussing how programs directors favor certain applicant attributes (AOA membership, USMLE scores, research), but there is no information on how many applications are required to gain enough interviews to successfully match in emergency medicine (EM). Similarly, there is no information on how different attributes affect where applicants will fall on their final rank order lists (ROL). This information would allow students to better tailor their applications such that they do not over apply, thus overspending, or under-apply, and risk not gaining a residency position.
The purpose of this survey was to obtain this information from fourth-year medical students applying to EM residency programs.
Methods
Through the online professional forum StudentDoctor.Net, fourth-year medical students applying to EM residencies were contacted anonymously. This year's survey is an extension of a similar one conducted last year, the results of which can be found here.
This year's participants hey were asked to provide the following information: USMLE I and II scores, class rank, the number of applications they sent out and the interview offers they received, their grade in their EM rotations at their home and away programs (if available). They were also asked to state if and how many research publications, presentations they had put on their applications, whether they were a member of AOA, or if they had a graduate degree. After the residency match, participants were contacted anonymously via the forum to provide where on their ROL they matched.
While this survey did not seek institutional review board approval, volunteers submitted this information to the author under pseudonyms; these submissions were subsequently de-identified prior to analysis.
Analysis of the data was completed using Microsoft Excel. Application effectiveness (AE) is defined as the number of offers received divided by the number of applications sent out. AE was calculated for each applicant, and correlated to USMLE scores and class rank. Correlations based on AE were converted into t-scores and then p-values to determine significance. The AE of sub-populations with dichotomous attributes (e.g., membership in AOA) was grouped and compared to the rest of the volunteers using 2-tailed t-tests with unequal variances. The total number of abstracts and presentations were treated as one value for each subject (referred in the paper as just "presentations"). The effect of publications and presentations were compared by subdividing the study population into groups who had 0,1-4, or 5+ publications or presentations. This approach was used by the NRMP in Charting Outcomes 2007.
All t-tests were conducted using an a priori alpha of 0.05 and were Bonferronni-corrected as appropriate for multiple observations.
Results
2009 Survey:
This year 56 people volunteered USMLE I scores, the number of applications they submitted, and the number of interview offers they received. Of those, I received have USMLE II scores from 51, and class rank from 54. 52 provided final ROL position.
The average USMLE I in this survey was 226 (±16), higher than both the nat'l average (215) and the EM-matched US senior average (219) for 2007 (Link here).
The study population sent out a total of 1727 applications and received 1056 interview offers. Mean number of applications sent out per person was 30 (±15), which garnered an average of 18 (±7) invites. Of those who reported their final position on their ROL, 54% respondents got their top choice, 23% got their second, 12% their third, and 12% matched to their fourth or higher place on their ROL (88% matched in their top 3).
As in the previous survey, the number of interviews offered per person correlated poorly with both USMLE scores and class rank (r^2 for all <0.01). The number of applications each senior sent out declined with increasing score.
The AE of applications correlated positively with USMLE Step I (r^2 = 0.12, df = 56, p-value = 0.0085), Step II (r^2 = 0.29, df = 51, p-value = 0.0004), and class rank (r^2 = 0.07, df = 54, p-value = 0.051). No significant differences between USMLE scores or rank were found amongst those who got their 1st, 2nd, 3rd, or fourth+ choice on their ROL.
The AE of subjects who were AOA members was significantly higher than those who were not members (p-value < 0.0001). All other attributes – presence of a grad degree, amount of research as evidenced by papers and presentations and EM rotation grades did not show significant differences in AE.
Grouped 2008 and 2009 Data:
This year's survey was more extensive than last year's, yet several sets of the results can be pooled between both years. The pooled data is presented here in this section. Both surveys collected a total of 106 submissions. 104 volunteered their Step I score, 85 their Step II, 99 their class rank, and 89 their final ROL position.
The average Step I was 228 (±16) and Step II was 237 (±17). The group sent out a total of 3137 applications and received 2054 interview offers, for a per-person average of 29 (±13) and 19 (±7), respectively. 59% respondents got their top choice, 22% got their second, 10% their third, and 9% matched to their fourth or higher place on their ROL (91% matched in their top 3).
The number of interviews offered per person correlated poorly with both USMLE scores and class rank (r^2 for all <0.01). The number of applications seniors sends out declines with increasing score.
Volunteer's AE correlated positively with USMLE Step I (r^2 = 0.19, df = 102, p-value < 0.0001), Step II (r^2 = 0.31, df = 83, p-value < 0.0001), and class rank (r^2 = 0.12, df = 97, p-value = 0.0005). No significant differences between USMLE scores or rank were found amongst those who got their 1st, 2nd, 3rd, or fourth+ choice on their ROL.
Discussion
As before, this survey was intended to create a tool to help future US seniors applying in EM. To that end, future applicants can use these results, their USMLE scores, and class rank, to gauge the AE of their application. From there, they should be able to predict how many applications they need to send out to get a desired number of interviews. Using Charting Outcomes data furnished by the NRMP, an applicant should be able to determine how many interviews they need to have a given likelihood of matching in EM. With the results of this survey, an applicant can determine, based on their medical school performance, how many applications they need to send out to get any desired chance of matching.
2009 Survey Discussion:
The rate of matching to one's top three choices was lower this year than in the grouped data (88% vs 91%). This may reflect a more competitive match, but it may also be a result in a variability in responses, or more trust on the part of volunteers as the survey enters it's second iteration. While the author has specifically asked for more ‘marginal' applicants with lower scores to participate, USMLE scores reported by volunteers was higher than the national average for all test-takers and all EM applicants. SDN itself may be a significant barrier to getting more realistic results, as only higher performing students may know about the website and are more willing to submit their medical school performance to an anonymous forum.
AOA status was the only attribute studied that had significantly higher application effetiveness. This was attributed to the small number of participants who had AOA status (n = 4) who all had relatively high AE values. Sub-groups divided on presence of research and graduate degrees, etc, did not have higher AE. It is likely that these attributes actually do make an applicant appear more promising to PDs, but they are simply a small part of the overall picture.
This survey had a much better rate of data completeness compared to the 2007 survey. This was attributed to an improved data submission form and instructions.
Grouped Data Discussion:
Interestingly, AE correlated the most with Step II score. USMLE II score was not correlated in the earlier survey because not at all programs require applicants to report this value when interview offers are made. I decided to calculate it this time given rumors of PDs placing increasing values on Step II. This result may be an artifact because out of the 85 subjects who submitted their Step II, 70 improved their scores an average of 16 points (±10). The old wisdom was to only report Step II scores when it benefits the applicant – those with high Step I's only needed to report their first score. Those who only reported Step I scores in these surveys had higher Step I scores (237, ±15). In post-hoc analysis, taking the applicant's highest score scores correlated with AE as well as Step II score alone (r^2 = 0.32 vs. r^2 = 0.31). This may show that PDs simply use the higher of the two scores when deciding who to interview.
Interestingly, only 62% of the difference between highly effective (or competitive) applications and less effective applications was accounted for by USMLE scores and class rank. Other factors not covered in this survey (letters of recommendation, extracurricular activities, and interview performance) may explain the remaining variance.
Conclusion
Students with high USMLE scores and class rank have higher AE when applying to EM. These values have been correlated for the convenience of readers who may be applying in the future and wish to predict their chances of a successful residency match. Step II score correlates the most strongly with AE, followed by Step I and class rank. These values have changed slightly between the two survey years, but more data is required to determine if these are real trends or simply random yearly variation. The survey has been expanded and improved greatly, and this will aid in determining the amount attributes such as research, AOA status, and graduate degrees affect AE.
To all of those who participated in the survey, thank you for your participation. Feel free to comment or ask questions.
