The Medical School Perspective
Having been chosen from thousands of applicants, each student represents a significant investment from the medical school’s perspective. Student affairs and curricular affairs staff know much more about students than their performance statistics may convey. School leadership is cognizant of stewardship in several areas, attending to multiple constituents’ concerns: current students’ success in the Match; the faculty’s and school’s reputation; admissions officers’ and future students’ opinions; and the public’s expectation of a highly competent physician workforce.
There are multiple new concerns in our current environment. Medical school applicants now routinely ask schools about both USMLE scores and match rates. All constituents are concerned with the increasing number of U.S. senior applicants to residency programs, increasing numbers of osteopathic and international applicants, and limited residency positions.
But which is a higher stewardship obligation for school officials: medical student advocate or objective reporter? There is often not consensus within an institution. Schools have access to information that may (unfairly) disadvantage an applicant, even if sound resolution has occurred. Some schools’ perception is that, at least initially, residency programs are looking for reasons to weed applicants out. Schools are concerned that the current application process does not provide adequate information about important qualities for a successful resident.
Striving for the best possible match for students is complex in the face of multiple considerations:
- Residency programs must review a large number of applications using limited resources, and determining best fit is difficult.
- Medical schools are expected to advocate for students to match with their desired program.
- The significance of past academic and professional concerns as predictors of future performance is unclear.
Further, there are multiple tensions at the school and administrator levels:
- The administrator’s “job” is to obtain the best match for an entire class, since the administrator’s performance evaluation may incorporate this outcome.
- Medical schools are expected to be stewards of public well-being and professional standards.
- There is a belief that all accepted students must finish medical school and match into residency programs.
- Students may have unrealistic expectations, at times fostered by faculty.
- Presenting comprehensive information about a student is challenging in light of the time crunch imposed by residency program requirements.
- Administrators may have incomplete or unreliable information about residency programs.
- Medical schools often have incomplete data about graduates’ subsequent performance (i.e., absence of a local “charting the outcomes” equivalent).
- Administrators and faculty have relationships and connections with students going back four years or more.
The goal, from a school, administrator, and student perspective, is a 100% match. Given the reality of the Match, particularly in highly competitive specialties, and sometimes unrealistic student expectations, this may remain an aspirational goal. Unless medical students’ specialty aspirations perfectly match the number and distribution of postgraduate training positions, disappointment is inevitable; however, this is not the consequence of residencies’ use of USMLE Step 1 scores or any other data in their decisions.
The Residency Program Perspective
Resident selection committees (RSCs) seek to identify and select applicants who are a “good fit” for their programs. They seek candidates who will benefit from, contribute to, and thrive in their training environments. RSC goals are to match individuals who will perform well during training and to equip them to succeed in their ultimate career paths.
Determining best fit and predicting resident performance are challenging. Multiple studies have attempted to determine which attributes predict success during residency, including the optimal method of selecting residents who possess these attributes. Factors that consistently rank as important in candidate selection include:
- Performance during medical school.
- USMLE Step 1 score.
- Alpha Omega Alpha Honor Medical Society and Gold Humanism Honor Society membership
- Medical school class rank
- Interview performance
- Letters of recommendation
- Extracurricular activities and leadership roles
- Personal statements
No consensus exists regarding the best predictors of resident success. RSCs necessarily rely on surrogate markers and a combination of objective and subjective data as listed above. Unfortunately, reliable objective data are extremely limited.
Use of USMLE Step 1 scores is common because the USMLE constitutes a national, standardized, objective measure. Frequently, students who have received highest honors have solid USMLE scores; however, unlike the USMLE, these other markers are not standardized across schools. While not designed to predict future residency or practice performance, in the absence of other standardized, objective data, USMLE scores have been used by programs as one surrogate marker of future performance. They have been correlated with medical school performance, In-Training Examination scores, and board certification pass rates. The last is important for the program as well as the future practitioner. Board pass rates are used to evaluate program effectiveness, and they affect accreditation and reputation.
Use of USMLE Step 1 scores or any single screening criterion for residency applicants may have unanticipated and undesirable effects. Standardized test scores often show significant variability across racial and ethnic groups. These differences may have the untoward effect of screening out underrepresented minority applicants to residency programs, despite the fact that differences in USMLE Step 1 scores do not predict performance on clinical task performance. This may impact an institution’s ability to fulfill its mission.
The MSPE is a critical component of residency applications, yet data contained within MSPEs can be incomplete and variable. Some medical schools do not follow AAMC guidelines for MSPEs. Also, because of the multiple and conflicting factors cited previously, often MSPEs are effectively another recommendation letter instead of an objective performance evaluation. Moving the MSPE release from November to October increased the likelihood that RSCs would include this letter in their initial reviews of applications. Early Match programs often still begin offering interviews before they receive applicants’ MSPEs. However, they can use the MSPE later when comparing applicants invited to interview and during subsequent ranking decisions.
The greatest value of the MSPE for the RSC is in its comparison of a student relative to his or her classmates at the same institution. However, across schools there are differences in grading scales and the percentage of students awarded the highest grades. Some schools have a pass/fail system. Not all schools rank their students. Understandably, it remains difficult to compare students from different schools using the MSPE.
In addition, it is equally difficult for RSCs to discern which applicants are most interested in their programs. Applicants submit a general application designed to appeal to a broad range of programs. The number of applications per applicant has increased dramatically in recent years and continues to rise. RSCs are more overwhelmed than ever by the volume of applications and supplemental materials received. They search for ways to compare candidates effectively and efficiently to determine which applicants to interview. Late interview cancellations and use of interview opportunities for “practice” by uninterested applicants hurt both the program and interested applicants who would have been eager to interview but did not receive an invitation.
Katsufrakis PJ, Uhler TA, Jones LD. The residency application process: pursuing improved outcomes through better understanding of the issues. Academic Medicine. 2016;20(10).
