These are great questions. I'll try to tackle them in depth, but it's going to take a while for me.
I thought it might help to understand process for selection for interview at my place. You can glean what I'm looking for at this stage of the application process--it only applies to selection for interview, not ranking after interview.
Here's what I do:
1. I download the ERAS application data into an excel spreadsheet which allows me to create custom parameters and is easier for me to filter/sort and quickly review. ERAS allows you to download certain parameters. Each applicant is a row, and I type notes and create formulas into custom columns that I create.
2. I personally select the resident applicants that we will interview from the hundreds of applications we get. It’s just too slow and hard to do this with a committee. At my old institution, this was also done by essentially 1 person (not me back then), but I don’t know if they have changed that. I have a “2nd reviewer” for borderline cases, maybe I use that for 3-4 applications each year.
3. If an applicant has taken USMLE Step 2, I average that score with Step 1. If an applicant hasn’t’ taken USMLE Step 2, I add 5 points to their Step 1 score and create a “derived Step 2” score (edit 3/2018--for the 2018 Match, we just used the Step 1 score as the person's Step 2 score for our spreadsheet/formulas--this really does underestimate the Step 2 score, since most people do better on Step 2 than Step 1). In our applicant pool, the average applicant has a Step 2 score that is about 7-10 points higher than Step 1. So it hurts you a little if you haven’t taken Step 2 because my assumed score for you is not as high as it statistically would be if you are the average applicant.
4. Turns out I apply a formula to the “combined USMLE score” that discounts super-high achievement. Essentially, as your score on Step 1 or Step 2 gets higher from 250, you get less added points. For example, someone who gets 250 on Step 1 and Step 2 has a combined score of 500 in my system. However, someone who gets 265 on Step 1 and Step 2 has a combined score of 515 in my system, not 530 (and the score is capped there, meaning even higher scores don't add points). I don’t want super-high achievement on USMLE to dominate an applicant’s eventual “overall score”, or to make up for lower clinical grades and interview scores. This last year, in our program, our applicants had a mean combined USMLE score of 494, and a median of 498, with a standard deviation of 19. Remember, those scores have been adjusted to discount performance substantially above 250 in a formulaic way that progressively discounts numbers the farther above 250.
5. We filter out the applications based on USMLE scores, but we use a really low threshold—in our case, we use a “soft” 220 USMLE step 1 score, which generally means I will consider applicants in the 215-220 range based on strength of school and how well they did on step 2, as well as other factors. Right now the average Step 1 score for all medical students is about 228 or so, I believe, so allowing someone to be as low as 215 in our case means you don’t necessarily have to be book smart to get into our program. However, radiology boards is now a computerized test, and we don’t want to worry about residents who may become outstanding radiologists but who might struggle with tests. There is too much of a penalty for our program in future applicant perception if one of our residents fails the boards. (edit 3/2018: despite our willingness to look at applicants with lower than average board scores, the average Step 1 score for the applicants we matched in March 2018 was 250, and the average Step 2 score was 258).
6. Our residency essentially filters out applicants who are IMGs—to be honest, I think there are some outstanding candidates in this group particularly from those individuals who are not from the US, but the problem is that it is really difficult to find those for me because we don’t use test scores as much in our ranking process. Communication skills are very important to us, and that can be a sticking point for some IMGs who did not grow up in the US that we don’t understand until the interview, and I don’t want to waste people’s time with interviews if there is a low chance of success. On the other hand, we understand there are some life circumstances and other legitimate reasons why some applicants who are US based ended up doing medical school internationally. So I download these into my spreadsheet, dig deeper at maybe the candidates with board scores above 250 to see if I recognize the school, review publications, special experiences, special circumstances, etc. Sometimes a colleague will ask me to look carefully at a person that someone in their field has highlighted for them. We do interview a few IMGs every year (< 5), and some are quite good. However, they face hurdles all along the way in our ranking process—just being honest.
7. We do the same as #6 for DO candidates. We do interview a few every year (< 3), but we believe there is a penalty for our program in future applicants if we have a number of DO residents, mainly because there is the perception that we couldn’t attract the best MD candidates. It’s unfortunate for some DO students who are going to be great, but it is reality.
8. One of the filters I use to select who to interview is 3rd year core clerkship performance. This is a really tough metric, since the schools are all over the place. Believe it or not, I spend the time to create a “translation formula” that looks at the % of Honors/High Pass/Pass from each US medical school from which we receive an application, so I can compare the performance of students from different schools—even then, it’s not easy and likely not accurate. For example, just looking at my spreadsheet for this year, I see that for the core clerkships that we review, at the University of Central Florida 52% of students got Honors and 48% got High Pass (no one just passed), whereas at Florida International University 14% got Honors and 29% got High Pass. I have a convoluted formula that tries to “normalize” this data, so that the student at FIU that got High Pass is given the same number of points as the student at UCF that got Honors.
9. Since I have to actually open up the application to get the 3rd year clerkship performance and % honors/high pass/pass data, I do quickly jot down a few notes about the candidate at this time—I’ll jot down a few sentences about the candidate about their particular timeline, skimming the personal statement, looking quickly at the research history, etc. I quickly look at the Dean’s letter (if available) to look for red flags (repeat courses). I do put down what “quartile” the Dean’s letter says you are in. I don’t have time to review LORs at this point UNLESS I can tell that the candidate is probably going to be on my “borderline” for selecting for interview. For example, here is a typical fictional set of notes that I might jot down in my “comment” box for a candidate at this point: “Brown undergrad; 1 yr gap spent as research intern for startup and also doing volunteer work; PS specifically mentions us”. In the “Dean’s letter” box, I might put “2nd quartile”.
10. My spreadsheet combines the “3rd year clerkship score” with the USMLE average score (either real or derived) in a way that weights the clerkship score. This gives each applicant a “non-interview” score—that is, their score without consideration of the interview. As you will see later, the eventual evaluation of a candidate relies more on the interview than this score. But this is the metric that helps us decide who to interview.
11. I sort my spreadsheet using the “non-interview” score to decide who to interview. For about 67% of our interview slots, I just take them from the top. For the final 33%, I use a different lower threshold for interviewing applicants who are considered “local” (from our med school and schools within about 100 miles of our urban program), “regional” (about 500 miles), and “national” (everyone else). The reason we do this is because we find that applicants are more likely to not cancel interviews and match with us if they are local or regional. We also don’t want to piss off our school/the local schools and not interview their students—to a degree. We won’t interview if someone is really not going to be up to snuff based on performance. If a national candidate has ties to our area that are obvious in the application (I look at permanent address and undergraduate location), then they get put in the local pile. Similarly, if the applicant did a rotation with us, we consider them in the local pile even if they aren’t. However, we don’t take that many medical students outside our own medical school for rotations. Along with the varying thresholds based on geography, I look at my comments and the “Dean’s letter comments” to decide who to select for these final 33%.
12. If the applicant is AOA, they almost always get an interview. Turns out they always score above my threshold on the non-interview score anyway (makes sense, since AOA status is generally a function of traits that are well reflected in the USMLE scores and core clerkship grades). However, we sometimes have an AOA student who I end up not interviewing, because of something in the application that is a red flag that I can easily see from my spreadsheet (repeating a course, something in the Dean’s letter).
13. We slightly “overinterview” in our program—basically, interview about 10-14 applicants for every spot, even though we typically fill our spots at the 5-8 applicant/spot filled mark—and even then, about 3/4ths of our class is filled before the 5 applicant/spot mark. The reason we do this is because we don’t trust that our combined “USMLE + clinical clerkship” score is so precise that we can rely on it, and we sometimes find that applicants we end up ranking fairly high would not have been offered an interview with us if we had not “overinterviewed”. (note: in 2018, we ended up filling at the 4 applicant/spot mark, so we are reducing the number of people we interview).
So, a few things should be evident so far, regarding “how to get an interview” at my program:
1. Do well on USMLE tests, but no need to ace it. Does it help to get a 270 vs. a 250?—not really.
2. Do well on 3rd year core clerkships.
3. Be local or communicate your connection to my community in some way—it lowers the threshold for you getting an interview. Say it in the personal statement if it is true. Even then, you might want to email me in advance if that local connection isn’t obvious in the application.
4. I don’t have time to consider whether you decided not to do a rotation with us at the “select for interview” stage. I don’t have time or an easy way to consider the strength of your research record. I don’t have time to look at your Dean’s letter in depth beyond just trying to make sure there is no coded “red flag” and to understand your relative performance. I don’t have time to consider your extracurricular activities.
Once you get selected for interview, the selection metrics become more nuanced—another long discussion for another day.
