Class of 2023 Step 1 Scores Possibly may be converted to P/F on Residency Application per USMLE town hall

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
Dude, there are dozens of threads like this every year in School X versus Y; it's more common than you think. Intuitively I'm sure you can grasp that of those with multiple acceptances, many will have a more well-known choice, and some will decline that choice. People choose less prestigious schools all of the time for a variety of reasons, like money, proximity to a support system, or just overall fit. The prospect of P/F Step 1 is another factor worth considering, and being blind to this when making that decision isn't fair to the class of 2023.
I've def read and participated in dozens of those threads, but we're talking about dozens in a sea of tens of thousands of people per year. The matriculating student questionnaire shows that only 20% of people get multiple MD admits. In most cases it will be 2 comparable options like getting into two instate campuses or two mid tier private schools. The group that are weighing huge merit packages against top 20 names are outliers.

Bobby who went to U of State undergrad and wants to remain at U of State for med school and residency and practice IM in a local city, well those are a huge proportion of the matriculants every year but they aren't the people who go on College Confidential or SDN. They don't need to, because there's no hypercompetitive obstacles in their path that they need to learn success formulas for.

Members don't see this ad.
 
  • Like
Reactions: 4 users
Very insightful look into a competitive program in mid-competitive field. However, I think these type of programs can be lumped with surgical subspecialties, but they still comprise a small proportion of all programs. Looking at that composite score, the average Step 1 for all applicants is at least 242, so not surprising that matched score is 250. That is not a representative mean applicant score for probably 75% of all programs.
Fair, but all of this debate is centered around the top quartile of most competitive fields or programs. The silent majority who just want to match a decent local IM/FM/Peds/ER/gas/gensurg spot don't have to do hundreds of thousands of flashcards.

At least, I assume they aren't doing that. What's it like in y'alls experience, it just the Ortho lads grinding zanki or does the average student interested in an average field have to do it too?
 
The silent majority who just want to match a decent local IM/FM/Peds/ER/gas/gensurg spot don't have to do hundreds of thousands of flashcards.

Okay, those people can do that already without succumbing to "mania".
 
  • Like
Reactions: 1 users
Members don't see this ad :)
Sure seems like overkill to study the way Ortho bros do when you can safely aim 40 percentile points lower

I mean maybe I’m wrong, but I don’t think most people who are scoring around the average are coasting into that. I see a lot of people working really hard and scoring average.
 
  • Like
Reactions: 3 users
I mean maybe I’m wrong, but I don’t think most people who are scoring around the average are coasting into that. I see a lot of people working really hard and scoring average.
Interesting. The only people I know who did hundreds of thousands of flashcards + additional qbanks beyond Uworld have all scored 250+, I didn't realize people were mastering all that and then still scoring 230.
 
  • Like
Reactions: 1 users
Interesting. The only people I know who did hundreds of thousands of flashcards + additional qbanks beyond Uworld have all scored 250+, I didn't realize people were mastering all that and then still scoring 230.

The problem is you’re assuming everyone doing all those cards is really mastering it. But I don’t have any data. Maybe everyone doing zanki at my school really are crushing it. It just doesn’t seem that way.
 
  • Like
Reactions: 4 users
Interesting. The only people I know who did hundreds of thousands of flashcards + additional qbanks beyond Uworld have all scored 250+, I didn't realize people were mastering all that and then still scoring 230.
I think you have a skewed population to begin with being at like a T5 school. Your reality (and your classmates') is not the reality for the vast majority of students.
 
  • Like
Reactions: 8 users
I think you have a skewed population to begin with being at like a T5 school. Your reality (and your classmates') is not the reality for the vast majority of students.
Yeah, but the skew I thought I was seeing was about where people wanted to match. Like I know Peds applicants who still aimed 250+ because they have their heart set on CHOP or Boston childrens. It's much harder to imagine someone doing thousands of extra questions and hours of daily flashcarding when they just want to match one of their decent local peds residencies.
 
Yeah, but the skew I thought I was seeing was about where people wanted to match. Like I know Peds applicants who still aimed 250+ because they have their heart set on CHOP or Boston childrens. It's much harder to imagine someone doing thousands of extra questions and hours of daily flashcarding when they just want to match one of their decent local peds residencies.
I was just responding to the point about everyone who does Zanki + qbanks scoring >250. I think that has more to do with the above average test taking skills of who you're surrounded with. There are lots of people over on the current Step 1 thread who have done all of those things and are still struggling.
 
  • Like
Reactions: 4 users
There are lots of people over on the current Step 1 thread who have done all of those things and are still struggling.
Man that is awful to hear. As mad as people are about P/F masking their success, it must be even more crushing to put in all those extra Qs and UFAPS hours, and still not find success with it in the first place.
 
  • Like
  • Dislike
Reactions: 2 users
Members don't see this ad :)

They just made an announcement and they still didn't say whether class of 2023 scores will be retroactively masked
 
  • Like
Reactions: 3 users
Is it safe to assume that if I am averaging 1.5 sd above class mean that I have a shot at 250+? I know I need to put in the work, but it seems like some people put in the work and still score at or slightly above the mean. Our class step 1 average is around the national avg I think, hence the extrapolation.
 

They just made an announcement and they still didn't say whether class of 2023 scores will be retroactively masked

For 2023, we're gonna have to grind like it's going to be a 3-digit score. Control the controllable.
 
  • Like
Reactions: 1 users
Potentially the final class to benefit from the Zanki legend.
Nah it is really useful even with a p/f step. I’ve used it for every module and it’s been a huge help. I just don’t keep up with the reviews, which makes it not much of a grind. I do like 75 new cards a day for this module and I have like 180 reviews today.
 
  • Like
Reactions: 1 user
Nah it is really useful even with a p/f step. I’ve used it for every module and it’s been a huge help. I just don’t keep up with the reviews, which makes it not much of a grind. I do like 75 new cards a day for this module and I have like 180 reviews today.

I know it's helpful but this sounds more epic - "The last Zanki, class of 2023"

But in reality, I do keep up with all the reviews and daily new cards. If step is P/F there's zero percent chance I'm going to punish myself with 1000+ cards/day if I don't need to. I would allocate my time and energy to other aspects of my application (e.g. research) that would allow me to accomplish my professional goals.
 
  • Like
Reactions: 2 users

They just made an announcement and they still didn't say whether class of 2023 scores will be retroactively masked
" ...while also retaining the ability of state medical licensing boards to use the exam results for the primary purpose of determining medical licensure eligibility"

Yeah, this sounds like a confirmation that they can't accelerate the timeline due to state legislatures, but doesn't address the score masking at all.
 
  • Like
Reactions: 2 users
Is it safe to assume that if I am averaging 1.5 sd above class mean that I have a shot at 250+? I know I need to put in the work, but it seems like some people put in the work and still score at or slightly above the mean. Our class step 1 average is around the national avg I think, hence the extrapolation.

Yes, you have a good shot. Preclinical performance has ~0.7 correlation with Step 1 performance, so the person struggling with class exams is going to have trouble crushing Step 1. On top of that, use flashcards and do as many unique questions as possible. These have been shown to work in the literature.
 
  • Like
Reactions: 1 user
With this whole S1 thing, we already have an example of what will happen: P/F preclinical years. Schools said they changed to P/F to allow students t focus on their interests, work together and collaborate, and decrease stress. Did it work? Seems like people now just skip lecture and cram anki decks. Perhaps it's more collaborative.

There's no one thing that will fix this. And there's no such thing as a "fix" -- if there are a limited number of Ortho spots, and more applicants than spots, then some people are not getting spots. If we change the system, the only possible outcome is different people getting spots -- whether you think that's good or not probably depends on whether you get a spot. No matter what we pick as our selection metrics, students will go bananas trying to maximize them. If I said that most programs review candidates alphabetically, people would change their last name to "Aardvark".

That said, I think we could do better. It would include some combination of:
A. USMLE quartiles, probably three groups: 1st, 2nd/3rd, and 4th (plus fail, of course).
B. An early application process where studnts could apply to a very limited number of programs, and programs could fill 20-40% of their spots.
C. Some way to better define program/applicant fit. One possibility would be a Gold/Silver/Bronze application process where each applicant gets a limited number of Gold and Silver apps, and an infinite number of bronze apps. Or, perhaps a better option, have a defined list of program strengths - might include research, med ed training, community based practice, fellowship prep, mentoring, etc. We'd need to come up with 10-15 options all of which would be of interest. Then, each program can pick a max of 3-5 of them that they specialize in -- this should be difficult, programs will often want to check them all. Each applicant then also chooses a limited number that are of interest to them. Ultimately programs can see how much overlap their applicants have with them. Both of these ideas have problems, and may be unworkable.
D. Increased clarity to applicants about their chances of getting a spot at programs. An MSAR for residency programs.

Applicants will hate C, because anything in that group will make your application to some programs "weaker"
Programs will hate D, we don't want to share this info.
Everyone will hate A, for different reasons.
If everybody is equally miserable, I call that a win.
 
  • Like
Reactions: 14 users
Got this back (name and USMLE ID removed).
 

Attachments

  • ECDC8BC5-4794-47A1-AE3F-C5F3724818BA.jpeg
    ECDC8BC5-4794-47A1-AE3F-C5F3724818BA.jpeg
    115.1 KB · Views: 237
  • Like
Reactions: 1 user
With this whole S1 thing, we already have an example of what will happen: P/F preclinical years. Schools said they changed to P/F to allow students t focus on their interests, work together and collaborate, and decrease stress. Did it work? Seems like people now just skip lecture and cram anki decks. Perhaps it's more collaborative.

There's no one thing that will fix this. And there's no such thing as a "fix" -- if there are a limited number of Ortho spots, and more applicants than spots, then some people are not getting spots. If we change the system, the only possible outcome is different people getting spots -- whether you think that's good or not probably depends on whether you get a spot. No matter what we pick as our selection metrics, students will go bananas trying to maximize them. If I said that most programs review candidates alphabetically, people would change their last name to "Aardvark".

That said, I think we could do better. It would include some combination of:
A. USMLE quartiles, probably three groups: 1st, 2nd/3rd, and 4th (plus fail, of course).
B. An early application process where studnts could apply to a very limited number of programs, and programs could fill 20-40% of their spots.
C. Some way to better define program/applicant fit. One possibility would be a Gold/Silver/Bronze application process where each applicant gets a limited number of Gold and Silver apps, and an infinite number of bronze apps. Or, perhaps a better option, have a defined list of program strengths - might include research, med ed training, community based practice, fellowship prep, mentoring, etc. We'd need to come up with 10-15 options all of which would be of interest. Then, each program can pick a max of 3-5 of them that they specialize in -- this should be difficult, programs will often want to check them all. Each applicant then also chooses a limited number that are of interest to them. Ultimately programs can see how much overlap their applicants have with them. Both of these ideas have problems, and may be unworkable.
D. Increased clarity to applicants about their chances of getting a spot at programs. An MSAR for residency programs.

Applicants will hate C, because anything in that group will make your application to some programs "weaker"
Programs will hate D, we don't want to share this info.
Everyone will hate A, for different reasons.
If everybody is equally miserable, I call that a win.
Love the idea of a residency MSAR. Sounds like what texas STAR has been trying to crowdsource. I still havent gotten a way to check that data out yet.

I'd be worried about how the early decision rounds might evolve over time. For competitive undergrads, it's become the norm to fill a majority of your class during the early decision round when you know your yield is higher. Usually the early admit rate is 3-5/x higher than the regular decision rate. Essentially you pick 1 target for your odds to go drastically up, in exchange for terrible regular decision odds everywhere else.

I'd worry about that evolving again and residency turning into college admissions 2.0. I really dont want to have to guess at 1 single program I expect to like best like people have to do with colleges these days.
 
Interesting. The only people I know who did hundreds of thousands of flashcards + additional qbanks beyond Uworld have all scored 250+, I didn't realize people were mastering all that and then still scoring 230.

Absolutely. This is a fantastic argument in favor of keeping step scored that I had trouble putting together, thank you.
 
Absolutely. This is a fantastic argument in favor of keeping step scored that I had trouble putting together, thank you.
What's the argument? Grinding cards and Qs works well for students at my school, therefor it's a great way for the nation to pick its surgeons?
 
  • Like
Reactions: 4 users
Love the idea of a residency MSAR. Sounds like what texas STAR has been trying to crowdsource. I still havent gotten a way to check that data out yet.

I'd be worried about how the early decision rounds might evolve over time. For competitive undergrads, it's become the norm to fill a majority of your class during the early decision round when you know your yield is higher. Usually the early admit rate is 3-5/x higher than the regular decision rate. Essentially you pick 1 target for your odds to go drastically up, in exchange for terrible regular decision odds everywhere else.

I'd worry about that evolving again and residency turning into college admissions 2.0. I really dont want to have to guess at 1 single program I expect to like best like people have to do with colleges these days.
In my proposal, I cap early spots at 20-40%. The NRMP would prevent programs from filling more than that.

Like any change, this creates winners and losers. Some people will get an early spot and "win". Many people will not, and the panic level will go through the roof. Even though the match is still basically the same, with a bunch of high achievers settled into spots. So although people would moan about "all the spots that were taken", there would be an equal number of applicants who were no longer competing for what's left.

People who applied early and didn't get a spot would still be considered for the main match, might actually have a better chance (if programs see it as higher interest).
 
  • Like
Reactions: 2 users
Fair, but all of this debate is centered around the top quartile of most competitive fields or programs. The silent majority who just want to match a decent local IM/FM/Peds/ER/gas/gensurg spot don't have to do hundreds of thousands of flashcards.

At least, I assume they aren't doing that. What's it like in y'alls experience, it just the Ortho lads grinding zanki or does the average student interested in an average field have to do it too?
I think a vast majority of people are giving it their most serious try in order to score as high as they can. Once results come out people start to self select into fields. I will say for the average applicant into fm/Im/ peds / obgyn/ neuro step doesn’t matter a whole lot.
 
  • Like
Reactions: 2 users
The problem is you’re assuming everyone doing all those cards is really mastering it. But I don’t have any data. Maybe everyone doing zanki at my school really are crushing it. It just doesn’t seem that way.
Iirc the studies that came out of mid tier schools still had a direct correlation with number of cards and step 1
 
  • Like
Reactions: 1 user
I remember seeing something about a good correlation between unique questions and step 1.
This was the same or very similar study.
I honestly solidified my study technique after reading those papers .
step one unlike the mcat can be brute forced to a degree .
 
  • Like
Reactions: 3 users
Iirc the studies that came out of mid tier schools still had a direct correlation with number of cards and step 1
There is little doubt in my mind that doing Zanki deck several times through will net you additional points. I think it's just really hard to cleanly prove so. One person might halfheartedly use flashcards only during the unit, while others might truly commit as much as they can to memory nonstop for 12+ months. No good way to control for whether it's being retained. I know I have friends who used it properly that can still recite all the First Aid tables cold.
 
There is little doubt in my mind that doing Zanki deck several times through will net you additional points. I think it's just really hard to cleanly prove so. One person might halfheartedly use flashcards only during the unit, while others might truly commit as much as they can to memory nonstop for 12+ months. No good way to control for whether it's being retained. I know I have friends who used it properly that can still recite all the First Aid tables cold.

Using that strategy (Zanki multiple times) is inefficient and takes way longer than studying the normal way. If step scores = hard work x natural ability, doubling the amount of effort will of course yield higher scores. If I were you, I’d be glad I didn’t waste time doing all that but still got a great score.
 
Some of the responses here really emphasize how out of touch SDN is with the real world. Literally 100% of medical students put their entire being into studying for Step 1 and <100% of them get an amazing score on it. The amount I studied and dedicated to Step 1 was orders of magnitude greater than the amount I have ever studied for anything else ever, and I got an average score - which probably has something to do with the fact that during the pre-clinical years, I was a fairly average medical student. I know many people who went to truly preposterous lengths and got scores in the 200s-210s. I also know many people who went to equally preposterous lengths and got obscenely high scores.
 
  • Like
Reactions: 11 users
With this whole S1 thing, we already have an example of what will happen: P/F preclinical years. Schools said they changed to P/F to allow students t focus on their interests, work together and collaborate, and decrease stress. Did it work? Seems like people now just skip lecture and cram anki decks. Perhaps it's more collaborative.

There's no one thing that will fix this. And there's no such thing as a "fix" -- if there are a limited number of Ortho spots, and more applicants than spots, then some people are not getting spots. If we change the system, the only possible outcome is different people getting spots -- whether you think that's good or not probably depends on whether you get a spot. No matter what we pick as our selection metrics, students will go bananas trying to maximize them. If I said that most programs review candidates alphabetically, people would change their last name to "Aardvark".

That said, I think we could do better. It would include some combination of:
A. USMLE quartiles, probably three groups: 1st, 2nd/3rd, and 4th (plus fail, of course).
B. An early application process where studnts could apply to a very limited number of programs, and programs could fill 20-40% of their spots.
C. Some way to better define program/applicant fit. One possibility would be a Gold/Silver/Bronze application process where each applicant gets a limited number of Gold and Silver apps, and an infinite number of bronze apps. Or, perhaps a better option, have a defined list of program strengths - might include research, med ed training, community based practice, fellowship prep, mentoring, etc. We'd need to come up with 10-15 options all of which would be of interest. Then, each program can pick a max of 3-5 of them that they specialize in -- this should be difficult, programs will often want to check them all. Each applicant then also chooses a limited number that are of interest to them. Ultimately programs can see how much overlap their applicants have with them. Both of these ideas have problems, and may be unworkable.
D. Increased clarity to applicants about their chances of getting a spot at programs. An MSAR for residency programs.
I think these are all great ideas. I especially like the early application idea. I know so many people where the gulf between their #1 and #2 was bigger than #2 and their last rank because of specific ties, location, family, etc. And I imagine it can get hard for programs to suss out what's real interest and what's people saying what programs want to hear. Obviously the greatest concern with early applications or the gold tokens is that it lays bare to every other program that they aren't your #1 choice and that could be challenging to navigate.

Programs will hate D, we don't want to share this info.
Honestly, why? I don't want to waste my time and money applying to a program where I'll just be screened out. Why do programs want people to apply if they're going to screen them out anyway? Transparency with the actual requirements seems like a win-win no-brainer to me.
 
  • Like
Reactions: 2 users
According to my schools admin:

NBME released a statement to admin confirming that Step 1 will not change to pass/fail until 2022 at the earliest. This means our class (2023) will still have numerical scores.
 
Last edited:
Some of the responses here really emphasize how out of touch SDN is with the real world. Literally 100% of medical students put their entire being into studying for Step 1 and <100% of them get an amazing score on it. The amount I studied and dedicated to Step 1 was orders of magnitude greater than the amount I have ever studied for anything else ever, and I got an average score - which probably has something to do with the fact that during the pre-clinical years, I was a fairly average medical student. I know many people who went to truly preposterous lengths and got scores in the 200s-210s. I also know many people who went to equally preposterous lengths and got obscenely high scores.

Say it louder for the people in the back!!!
 
  • Like
Reactions: 2 users
Some of the responses here really emphasize how out of touch SDN is with the real world. Literally 100% of medical students put their entire being into studying for Step 1 and <100% of them get an amazing score on it. The amount I studied and dedicated to Step 1 was orders of magnitude greater than the amount I have ever studied for anything else ever, and I got an average score - which probably has something to do with the fact that during the pre-clinical years, I was a fairly average medical student. I know many people who went to truly preposterous lengths and got scores in the 200s-210s. I also know many people who went to equally preposterous lengths and got obscenely high scores.
Do all students really put 100% in? I know some people who really didnt care that much. E.g. one friend from the midwest who wants psych straight up laughed at the idea of getting a 240+. They never touched anything but the school curriculum, were pretty average on the MCAT, and were totally at peace with being average on Step. I didnt realize these people were rare. I actually thought people putting in preposterous efforts to barely pass were the rare ones, based on all the AAMC success rate data.
 
  • Like
Reactions: 1 user
Do all students really put 100% in? I know some people who really didnt care that much. E.g. one friend from the midwest who wants psych straight up laughed at the idea of getting a 240+. They never touched anything but the school curriculum, were pretty average on the MCAT, and were totally at peace with being average on Step. I didnt realize these people were rare. I actually thought people putting in preposterous efforts to barely pass were the rare ones, based on all the AAMC success rate data.
I put in preposterous effort M2-M3 summer and my practice tests were too inconsistent to comfortably take it. Now I’m putting preposterous effort in again M3-M4 summer hoping to take it this time.

(DO student. 508 MCAT)
 
I put in preposterous effort M2-M3 summer and my practice tests were too inconsistent to comfortably take it. Now I’m putting preposterous effort in again M3-M4 summer hoping to take it this time.

(DO student. 508 MCAT)
The odds are very much in your favor! You got this
 
  • Like
Reactions: 1 user
Do all students really put 100% in? I know some people who really didnt care that much. E.g. one friend from the midwest who wants psych straight up laughed at the idea of getting a 240+. They never touched anything but the school curriculum, were pretty average on the MCAT, and were totally at peace with being average on Step. I didnt realize these people were rare. I actually thought people putting in preposterous efforts to barely pass were the rare ones, based on all the AAMC success rate data.
Your friend may very well be comfortable with being average on Step 1, which is completely reasonable since that's still a great score. I was extremely happy to get an average score on Step 1, but I can guarantee you that everybody, no matter how outwardly nonchalant or chill they seem, is working unbelievably hard to get whatever score they want. My partner's goal score was a 220 and she didn't get it despite putting in 12+ hour days for months and doing everything you're "supposed" to do, just for one example. Also, the number of people who get a 250+ is very similar to the number of people who get a 209 or less. But both are still extreme - it's just that one is represented much, much more on SDN.
 
  • Like
Reactions: 4 users
Honestly, why? I don't want to waste my time and money applying to a program where I'll just be screened out. Why do programs want people to apply if they're going to screen them out anyway? Transparency with the actual requirements seems like a win-win no-brainer to me.
Most programs don't want a ranking system. if we publish this data, for sure someone will take it and rank programs based upon it.

Some programs might advertise that their average S1 is better than it really is. Makes them look more competitive.

If I screen you out based on USMLE score, it's literally no work.
 
Do all students really put 100% in? I know some people who really didnt care that much. E.g. one friend from the midwest who wants psych straight up laughed at the idea of getting a 240+. They never touched anything but the school curriculum, were pretty average on the MCAT, and were totally at peace with being average on Step. I didnt realize these people were rare. I actually thought people putting in preposterous efforts to barely pass were the rare ones, based on all the AAMC success rate data.

Because getting a 220 from your program will get them into any specialty they want?
 
Most programs don't want a ranking system. if we publish this data, for sure someone will take it and rank programs based upon it.

Some programs might advertise that their average S1 is better than it really is. Makes them look more competitive.

If I screen you out based on USMLE score, it's literally no work.
I'm surprised they don't look with envious eyes at the amount of self-selection that goes on for med school admissions. For example Penn gets less than half as many applicants as Drexel does. If one of the major difficulties PDs face right now is overapplication, seems like it would help a great deal to let people better target their apps.
 
  • Like
Reactions: 1 users
According to my schools admin:

NBME released a statement to admin confirming that Step 1 will not change to pass/fail until 2022 at the earliest. This means our class (2023) will still have numerical scores.

we’ll get a score yes. But the argument here is that they would mask our transcripts summer of 2022 when we apply for residency. USMLE is still unsure of that

 
  • Like
Reactions: 4 users
Top