Questions about the USMLE

[SouthernGent]

---

Members don't see this ad.

So I keep hearing that the USMLE is changing and I'm not sure what this means. Has the exam changed in 2019 so far?

My friends who are preparing for Step 1 said that the NBMEs or UWorld are not an accurate predictor anymore since they've replaced 5 out of the 6 current NBMEs...can anyone comment on this and what it means for future test-takers? Thanks in advance!

Worry about this when you get there. Those exams were replaced for good reasons and Uworld is always updating. For those currently applying and are reading this: Worry about getting into medical school. surviving your first year, and then you can worry about STEP 1. You are currently doing the equivalent of a high school junior freaking out about changes to the MCAT. For those accepted but haven't started: Yes STEP is a thing, but worry about getting through first year first.

EDIT: Made this post assuming OP had not applied to med school yet; We all know what assuming does.

 

[Matthew9Thirtyfive]

Worry about this when you get there. Those exams were replaced for good reasons and Uworld is always updating. Worry about getting into medical school. surviving your first year, and then you can worry about STEP 1. You are currently doing the equivalent of a high school junior freaking out about changes to the MCAT.

I'm pretty sure OP has been accepted to med school and starts in a few months. It's not an unreasonable question when you're about to start M1.

 

[SouthernGent]

Tbh I hope it goes completely P/F.

No. Please no. My score is the only metric that I have to compare to students across the country. You get rid of that I lose any equal footing I might have gained. Go read the DO forums and residency threads where they get crapped on by programs simply for having a DO even when they got rock star scores. You think someone in their situation wants the one thing that shows their worth compared to all other medical students, MD and DO, removed? P/F is good for classes, and STEP does get blown out of proportion, but something needs to be there for comparing students.

 

[SouthernGent]

I'm pretty sure OP has been accepted to med school and starts in a few months. It's not an unreasonable question when you're about to start M1.

I had not seen his post history, thank you for the heads up. I still believe it is too early to start worrying about how to study for STEP, but I will respect your view on the matter and his.

 

[Splenda88]

Why is more emphasis put on Step 1 than Step 2?

Because it's not an easy test.

 

[SouthernGent]

Because it's not an easy test.

Understatement of the year right there. I cannot wait to be done with this thing.

 

[Med Ed]

I can see why students and residency directors love the scaled scores, but I can also see how stakeholders in good preclinical medical education are growing to hate it.

You make it sound like the hate isn't already fulminant.

Residency programs were capable of choosing highly qualified applicants in the decades before Step 1 became the be-all and end-all. The current arms race is largely a product of 1.) over-application to residency programs, and 2.) the further commercialization of the exam/exam prep industry.

I believe that they could make Step 1 P/F tomorrow and it would not have much effect on the outcome of the match. Programs would simply shift to other criteria that they already evaluate: Step 2 CK, clerkship grades, elective grades, shelf exams, sub-I grades, audition rotations, the MSPE, faculty LOR's, etc. Step 1 has never been the only piece of the puzzle, it's just the easiest one to obsess over.

 

[Splenda88]

Understatement of the year right there. I cannot wait to be done with this thing.

Glad that I am done with it, but still have to take step 3. Glad the latter is somewhat a P/F test. I was never obsessed anyway over step1 since I was not gunning for a competitive specialty from the get-go.

 

[907914]

Is it possible to not be a gunner and do top % on Step 1 as the exam stands? Say you are the kind of undergrad who, during premed years, only a few hours (if that) is needed per week to get a 4.0 in the sciences. When that translates to medical school (not that med school is comparable to undergrad), where that same student would presumably be putting in 40 hours a week for the same outcomes, would memorizing tens of thousands of little minutiae still be needed as opposed to just understanding what is going on?

 

[efle]

Is it possible to not be a gunner and do top % on Step 1 as the exam stands?

Well, I can give you my own experience as someone who didn't do zanki. I was top of class in undergrad, 100th percentile MCAT, and have been scoring at the top 10% marker on my Uworld blocks so far.

That last 10% though? That last 10% is a betch. Every block there are at least a few questions that will simply test whether you have a factoid memorized or not. For example the last block I did had a question that required I know that the conversion of oxaloacetate to phosphoenolpyruvate got its energy from GTP, and also which step of the citric acid cycle is the one that generates GTP. No amount of intuition can guide you through that. You either saw those two anki card enough times that it stuck, or you didn't.

Now don't get me wrong, anki gets a lot more credit than it should. You can't take someone that lacks understanding and get them a 270 by maturing Zanki. You do need to be good at test taking and able to reason through questions that can't be put on a flashcard. But yeah, getting a top 1% on the step1 requires both that skillset, and an encyclopedic knowledge base of minutiae.

The good news is, there are no residencies that require a top percentile step score, unlike with medical admissions. In fact the most competitive specialties have averages of about the top ~15% (a 250). And you absolutely can hit a 250 without doing a single flashcard.

 

[efle]

You make it sound like the hate isn't already fulminant.

Residency programs were capable of choosing highly qualified applicants in the decades before Step 1 became the be-all and end-all. The current arms race is largely a product of 1.) over-application to residency programs, and 2.) the further commercialization of the exam/exam prep industry.

I believe that they could make Step 1 P/F tomorrow and it would not have much effect on the outcome of the match. Programs would simply shift to other criteria that they already evaluate: Step 2 CK, clerkship grades, elective grades, shelf exams, sub-I grades, audition rotations, the MSPE, faculty LOR's, etc. Step 1 has never been the only piece of the puzzle, it's just the easiest one to obsess over.

So what's your take on how it used to be, back when all you had to worry about was solidly passing it? Was your school not a major determinant of where you were competitive? Joe Stateschool stood a great shot at surgical subspecialties in major cities, as long as he had great clinical evals?

 

[Lawpy]

Well, I can give you my own experience as someone who didn't do zanki. I was top of class in undergrad, 100th percentile MCAT, and have been scoring at the top 10% marker on my Uworld blocks so far.

The Step 1 discussion thread is stunned

And warmly suggests for tips once you come back with a strong score

 

[907914]

Well, I can give you my own experience as someone who didn't do zanki. I was top of class in undergrad, 100th percentile MCAT, and have been scoring at the top 10% marker on my Uworld blocks so far.

That last 10% though? That last 10% is a betch. Every block there are at least a few questions that will simply test whether you have a factoid memorized or not. For example the last block I did had a question that required I know that the conversion of oxaloacetate to phosphoenolpyruvate got its energy from GTP, and also which step of the citric acid cycle is the one that generates GTP. No amount of intuition can guide you through that. You either saw those two anki card enough times that it stuck, or you didn't.

Now don't get me wrong, anki gets a lot more credit than it should. You can't take someone that lacks understanding and get them a 270 by maturing Zanki. You do need to be good at test taking and able to reason through questions that can't be put on a flashcard. But yeah, getting a top 1% on the step1 requires both that skillset, and an encyclopedic knowledge base of minutiae.

The good news is, there are no residencies that require a top percentile step score, unlike with medical admissions. In fact the most competitive specialties have averages of about the top ~15% (a 250). And you absolutely can hit a 250 without doing a single flashcard.

Thank you for the thoughtful and full anecdote!

 

[907914]

Well, I can give you my own experience as someone who didn't do zanki. I was top of class in undergrad, 100th percentile MCAT, and have been scoring at the top 10% marker on my Uworld blocks so far.

That last 10% though? That last 10% is a betch. Every block there are at least a few questions that will simply test whether you have a factoid memorized or not. For example the last block I did had a question that required I know that the conversion of oxaloacetate to phosphoenolpyruvate got its energy from GTP, and also which step of the citric acid cycle is the one that generates GTP. No amount of intuition can guide you through that. You either saw those two anki card enough times that it stuck, or you didn't.

Now don't get me wrong, anki gets a lot more credit than it should. You can't take someone that lacks understanding and get them a 270 by maturing Zanki. You do need to be good at test taking and able to reason through questions that can't be put on a flashcard. But yeah, getting a top 1% on the step1 requires both that skillset, and an encyclopedic knowledge base of minutiae.

The good news is, there are no residencies that require a top percentile step score, unlike with medical admissions. In fact the most competitive specialties have averages of about the top ~15% (a 250). And you absolutely can hit a 250 without doing a single flashcard.

This is the route I am hoping (assuming I get in...) no real intent on being a gunner.

 

[Med Ed]

So what's your take on how it used to be, back when all you had to worry about was solidly passing it? Was your school not a major determinant of where you were competitive? Joe Stateschool stood a great shot at surgical subspecialties in major cities, as long as he had great clinical evals?

Are you talking 1970's or more 1990's?

 

[efle]

Are you talking 1970's or more 1990's?

However far back it was that a slightly above average score, say in the upper 230s, was good enough for any specialty. According to our faculty it wasn't all that long ago, many of them seem to still have that impression

 

[Splenda88]

Step1 20 years ago (in the 1990s) was an easy test where most of the question stems were 1-2 lines. Now some of the question stems will almost take a page, and you have to decipher what is relevant and what is not.

 

[Gurby]

The Step 1 discussion thread is stunned

View attachment 257287

And warmly suggests for tips once you come back with a strong score

 

[ricksimmonsrafowens]

Or maybe revamp Step exams to make it more strongly based on critical thinking just like the MCAT? Why get rid of the scores? It's really hard to just memorize your way to a high score in a critical thinking/reasoning based standardized exam. And that in turn could make preclinical years a lot more meaningful.

have you taken step 1? from my experience with 1, 2 CK, the USMLE exams are very much based on critical thinking. There were a lot of questions on Step 1 and 2 CK where I had to apply my knowledge to decide "what the best next step is" on 2CK or to answer high level questions about pathophys that definitely require a lot of critical thinking on Step 1. there were not that many straight up pure recall questions on both exams. the exams really do test clinical decision making and clinical acumen well.

also P/F grading to Step 1 is an AWFUL idea. The emphasis would then shift to Step 2 CK so you'd have a Zanki equivalent for Step 2 CK within a few months. Shelf exams are an okay marker but some schools don't even report the score for students on the MSPE. Clerkship grades are highly variable and subjective. At my school <10% receive Honors on Surgery whereas at other schools 80%+ of students receive Honors. Class rank usually correlates with exam performance, but top 1/3 at my school is in no way comparable in performance to top 1/3 of students at Johns Hopkins and the trend in med ed now is to eschew reporting class rank entirely. Prestige of medical schools is already incredibly important in residency selection and will become even more so going forward if P/F grading for Step 1 is adopted. so underprivileged students who say for socioeconomic reasons couldn't afford to go to "brand name" undergrad or medical schools are now even more penalized by the system with its already profound biases against those from low SES.

 

[Lucca]

have you taken step 1? from my experience with 1, 2 CK, the USMLE exams are very much based on critical thinking. There were a lot of questions on Step 1 and 2 CK where I had to apply my knowledge to decide "what the best next step is" on 2CK or to answer high level questions about pathophys that definitely require a lot of critical thinking on Step 1. there were not that many straight up pure recall questions on both exams. the exams really do test clinical decision making and clinical acumen well.

also P/F grading to Step 1 is an AWFUL idea. The emphasis would then shift to Step 2 CK so you'd have a Zanki equivalent for Step 2 CK within a few months. Shelf exams are an okay marker but some schools don't even report the score for students on the MSPE. Clerkship grades are highly variable and subjective. At my school <10% receive Honors on Surgery whereas at other schools 80%+ of students receive Honors. Class rank usually correlates with exam performance, but top 1/3 at my school is in no way comparable in performance to top 1/3 of students at Johns Hopkins and the trend in med ed now is to eschew reporting class rank entirely. Prestige of medical schools is already incredibly important in residency selection and will become even more so going forward if P/F grading for Step 1 is adopted. so underprivileged students who say for socioeconomic reasons couldn't afford to go to "brand name" undergrad or medical schools are now even more penalized by the system with its already profound biases against those from low SES.

I actually think the prejudice against low SES students is worse *without* P/F grading...the aid at virtually every “brand name” school that could significantly impact your residency prospects is actually fairly generous with several schools eliminating loans altogether for many if not most of their students. However, when every point counts people are expected to shell out thousands out of pocket for more and more third party resources or whatever board prep products are currently topping the metagame at that particular moment.

 

[aldol16]

Yeah I really feel for the test writers. They're supposed to make sure part of their exam is knowledge checks. But, the appropriate level of knowledge is so stupidly easy to achieve with modern resources that fair knowledge check questions are directly in conflict with a nice valid distribution. If they have to toss any questions that too many people get right, how can they ever ask someone to recognize an india-ink stain of Cryptococcus? It's just too easy, when 80% of students know which histone proteins are inside vs outside the core, 99% will be able to recognize that image. So instead they have to drop the image and go with a more ambiguous vignette, or ask followup questions about details of the mechanism of action for Crypto treatments. Is that really what we want the USMLE to be like? Shouldn't it be allowed to just straightforwardly confirm that an American physician can recognize Crypto?

I mean, that depends on the function of the test, right? If the function of the test is to establish a minimum knowledge that all US physicians should know, then that would be the correct solution. But the problem is, the function of the test is so much more than that nowadays, due to residencies relying on it heavily for quality. So for example, imagine that you can divide a class into top, middle, and bottom tertiles. And let's say you have a test right now that reliably distinguishes them, using whatever questions/methods. Then the people making the test want to make it P/F only. If you're in the bottom tertile, what do you say? You say, "Hell yeah!" because a "pass" makes you look better relative to where you would have scored otherwise. The middle third is probably mostly indifferent, depending on if you're upper middle or lower middle. What does the top tertile say? They say "Hell no!" Because a true P/F exam where the test is assessing only baseline knowledge doesn't help you. In fact, it hurts you by removing one more thing that could have been used to distinguish yourself from the rest of the pack.

So I certainly think that Step 1 is the best tool we currently have to distinguish top students from mediocre students. But I also think that Step 1 could be re-vamped to achieve this goal using better written questions. Presumably, if you relied more heavily on clinical reasoning and less on factoid recall, you'd still get that nice bell curve and you could still reliably distinguish top from mediocre students reliably (although the top students now may be slightly different from before). At the same time, you're emphasizing what you're really look for, which is clinical reasoning ability.

That last 10% though? That last 10% is a betch. Every block there are at least a few questions that will simply test whether you have a factoid memorized or not. For example the last block I did had a question that required I know that the conversion of oxaloacetate to phosphoenolpyruvate got its energy from GTP, and also which step of the citric acid cycle is the one that generates GTP. No amount of intuition can guide you through that. You either saw those two anki card enough times that it stuck, or you didn't.

I remember that exact question. I think it's realistic if you know your biochem. It's definitely not something that you would need Anki for. I think it's a good thing to understand the steps of the central metabolic pathways of life in detail (i.e. glycolysis, gluconeogenesis, TCA cycle, oxidative phosphorylation +/- urea cycle and fatty acid metabolism in less detail.

 

[rball]

OP I think you're getting a lot of advice from people who haven't taken the STEP1 and frankly don't know what they're talking about.

Replacing old, outdated NBMEs with newer ones is not a red flag. The test is still a good mix of having a broad, in depth knowledge base and being able to apply it to challenging scenarios on exam day. This has not changed and will likely not change for the next couple of years at least.

Also I think you should take advice from people about zanki who don't know much about zanki with a heavy grain of salt. Zanki isn't used to memorize unconnected minutae. Its a spaced repetition learning program designed to frequently review/revisit high yield concepts and their associated facts/minutae. People who just focus on using it to memorize facts will do very poorly on step; people who use it to retain the big picture and associated facts that they learned several months ago will end up doing very well. As you keep reviewing these concepts over and over again, you build intuition and form connections that can't be formed by just seeing something a couple of times and then forgetting it after your school's block exam. It takes a lot of willpower and discipline to do this type of spaced review, so naturally these individuals will tend to score higher and rightfully so. You don't have to use anki to do this type of spaced repetition, it just happens to be one way to do it.

 

[ciestar]

OP I think you're getting a lot of advice from people who haven't taken the STEP1 and frankly don't know what they're talking about.

Replacing old, outdated NBMEs with newer ones is not a red flag. The test is still a good mix of having a broad, in depth knowledge base and being able to apply it to challenging scenarios on exam day. This has not changed and will likely not change for the next couple of years at least.

Also I think you should take advice from people about zanki who don't know much about zanki with a heavy grain of salt. Zanki isn't used to memorize unconnected minutae. Its a spaced repetition learning program designed to frequently review/revisit high yield concepts and their associated facts/minutae. People who just focus on using it to memorize facts will do very poorly on step; people who use it to retain the big picture and associated facts that they learned several months ago will end up doing very well. As you keep reviewing these concepts over and over again, you build intuition and form connections that can't be formed by just seeing something a couple of times and then forgetting it after your school's block exam. It takes a lot of willpower and discipline to do this type of spaced review, so naturally these individuals will tend to score higher and rightfully so. You don't have to use anki to do this type of spaced repetition, it just happens to be one way to do it.

I didnt use Anki for Step 1 and I really wish I had. It is helping me with my shelf exams for third year, thankfully.

 

[efle]

I mean, that depends on the function of the test, right? If the function of the test is to establish a minimum knowledge that all US physicians should know, then that would be the correct solution. But the problem is, the function of the test is so much more than that nowadays, due to residencies relying on it heavily for quality. So for example, imagine that you can divide a class into top, middle, and bottom tertiles. And let's say you have a test right now that reliably distinguishes them, using whatever questions/methods. Then the people making the test want to make it P/F only. If you're in the bottom tertile, what do you say? You say, "Hell yeah!" because a "pass" makes you look better relative to where you would have scored otherwise. The middle third is probably mostly indifferent, depending on if you're upper middle or lower middle. What does the top tertile say? They say "Hell no!" Because a true P/F exam where the test is assessing only baseline knowledge doesn't help you. In fact, it hurts you by removing one more thing that could have been used to distinguish yourself from the rest of the pack.

So I certainly think that Step 1 is the best tool we currently have to distinguish top students from mediocre students. But I also think that Step 1 could be re-vamped to achieve this goal using better written questions. Presumably, if you relied more heavily on clinical reasoning and less on factoid recall, you'd still get that nice bell curve and you could still reliably distinguish top from mediocre students reliably (although the top students now may be slightly different from before). At the same time, you're emphasizing what you're really look for, which is clinical reasoning ability.



I remember that exact question. I think it's realistic if you know your biochem. It's definitely not something that you would need Anki for. I think it's a good thing to understand the steps of the central metabolic pathways of life in detail (i.e. glycolysis, gluconeogenesis, TCA cycle, oxidative phosphorylation +/- urea cycle and fatty acid metabolism in less detail.

If you don't need a flashcard to remember which of the last steps produce NADH vs FADH vs GTP, two years after your biochem unit, you're a freak of nature that will never need anki for anything

 

[efle]

OP I think you're getting a lot of advice from people who haven't taken the STEP1 and frankly don't know what they're talking about.

Replacing old, outdated NBMEs with newer ones is not a red flag. The test is still a good mix of having a broad, in depth knowledge base and being able to apply it to challenging scenarios on exam day. This has not changed and will likely not change for the next couple of years at least.

Also I think you should take advice from people about zanki who don't know much about zanki with a heavy grain of salt. Zanki isn't used to memorize unconnected minutae. Its a spaced repetition learning program designed to frequently review/revisit high yield concepts and their associated facts/minutae. People who just focus on using it to memorize facts will do very poorly on step; people who use it to retain the big picture and associated facts that they learned several months ago will end up doing very well. As you keep reviewing these concepts over and over again, you build intuition and form connections that can't be formed by just seeing something a couple of times and then forgetting it after your school's block exam. It takes a lot of willpower and discipline to do this type of spaced review, so naturally these individuals will tend to score higher and rightfully so. You don't have to use anki to do this type of spaced repetition, it just happens to be one way to do it.

What are your thoughts on the studies of step 1 performance that have found (a few times now) that flashcards are not a significant positive predictor? Meanwhile things like additional practice questions and MCAT are. Was surprising to me when I first saw the data because so many people swear by Anki

 

[ginsengreset]

If you don't need a flashcard to remember which of the last steps produce NADH vs FADH vs GTP, two years after your biochem unit, you're a freak of nature that will never need anki for anything

Or, you participate in an extracurricular activity that involves answering questions like these as early as possible:

 

[ricksimmonsrafowens]

I actually think the prejudice against low SES students is worse *without* P/F grading...the aid at virtually every “brand name” school that could significantly impact your residency prospects is actually fairly generous with several schools eliminating loans altogether for many if not most of their students. However, when every point counts people are expected to shell out thousands out of pocket for more and more third party resources or whatever board prep products are currently topping the metagame at that particular moment.

a lot of step 1 resources are free or can be free if the student is resourceful. I paid for UWorld and Pathoma. That was <$1000. Other things can be obtained for less or at no cost to student. there are many amazing free Anki decks on reddit for Step 1. there are a ton of free youtube USMLE review videos that are incredibly high quality. I think anyone who is advocating for P/F grading to Step 1 should take it and experience the impact the score has on residency applications before campaigning for this issue. Students of low SES and who are interested in competitive specialties may not have resources to do multiple away rotations ($$$), take unfunded research years, etc will absolutely suffer from P/F grading of Step 1. Dominating Step 1 is assuredly the most important aspect of anyone's application who wants to match a competitive specialty. It leverages the field for medical students who may be going anywhere from Harvard to Arkansas and gives a way for PDs to objectively compare a med student from Arkansas to a med student from Harvard.

 

[rball]

What are your thoughts on the studies of step 1 performance that have found (a few times now) that flashcards are not a significant positive predictor? Meanwhile things like additional practice questions and MCAT are. Was surprising to me when I first saw the data because so many people swear by Anki

Not very surprising. Practice questions are a more efficient way of doing spaced repetition than doing zanki - not only are you revising a concept, you are also seeing if you can apply it. Everytime you get a question wrong you revisit the concept and relearn it. It becomes very salient to you also, which makes it much harder to miss a subsequent question on that topic.

People who do a mass of questions (think 5,000+) prior to hitting UWorld/dedicated can only do this via spaced repetition (to make it feasible you will have to space it out so you only do x questions per day). Hence it is not surprising these people are doing well.

If you only do flashcards, you may understand every concept but may not be able to apply it. This is like simply memorizing and understanding UFAP cold, but then not doing any questions. Not a good idea.

Zanki (when used correctly) + mass of questions is especially powerful - zanki is a way to keep every concept that you have previously mastered from UFAP in your longterm memory, and questions ensure you can apply this knowledge too. No secret why people who do both these things do very well on the exam.

As far as mcat correlation also not surprising -both exams are heavy on critical thinking and test taking skills, though mcat moreso.

 

[efle]

Not very surprising. Practice questions are a more efficient way of doing spaced repetition than doing zanki - not only are you revising a concept, you are also seeing if you can apply it. Everytime you get a question wrong you revisit the concept and relearn it. It becomes very salient to you also, which makes it much harder to miss a subsequent question on that topic.

People who do a mass of questions (think 5,000+) prior to hitting UWorld/dedicated can only do this via spaced repetition (to make it feasible you will have to space it out so you only do x questions per day). Hence it is not surprising these people are doing well.

If you only do flashcards, you may understand every concept but may not be able to apply it. This is like simply memorizing and understanding UFAP cold, but then not doing any questions. Not a good idea.

Zanki (when used correctly) + mass of questions is especially powerful - zanki is a way to keep every concept that you have previously mastered from UFAP in your longterm memory, and questions ensure you can apply this knowledge too. No secret why people who do both these things so very well on the exam.

As far as mcat correlation also not surprising -both exams are heavy on critical thinking and test taking skills, though mcat moreso.

Yeah, makes sense that nothing is better than practice questions. What was weird is that zanki users vs non-zanki users don't have any difference between them. With the amount of hype that anki decks get among med students right now, you'd expect users to vastly outperform non-users.

 

[rball]

Yeah, makes sense that nothing is better than practice questions. What was weird is that zanki users vs non-zanki users don't have any difference between them. With the amount of hype that anki decks get among med students right now, you'd expect users to vastly outperform non-users.

Havent seen that study, so that is interesting.

I obviously havent used zanki myself, but from seeing others do it/tutoring students, hundreds if not thousands of those cards are poorly made, and lot of people do it incorrectly (ie rote memorizing minutae instead of using the card to briefly review the overarching concept in your head). The way to do well on step is to understand on a deep level every line in First aid + pathoma, commit that to longterm memory before dedicated, and do thousands of practice questions prior to dedicated. Simple as that.

Zanki is simply a vehicle to get that info into long term memory, but it is entirely possible to do this without zanki too. I think zanki works if done correctly, but other review methods are also effective.

 

[Med Ed]

However far back it was that a slightly above average score, say in the upper 230s, was good enough for any specialty. According to our faculty it wasn't all that long ago, many of them seem to still have that impression

Fifteen years ago the average Step 1 score was 217, so an upper 230's was not slightly above average. Also, the exam did not report any specific score above a 245.

It may be helpful to review the general history a bit. In the 1970's the exam was called NBME Part 1, and it was functionally pass/fail. People matched based on grades, evals, rank, and letters.

In the 1980's the NBME created a score scale for Part 1, which was initially ignored by most specialties, but after a few years some of the highly competitive fields began using it as a measure.

In the 1990's Part 1 was retired and USMLE Step 1 debuted, and thus began the slow but inexorable march toward our current state. The death knell occurred when program directors came under pressure to have 80% board passage rates in order to maintain ACGME accreditation, with Step 1 being seen as a good predictor of success on the speciality boards.

The key difference between the current situation and 10+ years ago isn't that Step 1 has really changed, or even that scores have increased. It's that residency programs no longer have the manpower to read all the applications they receive, so they use Step scores to whittle the number down to a manageable level. After that the rest of the package becomes more important.

This is somewhat anecdotal, but consistent across multiple program directors I know personally. The general feeling is that Step 1 scores have no real predictive power in terms of differentiating good residents from bad ones. One told me that the program's best residents are the ones with lower scores (meaning 220-240), because they tended to have strong interpersonal skills and could therefore function effectively in the clinical environment.

 

[efle]

Fifteen years ago the average Step 1 score was 217, so an upper 230's was not slightly above average. Also, the exam did not report any specific score above a 245.

It may be helpful to review the general history a bit. In the 1970's the exam was called NBME Part 1, and it was functionally pass/fail. People matched based on grades, evals, rank, and letters.

In the 1980's the NBME created a score scale for Part 1, which was initially ignored by most specialties, but after a few years some of the highly competitive fields began using it as a measure.

In the 1990's Part 1 was retired and USMLE Step 1 debuted, and thus began the slow but inexorable march toward our current state. The death knell occurred when program directors came under pressure to have 80% board passage rates in order to maintain ACGME accreditation, with Step 1 being seen as a good predictor of success on the speciality boards.

The key difference between the current situation and 10+ years ago isn't that Step 1 has really changed, or even that scores have increased. It's that residency programs no longer have the manpower to read all the applications they receive, so they use Step scores to whittle the number down to a manageable level. After that the rest of the package becomes more important.

This is somewhat anecdotal, but consistent across multiple program directors I know personally. The general feeling is that Step 1 scores have no real predictive power in terms of differentiating good residents from bad ones. One told me that the program's best residents are the ones with lower scores (meaning 220-240), because they tended to have strong interpersonal skills and could therefore function effectively in the clinical environment.

Wow, I didn't realize the score creep and modern role of step1 had all happened so recently. Makes sense why even our younger faculty are so dismissive of it.

Is an 80% board rate hard to achieve? Don't most established programs have pass rates in the mid 90s?

Also interesting to look at some of the data in that tableau link. Using Derm as an example, it's a massive deciding factor. If I'm reading this right, at a 230, half of programs hard screen you out. But at a 243, half of programs "almost always grant interview".

So going up a dozen points can move you from "need not apply" to almost certainly interviewed? No wonder it's become such an arms race

 

[Ad2b]

If you don't need a flashcard to remember which of the last steps produce NADH vs FADH vs GTP, two years after your biochem unit, you're a freak of nature that will never need anki for anything

FYI - I'm 7 years post biochem and I can remember it along with all the enzymes and pathways into/out of glycolysis and their enzymes and their NADPH or FADH2 or ... you get the idea

And I am no where, not even close, to being as smart as you

 

[Med Ed]

Wow, I didn't realize the score creep and modern role of step1 had all happened so recently. Makes sense why even our younger faculty are so dismissive of it.

Things have really only gone completely off the rails in the last 5 years.

Is an 80% board rate hard to achieve? Don't most established programs have pass rates in the mid 90s?

It's tough for very small programs, where even a single failure can put you in the red zone.

Also interesting to look at some of the data in that tableau link. Using Derm as an example, it's a massive deciding factor. If I'm reading this right, at a 230, half of programs hard screen you out. But at a 243, half of programs "almost always grant interview".

So going up a dozen points can move you from "need not apply" to almost certainly interviewed? No wonder it's become such an arms race

Derm is ever the outlier.

 

[efle]

Things have really only gone completely off the rails in the last 5 years.

Coincides exactly with the start of all these prep materials like Pathoma, Boards and Beyond, and Zanki.

I pity the med students of 2025 if this thing doesn't get changed Pass/Fail, or at least capped again to only reporting something like "245+"

 

[907914]

Things have really only gone completely off the rails in the last 5 years.



It's tough for very small programs, where even a single failure can put you in the red zone.



Derm is ever the outlier.

Is Derm just a lifestyle thing or is it genuinely a fulfilling pathway?

 

[efle]

FYI - I'm 7 years post biochem and I can remember it along with all the enzymes and pathways into/out of glycolysis and their enzymes and their NADPH or FADH2 or ... you get the idea

And I am no where, not even close, to being as smart as you

I honestly don't think I'll ever once in my life be able to draw the TCA cycle from scratch. I crammed it in undergrad, I crammed it for med school, and honestly I'm gonna look at it in the 48 hours before my step1 and then dump it one last time.

 

[efle]

Is Derm just a lifestyle thing or is it genuinely a fulfilling pathway?

Put it this way, I imagine the vast majority of derm applicants would lose their fascination with leather overnight if it dropped to 200k/yr. Just look at the competitiveness of something like PM&R for comparison. That has a plenty good lifestyle. It's the $$ on top of lifestyle that makes Derm so competitive, no doubt

 

[Splenda88]

Is Derm just a lifestyle thing or is it genuinely a fulfilling pathway?

Lifestyle ~ $$$$ >>> Fulfilling pathway

Nothing against derm.

Do you think it's just a coincidence that more US grads are applying to psych now?


By the way, FM is a lifestyle specialty...

 

[907914]

Put it this way, I imagine the vast majority of derm applicants would lose their fascination with leather overnight if it dropped to 200k/yr. Just look at the competitiveness of something like PM&R for comparison. That has a plenty good lifestyle. It's the $$ on top of lifestyle that makes Derm so competitive, no doubt

I have PM&R, Path and Optho on my list for the lifestyle and interest/fulfilling aspect. Regardless of money, Derm just seems boring to me...

 

[ginsengreset]

I honestly don't think I'll ever once in my life be able to draw the TCA cycle from scratch. I crammed it in undergrad, I crammed it for med school, and honestly I'm gonna look at it in the 48 hours before my step1 and then dump it one last time.

I'd've thought that cramming this stuff multiple times would help with retention, a la the Ebbinghaus forgetting curves...

 

[907914]

I honestly don't think I'll ever once in my life be able to draw the TCA cycle from scratch. I crammed it in undergrad, I crammed it for med school, and honestly I'm gonna look at it in the 48 hours before my step1 and then dump it one last time.

For TCA, bust out the Ochem molecule kit to help visualize it. I mean, you won’t need it ever after your Step1...but if you want to know it...

 

[efle]

I have PM&R, Path and Optho on my list for the lifestyle and interest/fulfilling aspect. Regardless of money, Derm just seems boring to me...

For some people, work is work, and skin is just as interesting or boring as livers and eyeballs. Might as well pick the one that lets you live the best and retire young.

I'd've thought that cramming this stuff multiple times would help with retention, a la the Ebbinghaus forgetting curves...

When I cram, I cram. But the corollary is when I dump, I dump

 

[aldol16]

If you don't need a flashcard to remember which of the last steps produce NADH vs FADH vs GTP, two years after your biochem unit, you're a freak of nature that will never need anki for anything

Or I'm good at biochem and remember the biochemical pathways. Strange how these same pathways keep coming up on MCAT and Step.

 

[efle]

Or I'm good at biochem and remember the biochemical pathways. Strange how these same pathways keep coming up on MCAT and Step.

Remind me in a couple years to ask for some anecdotes of you using this knowledge to help patients

 

[aldol16]

Remind me in a couple years to ask for some anecdotes of you using this knowledge to help patients

So I walked in on this patient doing his biochem homework. Needed to know what succinyl-CoA synthetase did. Patient satisfaction 100%.

Oh, and PhD in chemistry also helps.

 

[Gurby]

Back to the topic though, it looks like people on Reddit are getting wrecked by the new NBME's:

To the point about how NBME's typically are too easy and not representative of the actual test, it seems good that they have cranked up the difficulty. Hopefully the top end of the curve will have better resolution now. However, these new practice tests are unproven as an assessment tool (nobody knows how your score on the new NBME's will correlate to your actual step 1 score, whereas we a have a decent idea of how predictive the old ones are).

@efle which practice tests are you planning to take?

 

[Med Ed]

Coincides exactly with the start of all these prep materials like Pathoma, Boards and Beyond, and Zanki.

We had our own go-to board study materials back in the day. The two things that are truly novel are short online videos and Anki.

 

[efle]

We had our own go-to board study materials back in the day. The two things that are truly novel are short online videos and Anki.

What's up with First Aid tripling in size since Y2K? Are they testing that much more or was it just missing huge swaths of info back in the day

 

[efle]

Back to the topic though, it looks like people on Reddit are getting wrecked by the new NBME's:

To the point about how NBME's typically are too easy and not representative of the actual test, it seems good that they have cranked up the difficulty. Hopefully the top end of the curve will have better resolution now. However, these new practice tests are unproven as an assessment tool (nobody knows how your score on the new NBME's will correlate to your actual step 1 score, whereas we a have a decent idea of how predictive the old ones are).

@efle which practice tests are you planning to take?

Planning to take all the new and none of the old. Took 18 as baseline before starting dedicated and the curve was absurdly tight, I hit 90% correct and it said that was still only in the 240s. It's no wonder people started saying to expect it to feel a lot more like tough Uworld blocks and not like the NBMEs. On Uworld so far I do significantly lower (low 80s average) but rank much higher (top decile / mid to upper 250s)

Personally gonna trust the Uworld much more unless the new NBMEs have a VERY different curve