Are we about to witness a return to high Step 1 fail rates?

[FootballAndSkiing]

I'm just curious, to the doctors on here, do you think you could pass step 1 if you took it tomorrow? Kinda curious what you guys would get in an NBME or free 120. Someone take me up on that challenge, there will be no prizes.

---

Members don't see this ad.

 

[efle]

I'm just curious, to the doctors on here, do you think you could pass step 1 if you took it tomorrow? Kinda curious what you guys would get in an NBME or free 120. Someone take me up on that challenge, there will be no prizes.

It would vary a lot by subject section I think. Questions about next step in management are easy now when they were very challenging as an MS2, but for things like pathology findings my knowledge has dropped precipitously

 

[Dr. Anonymouss]

Hey, maybe they just keep up with the literature! Can't say I saw much management with lasix in my med schooling or IM year either.

touché.

 

[Neopolymath]

Ah yes, it's that funny feeling you get when the obvious consequences of decisions start to appear and you see people surprised and you are confused by their confusion. The Germans probably have a word for the sensation.

I feel that there are a lot of dumb things about med school that need to change but the one thing I don't think is that it should get even easier. I mean passing step 1 is a joke and now I'm hearing people are whining the passing bar is too high when it's now an effectively inconsequential exam for anyone who half-assed preclinical.

Honestly, the further I get from med school the harder I think med school should have been at some schools instead of what I saw from some of my top tier classmates in my TY year. I don't mean harder because of added PhD minutae or stupid extra busy work stuff but like 196 passing score? Really?! Lots of these people don't do that well on boards and then also go into intern year without super strong practical physician skills either. I mean what is the point? What a waste of 4 years and a competitive residency spot on someone whose only difference is basically gaming their undergrad GPA better than the other person when it comes down to it. That's where we are at these days. Running further away from meritocracy is just exacerbating this issue even more.

The theme I have seen from a lot of so-called good schools' curriculum is a focus on useless research, a chill preclinical period, and then a pretty undercover lukewarm clerkship situation where no one has any real responsibility and is constantly leaving for inappropriate didactics instead of actually doing patient care (but because it's at MD teaching hospitals it's curiously ignored). Are we really going to keep going down this road to making med school even less useful? Everything is just getting pushed back further and further in training. At some point we are going to have fellows learning things that med students used to do even 20-30 years ago. Is that really what we want? Apparently it is. The message is clear. Make a test that isn't even hard to pass even easier and remove the most immediate reason to do well. At least some some med students listened to the advice to try in M1 and M2 because it will set the trajectory for the rest of their performance and exams.

 

[Lawpy]

Hey, maybe they just keep up with the literature! Can't say I saw much management with lasix in my med schooling or IM year either.

The real question is why is there an IM year for some specialties that have nothing to do with IM

 

[Lawpy]

Ah yes, it's that funny feeling you get when the obvious consequences of decisions start to appear and you see people surprised and you are confused by their confusion. The Germans probably have a word for the sensation.

I feel that there are a lot of dumb things about med school that need to change but the one thing I don't think is that it should get even easier. I mean passing step 1 is a joke and now I'm hearing people are whining the passing bar is too high when it's now an effectively inconsequential exam for anyone who half-assed preclinical.

Honestly, the further I get from med school the harder I think med school should have been at some schools instead of what I saw from some of my top tier classmates in my TY year. I don't mean harder because of added PhD minutae or stupid extra busy work stuff but like 196 passing score? Really?! Lots of these people don't do that well on boards and then also go into intern year without super strong practical physician skills either. I mean what is the point? What a waste of 4 years and a competitive residency spot on someone whose only difference is basically gaming their undergrad GPA better than the other person when it comes down to it. That's where we are at these days. Running further away from meritocracy is just exacerbating this issue even more.

The theme I have seen from a lot of so-called good schools' curriculum is a focus on useless research, a chill preclinical period, and then a pretty undercover lukewarm clerkship situation where no one has any real responsibility and is constantly leaving for inappropriate didactics instead of actually doing patient care (but because it's at MD teaching hospitals it's curiously ignored). Are we really going to keep going down this road to making med school even less useful? Everything is just getting pushed back further and further in training. At some point we are going to have fellows learning things that med students used to do even 20-30 years ago. Is that really what we want? Apparently it is. The message is clear. Make a test that isn't even hard to pass even easier and remove the most immediate reason to do well. At least some some med students listened to the advice to try in M1 and M2 because it will set the trajectory for the rest of their performance and exams.

I mean my stance is pretty clear: change all curricula to 1 yr preclinical/3 yr clinical and go from there. I believe in an all Step P/F world, the people who want to learn and do well will excel and those who don’t care will likely end up weeded out anyways.

 

[Neopolymath]

The real question is why is there an IM year for some specialties that have nothing to do with IM

Cheap labor. It's the only answer that actually explains every angle.

 

[efle]

I don't think increased Step 1 studying is going to address people showing up clinically weak to their intern year. For that you'd need a time machine to the era when clerks functioned more like interns, being given the pager, following call schedule, being taught procedures etc.

 

[VA Hopeful Dr]

I don't think increased Step 1 studying is going to address people showing up clinically weak to their intern year. For that you'd need a time machine to the era when clerks functioned more like interns, being given the pager, following call schedule, being taught procedures etc.

Yep. My Step scores were never stellar but I had a pretty rigorous 3rd/4th year so came into intern year pretty prepared.

In contrast, several of my residency classmates had never done an overnight call in all of medical school and so the first several months of intern year were very rough for them.

 

[VA Hopeful Dr]

Cheap labor. It's the only answer that actually explains every angle.

Eh, I think it's more tradition at this point. That's why you don't see a massive expansion of transition years like we have residency programs in the last 10 years.

 

[Neopolymath]

I don't think increased Step 1 studying is going to address people showing up clinically weak to their intern year. For that you'd need a time machine to the era when clerks functioned more like interns, being given the pager, following call schedule, being taught procedures etc.

My argument would be that I think these problems both come from the same attitude and go hand in hand. There is a knowledge component to step 1 that has been approaching lower yield for sure but I think actually trying is something that sets the tone for being a doctor and is applicable to the rest of your training in more pertinent aspects even if an individual is one who doesn't believe that step one has any meaning.

 

[hallowmann]

I agree that this is something that doctors should know, but in year 2 of 7? To be honest I think having the knowledge level of a practicing NP at step 1 (of 3) is almost funny considering most schools dont let us treat a patient yet.

It's a progressive curriculum. You need the foundation by year 2 in order to advance. You need the medical knowledge base of physiology and pathology so that you can then study (and understand) the management in year 3/4, so that you can then understand the actual implementation of the management in residency. I get the frustration, we were all there with you back in preclinical when we understood very little of what medicine was. In retrospect though, this makes a lot more sense.

Another big aspect that this trains you for is learning how to process information. The education process forces this to develop. By far the biggest pitfall of anyone in medicine that is subpar is the information processing. The Step 1 actually does test a lot of this in 3rd and 4th level questioning, and it's very valuable.

Have you looked at any comments from attendings who are alarmed by the increasing craziness of Step 1 in recent years? Seriously even the FA size has increased many times and that’s not just memorizing more facts. The questions are magnitudes harder and more difficult compared to old exam forms and there is literally no reason for that to happen other than the score creep from Step 1 mania

Part of this issue is that medicine in general is more complex than it was decades ago. Hell, there are tons of new things even compared to the beginning of my training to now. Physicians learn constantly. Anyone who is practicing the same way they did immediately out of training with no updates to their practice is harming patients. Its as simple as that. You can't talk about the expansion of board prep without genuinely discussing the expansion of medical content/knowledge over time.

I also find it interesting how much easier it is to pass COMLEX. I passed a practice exam very comfortably at the beginning of dedicated. That same week I got a 50% on nbme. Passing both of these tests allow someone to be a doctor.

To be fair, passing the COMLEX is like 55% and the practice tests are wildly inaccurate representations of the real test. They often have themes that completely miss the broad information base that you might encounter on the reap exam. Is it easier, probably, but I'd also argue it's less predictable, so to reliably pass you'd have to much better.

Well, I just finished the exam today, and I'm not entirely sure that I passed it. If more people really are failing it this year, they're probably like me and didn't study very much for it. Most of the advice I've received about P/F STEP is that you can easily pass it if you just pay attention in class and do some basic studying. That turned out to be completely false lmao. I think people who took STEP I scored don't even realize just how much they studied for it in hindsight. "Just passing" definitely isn't easy by any means. Whether it should be easy or not is a whole different debate.

And as much as I hate to say this because I know it's just going to convince people to trash on DOs even more, I have to agree with the comments above about COMLEX vs STEP. I was passing COMLEX practice exams quite easily even before dedicated, but by the end of dedicated (admittedly I didn't do much studying during it) I was only getting like 60% average on UWorld practice questions, and the actual exam was slightly harder than UWorld on top of that. COMLEX is just so much easier, which came as a shock to me because I thought if I was passing COMLEX practice exams so effortlessly, I should be able to do the same for STEP I. Turns out that isn't the case at all.

Literally no one knows whether they passed after the exam. That feeling persists throughout the process. You'll never leave a medical exam feeling like you did great, because there will be tons of questions you missed and some you had no idea about. This was true even when people studied to get 250s. You probably passed.

I would say that a COMLEX score <450 is probably a failing USMLE score, but that's not surprising. 20% of the test is OMT afterall. It's why a lot of residencies will not interview people with COMLEX scores that low.

...The theme I have seen from a lot of so-called good schools' curriculum is a focus on useless research, a chill preclinical period, and then a pretty undercover lukewarm clerkship situation where no one has any real responsibility and is constantly leaving for inappropriate didactics instead of actually doing patient care (but because it's at MD teaching hospitals it's curiously ignored). Are we really going to keep going down this road to making med school even less useful? Everything is just getting pushed back further and further in training. At some point we are going to have fellows learning things that med students used to do even 20-30 years ago. Is that really what we want? Apparently it is. The message is clear. Make a test that isn't even hard to pass even easier and remove the most immediate reason to do well. At least some some med students listened to the advice to try in M1 and M2 because it will set the trajectory for the rest of their performance and exams.

Haha, glad I'm not the only one that's noticed this over the years

 

[efle]

I'm surprised at the criticism of university hospital clinical rotations. Aren't most DO schools sending people all over their region to small hospitals and clinics to piece together the requirements, often with preceptors that clearly arent in it for teaching? Is that really better for learning than rotating through the departments of a quaternary teaching hospital?

 

[efle]

My argument would be that I think these problems both come from the same attitude and go hand in hand. There is a knowledge component to step 1 that has been approaching lower yield for sure but I think actually trying is something that sets the tone for being a doctor and is applicable to the rest of your training in more pertinent aspects even if an individual is one who doesn't believe that step one has any meaning.

I quite like this take, because I find it much more honest than claims that a high step 1 is the basis for strong clinical work. But I also agree with Dr Carmody's argument ad absurdum - would a world where we stratify students on the number of pi digits they memorize not fulfill this goal of seeing who works hard too?

Plus, as I recounted earlier, I think someone can be a bright medical student who learns exactly what they're supposed to and still not do particularly well on Step. It's heavy studying of step-specific materials that drives up a score, which is far from synonymous with good foundations for the wards.

 

[wondervisions]

Yeah when I was chatting with some local MS3s who just tested they said pretty much everybody has been starting out with failing NBMEs / CBSEs and then cramming really hard in dedicated to get to a reliable pass. Lots of people postponing or having to test with a legit chance of failing if they're unlucky on their test form.

And this is at a school with graded clerkships so there is reason to do more than coast by on 70s every exam. In true unranked pass/fail curriculums there must be a lot of very nervous students and deans.

I failed my CBSE right before dedicated and ended up scoring ~240 after 7 weeks 🤷‍♀️

 

[DD214]

Are you a practicing physician?

Are you?

 

[Goro]

Are you?

Nope, but I have 20+ years in teaching at the pre-clinical level, and always amused at medical student who think that they know what on Step prior to taking it.

I notice that you didn't answer my question either. That confirms my suspicion.

 

[Neopolymath]

Haha, glad I'm not the only one that's noticed this over the years

I could write a long, long thread about how this is probably one of the few things that SDN dogma was straight up wrong about in my experience but I suppose it's only tangentially related to the bigger issues noted in this thread lol.

 

[arjusmc]

I also find it interesting how much easier it is to pass COMLEX. I passed a practice exam very comfortably at the beginning of dedicated. That same week I got a 50% on nbme. Passing both of these tests allow someone to be a doctor.

Comlex is a different exam, your med school(and mine) caters to it not step 1. I’m not surprised that within a week I got 65th percentile on a comsae and 31st on a uworld SA. Not to mention the obvious elephant in the room of the testing cohort of USMLE overall having higher aptitude.

 

[Goro]

I graduated residency in 2005 and have certified and recertified in my specialty

I guess your admission proves that those who can't, teach

And those who can't teach, teach gym!

 

[FootballAndSkiing]

Comlex is a different exam, your med school(and mine) caters to it not step 1. I’m not surprised that within a week I got 65th percentile on a comsae and 31st on a uworld SA. Not to mention the obvious elephant in the room of the testing cohort of USMLE overall having higher aptitude.

That's true, though my school didn't completely ignore step 1. At my school we still have to pass step 1 to graduate. I think it may have been a miscalculation by my school but we'll see.

 

[arjusmc]

That's true, though my school didn't completely ignore step 1. At my school we still have to pass step 1 to graduate. I think it may have been a miscalculation by my school but we'll see.

I can see how that would be different from most DO schools. I think most at my school could pass step 1 but would have to go well beyond their comlex prep to account for the knowledge gaps. I’m taking comlex on Friday but not taking step until late June.

 

[detritus]

150s!? Yeah, that was not the case in the past. 160s-170s were basically what people who we were worried wouldn't pass would get. Are people not using board study materials during 2nd year anymore? That's the only explanation, relying solely on the school curriculum.

Very few ppl at my school take step 1 seriously anymore. We’ll have to see the fail rate but I’d be shocked if it doesn’t go up

 

[Dr G Oogle]

Have you looked at any comments from attendings who are alarmed by the increasing craziness of Step 1 in recent years? Seriously even the FA size has increased many times and that’s not just memorizing more facts. The questions are magnitudes harder and more difficult compared to old exam forms and there is literally no reason for that to happen other than the score creep from Step 1 mania

Saying the test is harder because of score creep is just as simplistic as saying it’s harder because we have learned more about medicine in general. Both are probably true, favoring more the second point, and a sprinkling of other reasons. Dropping the fail threshold helps no one except maybe a struggling student who maybe should not be practice medicine. I definitely agree with the P/F route, the difference between barely passing and 258 could be just having a bad day; but passing versus failing is more likely due to a fundamental lack of basic knowledge necessary for all doctors.

 

[arjusmc]

I don’t think anyone can make a legitimate argument without knowing what’s going on with the tests. You literally would have to have all the information that USMLE has on the questions they are putting out compared to 5, 10, 15 years ago and the associated performance. If you do have that info and you’re commenting, you’re most likely breaking an NDA. The people that do have that info made the decision to go P/F. Like most decisions in bureaucratic organizations it was probably a few years too late. I’d imagine the pass line will be lowered but not to 2005 levels.

The increased failure rate is probably due to one of two thoughts. First some will assume that just because they passed their curriculum that they will pass boards without work or with minimal work. The second is due to competition being increased at schools where class rank will be more important without a board score. This may have people put off board studying to work on class tests more.

 

[Goro]

Not gloating, but kudos to the mods for another Banhammer smack.

 

[DOVinciRobot]

I'm surprised at the criticism of university hospital clinical rotations. Aren't most DO schools sending people all over their region to small hospitals and clinics to piece together the requirements, often with preceptors that clearly arent in it for teaching? Is that really better for learning than rotating through the departments of a quaternary teaching hospital?

I suspect it’s likely very highly dependent on the institutional culture. I have been extremely unimpressed by the clinical education the med students at my program get, and we’re a large regional tertiary care center with departments in everything. I was far more prepared to take care of patients as an intern coming from my DO school rotations than the med students here are. Most of the 4th years don’t even know how to write an adequate note or present a patient in a somewhat coherent manner…

 

[Billiam95]

Tested today, got one question right because I knew the feces of a specific bug species causes a certain pathology (thanks sketchy) and got another because I knew the description of the the histopathology (description, wasn't even shown the slide) of a parasitic infection (thanks uworld) - if I ever use this information again, I'll eat my sock.

 

[Ho0v-man]

I suspect it’s likely very highly dependent on the institutional culture. I have been extremely unimpressed by the clinical education the med students at my program get, and we’re a large regional tertiary care center with departments in everything. I was far more prepared to take care of patients as an intern coming from my DO school rotations than the med students here are. Most of the 4th years don’t even know how to write an adequate note or present a patient in a somewhat coherent manner…

I’ve had similar experiences. However, the DOs making such comments in this thread are sort of overachievers. Tiers are made by the bottom of the class, not the top. The worst students at an osteopathic school are probably worse than the worst students at a USMD school. Though I will concur that the significance of the clinical Ed stuff has been greatly overblown on SDN.

But this has been quite the derail of the thread.

 

[JP2740]

The fact that PDs abused Step 1 because they have no answer to the overapplication problem doesn’t justify keeping abnormally high Step 1 pass thresholds. Use the Steps for their intended purpose: to assess basic medical competency of doctors in training and nothing else. To have a Step 1 pass threshold in the 190s is completely absurd.

Abnormally high? A 190? If you just try you should get a 190.

 

[efle]

To bring it slightly back towards the rails then -

For people who dislike the idea of dropping the Pass mark back down, would it change your mind if a very large number, like 20-25% of people, turned out to be delaying their test or retaking it? Would that be attributable to a new zeitgeist of laziness? Or would numbers that high suggest the distribution had been pushed too far up by the pressure to maximize scores in prior years?

 

[Lawpy]

Abnormally high? A 190? If you just try you should get a 190.

I don’t think it’s hard but many students disagree. A 196 pass threshold is too close to 200s for an exam with large standard errors. That’s why i kept pushing to lower the pass threshold to 180 so that people scoring in the 200s-210s can feel comfortable that they’ll pass

 

[Lawpy]

Saying the test is harder because of score creep is just as simplistic as saying it’s harder because we have learned more about medicine in general. Both are probably true, favoring more the second point, and a sprinkling of other reasons. Dropping the fail threshold helps no one except maybe a struggling student who maybe should not be practice medicine. I definitely agree with the P/F route, the difference between barely passing and 258 could be just having a bad day; but passing versus failing is more likely due to a fundamental lack of basic knowledge necessary for all doctors.

I think people scoring in the 200-210s shouldn’t need to worry about failing which is what’ll happen with a pass threshold of 196. A 200-210s is not a bad score.

 

[Billiam95]

N=1 but I think my class spent more time on lecture material than previous years under the assumption that class rank may be more heavily weighted in match. The problem is that our lectures were inadequate preparation for step1, and there is just too much boards-specific information to learn in an 8 week dedicated.

 

[efle]

Abnormally high? A 190? If you just try you should get a 190.

Approximately half of your attendings who earned MDs in the 1990s scored at the modern fail threshold or lower, because the initial median was a 200. And as previously mentioned, until recently, a 196 in their day was supposed to be equivalent to now.

 

[Dr. Anonymouss]

N=1 but I think my class spent more time on lecture material than previous years under the assumption that class rank may be more heavily weighted in match. The problem is that our lectures were inadequate preparation for step1, and there is just too much boards-specific information to learn in an 8 week dedicated.

Lectures at my school were severely inadequate as well which is why I chose to pursue independent studies and relied on outside resources. One of our academic learning specialists said that the students who were struggling the most with step I prep were the ones who only attended lecture...

 

[Billiam95]

I think there's a great knowledge gap about the difficulty in step1 nowadays. Advice from admin who sat for step1 20+ years ago set a lot of my classmates up to fail. "Save uWorld for dedicated" might have worked years ago, but 3600 questions is just too much to cover and really learn in a short amount of time, especially with the need to do content review as well. What wasn't mentioned was how important spaced repetition, either through anki or some other method, is to retaining information from prior systems. Students who did go to the school's learning specialist after difficulty in dedicated were simply told that they should have learned the material better the first time - which is both possibly not true and definitely not helpful.
The best advice for step1 came from older students and from reddit, both of which have to be sought out and, in the case of reddit, require believing people on the internet.
I really think scoring a 215 approximates to foundational understanding of phys, path, micro, pharm, stats, biochem, genetics. However, I've seen the argument made that the confidence interval for step1 is +/- 20, which puts one at risk for failing. Moving to the 230s - 240s requires memorization of esoteric details that I don't think I'll ever use again. I think the time I spent pushing my practice test scores into the comfortably passing range via those esoteric details could have been better spent learning clinical medicine - I feel like I know virtually nothing about how I'll be expected to manage patients in about a month when all the information needed isn't presented in a clinical vignette.
What's the next step in managing a patient with any bread and butter disorder? No idea, but I could give you the intracellular pathway of Gs protein coupled receptors. I'm just not sure the juice is worth the squeeze.

 

[NotAProgDirector]

This may be a lazy observation, but my assumption is that people who make these tests have decided that the 93-96% pass rate seems to be ideal. It has nothing to do with being competent or having a certain level of comprehension of the material. PDs never looked COMLEX the same way they looked at STEP, thus that arms race never happened. People try harder on STEP thus the threshold to pass has to go up. Also I'm aware the populations are different, but have no idea how to factor that in. STEP has Harvard grads and Caribbean grads. COMLEX is only DO.

It's possible that the USMLE is targeting a 5% failure rate. That would be a norm referenced cut off -- define the mean and SD of the exam, then set the minimum pass at mean - 1.65 * SD (which is the 5th percentile). Agreed this is super lazy.

The other option is that they systematically determine a criterion based cutoff (using experts, there are all sorts of ways to do this) and then determine the minimum pass from that. Any criterion based system is open to bias of all sorts. Assuming that applicants to medical school "smarts" (at least as measured on a MCQ exam) remains stable over time, then the same percentage of people would be expected to fail over time.

What we are missing are raw scores. If the raw score to pass has gone up over time, then one explanation is that the testing system has become more strict over time to ensure the fail rate remains steady. There are other explanations -- such as that society demands more from physicians today than in the past (hence higher cutoffs). Or the nature / content of the exam has changed such that higher raw scores would be expected.

From my perspective, the most likely explanation for 80% of this phenomenon is just that people took their eye of the ball when the test went P/F. The next class will see this train wreck and make different choices. After about three cohorts we'll have a good idea of what the "new normal" looks like. Until then everyone just needs to take some deep breaths.

Interesting given that those who run the USMLE mentioned this possibility, and there was widespread condemnation about the statement.

True but don’t most PDs only look at USMLE anyways so it doesn’t really matter if the complex exam is easier?

It's COMLEX. Not all PD's ignore it.

I don’t think it’s hard but many students disagree. A 196 pass threshold is too close to 200s for an exam with large standard errors. That’s why i kept pushing to lower the pass threshold to 180 so that people scoring in the 200s-210s can feel comfortable that they’ll pass

Given the current nature of the exam. the mean is 228 with a SD of 18 or so. A 180 is a z score of (228-180)/18 = -2.67. That equates to a percentile of 0.38%. If the fail rate is that low, then there's almost no point in using the exam at all.

I really think scoring a 215 approximates to foundational understanding of phys, path, micro, pharm, stats, biochem, genetics. However, I've seen the argument made that the confidence interval for step1 is +/- 20, which puts one at risk for failing.

The confidence interval of the exam is not 20. The Standard Deviation of the distribution of exam scores is about 20. This describes the distribution of everyone's score, not the accuracy of any one score.

The metric you're (usually) looking for is the Standard Error of Measurement. This is a "standard deviation" measure of the accuracy of a test result. The SEM of Step 1 is 6. Therefore, if someone scores a 215, their "true score" is somewhere 215 +/- 12 95% of the time.

And it can get more complicated. You can calculate a Standard Error of the Estimate which helps answer: Given a score on an exam, what is the likely range of scores if the exam (with new items) is given to the same person with the same knowledge. This can get messy quickly as the confidence intervals on that are often asymmetric (due to regression to the mean, a repeat result is more likely to be between the given result and the mean rather than the other way). This quickly exhausts my statistical knowledge, and might be wrong -- anyone is free to chime in.

But I'm 100% certain you can't use the SD in the way you suggest. The actual accuracy of the test is much better.

 

[Lawpy]

The metric you're (usually) looking for is the Standard Error of Measurement. This is a "standard deviation" measure of the accuracy of a test result. The SEM of Step 1 is 6. Therefore, if someone scores a 215, their "true score" is somewhere 215 +/- 12 95% of the time.

This is what I was getting at. Someone scoring in 200s-210s is therefore hovering dangerously close to failing.

Given the current nature of the exam. the mean is 228 with a SD of 18 or so. A 180 is a z score of (228-180)/18 = -2.67. That equates to a percentile of 0.38%. If the fail rate is that low, then there's almost no point in using the exam at all.

This is true but the problem lies with the mean being in the upper 220s rather than in 200s. The exam should be recalibrated to adjust for this

 

[ace_inhibitor111]

To bring it slightly back towards the rails then -

For people who dislike the idea of dropping the Pass mark back down, would it change your mind if a very large number, like 20-25% of people, turned out to be delaying their test or retaking it? Would that be attributable to a new zeitgeist of laziness? Or would numbers that high suggest the distribution had been pushed too far up by the pressure to maximize scores in prior years?

What would be the utility of a test that needs 60% correct to pass, which would roughly be equivalent to the 190ish pass threshold. Could you even claim someone has basic knowledge at that point? The point of the test is to assess knowledge gathered over a prolonged period of rigorous study 1-2 years, which people are clearly not doing anymore. Otherwise any NP or PA nub can pick up First Aid, do an 8 week dedicated, and pass this test just as easily as well.

 

[Ho0v-man]

To bring it slightly back towards the rails then -

For people who dislike the idea of dropping the Pass mark back down, would it change your mind if a very large number, like 20-25% of people, turned out to be delaying their test or retaking it? Would that be attributable to a new zeitgeist of laziness? Or would numbers that high suggest the distribution had been pushed too far up by the pressure to maximize scores in prior years?

We shouldn’t lower the pass threshold. I get that our attendings weren’t scoring that high when they were students. But I don’t think that argument is very good. Our attendings did not have the resources we have. You can easily get to the pass threshold by just memorizing sketchy micro and pharm and having a basic understanding of pathophysiology from your classes that is then reinforced by uworld. So I do think we should be held to a higher standard because everything you need is at your fingertips. Heck, it’s all free to every med student in flash card form!

We’re literally talking about understanding how the human body works and how to pick an antibiotic. This is not a high bar.

 

[VA Hopeful Dr]

What would be the utility of a test that needs 60% correct to pass, which would roughly be equivalent to the 190ish pass threshold. Could you even claim someone has basic knowledge at that point? The point of the test is to assess knowledge gathered over a prolonged period of rigorous study 1-2 years, which people are clearly not doing anymore. Otherwise any NP or PA nub can pick up First Aid, do an 8 week dedicated, and pass this test just as easily as well.

Depends on the difficulty of the questions. Given the previously increasing score creep, it wouldn't surprise me if the questions got progressively harder after a certain percentage.

Honestly now that score doesn't matter it would be nice to have the test not as focused on minutiae as it seems to have been lately (at least per what I read here).

 

[Mean Green Raider]

Just throwing in my 2 cents:
Over the past few months I have been in meetings with various members of my school's academic affairs office, and one of the big issues that comes up is Step 1 and how students at our school and nationally are doing/expected to do. I think there are bits of truth to a lot of comments here, so I just wanted to throw out the few big reasons/thoughts I have had/heard:

1) COVID-19: The current class (2024) was in full COVID restrictions (I.e., virtual) their first year. This also extended to test taking, so there is a chance students could have cheated from home. It is also possible that students just got used to testing from home as opposed to testing in a school-type setting. (For example, if you are taking an exam at home, you can talk to yourself out-loud as you reason through a problem. You can't do this while at school).

2) Unrealistic Handed-Down Expectations: Many of the MS3s (Class of 2023) could have said to the MS2s (2024) "Oh yeah, passing Step 1 is so easy. All you need is a 196, that is a joke. Don't waste too much time studying!" However, since many, if not all, of those MS3s took a scored Step 1 (Which as we all understand has a significant impact on residency match odds), they likely spent much more time studying for Step 1 prior to dedicated. (For example, many in 2023 may have started studying for Step 1 on Day 1 on medical school or maybe they pushed it back a semester. While anecdotally, many in 2024 may not have started studying until this Spring).

3) Focusing on Non-Curriculum Things: I have noticed this especially with my class. I attend a P/F non-ranked pre-clinical medical school. As a result, many students have a mentality of "All I have to do is make a 70 on this exam to pass. I can now spend a lot more time doing research/volunteering/etc., instead of studying, to bolster my residency application!" Which in theory is true, but many students may have also struck a poor balance in the process. As a result, they may have neglected studying so much that they put themselves into a deep hole that is hard to dig out of in just the time of dedicated (Approximately 8 weeks). This also extends to Step 1 being P/F. Since score doesn't matter, the idea is to get the minimum score necessary to pass and then spend all of your remaining time doing other residency-app-bolstering activities. I think some have just struck a poor balance/underestimated what it really takes to hit that 196 number.

Would love any other thoughts anyone has on this/relevant topics/experiences at their school!

 

[efle]

What would be the utility of a test that needs 60% correct to pass, which would roughly be equivalent to the 190ish pass threshold. Could you even claim someone has basic knowledge at that point? The point of the test is to assess knowledge gathered over a prolonged period of rigorous study 1-2 years, which people are clearly not doing anymore. Otherwise any NP or PA nub can pick up First Aid, do an 8 week dedicated, and pass this test just as easily as well.

Now that's an interesting sidebar. If an NP did pick up FA and drop a 230, are they equivalent to an MD student with the same score? Because we all seem to believe a 230 is best gotten from UFAPS instead of a med school curriculum, and anyone can access UFAPS. Starting to realize why the exam can only be taken by currently enrolled medical students....

 

[ChymeofPassion]

Now that's an interesting sidebar. If an NP did pick up FA and drop a 230, are they equivalent to an MD student with the same score? Because we all seem to believe a 230 is best gotten from UFAPS instead of a med school curriculum, and anyone can access UFAPS. Starting to realize why the exam can only be taken by currently enrolled medical students....

Change all medical school curriculums to require a passing step score for entrance then do 2 years of clinicals. Get rid of the MCAT, the new dogma is premeds take a year off (or do it during college) to grind pathoma/BnB and pass Step 1, then apply for the 2 year of clinicals with a 4 week crash course on how to interview and present patients.

Would save 2 years of tuition and the remaining two years would be decreased due to not needing to pay professors to teach to auditoriums of 7 students.

/s but also preclinical is such a waste of time so maybe not /s

 

[efle]

Change all medical school curriculums to require a passing step score for entrance then do 2 years of clinicals. Get rid of the MCAT, the new dogma is premeds take a year off (or do it during college) to grind pathoma/BnB and pass Step 1, then apply for the 2 year of clinicals with a 4 week crash course on how to interview and present patients.

Would save 2 years of tuition and the remaining two years would be decreased due to not needing to pay professors to teach to auditoriums of 7 students.

/s but also preclinical is such a waste of time so maybe not /s

Flash forward to SDN in the year 2040, consensus opinion becomes -

MS3-4 are glorified useless shadowing, they just want our tuition like they used to get from preclinicals. The real foundation for success as an intern is avoiding the resident room to do more anCKi deck flashcards and at least two Uworld CK passthroughs. What's that grandpa? When you graduated in 2020 many people didn't even take CK until it was an afterthought past their residency interviews, and the average Derm match was at our 20th percentile? Well, you were in a different time when you didn't have to know half as much to be a decent doctor. These days, everyone knows Step 2 performance is the way to show you'll be a capable and hard working resident, and anyone scoring below a 242 deserves to fail they're clearly deficient.

 

[efle]

You can easily get to the pass threshold by just memorizing sketchy micro and pharm and having a basic understanding of pathophysiology from your classes that is then reinforced by uworld.

We’re literally talking about understanding how the human body works and how to pick an antibiotic. This is not a high bar.

Clearly, your attendings knew enough about physio to function as residents and fellows in the 90s-2000s.

Is the idea that they learned it all on the job? Because this interpretation of a ~200 would imply half of them snuck through med school without learning to pick an antibiotic. You can blame their lack of resources instead of blaming them, but ultimately that would have to be your claim.

Personally, I think it's just that nobody expected them to memorize whether Caliciviridae was enveloped or not, and it made them none the worse at holding the pager

 

[VA Hopeful Dr]

Clearly, your attendings knew enough about physio to function as residents and fellows in the 90s-2000s.

Is the idea that they learned it all on the job? Because this interpretation of a ~200 would imply half of them snuck through med school without learning to pick an antibiotic. You can blame their lack of resources instead of blaming them, but ultimately that would have to be your claim.

Personally, I think it's just that nobody expected them to memorize whether Caliciviridae was enveloped or not, and it made them none the worse at holding the pager

I've lost 87 patients to calciviridae this year.

Or wait, is it that I've never heard of that virus before?

 

[Ho0v-man]

Clearly, your attendings knew enough about physio to function as residents and fellows in the 90s-2000s.

Is the idea that they learned it all on the job? Because this interpretation of a ~200 would imply half of them snuck through med school without learning to pick an antibiotic. You can blame their lack of resources instead of blaming them, but ultimately that would have to be your claim.

Personally, I think it's just that nobody expected them to memorize whether Caliciviridae was enveloped or not, and it made them none the worse at holding the pager

We have a much more comprehensive knowledge of physiology than attendings who were in school at that time. We have way more meds and way more treatments. In the 90s we didn’t give beta blockers in heart failure and you essentially always admitted uncomplicated pneumonia. We’re expected to hit wards knowing a ton of physiology and pharm that was poorly understood then or (in the case of pharm) didn’t even exist. So yeah, I think the knowledge bar is higher now for what we’re expected to know when we hit wards. The standard is higher.

We definitely learn clinically irrelevant minutiae, but as I’ve already said, no one’s failing based on that if they know the important stuff. As an aside, the micro minutiae was nothing on step 1 compared to comlex.

Now something we’ll both likely agree on is that when someone hit wards in 3rd year 20 years ago with knowledge deficits, the information was drilled into them. That’s not the case anymore. So I firmly believe our attendings knew the phys and pharm they we’re supposed to know by the end of med school. But they learned it during rotations instead of showing up with it all memorized as is the norm now.

 

[VA Hopeful Dr]

We have a much more comprehensive knowledge of physiology than attendings who were in school at that time. We have way more meds and way more treatments. In the 90s we didn’t give beta blockers in heart failure and you essentially always admitted uncomplicated pneumonia. We’re expected to hit wards knowing a ton of physiology and pharm that was poorly understood then or (in the case of pharm) didn’t even exist. So yeah, I think the knowledge bar is higher now for what we’re expected to know when we hit wards. The standard is higher.

We definitely learn clinically irrelevant minutiae, but as I’ve already said, no one’s failing based on that if they know the important stuff. As an aside, the micro minutiae was nothing on step 1 compared to comlex.

Now something we’ll both likely agree on is that when someone hit wards in 3rd year 20 years ago with knowledge deficits, the information was drilled into them. That’s not the case anymore. So I firmly believe our attendings knew the phys and pharm they we’re supposed to know by the end of med school. But they learned it during rotations instead of showing up with it all memorized as is the norm now.

I thought this way in med school and residency, especially in talking with my FP uncle.

But then I got a copy of my grandfather's Grey's Anatomy from the 40s and talked to said uncle and my OB/GYN father-in-law. That anatomy book had significantly more detail than my anatomy textbook from 2005. And the physical exam skills they learned in the 70s were significantly more involved that what I learned. The amount of stuff that had to be learned I don't believe was that different. but the focus is on whatever is known at the time. Fewer medications were around in the 70s, so more emphasis on anatomy and exam skills.