the state of step 1

[Med Ed]

---

Members do not see this ad.

They don’t want their match rate to go down.

Meh, most of them would get mopped up in the SOAP and count as matched.

 

[Med Ed]

Yeah I can definitely see this as a downside that I hadn’t considered. I don’t have a solution, and there is no perfect solution available, as some population will be upset no matter what happens.

Indeed, this is a zero sum game. In the current system the anecdotes are students who are gunning for competitive fields but do badly on Step 2. They get hurt. But every time that happens an opportunity opens up for someone else.

When Step 1 was scored I had many students who would squeak past it and then blossom in the wards, often getting respectable Step 2 scores. In spite of this, many of them remained hamstrung by their Step 1 scores in terms of specialty choice or program. Those students now benefit from the current system.

 

[eyeeye_captain]

Indeed, this is a zero sum game. In the current system the anecdotes are students who are gunning for competitive fields but do badly on Step 2. They get hurt. But every time that happens an opportunity opens up for someone else.

When Step 1 was scored I had many students who would squeak past it and then blossom in the wards, often getting respectable Step 2 scores. In spite of this, many of them remained hamstrung by their Step 1 scores in terms of specialty choice or program. Those students now benefit from the current system.

You basically described my application back in the graded days. Good preclinical grades, then a hair to two hairs below average Step 1 for the really competitive stuff, nailed the wards, and got like a 270 Step 2 (taking it as early as I possibly could). I would’ve had a much easier go if it were P/F as I’m more clinically oriented. For people who are more preclinically oriented, having a graded Step 1 where you could crush it and put Step 2 off until after apps were in was a big advantage.

The opportunity for someone still comes from a pool that was already building an application for those specialties to some degree. It’s harder to suddenly get connections and LORs later in the game.

 

[Med Ed]

You basically described my application back in the graded days. Good preclinical grades, then a hair to two hairs below average Step 1 for the really competitive stuff, nailed the wards, and got like a 270 Step 2 (taking it as early as I possibly could). I would’ve had a much easier go if it were P/F as I’m more clinically oriented. For people who are more preclinically oriented, having a graded Step 1 where you could crush it and put Step 2 off until after apps were in was a big advantage.

The opportunity for someone still comes from a pool that was already building an application for those specialties to some degree. It’s harder to suddenly get connections and LORs later in the game.

True. The simple fact of the matter is there is no universally ideal time to schedule these exams. Perhaps medical school should be 5 years.

Back in my day M3 ended in July, and there were zero electives permitted before that time. If you wanted a specialty outside the core clerkships you had to hit the ground running the moment M4 started.

 

[ceo_of_stonks]

So the dean is incompetent and the student was a fool. That's hardly a systems-level issue.


That may be true at your institution. My own experience has been quite different.

I would love to hear your own experience then

 

[Lost in Translation]

You basically described my application back in the graded days. Good preclinical grades, then a hair to two hairs below average Step 1 for the really competitive stuff, nailed the wards, and got like a 270 Step 2 (taking it as early as I possibly could). I would’ve had a much easier go if it were P/F as I’m more clinically oriented. For people who are more preclinically oriented, having a graded Step 1 where you could crush it and put Step 2 off until after apps were in was a big advantage.

The opportunity for someone still comes from a pool that was already building an application for those specialties to some degree. It’s harder to suddenly get connections and LORs later in the game.

Crazy idea just crossed my mind: have the applicant decide which Step sequence of 1 or 2 they want to be graded P/F, and the other one is graded on the standard numerical scale. Those who are aiming for competitive specialties would benefit from a numerical Step 1 (and those programs would probably "screen" applicants that way), and those who choose to have Step 1 be P/F and Step 2 be numerical can have less stress during Step 1 studying. Nothing stops the latter group from applying to competitive specialties, just like nothing stops those programs from reviewing select applicants with a P/F Step 1 and graded Step 2.

This solution allows for the hyper-competitive to distinguish themselves on a[n irrelevant] test but also lets the not-as-good test takers avoid a less-than-stellar result that, while numerically passing, would diminish their chances of matching in even a "mid-tier" program in their desired field. Or of they're still aiming for a competitive specialty, they can hedge all their bets on a killer Step 2. Best of both worlds.

This is not a serious idea. Unless...

 

[Nontrad_FL_LGBT]

Crazy idea just crossed my mind: have the applicant decide which Step sequence of 1 or 2 they want to be graded P/F, and the other one is graded on the standard numerical scale. Those who are aiming for competitive specialties would benefit from a numerical Step 1 (and those programs would probably "screen" applicants that way), and those who choose to have Step 1 be P/F and Step 2 be numerical can have less stress during Step 1 studying. Nothing stops the latter group from applying to competitive specialties, just like nothing stops those programs from reviewing select applicants with a P/F Step 1 and graded Step 2.

This solution allows for the hyper-competitive to distinguish themselves on a[n irrelevant] test but also lets the not-as-good test takers avoid a less-than-stellar result that, while numerically passing, would diminish their chances of matching in even a "mid-tier" program in their desired field. Or of they're still aiming for a competitive specialty, they can hedge all their bets on a killer Step 2. Best of both worlds.

This is not a serious idea. Unless...

I feel like this wouldn't work simply because PDs wouldn't be able to decide on which one matters most and you'd be DOA at some top programs either way. But conceptually cool idea haha.

 

[ron_johnson]

P/F school grading + no AOA makes for a better educational experience in terms of collaborative classmates that know that the rising tide lifts all ships, but I think that P/F Step 1 makes for an incentive system that the average patient would find patently insane. I have classmates grinding meaningless research and EC's 25/30 days of the block and cramming for a quiz / test the other 5 days since Step is no longer graded and they are smart enough to get the ≥70 on the test. This seems very backwards for folks that want to become doctors and treat patients, at least to me.

Totally agree I think the research stuff is crazy. The meta now for matching competitive specialties is to do enough studying to pass comfortably and then grind as many research items as possible, which is totally backward since 99% of us are in school to become clinicians and not researchers. Ik people counter this by saying that a lot of the content on step1 is just garbage (muh acute angle vs 90 deg branching hyphae), but in that case the answer is to improve curricula or just remove the crap content and shorten the degree program, not add meaningless research. Like the fact that all these people are doing research years to match ortho or whatever is pretty rich considering the vast majority of them are going to be community surgeons some day. honestly seems exploitative given the opportunity cost of delaying attendinghood.

 

[Med Ed]

I would love to hear your own experience then

In my experience most Step 1 failures don't occur in a vacuum. There is a history of persistent bottom quartile exam performance, and students in this situation usually begin to select themselves out of the running for highly competitive fields. But once in awhile someone gets completely blindsided.

After the Step 1 failure hits, you still have to sit down with them and find out what they are looking for in a specialty: cognitive vs. procedural, inpatient vs. outpatient, acute vs. chronic, etc. While FM is a common destination, I have seen students match into fields like IM, EM, psych, neurology, and PM&R. IM in particular is a nice gateway to a whole bunch of subspecialties.

In the immediate wake it is still unknown how the student will respond in the coming year. If they are able to successfully shift over to clerkships and show aptitude for clinical practice, that keeps doors open.

 

[AJS59]

Some schools have done this in concert with earlier clinicals. Finish clinicals at end of year two then they basically get two blocks for studying for step 1 and step 2 mid year 3. Sub Is late year 3 early year 4.

FWIW, this is exactly how my school does it. You have 3 months after the end of rotations (beginning of October-beginning of January M3 year) to get both done in whatever order you want. Then people customarily "front-load" their schedules where possible to get sub I's and most electives done by end of spring M3 year, away rotations over the summer and early fall. By the time interview season rolls around, many (at least those not in dual degree programs, where schedules get much tighter) have effectively completed their degree requirements.

I think there's some benefits to this schedule: Namely the fact that you can pick and choose when to take both exams, and that you have a 9 month period from when you have your board scores in hand to when you submit applications where you can remold your application as you see fit.

 

[Goro]

In my experience most Step 1 failures don't occur in a vacuum. There is a history of persistent bottom quartile exam performance, and students in this situation usually begin to select themselves out of the running for highly competitive fields. But once in awhile someone gets completely blindsided.

After the Step 1 failure hits, you still have to sit down with them and find out what they are looking for in a specialty: cognitive vs. procedural, inpatient vs. outpatient, acute vs. chronic, etc. While FM is a common destination, I have seen students match into fields like IM, EM, psych, neurology, and PM&R. IM in particular is a nice gateway to a whole bunch of subspecialties.

In the immediate wake it is still unknown how the student will respond in the coming year. If they are able to successfully shift over to clerkships and show aptitude for clinical practice, that keeps doors open.

That's almost word for word I would say about our students who fail Level 1!!

 

[Ho0v-man]

In my experience most Step 1 failures don't occur in a vacuum. There is a history of persistent bottom quartile exam performance, and students in this situation usually begin to select themselves out of the running for highly competitive fields. But once in awhile someone gets completely blindsided.

After the Step 1 failure hits, you still have to sit down with them and find out what they are looking for in a specialty: cognitive vs. procedural, inpatient vs. outpatient, acute vs. chronic, etc. While FM is a common destination, I have seen students match into fields like IM, EM, psych, neurology, and PM&R. IM in particular is a nice gateway to a whole bunch of subspecialties.

In the immediate wake it is still unknown how the student will respond in the coming year. If they are able to successfully shift over to clerkships and show aptitude for clinical practice, that keeps doors open.

Easy to be blindsided when your whole curriculum is pass fail

 

[ceo_of_stonks]

Easy to be blindsided when your whole curriculum is pass fail

this. I wasn't top student, but I was doing better than some I know. Alot of our lower performing students haven't even taken step 1 yet. I also hit my school's qualifying score, so I took. I know I wasn't the best, but not the worst either (I only failed one system because I got COVID really bad for 1 week of the 3 weeks of the system- this isn't reported, because I passed the 'thread' aka semester)

 

[ceo_of_stonks]

In my experience most Step 1 failures don't occur in a vacuum. There is a history of persistent bottom quartile exam performance, and students in this situation usually begin to select themselves out of the running for highly competitive fields. But once in awhile someone gets completely blindsided.

After the Step 1 failure hits, you still have to sit down with them and find out what they are looking for in a specialty: cognitive vs. procedural, inpatient vs. outpatient, acute vs. chronic, etc. While FM is a common destination, I have seen students match into fields like IM, EM, psych, neurology, and PM&R. IM in particular is a nice gateway to a whole bunch of subspecialties.

In the immediate wake it is still unknown how the student will respond in the coming year. If they are able to successfully shift over to clerkships and show aptitude for clinical practice, that keeps doors open.

I wish someone would have sat down with me and explored what I was looking for in a speciality- I still feel lost and don't know what's open to me. Thank you for doing this with your students, from someone's who's dean left them to flounder on their own

 

[Med Ed]

Easy to be blindsided when your whole curriculum is pass fail

Students should still be getting numeric performance data from their exams. And there are presently more tools to estimate one’s USMLE readiness than at any point in human history.

 

[Goro]

Easy to be blindsided when your whole curriculum is pass fail

Our exams are graded.
Courses are P/F.
One gets out of med school with what one puts into med school

 

[Taddy Mason]

Easy to be blindsided when your whole curriculum is pass fail

My school was P/F but we still got our raw scores, class distribution, mean/median/mode, and tons of other stats so we could individually gauge our performance. Plus, as others have said there are plenty of well validated step 1 calculators that can predict performance based on UWorld and practice test performance.

 

[DarkHorizon]

Totally agree I think the research stuff is crazy. The meta now for matching competitive specialties is to do enough studying to pass comfortably and then grind as many research items as possible, which is totally backward since 99% of us are in school to become clinicians and not researchers. Ik people counter this by saying that a lot of the content on step1 is just garbage (muh acute angle vs 90 deg branching hyphae), but in that case the answer is to improve curricula or just remove the crap content and shorten the degree program, not add meaningless research. Like the fact that all these people are doing research years to match ortho or whatever is pretty rich considering the vast majority of them are going to be community surgeons some day. honestly seems exploitative given the opportunity cost of delaying attendinghood.

Getting rid of step 1 score was a huge blow to students from lower tier/DO schools IMO. High step 1 score opened many doors for these students, as they didn’t have the research/networking opportunities as students from top schools. My $0.02

 

[Starr_key]

I HATE that they made STEP1 P/F.
Reason being that the NBME forms (which are supposed to be the most predictive) are not as predictive in my opinion. I know students like OP who got 70% on the forms and still failed, but students who got high 50s and passed. Tf kinda nonsense is that.

I still think STEP1 failure is still a red flag, but less now that is P/F and the fail rate has gone up, a lot can be forgiven with a good STEP2 score and good clinical grades.

 

[Starr_key]

In my experience most Step 1 failures don't occur in a vacuum. There is a history of persistent bottom quartile exam performance, and students in this situation usually begin to select themselves out of the running for highly competitive fields. But once in awhile someone gets completely blindsided.

After the Step 1 failure hits, you still have to sit down with them and find out what they are looking for in a specialty: cognitive vs. procedural, inpatient vs. outpatient, acute vs. chronic, etc. While FM is a common destination, I have seen students match into fields like IM, EM, psych, neurology, and PM&R. IM in particular is a nice gateway to a whole bunch of subspecialties.

In the immediate wake it is still unknown how the student will respond in the coming year. If they are able to successfully shift over to clerkships and show aptitude for clinical practice, that keeps doors open.

I disagree. I know students who did well on exams and still failed. I know students who were constantly in the bottom or scraped by and they passed.

Usually, the main thing I've seen is students rushing to take STEP1. This is either because they want to use the rest of dedicated to take a preplanned vacation (bad idea) or, more often than not, the schools are not giving them enough time. Some schools will sort of force students to take it by threatening to delay clinicals or whatever, so students are forced to take it by a deadline.

 

[Ho0v-man]

Getting rid of step 1 score was a huge blow to students from lower tier/DO schools IMO. High step 1 score opened many doors for these students, as they didn’t have the research/networking opportunities as students from top schools. My $0.02

And honestly, a low step 1 could be somewhat made up for with a high step 2. Only the most competitive fields were off the table as long as you were geographically flexible.

The real answer is to just make the tests better. Stop forcing students to regurgitate chromosome numbers and gene names that aren’t even real words. Nbme wants to test obscure information in the hopes that people miss these questions so the tests can stay relevant. But medicine is complex enough to test relevant things and still be challenging.

 

[Med Ed]

I disagree. I know students who did well on exams and still failed. I know students who were constantly in the bottom or scraped by and they passed.

Is this your first encounter with the concept of uncertainty?

Usually, the main thing I've seen is students rushing to take STEP1. This is either because they want to use the rest of dedicated to take a preplanned vacation (bad idea) or, more often than not, the schools are not giving them enough time. Some schools will sort of force students to take it by threatening to delay clinicals or whatever, so students are forced to take it by a deadline.

Yes, this is how a curriculum works. Things are placed into different time periods, and if one thing is not finished in the time allotted it impacts the future. How much time do you think students should get for Step 1?

 

[Starr_key]

Is this your first encounter with the concept of uncertainty?


Yes, this is how a curriculum works. Things are placed into different time periods, and if one thing is not finished in the time allotted it impacts the future. How much time do you think students should get for Step 1?

You said "there is a history of persistent bottom quartile performance" and I was refuting that. I know people who were nowhere near the bottom who failed, while others who got flagged for potential failure passed. You seem to have a narrow perspective, so I was just trying to open it 🙂

You do realize that schools have different policies right?? Some schools give their students 8 weeks, others 4 weeks. I also mentioned some students been given enough time but rushing, which is a fault of the student, not the school.

 

[Starr_key]

Getting rid of step 1 score was a huge blow to students from lower tier/DO schools IMO. High step 1 score opened many doors for these students, as they didn’t have the research/networking opportunities as students from top schools. My $0.02

Do you know if it's also been hurting IMGs? I haven't seen much data for them.

 

[Goro]

I HATE that they made STEP1 P/F.
Reason being that the NBME forms (which are supposed to be the most predictive) are not as predictive in my opinion. I know students like OP who got 70% on the forms and still failed, but students who got high 50s and passed. Tf kinda nonsense is that.

I still think STEP1 failure is still a red flag, but less now that is P/F and the fail rate has gone up, a lot can be forgiven with a good STEP2 score and good clinical grades.

Our current thinking is that the fail rate has gone up is due to the lingering effects of COVID, and also a mindset that one doesn’t have to study as hard for the PF Level 1 and Step 1.

That pernicious mindset seems fixed in our weakest students.

 

[Goro]

I disagree. I know students who did well on exams and still failed. I know students who were constantly in the bottom or scraped by and they passed.

Please restrain the sin of solipsism.

Our low-risk students who failed Boards either got sick just before the exam, or had some life event intrude.

 

[Med Ed]

You said "there is a history of persistent bottom quartile performance" and I was refuting that. I know people who were nowhere near the bottom who failed, while others who got flagged for potential failure passed. You seem to have a narrow perspective, so I was just trying to open it 🙂

Ah, I see the issue. When I was writing the post in question, I revised the sentence you quoted and left out an important word: "there is typically a history of persistent bottom quartile performance..." My mistake.

In my experience poring over pre-clerkship performance data and USMLE outcomes, there is a threshold below which the risk of Step 1 failure goes up. Not to 100%, but way up. Those students are in the danger zone and need extra help.

Conversely, above that threshold the risk of Step 1 failure goes down. Not to 0%, but way down.

 

[Starr_key]

Ah, I see the issue. When I was writing the post in question, I revised the sentence you quoted and left out an important word: "there is typically a history of persistent bottom quartile performance..." My mistake.

In my experience poring over pre-clerkship performance data and USMLE outcomes, there is a threshold below which the risk of Step 1 failure goes up. Not to 100%, but way up. Those students are in the danger zone and need extra help.

Conversely, above that threshold the risk of Step 1 failure goes down. Not to 0%, but way down.

That's fair actually. I do see some students who could use extra help beyond just dedicated period, but AFAIK, schools don't give that. Maybe some schools do, and if they do, kudos to them.

But I've seen a lot of those students get punted to shortly before or during dedicated and then they're in trouble. If they just been targeted and given extra help early on, it would have been just fine.

 

[surelycalm]

Easy to be blindsided when your whole curriculum is pass fail

How so? My program is P/F but all of our tests are numerically graded. If you are passing by a few points you know you are just getting by.

 

[Ho0v-man]

How so? My program is P/F but all of our tests are numerically graded. If you are passing by a few points you know you are just getting by.

My statement was in regard to a scenario where you have no idea where you stand relative to your peers, which isn’t the case in all p/f programs apparently.

 

[Nontrad_FL_LGBT]

My statement was in regard to a scenario where you have no idea where you stand relative to your peers, which isn’t the case in all p/f programs apparently.

My program is probably amongst the cushiest P/F and class rank wise in that everything except AIs are P/F and they don't track a class rank metric. They still give us our numerical score out of 100 for every exam and do an exam review session where you see your answers compared to a key about a week after the exam. I can't imagine a program where you'd have literally no idea if you're doing well or not tbh.