Has Step I scoring changed?

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
Status
Not open for further replies.

zanzizic

Full Member
10+ Year Member
Joined
Jul 5, 2010
Messages
123
Reaction score
247
I’m a private practice attending (dermatology) and I’m reading the medical student threads out of curiosity and seeing that quite a few things have changed. I was a med student in the 90s and back then MCAT scoring was a lot different. Your typical state school had an average MCAT of 28-29 or so, and now I see the scoring is in the 500s.

I took Step 1 in 1995 and back then the average if I remember correctly was down in the low 200s. I scored a 229 in 1995 (maybe the third year the Step 1 was administered) which was pretty high. Now I see people routinely throwing around much higher scores than I ever saw when I was in my derm residency and I was having to sort through the copious applications. My chairman had the first year derm residents look through 1000 applications and pick out 40 to interview. We wanted to see scores >230 which was up near 90th percentile I believe.

I was looking at the Wikipedia entry on Step 1 and they say that the average score in 2008 was 221 but by 2016 it was up to 228, so clearly things are changing. I’m just wondering if the scoring has changed or have study aids made it easier than ever for students to access and study the testable material?

Members don't see this ad.
 
  • Wow
Reactions: 3 users
I think the score creep is mainly due to a great increase in quality of study resources, from Anki to Sketchy, along with a greater overall awareness of how to do well on step.
 
  • Like
  • Haha
Reactions: 2 users
1566759980620.png

Its not that the test is easier, its just that the competition is fierce and students realize that geting a good step score is the number 1 focus of the first two years of medical school. This has resulted in students enmass abandoning school curricula and lectures.

Also the current 90th percentile on step 1 is around the 250-255 mark.
 
Last edited:
  • Like
  • Wow
Reactions: 6 users
Members don't see this ad :)
View attachment 277741
Its not that the test is easier, its just that the competition is fierce and students realize that geting a good step score is the number 1 focus of the first two years of medical school. This has resulted in students enmass abandoning school curricula and lectures.
Maybe that is why they are conversing on changing step 1 to pass/fail;
 
  • Like
Reactions: 1 user
Maybe that is why they are conversing on changing step 1 to pass/fail;
They are doing that because first two years of medical school have become an anxiety provoking meat grinder for students wanting to keep all specialty doors open. I still feel guilty every moment of every day that I am not studying and i took step 1 greater than 3 months ago.
The problem is that there isnt a standardized objective way to evaluate students besides this currently. It is incredible the amount of effort students exert towards this goal. ANKI reviews of 500,000 and 10,000 unique questions are not unheard of if you look towards people who score 250+
 
Last edited:
  • Like
Reactions: 4 users
I’m a private practice attending (dermatology) and I’m reading the medical student threads out of curiosity and seeing that quite a few things have changed. I was a med student in the 90s and back then MCAT scoring was a lot different. Your typical state school had an average MCAT of 28-29 or so, and now I see the scoring is in the 500s.

I took Step 1 in 1995 and back then the average if I remember correctly was down in the low 200s. I scored a 229 in 1995 (maybe the third year the Step 1 was administered) which was pretty high. Now I see people routinely throwing around much higher scores than I ever saw when I was in my derm residency and I was having to sort through the copious applications. My chairman had the first year derm residents look through 1000 applications and pick out 40 to interview. We wanted to see scores >230 which was up near 90th percentile I believe.

I was looking at the Wikipedia entry on Step 1 and they say that the average score in 2008 was 221 but by 2016 it was up to 228, so clearly things are changing. I’m just wondering if the scoring has changed or have study aids made it easier than ever for students to access and study the testable material?
There has been a gradual arms race in pre-med and medical student stats since the time you matriculated. DO schools have seen this as well.
 
  • Like
Reactions: 1 users
Very interesting. Yes, there were no Step 1 specific materials when I was taking the exam. First aid for the Boards came out in time for Step II.

We didn’t know how important the scores were when we took it. I remember being told the scores didn’t really matter. We were studying by buying Lippincott review books of the different subjects and reviewing them.
 
  • Like
Reactions: 1 user
They are doing that because first two years of medical school have become an anxiety provoking meat grinder for students wanting to keep all specialty doors open. I still feel guilty every moment of every day that I am not studying and i took step 1 greater than 3 months ago.
The problem is that there isnt a standardized objective way to evaluate students besides this currently. It is incredible the amount of effort students exert towards this goal to. ANKI reviews of 500,000 and 10,000 unique questions are not unheard of if you look towards people who score 250+
True, medical students spending most of their time/ efforts to pass this standardized test while I think students should concentrate more on learning common diseases. you mean you haven't started studying for step 2 ck?. I'm finishing uworld step 2 ck soon( my first pass), but I feel I need to study something else besides Uworld as the vagueness of the questions is becoming greater.So, maybe I will add MTB, but I'm not sure. What will be your resources?
 
True, medical students spending most of their time/ efforts to pass this standardized test while I think students should concentrate more on learning common diseases. you mean you haven't started studying for step 2 ck?. I'm finishing uworld step 2 ck soon( my first pass), but I feel I need to study something else besides Uworld as the vagueness of the questions is becoming greater.So, maybe I will add MTB, but I'm not sure. What will be your resources?
so far, dorian deck for anki, OME (quick watch), Emma holiday videos(quick watch), Uworld and Amboss, come join us in the step 2 2020 thread for more breakdown of people's strategies. Since i have a good step 1 score I will be delaying taking the thing as far as i can so I dont have to disclose it to programs.
 
  • Like
Reactions: 2 users
so far, dorian deck for anki, OME (quick watch), Emma holiday videos(quick watch), Uworld and Amboss, come join us in the step 2 2020 thread for more breakdown of people's strategies. Since i have a good step 1 score I will be delaying taking the thing as far as i can so I dont have to disclose it to programs.
Thank you, I just found it.
 
True, medical students spending most of their time/ efforts to pass this standardized test while I think students should concentrate more on learning common diseases. you mean you haven't started studying for step 2 ck?. I'm finishing uworld step 2 ck soon( my first pass), but I feel I need to study something else besides Uworld as the vagueness of the questions is becoming greater.So, maybe I will add MTB, but I'm not sure. What will be your resources?

OMG.
 
  • Like
Reactions: 1 users
I think med school curriculum has changed as well with more of an effort of getting good scores on your standardized tests (which is kinda unfortunate imo)
 
I’m a private practice attending (dermatology) and I’m reading the medical student threads out of curiosity and seeing that quite a few things have changed. I was a med student in the 90s and back then MCAT scoring was a lot different. Your typical state school had an average MCAT of 28-29 or so, and now I see the scoring is in the 500s.

I took Step 1 in 1995 and back then the average if I remember correctly was down in the low 200s. I scored a 229 in 1995 (maybe the third year the Step 1 was administered) which was pretty high. Now I see people routinely throwing around much higher scores than I ever saw when I was in my derm residency and I was having to sort through the copious applications. My chairman had the first year derm residents look through 1000 applications and pick out 40 to interview. We wanted to see scores >230 which was up near 90th percentile I believe.

I was looking at the Wikipedia entry on Step 1 and they say that the average score in 2008 was 221 but by 2016 it was up to 228, so clearly things are changing. I’m just wondering if the scoring has changed or have study aids made it easier than ever for students to access and study the testable material?
229 in Derm? Can I time travel. Sheesh old people had it so much easier
 
  • Like
Reactions: 1 users
Members don't see this ad :)
Not sure if you understood my post. 229 was around 90% percentile in 1995

Not sure old people had anything easier back in the day...especially since there are about twice as many derm residency spots now as back then
 
  • Like
Reactions: 5 users
Not sure if you understood my post. 229 was around 90% percentile in 1995

Not sure old people had anything easier back in the day...especially since there are about twice as many derm residency spots now as back then

This isn't well understood (maybe because the MCAT isn't scored like this), but Step 1 is not a scaled score - it's a score based on how many questions you get right and wrong. This number may have changed somewhat over the years, but I doubt it has changed a lot, or the numbers wouldn't be creeping up every year. What has changed is related to importance of step 1 by residency directors, and the arms race between questions getting more difficult and review resources getting better.
 
  • Like
Reactions: 1 users
The Step 1 is not a norm-referenced test, it's a criterion-referenced test. The fact that a 200 was median in the early 1990s and is now roughly the same as the failing threshold? Yeah, that literally does mean that half the practicing physicians who went to med school ~25 years ago would today be deemed too dangerously deficient in basic science knowledge to be licensed.

You can read some good stuff on the topic on this dude's blog
 
  • Like
Reactions: 6 users
The Step 1 is not a norm-referenced test, it's a criterion-referenced test. The fact that a 200 was median in the early 1990s and is now roughly the same as the failing threshold? Yeah, that literally does mean that half the practicing physicians who went to med school ~25 years ago would today be deemed too dangerously deficient in basic science knowledge to be licensed.

You can read some good stuff on the topic on this dude's blog

I think there’s a more well-known body of knowledge that is has solidified and is considered to be the basic science that med students need to know. As the test has become more important, students study the basic sciences with more test prep material. The test has become a self-fulfilling prophecy. Would those students of yesterday be deemed deficient if they took the test today? Well, no. If they took it today they’d be a part of the USMLE “arms race” that has taken place.

During the years 1 and 2, my classmates and I studied lecture notes and read textbooks. When my class took Step I in the spring of 1995, no test prep for the Boards was commercially available. There were no test banks/sample questions. Some of my classmates were using their med school lecture notes to review the basic sciences to prepare for Step 1.

My opinion is that med students of that era knew as much basic science as students do today. They just weren’t versed in the format of Step I and weren’t as proficient in ‘high yield’ Board topics.
 
  • Like
Reactions: 1 user
I think there’s a more well-known body of knowledge that is has solidified and is considered to be the basic science that med students need to know. As the test has become more important, students study the basic sciences with more test prep material. The test has become a self-fulfilling prophecy. Would those students of yesterday be deemed deficient if they took the test today? Well, no. If they took it today they’d be a part of the USMLE “arms race” that has taken place.

During the years 1 and 2, my classmates and I studied lecture notes and read textbooks. When my class took Step I in the spring of 1995, no test prep for the Boards was commercially available. There were no test banks/sample questions. Some of my classmates were using their med school lecture notes to review the basic sciences to prepare for Step 1.

My opinion is that med students of that era knew as much basic science as students do today. They just weren’t versed in the format of Step I and weren’t as proficient in ‘high yield’ Board topics.
Well, I think your generation knew less, not because you put in less effort, but because you didn't have the toolkit we do now. Someone that spends 10 hours slamming First Aid into their long-term memory with Anki is going to walk away with more retained knowledge than someone could have squeezed out of 10 hours in 1990.

I agree the work ethic of students back then was probably just as impressive
 
  • Like
Reactions: 7 users
Well, I think your generation knew less

I was like yeah, not even close. Just compare FA even from the mid 2000s to today. Forget the 90s. Here's a pic for reference.

Screenshot_20190827-214527.png
 
I sincerely doubt I could have hit high 220s on just lecture slides alone. OP’s probably a genius.
 
  • Like
Reactions: 8 users
Not sure if you understood my post. 229 was around 90% percentile in 1995

Not sure old people had anything easier back in the day...especially since there are about twice as many derm residency spots now as back then
Just for reference this paper has the scores from 1995. National mean was 204 with std of 20. So with a roughly normal distribution, 229 would be 90th percentile.

 
Well, I think your generation knew less, not because you put in less effort, but because you didn't have the toolkit we do now. Someone that spends 10 hours slamming First Aid into their long-term memory with Anki is going to walk away with more retained knowledge than someone could have squeezed out of 10 hours in 1990.

I agree the work ethic of students back then was probably just as impressive

Disagree. I think there’s a core of high yield Board topics that we didn’t know as well as current students, but considering the breadth of relevant basic science I think we worked as hard and learned just as much as current students.

Consider this (imperfect) analogy. Let’s say there’s a really important standardized test for literature grad students over the novel Les Miserables by Victor Hugo. At 1400 pages, it’s a big book, and the students have six weeks to study for it. In the first few years of the administration of the Les Mis test, students would study by carefully reading the novel and read other commentary books about the novel. They spend an average of 6hrs/day doing this.

After a few years, it becomes apparent that certain kinds of questions and facts about the novel are considered more important than others as they are asked frequently on the test. Soon Les Mis test prep books proliferate and students spend less time reading the novel and just reviewing high yield facts and concepts from the book. They abandon reading commentaries on the novel altogether. Test prep books get thicker over time but students still spend an average of 6hrs/day on studying this high yield material.

Over time, scores rise on the Les Mis test. Does that mean the students actually know more about the novel, or do they actually just know more about the test and the type of questions it contains and the type of facts it has come to test?

Does the fact that they do better on a test of narrower (high yield) topics mean they actually know more than the students who actually spent time reading the novel?

Consider an example from basic science: autoimmune neuromuscular diseases. The classic one is myasthenia gravis. Board prep books will assure that you know the relevant facts because it’s likely you’ll see a question on MG. But others exist: Lambert-Eaton myasthenic syndrome, Isaacs syndrome, etc. If you’re a student in the early 90s, how do you know they won’t ask about the others? You don’t, so you review them all.

So who actually knows more basic science...the 2019 student who definitely won’t miss the myasthenia gravis question because he knows Step I has asked about that disease before or the 1992 student who spent time studying all of these neuromuscular conditions because he didn’t know which topics were more important than any other?
 
  • Like
Reactions: 2 users
Disagree. I think there’s a core of high yield Board topics that we didn’t know as well as current students, but considering the breadth of relevant basic science I think we worked as hard and learned just as much as current students.

Consider this (imperfect) analogy. Let’s say there’s a really important standardized test for literature grad students over the novel Les Miserables by Victor Hugo. At 1400 pages, it’s a big book, and the students have six weeks to study for it. In the first few years of the administration of the Les Mis test, students would study by carefully reading the novel and read other commentary books about the novel. They spend an average of 6hrs/day doing this.

After a few years, it becomes apparent that certain kinds of questions and facts about the novel are considered more important than others as they are asked frequently on the test. Soon Les Mis test prep books proliferate and students spend less time reading the novel and just reviewing high yield facts and concepts from the book. They abandon reading commentaries on the novel altogether. Test prep books get thicker over time but students still spend an average of 6hrs/day on studying this high yield material.

Over time, scores rise on the Les Mis test. Does that mean the students actually know more about the novel, or do they actually just know more about the test and the type of questions it contains and the type of facts it has come to test?

Does the fact that they do better on a test of narrower (high yield) topics mean they actually know more than the students who actually spent time reading the novel?

Consider an example from basic science: autoimmune neuromuscular diseases. The classic one is myasthenia gravis. Board prep books will assure that you know the relevant facts because it’s likely you’ll see a question on MG. But others exist: Lambert-Eaton myasthenic syndrome, Isaacs syndrome, etc. If you’re a student in the early 90s, how do you know they won’t ask about the others? You don’t, so you review them all.

So who actually knows more basic science...the 2019 student who definitely won’t miss the myasthenia gravis question because he knows Step I has asked about that disease before or the 1992 student who spent time studying all of these neuromuscular conditions because he didn’t know which topics were more important than any other?
It's about the efficiency. Trust me I'm familiar with what it's like to learn from lectures and books, we do still have those tools from the 90s. But the quality and efficiency is simply not comparable. The same amount of material that would be covered in a 1 hour lecture from a cardiology professor is covered better and faster in a 20 minute Boards&Beyond module that students will then watch at 1.5x to increase their efficiency even further. Then, instead of revisiting the slide deck to reinforce, they can use a spaced-repetition Anki tool that only repeatedly shows them info if they fail to recall it correctly, so that they can have the same material learned faster and more permanently.

To apply this to your analogy, the best way to prepare for a top 10% score back in the day would be carefully reading Les Mis for 6-8 weeks and perhaps listening to a local professor walk you through the toughest parts to master.

In the modern climate, the practice isn't just to redirect the 6-8 weeks onto the most important chapters. It is to spend 1-2 years memorizing the entire thing verbatim, or as close as possible, while also listening to the highest regarded professors' walkthroughs at higher speed and doing many thousands of practice questions.

It's simply not true that medicine has been boiled down to the Step topics for us, and a bunch of other things cut out. We're responsible for all of it. On a modern test a question asking to distinguish Lambert-Eaton from Myasthenia Gravis would be a gimme joke of a question. An example of one I actually had on mine was a vignette hinting at a far more rare syndrome (think anti-cerebellar, that kind of thing) and then asking me what the second most common type of neoplasm causing it is. That is the kind of thing they have to use now to differentiate the top ~10%, which I where I landed too. I had questions asking me the specific type of CYP enzyme that metabolizes a specific opiate drug (e.g. is codeine metabolized by 3A4 or 2D6). That is the degree of minutiae that is fair game for the modern folks who want to be competitive for derm.

To give a much more straightforward thought experiment, since we scored similar percentiles, pretend we're given this challenge. We're told to go and brush up on autoimmune diseases for a weekend and then we're going to have an oral exam with a panel of rheum/immuno docs. You get to use physical textbooks, and use slide decks from any med school curriculum you want. I get to use the modern internet, instantly searchable PDF versions of those books, the Anki tool for making sure I've got what I want memorized down cold, and can stream concise review modules on autoimmune conditions at 2x speed. We both study for 20 hours.

There is no chance you'd come out as prepared. If we are both equally hard working and focused and smart, I'm just going to cover more ground.
 
Last edited:
  • Like
Reactions: 6 users
Disagree. I think there’s a core of high yield Board topics that we didn’t know as well as current students, but considering the breadth of relevant basic science I think we worked as hard and learned just as much as current students.

Consider this (imperfect) analogy. Let’s say there’s a really important standardized test for literature grad students over the novel Les Miserables by Victor Hugo. At 1400 pages, it’s a big book, and the students have six weeks to study for it. In the first few years of the administration of the Les Mis test, students would study by carefully reading the novel and read other commentary books about the novel. They spend an average of 6hrs/day doing this.

After a few years, it becomes apparent that certain kinds of questions and facts about the novel are considered more important than others as they are asked frequently on the test. Soon Les Mis test prep books proliferate and students spend less time reading the novel and just reviewing high yield facts and concepts from the book. They abandon reading commentaries on the novel altogether. Test prep books get thicker over time but students still spend an average of 6hrs/day on studying this high yield material.

Over time, scores rise on the Les Mis test. Does that mean the students actually know more about the novel, or do they actually just know more about the test and the type of questions it contains and the type of facts it has come to test?

Does the fact that they do better on a test of narrower (high yield) topics mean they actually know more than the students who actually spent time reading the novel?

Consider an example from basic science: autoimmune neuromuscular diseases. The classic one is myasthenia gravis. Board prep books will assure that you know the relevant facts because it’s likely you’ll see a question on MG. But others exist: Lambert-Eaton myasthenic syndrome, Isaacs syndrome, etc. If you’re a student in the early 90s, how do you know they won’t ask about the others? You don’t, so you review them all.

So who actually knows more basic science...the 2019 student who definitely won’t miss the myasthenia gravis question because he knows Step I has asked about that disease before or the 1992 student who spent time studying all of these neuromuscular conditions because he didn’t know which topics were more important than any other?
1567028430134.png

Medical literature has increased exponentially over the past few decades, and so has the cannon considered basic science for step 1. I am not discounting the fact that previous generations worked hard. In your example it is like now they discovered a new volumes of les miserables that were unpublished and is now widely available in circulation.
1567028759064.png

New Drugs including HIV and anticoag options and lets not forget the monoclonal antibodies with endless names and receptors.

There is definately an aspect of medical students focusing on board exams, but the other aspect is the rampant increase in medical knowledge has made it more difficult retain everything.

Now they are not going to ask you about mysthenia gravis, they are going to ask which receptor is impacted, what is the name of the channel, what is the mechanism, what are the drug options, and what is their moa.
 
  • Like
Reactions: 3 users
View attachment 278081
Medical literature has increased exponentially over the past few decades, and so has the cannon considered basic science for step 1. I am not discounting the fact that previous generations worked hard. In your example it is like now they discovered a new volumes of les miserables that were unpublished and is now widely available in circulation.
View attachment 278082
New Drugs including HIV and anticoag options and lets not forget the monoclonal antibodies with endless names and receptors.

There is definately an aspect of medical students focusing on board exams, but the other aspect is the rampant increase in medical knowledge has made it more difficult retain everything.

Now they are not going to ask you about mysthenia gravis, they are going to ask which receptor is impacted, what is the name of the channel, what is the mechanism, what are the drug options, and what is their moa.
Oh yeah that reminds me of another pharma question where they asked about a drug not in SketchyPharm and wanted to know what ion channel type it affects...god this test HAS to stop reporting numerical scores.
 
  • Like
Reactions: 1 user
Oh yeah that reminds me of another pharma question where they asked about a drug not in SketchyPharm and wanted to know what ion channel type it affects...god this test HAS to stop reporting numerical scores.

*insert usual SDN argument spanning several pages why Step scores are necessary because matching*

/runs away
 
  • Like
Reactions: 1 users
View attachment 278081
Medical literature has increased exponentially over the past few decades, and so has the cannon considered basic science for step 1. I am not discounting the fact that previous generations worked hard. In your example it is like now they discovered a new volumes of les miserables that were unpublished and is now widely available in circulation.
View attachment 278082
New Drugs including HIV and anticoag options and lets not forget the monoclonal antibodies with endless names and receptors.

There is definately an aspect of medical students focusing on board exams, but the other aspect is the rampant increase in medical knowledge has made it more difficult retain everything.

Now they are not going to ask you about mysthenia gravis, they are going to ask which receptor is impacted, what is the name of the channel, what is the mechanism, what are the drug options, and what is their moa.

It’s always been this way. As knowledge advances, there are more things to learn. But we are only human with a finite ability to integrate knowledge. As the corpus of medical knowledge expands, less important things fall from the medical curriculum.

When I was in gross anatomy the head anatomy professor had written our dissector decades before. It was massive, and we covered a minority of the minutiae contained within. Previous decades of medical students did all of it and crammed their heads with extensive anatomical knowledge. Gross anatomy had been a year long course in previous decades and was only four months by the time I took it in ‘93. A quick look online shows some schools now cover it in 7 weeks? Don’t tell me you’re responsible for the anatomical knowledge that medical students of the 1960s had command of. A brief look through Grays anatomy shows that.

As new drugs appear, old obsolete ones fall off. As medical imaging gets better, our learning of arcane physical examination techniques diminish. Cardiologists of today read echocardiograms but can’t do nearly what their forebears could with a stethoscope.

You could argue that some of this was clinically useless. You could argue the same about the names of enzymes you’re currently memorizing.

Yes, learning in medical school has been likened to drinking from a firehose. This is nothing new.
 
  • Like
Reactions: 2 users
I think this conversation is kind of silly. What kind of pissing contest are we in now were students today want to make sure students from the 90's know that we are working harder, smarter, more efficient, blah blah. I'm of the opinion that yeah, sure medical knowledge has increased thus the testing pool of potential questions increased too. But without the resources we have today, students before somehow made it work. And for that, I applaud them. I think students in our generation shouldn't be trying to dismantle that notion. But that's just my $0.02
 
  • Like
Reactions: 3 users
I think this conversation is kind of silly. What kind of pissing contest are we in now were students today want to make sure students from the 90's know that we are working harder, smarter, more efficient, blah blah. I'm of the opinion that yeah, sure medical knowledge has increased thus the testing pool of potential questions increased too. But without the resources we have today, students before somehow made it work. And for that, I applaud them. I think students in our generation shouldn't be trying to dismantle that notion. But that's just my $0.02
Hey man, they wanted to know why the average Step score today is higher than their 90th percentile score from decades ago. I'm just explaining why. I don't for a second think that knowing more esoteric factoids about 1 in a million paraneoplastic syndromes will make us any better on the wards.

*insert usual SDN argument spanning several pages why Step scores are necessary because matching*

/runs away
I can just link to this page instead now. The whole discussion I had last time with AnatomyGrey is pretty much identical to his take on "The American Dreamer"
 
  • Like
Reactions: 4 users
That guy has trash arguments. I wont get into it in this thread but he is far from compelling
Some stronger than others for sure, but I can't say I've heard a good defense of Step 1 in it's current format and usage that is any more convincing. I sympathize most with people at DO or international schools that (correctly) feel it's the last shot at meritocracy opening doors in the match, but I don't think we can hold the entire system hostage for outliers.
 
  • Like
Reactions: 2 users
Some stronger than others for sure, but I can't say I've heard a good defense of Step 1 in it's current format and usage that is any more convincing. I sympathize most with people at DO or international schools that (correctly) feel it's the last shot at meritocracy opening doors in the match, but I don't think we can hold the entire system hostage for outliers.
Eventually there has to be stratification based on merit. There just has to be. I don’t benefit from it cuz I’m average but you can’t have a system with entirely pass fail anymore.

Rework the test sure, but all it would do as P/F is shift importance to a different board exam
 
  • Like
Reactions: 1 users
I don't for a second think that knowing more esoteric factoids about 1 in a million paraneoplastic syndromes will make us any better on the wards.

This is correct. The vast majority of what makes you a good doctor is not what you learn in medical school. Most of it is what was already inside of you before you even got there.

Man, that is so damn corny. But it’s the truth.
 
  • Like
Reactions: 4 users
Some stronger than others for sure, but I can't say I've heard a good defense of Step 1 in it's current format and usage that is any more convincing. I sympathize most with people at DO or international schools that (correctly) feel it's the last shot at meritocracy opening doors in the match, but I don't think we can hold the entire system hostage for outliers.

Step 1’s best use is as an objective equalizer among residency applicants.

As I said before I used to review applications for a dermatology residency. How the heck are you going to compare a guy who went to an Ivy League medical school with all pass/fail grades versus a guy from a lower rung state school with an A average?

Plus it tells their future residency program director how likely they are to fail their specialty board examination. Residents who fail their board examinations make their program look really bad.
 
  • Like
Reactions: 1 user
Eventually there has to be stratification based on merit. There just has to be. I don’t benefit from it cuz I’m average but you can’t have a system with entirely pass fail anymore.

Rework the test sure, but all it would do as P/F is shift importance to a different board exam
Hey it's an improvement just to use Step2 CK instead. At least then they're asking questions about management of a disease like what drug combination to give, instead of asking questions about which specific CYP enzymes chews those drugs up.

I think there's a middle ground we could use, like tertiles. That way nobody has to worry about gains beyond 240+ and doesn't have to worship UFAPS until reaching dedicated. Also forces residency directors in surgical subspecialties to stop abusing it to the current degree. It's true that clinical grades aren't very objective but they're a step up (pun intended) from the basic science trivia contest.
 
  • Like
Reactions: 1 users
This is correct. The vast majority of what makes you a good doctor is not what you learn in medical school. Most of it is what was already inside of you before you even got there.

Man, that is so damn corny. But it’s the truth.
Step 1’s best use is as an objective equalizer among residency applicants.

As I said before I used to review applications for a dermatology residency. How the heck are you going to compare a guy who went to an Ivy League medical school with all pass/fail grades versus a guy from a lower rung state school with an A average?

Plus it tells their future residency program director how likely they are to fail their specialty board examination. Residents who fail their board examinations make their program look really bad.
It's not useful for the purpose of board pass rates at the current extremes. You simply don't get enough gains from a 250 over a 230 for that to be the #1 filter mechanism for derm residency:
img_0115-1.jpg

This test was meant to be a Pass/Fail licensing exam like the bar exam. The authors (NBME) explicitly state that it is NOT supposed to be used as it currently is.
And that's just for peds. For other cases the association is even less relevant. For IM, "99% of residents who had USMLE Step 1 scores >211 passed the internal medicine boards."

See more here: The Mythology of USMLE Step 1 Scores and Board Certification

I agree it's ridiculous how derm residencies now get hundreds of applicants to fill a handful of spots. I can sympathize with the desire to have some kind of meritocratic comparison tool to sort people and make it easier to determine interview invites.

But ask yourself this - if you were in modern medical school and scored in the 220s again, would this system make sense to you? Should you be denied access to dermatology because of Step 1? Would someone else who scored higher really have made a better derm resident or practicing doc than yourself?
 
Last edited:
  • Like
Reactions: 1 user
Hey it's an improvement just to use Step2 CK instead. At least then they're asking questions about management of a disease like what drug combination to give, instead of asking questions about which specific CYP enzymes chews those drugs up.

I think there's a middle ground we could use, like tertiles. That way nobody has to worry about gains beyond 240+ and doesn't have to worship UFAPS until reaching dedicated. Also forces residency directors in surgical subspecialties to stop abusing it to the current degree. It's true that clinical grades aren't very objective but they're a step up (pun intended) from the basic science trivia contest.

How useful was Step 1 knowledge for the wards? Was it completely useless or did it help you nail down some pimp questions and/or ace the shelf?
 
It's not useful for the purpose of board pass rates at the current extremes. You simply don't get enough gains from a 250 over a 230 for that to be the #1 filter mechanism for derm residency:
img_0115-1.jpg

This test was meant to be a Pass/Fail licensing exam like the bar exam. The authors (NBME) explicitly state that it is NOT supposed to be used as it currently is.
See more here: The Mythology of USMLE Step 1 Scores and Board Certification

I agree it's ridiculous how derm residencies now get hundreds of applicants to fill a handful of spots. I can sympathize with the desire to have some kind of meritocratic comparison tool to sort people and make it easier to determine interview invites.

But ask yourself this - if you were in modern medical school and scored in the 220s again, would this system make sense to you? Should you be denied access to dermatology because of Step 1? Would someone else who scored higher really have made a better derm resident or practicing doc than yourself?
I scored a 230 ish and I absolutely should be prevented from doing derm, neurosurg, etc. it’s not that anyone will make a better resident, they’re simply smarter and have better applications. That’s how it does.
 
  • Like
  • Sad
Reactions: 4 users
How useful was Step 1 knowledge for the wards? Was it completely useless or did it help you nail down some pimp questions and/or ace the shelf?
Largely useless. No attending is ever going to pimp you on CYP enzymes or whether a hormone binds Gs or Gq receptors. You're never going to draw the metabolism pathways again.

You're much more likely to get pimped on Step 2 CK topics
 
I scored a 230 ish and I absolutely should be prevented from doing derm, neurosurg, etc. it’s not that anyone will make a better resident, they’re simply smarter and have better applications. That’s how it does.
Are they better applicants, though? Suppose you honor all your clerkships and they don't, they only honor half. Does screening you out and giving the interview to them really make sense?

And we've got to dispel this idea that students scoring higher means they are smarter or harder working. At the risk of sounding like a narcissist and dingus, in recent years Hopkins medicine had an average step 1 of 235, barely above average. Compare to Mizzou that was at 240 that year. Smarter students? Harder working? Or just prioritized step 1 differently?
 
I mean my only objection to getting rid of Step 1 scores is because of the even greater emphasis placed on school name for matching. And that'll get even worse for the competitive specialties and top tier IM programs (needed for strong fellowships).

Step 2 CK seems pretty inflated though. Wasn't the national median like 240 or something?
 
  • Like
Reactions: 1 users
How useful was Step 1 knowledge for the wards? Was it completely useless or did it help you nail down some pimp questions and/or ace the shelf?
Absolutely and utterly useless. No one is going to ask you the p glycoprotein entry mechanism associated with parvo. Most residents pimping you dont even remember the trivia let alone the attending.I get pimping questions wrong all the time because they are asked abruptly and I am in that moment practicing my presentation in my head or still processing the management .
Also people dont remember this stuff. I was trying to talk to a senior resident about a patient with a patent ductus and how it may result in increased likelihood of infection and chronic lung disease. The resident looked me dead in the eye like i was a stupid person and said no pda does not cause those.
Most of the time they are simple questions, or mechanisms that are very basic.

Step minutiae , making you look stupid infront of people everyday.
 
  • Like
  • Haha
Reactions: 3 users
I mean my only objection to getting rid of Step 1 scores is because of the even greater emphasis placed on school name for matching. And that'll get even worse for the competitive specialties and top tier IM programs (needed for strong fellowships).

Step 2 CK seems pretty inflated though. Wasn't the national median like 240 or something?
I think that fear is generally overhyped. Look up some faculty at major research centers, a lot of them went to "top 20 NIH" medical schools but a lot of them went to their state schools, too. There was no step 1 screening or ranking in their day.
 
I think that fear is generally overhyped. Look up some faculty at major research centers, a lot of them went to "top 20 NIH" medical schools but a lot of them went to their state schools, too. There was no step 1 screening or ranking in their day.
IDK your from at top tier school, so probably not the appropriate person to be commenting on that
 
  • Like
Reactions: 1 users
Absolutely and utterly useless. No one is going to ask you the p glycoprotein entry mechanism associated with parvo. Most residents pimping you dont even remember the trivia let alone the attending.I get pimping questions wrong all the time because they are asked abruptly and I am in that moment practicing my presentation in my head or still processing the management .
Also people dont remember this stuff. I was trying to talk to a senior resident about a patient with a patent ductus and how it may result in increased likelihood of infection and chronic lung disease. The resident looked me dead in the eye like i was a stupid person and said no pda does not cause those.
Most of the time they are simple questions, or mechanisms that are very basic.

Step minutiae , making you look stupid infront of people everyday.
Not all of it. Some of the Boards and Beyond topics contain some highly clinically relevant information.
 
  • Like
Reactions: 1 user
IDK your from at top tier school, so probably not the appropriate person to be commenting on that
On the flip side, I scored in the 250s and a switch to Pass/Fail or tertiles would take away my single biggest advantage in the match. I still think it needs to happen. I don't think I can be dismissed for bias when I'm arguing against my own self interest.
 
On the flip side, I scored in the 250s and a switch to Pass/Fail or tertiles would take away my single biggest advantage in the match. I still think it needs to happen. I don't think I can be dismissed for bias when I'm arguing against my own self interest.
You probably mean well but, the difference you'd still come out with a positive advantage regardless. It's like listening to wall street about who caused the financial recession in 2008
 
  • Like
Reactions: 1 users
Not all of it. Some of the Boards and Beyond topics contain some highly clinically relevant information.
There's definitely some clinically useful stuff tested on Step 1, but the preclinical curriculums at most med schools and/or Step 2 CK are generally a lot more clinically relevant. "Teaching to the step" usually means teaching a bunch of esoteric basic science details that nobody really needs to know in a hospital.
 
Status
Not open for further replies.
Top