Step 1 P/F: Decision

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
No, dude, no. There's a reason they are entirely separated Qbanks and First Aid books and why everyone watches OnlineMedEd instead of Boards and Beyond to review. They are different. Not 20% different. Different.

Anecdotal, but I averaged 82% on medicine on Uworld a couple months after step 1 with no outside reading or studying outside of being in the hospital every day.

ObGyn, though - that was a different monster.
 
I don't even know what to say to this. They're two separate exams with two separate sets of materials and two fundamentally different areas assessed as described by their authors. I might as well argue with you about whether the neuro and OBGYN shelves tested 80% of the same stuff because they're both medical topics.

Okay man, that’s why the NBME has ONE content outline for all three Steps. You can take a look here and see the overlap


Just took a Step 2CK practice test now and here were the first 20 topics: fluoroquinolones to treat Legionella, Diagnose PE, calcium chelation transfusion, contact dermatitis, parvovirus infection in adults, diagnose OA, pathophys SLE thrombocytopenia, protein c deficiency, diagnose NPH, diagnose Syphilis, acute graft rejection, central retinal vein occlusion, work-up DVT, etc.

17/18 are fact that can be answered straight from First said. If you think Step 2 CK and Step 1 test completely different concepts you’re not paying close enough attention.

EDIT: The R^2 between Step 1 and Step 2 is 0.51, which is greater than most practice tests and considerably greater than all shelves except for the IM shelf. Take a hard look at the data

hxxps://www.reddit.com/r/Step2/comments/ezdgp9/step2_ck_2019_survey_results/
 
Last edited:
Okay man, that’s why the NBME has ONE content outline for all three Steps. You can take a look here and see the overlap


Just took a Step 2CK practice test now and here were the first 20 topics: fluoroquinolones to treat Legionella, Diagnose PE, calcium chelation transfusion, contact dermatitis, parvovirus infection in adults, diagnose OA, pathophys SLE thrombocytopenia, protein c deficiency, diagnose NPH, diagnose Syphilis, acute graft rejection, central retinal vein occlusion, work-up DVT, etc.

17/18 are fact that can be answered straight from First said. If you think Step 2 CK and Step 1 test completely different concepts you’re not paying close enough attention.

EDIT: The R^2 between Step 1 and Step 2 is 0.51, which is greater than most practice tests and considerably greater than all shelves except for the IM shelf. Take a hard look at the data

hxxps://www.reddit.com/r/Step2/comments/ezdgp9/step2_ck_2019_survey_results/
They correlate well because they're similarly constructed tests that are studied for with similar methods. I guarantee the correlation between shelves for an individual is high, but that's not because of the content being similar in OBGYN and Neuro shelves. Even when topic is the same (e.g. pick any from that list) the step 1 questions emphasize pathophys causing disease to arise while step 2 asks for what to order/test/treat with next.

Did you even read the opening blurb for that document? It says each Step tests is built around different parts of the global outline. They have individual pages for each Step on the USMLE site that detail how they differ. For christ's sake it's self evident just in the fact that they break it into different exams to be taken at different points in the pathway.

I wish you good luck in using First Aid for the USMLE step 1 as your primary resource for Step 2 CK.
 
Anecdotal, but I averaged 82% on medicine on Uworld a couple months after step 1 with no outside reading or studying outside of being in the hospital every day.

ObGyn, though - that was a different monster.
I don't think its impossible for someone to enter the wards ready to do well on both tests, but that'd be from having picked up a lot outside of UFAPS (like from school curriculum). I do think it's impossible for someone to believe that Uworld banks and First Aid books and online lecture packages and the USMLEs themselves are spuriously divided with an 80% overlap.
 
Asked elsewhere but the conversation didn't go anywhere...

Anyone know if there have ever been attempts to employ standardized testing that provides meaningful insight for residency match? Thinking something that assesses for resilience, checks how one's values and work ethic actually align with a given specialty, and maybe includes a personality inventory. Those things tend not to change with repeat testing.

If step 1 is intended as a check of minimum knowledge for licensure (and NOT a means of stratifying applicants or determining specialty fit), then it stands to reason there should be another type of assessment to evaluate those qualities. Thoughts?
 
Asked elsewhere but the conversation didn't go anywhere...

Anyone know if there have ever been attempts to employ standardized testing that provides meaningful insight for residency match? Thinking something that assesses for resilience, checks how one's values and work ethic actually align with a given specialty, and maybe includes a personality inventory. Those things tend not to change with repeat testing.

If step 1 is intended as a check of minimum knowledge for licensure (and NOT a means of stratifying applicants or determining specialty fit), then it stands to reason there should be another type of assessment to evaluate those qualities. Thoughts?

Clinical grades, 4th/aways/subi year grades?
 
Clinical grades, 4th/aways/subi year grades?
That's a great way to get a look at how an applicant works, but school grading is prone to wild inconsistency and the expectations are all over the place. I'm sure folks here can attest - there are preceptors who are brutal and others who give everyone a trophy and a pat on the back for showing up. Add to that, the push to move toward P/F for everything, and objectivity is out the window.

When I was in the Marine Corps, I was evaluated using a process/tool called a fitness report. Check out an example here: FITREP. It's a beast of a document and highly structured, but it does allow for narrative information. There are three crucial elements:
  • Descriptive metrics with ordinal data points
  • Consideration of whether a Marine's job was above/below pay grade (assigned to a difficult job or an easy job)
  • Consideration of how the supervisor grades everyone else (how you compare to others with this supervisor)
While something like this implemented across all clinical rotations for all schools could be helpful, it's a lot of work on the preceptors. Even a trimmed down version requires careful consideration because, at least in the Marine Corps, if you give everyone high marks then it's understood your evaluations are worthless. This data is available for comparison in the FITREP system and you get to see your report when it's done.

This is basically what EM does with their SLOE and they consider it to be really effective. That said, students still need to get accepted to then schedule & pay for away rotations, and programs don't always complete SLOEs in a timely manner. While programs only request 1-2 SLOEs for residency application, it's not uncommon for students to set up 4+ EM away rotations. Plus, the SLOE is blind and students apply broadly. Those issues are exacerbated if the process is blown up to include all specialties.
 
Personality testing is fun, because it's low stakes. Once you make it matter, everyone will want to know what the "right personality" is and how you have to answer the questions to get it. It won't be helpful.

The linked document is interesting. We use anchored likert scale evaluations like that all time time. On a 5 point likert, everyone gets 4-5 no matter what we do. So also, not terribly useful.

The value of SLOE's is that they (theoretically) force the evaluator to compare this student to all of the others. Whether that's good (peer based criterion) vs bad (objectively based criterion) is up for debate.
 
Asked elsewhere but the conversation didn't go anywhere...

Anyone know if there have ever been attempts to employ standardized testing that provides meaningful insight for residency match? Thinking something that assesses for resilience, checks how one's values and work ethic actually align with a given specialty, and maybe includes a personality inventory. Those things tend not to change with repeat testing.

If step 1 is intended as a check of minimum knowledge for licensure (and NOT a means of stratifying applicants or determining specialty fit), then it stands to reason there should be another type of assessment to evaluate those qualities. Thoughts?
Well, when it happened to Dental they just started accepting other standardized exams like CBSE and even GREs.

Capping applications to stop the problem at its source makes more sense than trying to find a new test for the same role imo
 
Well, when it happened to Dental they just started accepting other standardized exams like CBSE and even GREs.

Capping applications to stop the problem at its source makes more sense than trying to find a new test for the same role imo
I agree. It seems the response to this decision has focused on the need for more objective criteria, for some reason. But yes definitely too much applying overall.
 
The linked document is interesting. We use anchored likert scale evaluations like that all time time. On a 5 point likert, everyone gets 4-5 no matter what we do. So also, not terribly useful.
I don’t disagree. The value of the FITREP system is the weight applied to the rating because all evaluators are compared to one another. If you consistently fluff your evaluations, you’re not doing anyone any good. That alone can’t fix how candidates are evaluated, but something along those lines seems more attractive than P/F grades or a glowing LOR that is identical to someone else’s glowing LOR.
 
I still think clinical grades are the single worst grading system in all of med school (and much of education) - definitely an already existing problem that is only made worse by the step 1 change, not a solution.
I very much agree. Furthermore, some are much harsher than others. I had one evaluate me “below expectations” while the other people (3 of them i think) that evaluated me all gave me “far exceeds”. That evaluation was tossed... and other students had the same issue with this person.
 
I very much agree. Furthermore, some are much harsher than others. I had one evaluate me “below expectations” while the other people (3 of them i think) that evaluated me all gave me “far exceeds”. That evaluation was tossed... and other students had the same issue with this person.

Yep. Have a buddy who got worked waaaaaay harder on psychiatry than the other people on his rotation because he got assigned to a particular attending. And he got a lower eval than they did. It's ridiculous
 
Yep. Have a buddy who got worked waaaaaay harder on psychiatry than the other people on his rotation because he got assigned to a particular attending. And he got a lower eval than they did. It's ridiculous
Yep. On the other hand, some dont care and give honors to everyone.
or my favorite, give you ZERO feedback, your evaluation has nothing to improve on, and you still don’t honor it. So irritating.
 
Millennials did this. They complain hard and now they/we have a lot of influence in these matters. We have put institutions (schools and residencies) in trouble with complaining, strictly enforce training hours and other things for “mental health” reasons. Now we got rid of the only standardized way to test someone basic medical knowledge. We replace it with some nonsense. Then when a med student or even a resident who felt no need to study during the first half of school doesn’t know anything during clinical rotations or residency and someone call them out who will be in trouble, the student or the one calling them out? This is so wrong.
Are millennials killing the step 1 industry?
The oldest millennial would have been 22 when acgme duty hours restrictions went in place. Well before residency.
And step 1 didn’t even exist before the 90s and yet somehow going to P/f is the end of the world.
Ok, boomer. Stop blaming millennials for everything .
 
Last edited:
Are millennials killing the step 1 industry?
The oldest millennial would have been 22 when acgme duty hours restrictions went in place. Well before residency.
And step 1 didn’t even exist before the 90s and yet somehow going to P/f is the end of the world.
Ok, boomer. Stop blaming millennials for everything .
Lol I’m far from boomer but I see what people in my generation and few years younger and older are doing. This emphasis on “life balance” and “mental health” has resulted in docs that are too sensitive. I see them cry over scores, an attending not giving enough praise, and threatening to report their own programs to LCME/ACGME like a spoiled kid telling their parents that they’ll report them for disciplining over anything. This doesn’t help produce better docs or improve medical care IMO. Step 1 (or NBME) has been around for a very long time and has helped PDs screen people in and out on both extremes of the scores.
 
Lol I’m far from boomer but I see what people in my generation and few years younger and older are doing. This emphasis on “life balance” and “mental health” has resulted in docs that are too sensitive. I see them cry over scores, an attending not giving enough praise, and threatening to report their own programs to LCME/ACGME like a spoiled kid telling their parents that they’ll report them for disciplining over anything. This doesn’t help produce better docs or improve medical care IMO. Step 1 (or NBME) has been around for a very long time and has helped PDs screen people in and out on both extremes of the scores.
You might not be a boomer in age, but sure sound like one in spirit. Step 1 didnt exist in this format before 92 , yet going pass fail is the end of the world? Why wasnt the world ending before 92?
Im sorry, were you around in the 80's on wards to know what people were doing or how often they were making reports to the LCME acgme ? Plus frankly, you have zero evidence of what it does in terms of making better doctors or not. People made the same claims about the 80 hour work week, and somehow the sky did not fall and medical errors did not magically increase , nor did we have a string of incompetent doctors going out into the world .
If anything training is longer now than it has ever been with increase in years or "mandatory" fellowship. and have some of the highest achieving students in raw metrics of step 1, mcat and grades, and even research.
Just because you see some of the outliers doesnt mean that a vast majority of us are not as hardworking as the previous classes or generations.
 
Last edited:
Lol I’m far from boomer but I see what people in my generation and few years younger and older are doing. This emphasis on “life balance” and “mental health” has resulted in docs that are too sensitive. I see them cry over scores, an attending not giving enough praise, and threatening to report their own programs to LCME/ACGME like a spoiled kid telling their parents that they’ll report them for disciplining over anything. This doesn’t help produce better docs or improve medical care IMO. Step 1 (or NBME) has been around for a very long time and has helped PDs screen people in and out on both extremes of the scores.

I'd argue that what we're seeing is just a byproduct of the greater hypersensitivity/outrage culture as a whole. We have been encouraged to embrace narcissistic tendencies. We live in a post-fact society where feelings are supreme.
 
1582489092120.png

1582489130222.png

Here are a bunch of citations from the 80's and before talking about burnout as well. Just incase you think millennials made this **** up .
 
You might not be a boomer in age, but sure sound like one in spirit. Step 1 didnt exist in this format before 92 , yet going pass fail is the end of the world? Why wasnt the world ending before 92?
Im sorry, were you around in the 80's on wards to know what people were doing or how often they were making reports to the LCME acgme ? Plus frankly, you have zero evidence of what it does in terms of making better doctors or not. People made the same claims about the 80 hour work week, and somehow the sky did not fall and medical errors did not magically increase , nor did we have a string of incompetent doctors going out into the world .
If anything training is longer now than it has ever been with increase in years or "mandatory" fellowship. and have some of the highest achieving students in raw metrics of step 1, mcat and grades, and even research.
Just because you see some of the outliers doesnt mean that a vast majority of us or not as hardworking as the previous classes or generations.
Older physicians always use these tactics to defend the current antiquated medical system. Nothing new!
 
It's pretty dumb to credit any crying millenials for this change. I said it before and I'll say it again, the NBME does not serve educators or students, it serves the NBME. Either there's a financial angle in the works, or they simply couldn't make the stats work for a test trying to fill two contradictory roles.
 
It's pretty dumb to credit any crying millenials for this change. I said it before and I'll say it again, the NBME does not serve educators or students, it serves the NBME. Either there's a financial angle in the works, or they simply couldn't make the stats work for a test trying to fill two contradictory roles.
This will somehow make the NBME more money. Maybe they’re planning on selling more shelf exams or something.
 
This will somehow make the NBME more money. Maybe they’re planning on selling more shelf exams or something.
Reporting preclinical and clinical shelf scores in MSPEs could be a decent way forward. Much better at placing people across many thousands of test items, and highlighting strong areas. Gives people a way to stand out in their subject of interest instead of all battling it out over First Aid.
 
Reporting preclinical and clinical shelf scores in MSPEs could be a decent way forward. Much better at placing people across many thousands of test items, and highlighting strong areas. Gives people a way to stand out in their subject of interest instead of all battling it out over First Aid.
That seems like a good idea if everyone had the same third year experience. But that’s not the case. Some people do 40 hours no call no weekends on surgery while others do 80 hrs/week with a few 24 hour calls sprinkled in. Also the order of your rotations can really screw you here. For instance, the family med shelf is pretty chill if you’ve already had ob, peds, and im. It’s a monster if you haven’t.

If they don’t want to have students stratified by this or any other USMLE exam, then they need to come up with one to do just that.
 
That seems like a good idea if everyone had the same third year experience. But that’s not the case. Some people do 40 hours no call no weekends on surgery while others do 80 hrs/week with a few 24 hour calls sprinkled in. Also the order of your rotations can really screw you here. For instance, the family med shelf is pretty chill if you’ve already had ob, peds, and im. It’s a monster if you haven’t.

If they don’t want to have students stratified by this or any other USMLE exam, then they need to come up with one to do just that.

The exams are standardized regardless. What about the study time students have in mandatory attendance vs. non-mandatory attendance lectures? It doesn’t seem like too much of a difference looking at it broadly.
 
Reporting preclinical and clinical shelf scores in MSPEs could be a decent way forward. Much better at placing people across many thousands of test items, and highlighting strong areas. Gives people a way to stand out in their subject of interest instead of all battling it out over First Aid.
I am honestly surprised why more people and program directors are not more interested in this. Why rely on one test day when you have dozens of data points? Its not like you couldnt import all the applicants into a spread sheet and apply custom formulas with higher weight given to certain exams. Want to do radiology? what was your path, anatomy, IM and surgery shelf score?
That seems like a good idea if everyone had the same third year experience. But that’s not the case. Some people do 40 hours no call no weekends on surgery while others do 80 hrs/week with a few 24 hour calls sprinkled in. Also the order of your rotations can really screw you here. For instance, the family med shelf is pretty chill if you’ve already had ob, peds, and im. It’s a monster if you haven’t.

If they don’t want to have students stratified by this or any other USMLE exam, then they need to come up with one to do just that.
Yes, but people have different preclinical schedules with varied amount of time for dedicated. Schools also vary on how much they teach to the test, etc. Also even on the same service students may have a completely different experience in terms of hours. This variability is part of life, but i feel like the average of shelf exams or looking at individual shelves is not terrible considering the medians and percentiles on the shelf change over the course of the year as well.
 
Reporting preclinical and clinical shelf scores in MSPEs could be a decent way forward. Much better at placing people across many thousands of test items, and highlighting strong areas. Gives people a way to stand out in their subject of interest instead of all battling it out over First Aid.

Students will just binge and purge information, even worse than they do now. With a comprehensive test at least there’s some semblance of trying to remember information long term.
 
Students will just binge and purge information, even worse than they do now. With a comprehensive test at least there’s some semblance of trying to remember information long term.
People are already binging and purging for step. You think I legit remember the Krebs cycle. All I know is that the mitochondria is the powerhouse of the cell.
 
Students will just binge and purge information, even worse than they do now. With a comprehensive test at least there’s some semblance of trying to remember information long term.
Shelf studying a couple hours daily, on material related to what I'm seeing on the wards that month, is a WAY better educational experience. My school giving me an empty block of 8-10 weeks to cram 24/7 was more binge-and-purge style, not less.
 
Shelf studying a couple hours daily, on material related to what I'm seeing on the wards that month, is a WAY better educational experience. My school giving me an empty block of 8-10 weeks to cram 24/7 was more binge-and-purge style, not less.

With the latter you’ll have learned the material once before, and then reviewing the material again during dedicated. If you just studied for the shelf or whatever and then didn’t care about any comprehensive exam, you won’t bother reviewing previous information you frankly won’t need to use. If Step 2 became pass/fail I wouldn’t bother reviewing for OB or pediatrics like I’m doing now. It would be effectively purged from my memory from seeing it once then done.
 
Apparently everyone remembers the Kreb cycle pretty damn well given how often they bring it up.
I know of the krebs cycle, I also know of nuclear fusion/fission, doesnt mean i know anything meaningful about it or i could rattle off the equations/intermediate steps of either.
 
I know of the krebs cycle, I also know of nuclear fusion/fission, doesnt mean i know anything meaningful about it or i could rattle off the equations/intermediate steps of either.

I was kidding. But regardless, I’ve said before that Step 1/First Aid isn’t all biochem. 80% of the book are diseases and pharmaceuticals by organ system.
 
Not that it matters but the Krebs cycle thing is a strawman. it takes about 1 BnB video worth of time to learn what you need of the Krebs cycle for step

Another more relevant pathway might be glycolysis, knowing the basics of glycolysis seems to entirely relevant, even if the enzyme names don't stick, having an understanding of the basics and being able to reason through its related pathologies seperates us from the low-level providers. In p/f world you likely can just blow off all of biochem and still comfortably pass

Entirely possible that less focus on preclinical stuff will produce physicians that are competent enough but this move lessens their importance for sure. Maybe that's a good thing but I don't get why anyone would do more than the bare minimum in p/f world
 
Not that it matters but the Krebs cycle thing is a strawman. it takes about 1 BnB video worth of time to learn what you need of the Krebs cycle for step

Another more relevant pathway might be glycolysis, knowing the basics of glycolysis seems to entirely relevant, even if the enzyme names don't stick, having an understanding of the basics and being able to reason through its related pathologies seperates us from the low-level providers. In p/f world you likely can just blow off all of biochem and still comfortably pass

Entirely possible that less focus on preclinical stuff will produce physicians that are competent enough but this move lessens their importance for sure. Maybe that's a good thing but I don't get why anyone would do more than the bare minimum in p/f world
obviously I am exaggerating for the sake of making a point and in jest.
 
With the latter you’ll have learned the material once before, and then reviewing the material again during dedicated. If you just studied for the shelf or whatever and then didn’t care about any comprehensive exam, you won’t bother reviewing previous information you frankly won’t need to use. If Step 2 became pass/fail I wouldn’t bother reviewing for OB or pediatrics like I’m doing now. It would be effectively purged from my memory from seeing it once then done.
Bro, if you try asking people at the end of their surgical subspecialty residency to answer questions from the peds shelf, they'll bomb them whether they took scored step exams or not. Not knowing information we don't need to use is how it works. It atrophies. See: cardiologist who did an IM residency and yet doesn't remember pancreatitis.
 
Bro, if you try asking people at the end of their surgical subspecialty residency to answer questions from the peds shelf, they'll bomb them whether they took scored step exams or not. Not knowing information we don't need to use is how it works. It atrophies. See: cardiologist who did an IM residency and yet doesn't remember pancreatitis.

I mean, I agree that much of third year and Step 2 is useless especially for surgical subspecialties. But surely some people and specialties think it’s relevant, and I still remember the information better having gone through it twice and reinforcing it throughout the year rather than a 6 week bootcamp and then that’s it.
 
Reporting preclinical and clinical shelf scores in MSPEs could be a decent way forward. Much better at placing people across many thousands of test items, and highlighting strong areas. Gives people a way to stand out in their subject of interest instead of all battling it out over First Aid.
I have some worry that this is the "secret plan" by the NBME. Most DO students, and all IMG students, do not take the shelf exams. Ring the $$$$ bell for the NBME.
 
I have some worry that this is the "secret plan" by the NBME. Most DO students, and all IMG students, do not take the shelf exams. Ring the $$$$ bell for the NBME.
why do you worry in regards to this plan , yes the NBME makes more money, but spread out testing and more data points seems like a win for both applicants and program directors.
 
I wonder whether this will benefit or hurt the Step tutoring industry.

Will more students be willing to pay $275/hr for tutors, given that there will only be one test whose score matters? Or will students prefer to just get started on Step 2CK studying earlier in med school and forego tutoring?
 
I wonder whether this will benefit or hurt the Step tutoring industry.

Will more students be willing to pay $275/hr for tutors, given that there will only be one test whose score matters? Or will students prefer to just get started on Step 2CK studying earlier in med school and forego tutoring?
I think tutoring industry is going to through major changes. Step 2 CK will obviously be acting monarch until the system achieves stability again. If it goes towards shelves, now they will probably pay for shelves tutoring.
 
I have some worry that this is the "secret plan" by the NBME. Most DO students, and all IMG students, do not take the shelf exams. Ring the $$$$ bell for the NBME.
Figured there was something cooking behind the scenes. Even many MD schools do not use shelves for preclinical, only clinical. If shelves became the central metric surely they'd start buying the preclinical ones to better prep us for NBME exams.

Seems like they do stand to make a whole lot more.
 
We also have to keep in mind that medical treatments, understanding, and diagnostics have exploded over the last 10-20+ years. I imagine med school was easier in 1985 when there were 1/3 of drugs and 1/2 of the diseases etiologies were logged as “idiopathic”.
 
We also have to keep in mind that medical treatments, understanding, and diagnostics have exploded over the last 10-20+ years. I imagine med school was easier in 1985 when there were 1/3 of drugs and 1/2 of the diseases etiologies were logged as “idiopathic”.
Be careful man. These SDNers are going to burn you at the stake for saying this. I had to lock my post earlier because they all came after me.
 
Top