MD Heard a rumor that Step 1 (and maybe Step 2 CK) may change from scores to P/F. Is that true?

[CyrusHabs7695]

I’m wondering if some group believes that the most competitive specialties and programs are too populated with overly-represented groups?

If we look at the residents’ pics posted on top programs and competitive specialties, will we see too many ORMs who test very well?

And why is having a score for Step 1 “bad,” but it’s ok to have scores for MCAT and Step 2?

Dude don't even try to pull this. People are not landing neurosurgery or plastics residencies because they are white.

---

Members don't see this ad.

 

[Goro]

What do you mean by "current incarnation"?

and yes that type of situation did actually exist prior to residency applications becoming meritocratic thru the NRMP algorithm and numerical scoring of USMLE exams

*sigh*, Rick.....This is why there's a CARS section in MCAT.
cur·rent
/ˈkərənt/
adjective

1. 1.
   belonging to the present time; happening or being used or done now.
   incarnation
   noun
   
   a particular physical form or state
   Wise @gyngyn, @Med Ed @Pathdocmd @VA Hopeful Dr ...was this indeed how things were done in the past?
   
   
   
   
   

 

[863168]

You would have though it was going to be in Korean, the way people carried on!

The test has as much chance of being written in hangumal as it does of being written in hebrew, it's far more likely that administrators find a way for it to be written in guyanese or seychellois to even out representation in their proverbial coloring box.

 

[dohopeful13]

I still think this is not a forgone conclusion yet despite the momentum. The language regarding the meeting has been pretty wishywashy. For all we know, the recommendation will be for a study to be conducted on possibly changing the exam, or may include High Pass/P/Fail or something. I think there will be pushback from PDs and the organizations that make a lot of money off test prep. I’m sure it will happen eventually, but it may not be as immediate and concrete as we fear or expect.

I also can’t help but wonder what happens to re-applicants who have a scored USMLE. Are they just given an advantage against everyone else or is that score blinded?

 

[Oh_Gee]

At some point PDs need an objective way to stratify applicants for interview invites. If Step 1 goes P/F it becomes a checkbox like Step 2CS, and an expensive one at that. The buck effectively gets kicked down a rung to the freaking MCAT which obviously plays a significant role in what tier of med school one can attend. I’d much rather be judged for my monumental effort on a sub-optimal exam in conjunction with, instead of only, the bull**** subjectiveness that is research output (I.e. most often useless noise), LORs (who you know), clinical evals, and tier of med school. They could easily change the exam to be more useful with clinical relevance but instead have gone down the rabbit hole of PhD mental-masturbation for score control that has lead to the situation we are in now.

On one hand I think med schools have an interest in de-emphasizing Step 1 in order to keep their lecturers relevant. I think most of my classmates had learned most of the requisite USMLE knowledge on their own and could just as easily done preclinical years on their own. On the other hand you have the USMLE that makes an absurd amount of money off of their exams and NBMEs. In both cases there are some people making lots of money. More than you reading this is likely to make. The dean at my med school makes 7 figures and the AAMC president makes 7 figures.

Medical school education has been consumed by greedy parasites who preach to us the importance of being selfless. I’m not sure what will happen with Step 1 but I’m pretty confident it will be in the administrators’ best financial interest. Post-secondary education, and especially medical education, is one of the biggest rackets in existence. Needs to be torn down then rebuilt from the ground up. Until then... get back to your Anki!

i wonder how many people would buy the NBME's if step1 became p/f. i for sure wouldn't

 

[VA Hopeful Dr]

*sigh*, Rick.....This is why there's a CARS section in MCAT.
cur·rent
/ˈkərənt/
adjective

1. 1.
   belonging to the present time; happening or being used or done now.
   incarnation
   noun
   
   a particular physical form or state
   Wise @gyngyn, @Med Ed @Pathdocmd @VA Hopeful Dr ...was this indeed how things were done in the past?
   
   
   
   

.... how old do you think I am?

I took Step 1 in 2008, I wasn't aware it had changed much since then.

 

[dohopeful13]

Dude don't even try to pull this. People are not landing neurosurgery or plastics residencies because they are white.

I don’t think the previous poster is saying they landed the residency because they are white. He’s saying the push for P/F may be partially because some believe ORMs/Whites are too prevalent at the top residencies.

And if you look at the InCus conference pro/con section, one of their cons is actually “lack of diversity” due to numeric scoring. That’s straight from the horses’ mouth. I mean, think about if the MCAT was P/F. URMs already statiscally get in with lower scores on average. Take away numeric scoring completely and they will absolutely be preferred over those over represented groups purely in the name of diversity.


(For the record, I am not bashing affirmative action or saying it’s wrong. But to pretend P/F isn’t going to make it more of an influence is naive)

 

[863168]

(For the record, I am not bashing affirmative action or saying it’s wrong. But to pretend P/F isn’t going to make it more of an influence is naive)

Kaplan made an interesting comment in his interview awhile back with The New Yorker where he stated that he thought standardized testing was a wonderful system because a standardized test was blind to your ethnic background. Another interesting fact was that he felt that he was denied entry into medical school because they had already filled up their ethnic quota on Jewish applicants. Julius Axelrod had also referenced that he was likely systematically rejected due to ethnic quotas, but went on to win a Nobel prize for his discovery of catecholamines.

If anything recent scandals show that administrators get away with pushing sociopolitical experiments because the bench of qualified applicants is so deep that they could select off the wrong end and still get away with it. The whole point of quantitative metrics was to establish some degree of an even playing field that could not be shaped by people in power who want to implement a selective ideology. It's too bad that almost a century later they are still attempting to implement the same policies through obfuscation in order to make the bench marks less clear for who is and who is not a stronger applicant when it comes to working within an established system.

 

[Med Ed]

I would like to think that admins are seeing systemic issues with the current step 1 process that students aren’t aware of, if they are aiming for such a drastic change.

Yes, we are seeing systemic issues with Step 1 that have worsened exponentially even in the last 2-3 years. It takes five acts of God to get the NMBE to consider changing anything. The fact that InCUS was convened with any rapidity should give us all some idea of how bad things have gotten. The students obviously don't know any different, but those of us with a more longitudinal perspective are witnessing a nuclear meltdown.

What were PDs doing to determine competitive candidates prior to having such a huge emphasis on Step 1? How were people matching into competitive specialties back then?

Screening with Step 1 has become the standard method to make application pools manageable. After the pool has been adequate reduced in size, the PD & Co. then has to do something truly remarkable: read the applications. All the subjective elements that people are complaining about were already an important part of the process, it's just much easier to fixate on the clean, three digit number that keeps you in the running.

If Step 1 goes P/F I predict that Step 2 CK will become the new screening tool. While still not perfect, this would be a marked improvement over the current situation.

 

[Chibucks15]

Yes, we are seeing systemic issues with Step 1 that have worsened exponentially even in the last 2-3 years. It takes five acts of God to get the NMBE to consider changing anything. The fact that InCUS was convened with any rapidity should give us all some idea of how bad things have gotten. The students obviously don't know any different, but those of us with a more longitudinal perspective are witnessing a nuclear meltdown.



Screening with Step 1 has become the standard method to make application pools manageable. After the pool has been adequate reduced in size, the PD & Co. then has to do something truly remarkable: read the applications. All the subjective elements that people are complaining about were already an important part of the process, it's just much easier to fixate on the clean, three digit number that keeps you in the running.

If Step 1 goes P/F I predict that Step 1 CK will become the new screening tool. While still not perfect, this would be a marked improvement over the current situation.

I fail to see why people think this is a worse idea (besides the fact that SDN is filled with gunners). Getting tested on clinical situations is about as doctory as we can get. Why should that not be the measuring stick? If we wanted to learn the crazy amount of material, with a healthy chunk of it past everyday clinical scope, then why didn't we all go get a PhD?

 

[deleted480308]

it's an arms race, the only thing changing is the weapon

Step 2 will get crazy if step1 goes p/f and if that goes p/f get ready for something else to get absurd

 

[Med Ed]

I’d much rather be judged for my monumental effort on a sub-optimal exam

I don't think it's adequate to call Step 1 "sub-optimal" for purposes of resident selection. The (mis)use of Step 1 in this context is more like giving laxatives to someone with pneumonia. It won't accomplish anything useful, but at least something is being done.

That a licensing exam is a poor tool for other purposes is not a new observation. Here is part of a paper published almost 30 years ago about the utility of score reporting in the old NBME examination:

"The general purpose of an examination should be the primary factor in determining how scores are defined and reported. That the purpose of certification examinations is to differentiate between examinees who possess necessary knowledge and skills and those who do not is consistent with the use of a pass/fail classification. For such an examination there should be maximum discrimination among examinees and a high level of precision of measurement at the pass/fail point. When used to measure academic achievement, and examination should discriminate among examinees and measure with equal precision throughout the score distribution and reflect were, along a continuum, the examinee's level of achievement on the examination resides." - Academic Medicine. 65(12):723-9, 1990.

Given Step 1's purpose, attempting to apply meaning to scores that are many standard deviations above the passing point is, in a word, stupid.

 

[Chibucks15]

it's an arms race, the only thing changing is the weapon

Step 2 will get crazy if step1 goes p/f and if that goes p/f get ready for something else to get absurd

whats the solution then? Besides eventual catastrophic explosion? There's a tipping point and its coming soon with how fast medical knowledge is expanding and how insanely competitive everything is (and honestly how fragile some peoples egos are)

 

[Nurse2MD2018]

Kaplan made an interesting comment in his interview awhile back with The New Yorker where he stated that he thought standardized testing was a wonderful system because a standardized test was blind to your ethnic background

My memory may be failing me but I remember reading findings back in hs sociology about this being false but I think most of the studies were in relation to the SAT.

 

[deleted480308]

whats the solution then? Besides eventual catastrophic explosion? There's a tipping point and its coming soon with how fast medical knowledge is expanding and how insanely competitive everything is (and honestly how fragile some peoples egos are)

the solution is it will stay absurdly competitive for desirable specialties because no residency staff is going to thoroughly read 6000 applications

if you want the thing all the other students want, get better at whatever PDs use to screen

 

[Chibucks15]

the solution is it will stay absurdly competitive for desirable specialties because no residency staff is going to thoroughly read 6000 applications

if you want the thing all the other students want, get better at whatever PDs use to screen

yeah I get that, and personally I'm not going for anything nuts to begin with so it doesn't really matter to me, but it just seems like basing your entire future on a multiple choice test of esoteric facts that vaguely have to do with a clinical vignette given is not the most ideal measure.

 

[deleted480308]

yeah I get that, and personally I'm not going for anything nuts to begin with so it doesn't really matter to me, but it just seems like basing your entire future on a multiple choice test of esoteric facts that vaguely have to do with a clinical vignette given is not the most ideal measure. Not to mention the stats that back up the fact that step 1 doesn't predict future performance by any significant measure more than the other exams

but step 1 sure as hell correlates with getting an ortho/derm interview and it doesn't take 3000man hours of a residency staff to narrow the list

 

[Chibucks15]

but step 1 sure as hell correlates with getting an ortho/derm interview and it doesn't take 3000man hours of a residency staff to narrow the list

so let's figure out a better system? I get the high scorers want this to stay but when the average score keeps creeping higher eventually either the test has to get harder (which doesn't make sense because its already pretty damn extensive) or there has to be a new structure in place

 

[Goro]

I fail to see why people think this is a worse idea (besides the fact that SDN is filled with gunners).

For a depressingly large number of SDNers, medical education ends with Step I and the exam is an affirmation of their self worth and being.

 

[Goro]

but step 1 sure as hell correlates with getting an ortho/derm interview and it doesn't take 3000man hours of a residency staff to narrow the list

More accurate to say that Step I correlates with NOT getting an interview if you fall below an arbitrary line, but scoring above the line does not guarantee an interview.

 

[jhenj529]

I actually don't like this. Step 1 was the thing I had control over. I fought tooth and nail to get a UNPAID and minor commitment research position this summer and was rejected by 4+ different opportunities. Can't imagine having that play a greater role in residency. Hopefully they'll make these changes for a fresh class of first year students. Could you imagine the riots if they made the change halfway through people's medical education???

 

[Chibucks15]

I actually don't like this. Step 1 was the thing I had control over. I fought tooth and nail to get a UNPAID and minor commitment research position this summer and was rejected by 4+ different opportunities. Can't imagine having that play a greater role in residency. Hopefully they'll make these changes for a fresh class of first year students. Could you imagine the riots if they made the change halfway through people's medical education???

I mean they changed the MCAT, added sections and made it longer, and there weren't any riots besides maybe online forums...

 

[deleted480308]

so let's figure out a better system? I get the high scorers want this to stay but when the average score keeps creeping higher eventually either the test has to get harder (which doesn't make sense because its already pretty damn extensive) or there has to be a new structure in place

no, the test doesn't have to get harder. The minimum scores just keep going up

name the better system, and don't forget that it can't require any more time from residency staff for screening

 

[ciestar]

For a depressingly large number of SDNers, medical education ends with Step I and the exam is an affirmation of their self worth and being.

Yeah, because we’re taught before we even start med school if you do poorly on step 1 you’re destined for nothing and will be a bad physician.

 

[Chibucks15]

no, the test doesn't have to get harder. The minimum scores just keep going up

name the better system, and don't forget that it can't require any more time from residency staff for screening

I'm not saying I have a better one personally, I figured the hivemind could come up with something. And sure the minimum scores going up actually makes my point for me...rather than just being below average and still getting to be a physician somewhere, you make it so more people fail? Now that just doesn't make sense in any context

If we were actually searching for the best doctor It just makes sense to have the focus on actually being a doctor rather than what was originally built as a competency exam.

 

[lj_99]

what are the chances this is pass/fail for 2020 test takers?

 

[deleted480308]

I'm not saying I have a better one personally, I figured the hivemind could come up with something. And sure the minimum scores going up actually makes my point for me...rather than just being below average and still getting to be a physician somewhere, you make it so more people fail? Now that just doesn't make sense in any context

If we were actually searching for the best doctor It just makes sense to have the focus on actually being a doctor rather than what was originally built as a competency exam.

I meant the minimum screening for a program to interview

 

[gyngyn]

what are the chances this is pass/fail for 2020 test takers?

Vanishingly small.

 

[7331poas]

No one does.

BTW, the angst about this subject reminds of the time when AAMC changed the MCAT.

You would have though it was going to be in Korean, the way people carried on!

The core concept of the mcat was still the same. Still graded and was still an objective measure of your knowledge

 

[ricksimmonsrafowens]

it's an arms race, the only thing changing is the weapon

Step 2 will get crazy if step1 goes p/f and if that goes p/f get ready for something else to get absurd

Critical thinking is not my strong suit. Someone please explain to me... If Step 1 is pass/fail, then Step 2 CK will be hugely important and essentially equivalent to Step 1... at least with current system if a student underscores on Step 1, they can demonstrate improvement on Step 2. Eliminating numerical grading for Step 1 makes CK a single extremely high stakes exam... I can't see any benefit to current students in having a single high stakes exam like CK.

 

[Goro]

The core concept of the mcat was still the same. Still graded and was still an objective measure of your knowledge

But people refused to believe that.

 

[Goro]

I actually don't like this. Step 1 was the thing I had control over. I fought tooth and nail to get a UNPAID and minor commitment research position this summer and was rejected by 4+ different opportunities. Can't imagine having that play a greater role in residency. Hopefully they'll make these changes for a fresh class of first year students. Could you imagine the riots if they made the change halfway through people's medical education???

Schools make changes to curricula mid-stream all the time. At my school alone, we've switched from classic Flexner to systems-based, and then to TBL. The students survived, and didn't raise an eyebrow.

You have control over all the things you learn about during clinicals, your behavior and your ability to network...ie, all the humanistic domains you're required to master, and make up the other stuff that residencies look at besides Step scores.

 

[ricksimmonsrafowens]

single extremely high stakes exam...it's Step 1

Yeah but there's Step 2 CK if someone doesn't do well on Step 1

for example, say you attend my school with it's totally garbage pre-clinical curriculum and below the national average Step 1. You get destroyed on Step 1, but then improve on Step 2 and match that "Ivy" residency program that ranked you to match because you showed improvement on 2CK. This is multiple people at my school btw.

what if you took only the high stakes 2CK with the same garbage prep that my school provides... You'd probably underperform because you didn't have the experience of taking Step 1 and doing poorly on that.

Also, I strongly disagree with your implication that Step 1 doesn't test clinical gestalt. It's important to know pathophysiology of disease before knowing what's the next best step is in a purely clinical scenario. You have to know how to crawl (knowing pathophysiology and like what orphan annie nuclei indicate in thyroid tissue ie Step 1) before walking (whether to do a neck dissection on someone with follicular cx ie Step 2). I felt that when I took Step 1 it definitely tested "clinical gestalt" and my understanding of pathphys and foundation of disease very well.

 

[sloop]

I mean, we already have a single extremely high stakes exam...it's Step 1. I feel like if we have to have an objective, score-based measure upon which residency selection should be based, CK is a hell of a lot better than Step 1. MUCH more clinically relevant and directed towards clinical-decision making vs. knowing minute details that most of us don't even remember by the time we start residency. Putting myself in the shoes of a patient, employer, etc. I care much more about whether a physician can get a clinical gestalt and decide on the next step (a Step 2 style question) rather than whether they know what some zebra disease looks like under the microscope when they're never even going to look at a pathology slide after graduating from med school (a Step 1 style question). Not saying we shouldn't learn the step 1 stuff or at least be aware that that information exists, but there's truly no data showing that it predicts clinical competence or performance in residency and beyond. We might as well be sorting people into specialties based on their performance on an art history quiz (I'm exaggerating a bit, but still).

At least among my classmates that I know of, the people who had the best pre-clinical grades and step 1 scores were not always the best clinically. The people who did well on Step 2 were generally pretty solid clinically.

Most people who do well on Step 1 do well on Step 2 CK. The truth about these exams is that they all probably mostly test a common factor. We can argue about what that factor is or what it means, but the exams are not particularly unique.

 

[7331poas]

This change will eliminate the upward mobility of an applicant for a lower tier medical school, especially in specialties where aways are discouraged

If step 2 becomes the new test that PDs look at, then I will simply just switch to that test and start gunning. The problem remains the same.

 

[jhenj529]

Schools make changes to curricula mid-stream all the time. At my school alone, we've switched from classic Flexner to systems-based, and then to TBL. The students survived, and didn't raise an eyebrow.

You have control over all the things you learn about during clinicals, your behavior and your ability to network...ie, all the humanistic domains you're required to master, and make up the other stuff that residencies look at besides Step scores.

Yeah but that's usually for the next matriculating class

 

[jhenj529]

But if you can do well on CK anyway, then what does it matter if Step 1 is P/F? And maybe it's just me, but I don't need to have done poorly on a prior exam to kick me in the butt to work harder next time. I think med students are overall a hardworking group and generally do their best all the time.

And plenty of competitive residencies screen based on Step 1 score. A low Step 1 will get you auto-screened out of an interview even with a stellar CK score.


Again, I'm not saying that we shouldn't learn or be tested on the building blocks - just that such an exam shouldn't be the end-all be-all of what specialty you can realistically get into. Pathophysiology is very important, but if you've taken Step 1 I'm sure we both know that the stuff that gets tested isn't always clinically relevant.


And lots of people who don't do well on Step 1 end up doing very well on CK (myself included). There's some overlap - both obviously require you to have a strong work ethic, some intelligence, etc to do well, but I think CK tests critical thinking and big picture stuff much more than Step 1.

What was your s1 and 2 score?

 

[ricksimmonsrafowens]

But if you can do well on CK anyway, then what does it matter if Step 1 is P/F? And maybe it's just me, but I don't need to have done poorly on a prior exam to kick me in the butt to work harder next time. I think med students are overall a hardworking group and generally do their best all the time.

And plenty of competitive residencies screen based on Step 1 score. A low Step 1 will get you auto-screened out of an interview even with a stellar CK score.


Again, I'm not saying that we shouldn't learn or be tested on the building blocks - just that such an exam shouldn't be the end-all be-all of what specialty you can realistically get into. Pathophysiology is very important, but if you've taken Step 1 I'm sure we both know that the stuff that gets tested isn't always clinically relevant.


And lots of people who don't do well on Step 1 end up doing very well on CK (myself included). There's some overlap - both obviously require you to have a strong work ethic, some intelligence, etc to do well, but I think CK tests critical thinking and big picture stuff much more than Step 1.

I feel like we took very different exams. Step 1 and Step 2 both tested critical thinking and big picture stuff. Best prep for Step 2 CK was.... still FA for Step 1. I'm sorry you didn't do well on Step 1, but wouldn't it be awful if your first high stakes standardized exam in med school was Step 2CK? Didn't you change the way you studied for Step 2 CK based on how you performed on Step 1 and knowing you could've done better? Didn't you realize you needed a better foundation for practicing medicine and then accordingly improved your study habits to score better on Step 2 CK?

What is ethical problems with Step screens? They are usually low... not even derm screens above 235 or something on Step 1 and students definitely need above a 235 to match in derm. Medical schools screen for admission with MCAT and no one cares.

also just as high Step 1 scorers do not want the test to become pass/fail and invalidate their achievements, so do those who improve on Step 2 CK relative to Step 1 want other people to believe that's Step 2 is a more important exam.

 

[NotAProgDirector]

it's an arms race, the only thing changing is the weapon

Step 2 will get crazy if step1 goes p/f and if that goes p/f get ready for something else to get absurd

Totally agree. We'd require S2 for an interview offer. All the insanity over S1 would shift to S2.

I don't think it's adequate to call Step 1 "sub-optimal" for purposes of resident selection. The (mis)use of Step 1 in this context is more like giving laxatives to someone with pneumonia. It won't accomplish anything useful, but at least something is being done.

That a licensing exam is a poor tool for other purposes is not a new observation. Here is part of a paper published almost 30 years ago about the utility of score reporting in the old NBME examination:

"The general purpose of an examination should be the primary factor in determining how scores are defined and reported. That the purpose of certification examinations is to differentiate between examinees who possess necessary knowledge and skills and those who do not is consistent with the use of a pass/fail classification. For such an examination there should be maximum discrimination among examinees and a high level of precision of measurement at the pass/fail point. When used to measure academic achievement, and examination should discriminate among examinees and measure with equal precision throughout the score distribution and reflect were, along a continuum, the examinee's level of achievement on the examination resides." - Academic Medicine. 65(12):723-9, 1990.

Given Step 1's purpose, attempting to apply meaning to scores that are many standard deviations above the passing point is, in a word, stupid.

Sorry to torture you twice, I commented on this on the pre-med thread on the same topic.

I disagree. The test was originally designed to determine adequacy for licensure. Students with higher scores have more knowledge, or at least can use that knowledge to answer questions. USMLE scores trend with ITE scores, which trend with ABIM scores.

I see no reason why the USMLE scores won't discriminate over the entire range. If the exam were truly designed ONLY to test for minimum knowledge base, the questions woulb be all very basic -- those with enough knowledge would get most/all of them correct, those whose knowledge was poor would get questions wrong. I expect that's not the case. So, I guess I'm saying that I think the USMLE "measure with equal precision throughout the score distribution and reflect were, along a continuum, the examinee's level of achievement", as per your above quote.

It is often said that USMLE scores don't predict residency performance. My experience this is untrue, it's clearly not the only predictor, but in general residents with higher scores are better residents.

Critical thinking is not my strong suit. Someone please explain to me... If Step 1 is pass/fail, then Step 2 CK will be hugely important and essentially equivalent to Step 1... at least with current system if a student underscores on Step 1, they can demonstrate improvement on Step 2. Eliminating numerical grading for Step 1 makes CK a single extremely high stakes exam... I can't see any benefit to current students in having a single high stakes exam like CK.

Yes, that's exactly what will happen. So then you'll need to study for CK during clerkships, and you'll need a score by early September.

Honestly, I'm so frustrated by the NBME at the moment that I'm ready to recommend that IM just create it's own exam. Then the USMLE can just do whatever it wants. But unfortunately students will be stuck taking, and paying for, yet another exam. So much for "decreasing stress for students". If the USMLE is P/F for S1, it's only a matter of time they do it for S2 also, so we might as well do this.

 

[jhenj529]

Totally agree. We'd require S2 for an interview offer. All the insanity over S1 would shift to S2.



Sorry to torture you twice, I commented on this on the pre-med thread on the same topic.

I disagree. The test was originally designed to determine adequacy for licensure. Students with higher scores have more knowledge, or at least can use that knowledge to answer questions. USMLE scores trend with ITE scores, which trend with ABIM scores.

I see no reason why the USMLE scores won't discriminate over the entire range. If the exam were truly designed ONLY to test for minimum knowledge base, the questions woulb be all very basic -- those with enough knowledge would get most/all of them correct, those whose knowledge was poor would get questions wrong. I expect that's not the case. So, I guess I'm saying that I think the USMLE "measure with equal precision throughout the score distribution and reflect were, along a continuum, the examinee's level of achievement", as per your above quote.

It is often said that USMLE scores don't predict residency performance. My experience this is untrue, it's clearly not the only predictor, but in general residents with higher scores are better residents.



Yes, that's exactly what will happen. So then you'll need to study for CK during clerkships, and you'll need a score by early September.

Honestly, I'm so frustrated by the NBME at the moment that I'm ready to recommend that IM just create it's own exam. Then the USMLE can just do whatever it wants. But unfortunately students will be stuck taking, and paying for, yet another exam. So much for "decreasing stress for students". If the USMLE is P/F for S1, it's only a matter of time they do it for S2 also, so we might as well do this.

Do you think the changes would go into affect for 2020 test takers?

 

[jhenj529]

Why do you ask?

So we can get a better understanding of your argument

 

[carrots28]

Having worked before going to school, I have come to the conclusion that work life is mostly subjective. Evaluations are based on how well you work with your team members and how effectively you get that done. I imagine they are moving to that for the reality of the situation if in fact high STEP scores don’t correlate with success in residency. A better testament to that potential for success if they do change it to p/f will be how well your recommendations letters are written which is independent of how highly ranked your school is.

 

[lj_99]

I disagree — letters of recommendation have everything to do with how highly ranked your school is. Bigger names at more prestigious institutions.

Having worked before going to school, I have come to the conclusion that work life is mostly subjective. Evaluations are based on how well you work with your team members and how effectively you get that done. I imagine they are moving to that for the reality of the situation if in fact high STEP scores don’t correlate with success in residency. A better testament to that potential for success if they do change it to p/f will be how well your recommendations letters are written which is independent of how highly ranked your school is.

 

[Chibucks15]

So we can get a better understanding of your argument

It’s irrelevant to the argument though haha you just wanna know so you can beat it

 

[863168]

Doesn't a P/F Step 1 absolve medical schools from the weight of having to actually provide quality instruction in the basic sciences for the first two years of education? The bar changes from being a performance metric to just designing a curriculum where the majority of students can pass S1.

 

[Goro]

Doesn't a P/F Step 1 absolve medical schools from the weight of having to actually provide quality instruction in the basic sciences for the first two years of education? The bar changes from being a performance metric to just designing a curriculum where the majority of students can pass S1.

Why do I have to keep repeating this? Your medical education doesn't stop with Step I, and we teach you for both Boards and Wards.

 

[ciestar]

Why do I have to keep repeating this? Your medical education doesn't stop with Step I, and we teach you for both Boards and Wards.

Tell that to people forced into specialties they have no interest in. The ortho (or any other specialty) or bust crowd loses their mind. Im just glad I was interested in FM anyway. I like OB but I am really not even competitive enough for that anymore.

It sucks having your entire career outcome being dictated by an exam score.

 

[ricksimmonsrafowens]

Why do you ask?


I think it's more awful that the first high stakes exam in med school is Step 1. One that's less clinically relevant, and the first board exam that most of us have ever taken. I'd much rather it was CK - once we've taken a similar board exam already, have some additional experience under our belt (clinicals, shelf exams, etc). Like I genuinely, truly think that Step 2 CK is a way better and more relevant exam than Step 1 various ways so yes, I would 1000% rather have CK as the first high stakes exam.

Yes, I did change the way I studied for CK, but having a numeric score that was reported to residencies wouldn't have changed that. And I disagree that scoring lower on Step 1 means that your foundation is adequate. It's a licensure exam. By definition, if you pass, your knowledge is considered adequate to do the job. If that's not the case, why are we graduating physicians who don't have adequate knowledge? Again, there's no evidence your Step 1 score is any indicator of your clinical acumen.

Re: the MCAT, don't really want to dig into this argument here, but I'd actually be fine with a P/F MCAT where the passing line is set at whatever score is consistent with academic success in med school.

Didn't say it was an "ethical" problem, I'm just arguing that step 1 is not a useful test beyond assigning a numerical score that doesn't really mean anything outside of the residency application process. Do you really think a dermatologist needs to know more basic science than a hospitalist or PCP?

And my argument here is not based on my personal experiences. I am going exactly where I want to be for residency and my step 1 score literally does not matter anymore. All I'm saying is that we should base residency selection on things that are actually relevant to residency performance, and not on arbitrary measures that have not been shown to predict residency performance, how good of a doctor you'll be, etc.

These are really strong statements. Step 1 score is correlated to performance in-training exams and passing specialty boards. Passing it equates to minimal competence (which means the prospective resident has minimal competence to be trainable). A 260 scorer definitely has a better knowledge base in medicine than a 205 scorer. That's why it is a standardized exam.... APD says essentially that in a post following yours. You cannot be serious that you'd rather have a SINGLE high stakes exam (doen't matter is it's clinically vs basic science focused) than a SERIES of high stakes exams testing your knowledge in many different areas of medicine. I'd rather take a class with a midterm and a final rather than just a final that's worth 100% of the classes grade. I would've failed a lot of pre-clinical classes if there was only 1 high stakes exam per class.

 

[wamcp]

These are really strong statements. Step 1 score is correlated to performance in-training exams and passing specialty boards. Passing it equates to minimal competence (which means the prospective resident has minimal competence to be trainable). A 260 scorer definitely has a better knowledge base in medicine than a 205 scorer. That's why it is a standardized exam.... APD says essentially that in a post following yours. You cannot be serious that you'd rather have a SINGLE high stakes exam (doen't matter is it's clinically vs basic science focused) than a SERIES of high stakes exams testing your knowledge in many different areas of medicine. I'd rather take a class with a midterm and a final rather than just a final that's worth 100% of the classes grade. I would've failed a lot of pre-clinical classes if there was only 1 high stakes exam per class.

What about changing it so if you score one standard deviation above (say, 245) then that is the only distinguishing factor?

Score 260, you only get a score that says >=245

Score 240 or 230, that still gets reported to you and residencies.

Or how about making step 1 and 2 pass fail only, AND get rid of AOA status...because resiedency match gets changed into a strict lottery determined by chance and nothing else. You went to harvard and you want derm? Well, your match lotto ticket number selected you into FM in Nebraska, have fun!

 

[Chibucks15]

These are really strong statements. Step 1 score is correlated to performance in-training exams and passing specialty boards. Passing it equates to minimal competence (which means the prospective resident has minimal competence to be trainable). A 260 scorer definitely has a better knowledge base in medicine than a 205 scorer. That's why it is a standardized exam.... APD says essentially that in a post following yours. You cannot be serious that you'd rather have a SINGLE high stakes exam (doen't matter is it's clinically vs basic science focused) than a SERIES of high stakes exams testing your knowledge in many different areas of medicine. I'd rather take a class with a midterm and a final rather than just a final that's worth 100% of the classes grade. I would've failed a lot of pre-clinical classes if there was only 1 high stakes exam per class.

There was a study where the link was posted on reddit which states that step 1 relationship with future performances is not statistically significant. I’ll try to find it