USMLE versus MCAT difficulty?

[efle]

I'm a little lost, so help me out here. Step 1 right now is focusing heavily on memorizing stuff. And getting high scores require memorizing more of the content than needed. Do you believe that someone who just barely passed Step 1 has the necessary knowledge base to function clinically well?

I personally wouldn't be thrilled if I found out my family doc had barely passed his licensing exams lol, but yeah, that is exactly what it means.

Of note, the exam is not referenced against a static standard, it's norm-referenced. In other words, its your percentile that determines if you pass, rather than your percentage correct. To put it a third way, the US medical licensing board doesn't say there's a number of questions you need to get correct that means you're good to go; instead they just say that 6% of US medical students should fail and use that to determine the fail threshold.

---

Members don't see this ad.

 

[Lawpy]

According to the licensing board they do; that's the entire point of the test. Your own standards and opinions not withstanding.

Tbh, I haven't read your posts, but I'm focusing mainly on the Step 1 issues and suggestions to fix them without changing the scoring.

 

[Raryn]

Did/do you have a very strong English/Humanities background?

I mean, I was an honors student in undergrad so did have to take a higher than typical number of humanities GE courses, but otherwise not really. I'm a traditional student and was a Biology/Chemistry double major.

I'm a little lost, so help me out here. Step 1 right now is focusing heavily on memorizing stuff. And getting high scores require memorizing more of the content than needed. Do you believe that someone who just barely passed Step 1 has the necessary knowledge base to function clinically well?

Yes. Passing Step 1+CK+CS+3 is literally the minimum bar to get independent practice (in conjunction with finishing school and at least 1 year of residency). My father passed Step 1 by 1 point back when the passing score was in the 170s - he functions clinically just fine 3 decades later. I had friends who also barely passed with scores in the 180s - finished residency and thriving in practice today.

 

[Lawpy]

I personally wouldn't be thrilled if I found out my family doc had barely passed his licensing exams lol, but yeah, that is exactly what it means.

Of note, the exam is not referenced against a static standard, it's norm-referenced. In other words, its your percentile that determines if you pass, rather than your percentage correct. To put it a third way, the US medical licensing board doesn't say there's a number of questions you need to get correct that means you're good to go; instead they just say that 6% of US medical students should fail and use that to determine the fail threshold.

Yes. Passing Step 1+CK+CS+3 is literally the minimum bar to get independent practice (in conjunction with finishing school and at least 1 year of residency). My father passed Step 1 by 1 point back when the passing score was in the 170s - he functions clinically just fine 3 decades later. I had friends who also barely passed with scores in the 180s - finished residency and thriving in practice today.

Ok so the Step forums might've misled me but apparently, I found out that barely passing Step 1 = gaps in knowledge base that somehow become apparent in clinical years (so using Anki more = boosting your scores, which tbh feels weird but that's just me). Which fit with the theme that Step 1 as it is = memorizing/content heavy.

But the focus I was getting at is the obsession with Step 1 scores in terms of residency selection. For whatever reason, programs are focusing heavily on Step 1 scores in a similar pattern seen with MCAT obsession and massive score creep over the past decade. Hence why I was suggesting to rework the Step to make it like the MCAT.

 

[notretired]

Tbh, I haven't read your posts, but I'm focusing mainly on the Step 1 issues and suggestions to fix them without changing the scoring.

Looks like you've read them. I think arbitrarily moving a test away from it's actual purpose to simply make it "harder" is a shortsighted view of how to address your goal.

 

[Med Ed]

Of note, the exam is not referenced against a static standard, it's norm-referenced. In other words, its your percentile that determines if you pass, rather than your percentage correct. To put it a third way, the US medical licensing board doesn't say there's a number of questions you need to get correct that means you're good to go; instead they just say that 6% of US medical students should fail and use that to determine the fail threshold.

Yes and no. The NBME will tell you directly (I know because I have asked them) that the Step exams are criterion-referenced. But they keep upping the passing score, so the result is that it behaves like a norm-referenced test.

 

[907914]

Did/do you have a very strong English/Humanities background?

I have zero English/humanities background and I got a 521 (99%) MCAT with less than 200 total study hours. I think your theory falls short.

 

[Lawpy]

Looks like you've read them. I think arbitrarily moving a test away from it's actual purpose to simply make it "harder" is a shortsighted view of how to address your goal.

Step scores aren't used for their intended purpose anyways for reasons similar to why people don't believe MCAT 500+ = pass (and the pass/fail implications) seriously. The scores get their value for residency purposes which is what i'm focusing on

 

[Lawpy]

Yes and no. The NBME will tell you directly (I know because I have asked them) that the Step exams are criterion-referenced. But they keep upping the passing score, so the result is that it behaves like a norm-referenced test.

What are your thoughts on the high Step 2 averages?

 

[efle]

Yes and no. The NBME will tell you directly (I know because I have asked them) that the Step exams are criterion-referenced. But they keep upping the passing score, so the result is that it behaves like a norm-referenced test.

That's hilarious, the average has crept 10 points in the last decade and the same percent are still failing it. How can they pretend that's anything other than norm referenced

 

[notretired]

Step scores aren't used for their intended purpose anyways for reasons similar to why people don't believe MCAT 500+ = pass (and the pass/fail implications) seriously. The scores get their value for residency purposes which is what i'm focusing on

So we should just move them even further away? At that point we might as well scrap them all together.

 

[Lawpy]

So we should just move them even further away? At that point we might as well scrap them all together.

Exams get regularly changed. Happened to the MCAT. Can happen to Steps. Its a different reality now from what was decades ago

 

[Raryn]

Ok so the Step forums might've misled me but apparently, I found out that barely passing Step 1 = gaps in knowledge base that somehow become apparent in clinical years (so using Anki more = boosting your scores, which tbh feels weird but that's just me). Which fit with the theme that Step 1 as it is = memorizing/content heavy.

But the focus I was getting at is the obsession with Step 1 scores in terms of residency selection. For whatever reason, programs are focusing heavily on Step 1 scores in a similar pattern seen with MCAT obsession and massive score creep over the past decade. Hence why I was suggesting to rework the Step to make it like the MCAT.

A knowledge base reflected by a barely passing step score might give you some trouble in clinical years, but it still reflects a sufficient fund of knowledge and ability to learn information that you'll be fine being a physician *in general*. The minimum is the minimum for a reason.

 

[Lawpy]

A knowledge base reflected by a barely passing step score might give you some trouble in clinical years, but it still reflects a sufficient fund of knowledge and ability to learn information that you'll be fine being a physician *in general*. The minimum is the minimum for a reason.

Why does the minimum increase over time?

 

[Raryn]

Why does the minimum increase over time?

All the NBME says is

The USMLE program provides a recommended pass or fail outcome on all Step examinations. Recommended performance standards for the USMLE are based on a specified level of proficiency. As a result, no predetermined percentage of examinees will pass or fail the examination. The recommended minimum passing level is reviewed periodically and may be adjusted at any time. Notice of such review and any adjustments will be posted at the USMLE website. On the examinations containing multiple-choice items, the percentages of correctly answered items required to pass varies from form to form. However, examinees typically must answer 60 to 70 percent of items correctly to achieve a passing score.

So they determine the minimum standard does change over time.

 

[efle]

All the NBME says is



So they determine the minimum standard does change over time.

And coincidentally, its ALWAYS 5-6% that fail to meet that year's proficiency...

 

[Med Ed]

That's hilarious, the average has crept 10 points in the last decade and the same percent are still failing it. How can they pretend that's anything other than norm referenced

Contact | NBME

Our support team is here to help. Find contact options for when you have a question about an NBME assessment or service.

www.nbme.org

 

[Lucca]

Tbh, I haven't read your posts, but I'm focusing mainly on the Step 1 issues and suggestions to fix them without changing the scoring.

The problematic thinking with your approach is that the only thing we can do is tweak Step 1 but it isn’t. What leadership should be doing is thinking about new standardized metrics to use in addition to Step 1 to improve the ability of PDs to make meaningful decisions when choosing residents. National implementation of SLOEs across all specialties seems like the most obvious solution to start thinking about, for example.

If your goal is to do nothing to Step scoring while simultaneously creating some magical exam that can somehow measure everything you could ever want to know about a medical student then you will get nowhere very quickly. Second, nothing will ever change about anything if Step scoring doesn’t change. You need to disrupt a system if you want it to adapt because the preference will always be for doing nothing even when doing nothing is itself an active decision with consequences (e.g inexorable step creep, app proliferation, taking step later and later, taking research years to better prep for step/make up for lower step in spite of no interest in research, etc)

 

[AnatomyGrey12]

Second, nothing will ever change about anything if Step scoring doesn’t change. You need to disrupt a system if you want it to adapt because the preference will always be for doing nothing even when doing nothing is itself an active decision with consequences (e.g inexorable step creep, app proliferation, taking step later and later, taking research years to better prep for step/make up for lower step in spite of no interest in research, etc)

How stupid. Changing Step scoring just to “disrupt the system” is stupid and would have drastic negative effects for thousands of medical students. Explain to me how not changing Step somehow decreases the need for research years when it’s exactly that kind of arbitrary junk that would become more emphasized if a scores Step went away.

Change for the sake of change is always a bad idea. First we need to study what changes would achieve the desired result, instead of just flinging monkey **** at the wall hoping it sticks.

Just finally getting back to this thread. I’ll address some of the stuff I was discussing with other posters before a little later when I have time.

 

[efle]

How stupid. Changing Step scoring just to “disrupt the system” is stupid and would have drastic negative effects for thousands of medical students. Explain to me how not changing Step somehow decreases the need for research years when it’s exactly that kind of arbitrary junk that would become more emphasized if a scores Step went away.

Change for the sake of change is always a bad idea. First we need to study what changes would achieve the desired result, instead of just flinging monkey **** at the wall hoping it sticks.

Just finally getting back to this thread. I’ll address some of the stuff I was discussing with other posters before a little later when I have time.

Take the suggestion in context, though. Imagine switching the USMLE, a knowledge-check exam, back to its original purpose as a pass/fail licensure minimum.

Then introduce a different exam, one built to assess clinical reasoning with a tight confidence interval, instead of assessing flashcard recalls with a huge sloppy CI. Let that new test be the percentile based report that residencies screen with.

This isn't some kind of outlandish novel system, either. For heck's sake this is what we do with high schoolers, give them a GED/exit exam to prove they got the minimum, but then use the SAT to actually assess reasoning and aptitude. We even do it on the level of individual courses, having high schoolers take in-class exams to get credit on the transcript, but then offering national AP tests to show mastery on a curve.

I honestly used to be fine with the idea of the current system until I actually got to dedicated. Shocking amounts of the material I was supposed to learn were clinically useless, most of the questions hinged on recall of some esoteric fact, there are wild swings in score between practice tests due to the massive CI, and at least for the upper half of the curve theres absolutely no link I can see between step performance and who functions best on the wards. Hitting my target score didn't even feel good, it doesnt boost your ego, it just gets a big sigh of relief that this stupid-ass hurdle wont get in the way of my match. I get that people want a way to identify themselves as outliers for PDs, but were deluding ourselves if we think this is a good option to do so.

 

[notretired]

Take the suggestion in context, though. Imagine switching the USMLE, a knowledge-check exam, back to its original purpose as a pass/fail licensure minimum.

Then introduce a different exam, one built to assess clinical reasoning with a tight confidence interval, instead of assessing flashcard recalls with a huge sloppy CI. Let that new test be the percentile based report that residencies screen with.

This isn't some kind of outlandish novel system, either. For heck's sake this is what we do with high schoolers, give them a GED/exit exam to prove they got the minimum, but then use the SAT to actually assess reasoning and aptitude. We even do it on the level of individual courses, having high schoolers take in-class exams to get credit on the transcript, but then offering national AP tests to show mastery on a curve.

This is one of the best ideas I've heard from anyone thus far. Negates the need to constantly shift step 1 away from it's purpose while simultaneously ensuring that objective metrics remain in place.

 

[notretired]

Exams get regularly changed. Happened to the MCAT. Can happen to Steps. Its a different reality now from what was decades ago

And what was gained exactly? The mcat-->step correlation didn't change much. The percentage of students completing pre-clin for a given percentile didn't change more than 1-2% in either direction. I took a couple of the old exams for practice and scored within +/- 1 of my exact percentile on the new test. Only change Imo was the additional pain in the ass of rote memorizing useless/disproven psych theories.

 

[TheIllusionist]

Take the suggestion in context, though. Imagine switching the USMLE, a knowledge-check exam, back to its original purpose as a pass/fail licensure minimum.

Then introduce a different exam, one built to assess clinical reasoning with a tight confidence interval, instead of assessing flashcard recalls with a huge sloppy CI. Let that new test be the percentile based report that residencies screen with.

This isn't some kind of outlandish novel system, either. For heck's sake this is what we do with high schoolers, give them a GED/exit exam to prove they got the minimum, but then use the SAT to actually assess reasoning and aptitude. We even do it on the level of individual courses, having high schoolers take in-class exams to get credit on the transcript, but then offering national AP tests to show mastery on a curve.

I honestly used to be fine with the idea of the current system until I actually got to dedicated. Shocking amounts of the material I was supposed to learn were clinically useless, most of the questions hinged on recall of some esoteric fact, there are wild swings in score between practice tests due to the massive CI, and at least for the upper half of the curve theres absolutely no link I can see between step performance and who functions best on the wards. Hitting my target score didn't even feel good, it doesnt boost your ego, it just gets a big sigh of relief that this stupid-ass hurdle wont get in the way of my match. I get that people want a way to identify themselves as outliers for PDs, but were deluding ourselves if we think this is a good option to do so.

reasoning is overrated for clinical practicing

 

[efle]

reasoning is overrated for clinical practicing

Yeah but the guy replacing my knee might need to dust off his brain once or twice a year. Drawing the TCA cycle has been helpful in the OR exactly 0 times.

 

[el_duderino]

Adding another exam on top of the USMLE exams is one of the most face-palming ideas I've ever heard.

 

[efle]

Adding another exam on top of the USMLE exams is one of the most face-palming ideas I've ever heard.

No no, but the thing is, nobody will give a **** about the USMLE any more. It'll be a replacement, not an addition. Like, did you worry about your high school exit exam? Of course not, everyone knows only the SAT mattered at all, unless you somehow managed to fail your exit exam and not even graduate high school on time. It'd be the same deal

 

[Raryn]

Adding another exam on top of the USMLE exams is one of the most face-palming ideas I've ever heard.

No no, but the thing is, nobody will give a **** about the USMLE any more. It'll be a replacement, not an addition. Like, did you worry about your high school exit exam? Of course not, everyone knows only the SAT mattered at all, unless you somehow managed to fail your exit exam and not even graduate high school on time. It'd be the same deal

I think adding another exam is a wonderful idea. They should make sure it's well crafted and perhaps say... three days. 1000 questions. Very high stakes. Might be expensive to put another one together though, but I'm sure the NBME would be happy to do so for the small small sum of... hmm... how's $3,000 a student sound?

We should pitch it to the NBME, they might take some convincing because of the altruistic nature of their mission, but I think the strong arguments here would be sufficient to turn them around.

/s

 

[efle]

I think adding another exam is a wonderful idea. They should make sure it's well crafted and perhaps say... three days. 1000 questions. Very high stakes. Might be expensive to put another one together though, but I'm sure the NBME would be happy to do so for the small small sum of... hmm... how's $3,000 a student sound?

We should pitch it to the NBME, they might take some convincing because of the altruistic nature of their mission, but I think the strong arguments here would be sufficient to turn them around.

/s

Hey man, I'd honestly take the 3-day 1000 question test over the current version. A 1 in 3 chance of scoring 10+ points outside my actual range was absolutely terrifying. There are literally thousands of students every year suffering 10+ point drops in their score out of pure bad luck.

 

[907914]

No no, but the thing is, nobody will give a **** about the USMLE any more. It'll be a replacement, not an addition. Like, did you worry about your high school exit exam? Of course not, everyone knows only the SAT mattered at all, unless you somehow managed to fail your exit exam and not even graduate high school on time. It'd be the same deal

Is the high school exit exam a common thing? I did not have this in 2013.

 

[Alkain]

Honestly, I feel the best way to deal with boards is to get rid of the first two years of medical school and then add a clinical base year where they teach you basic clinical skills so medical school winds up being 3 years total.

I don't see any reason why I had to pay $50k a year to sit at home watching b&b/pathoma/sketchy and doing anki.

Use board scores instead of MCAT/GPA for med school admissions. Standardize clinical grades and then use those for residency.


Also in terms of difficulty USMLE > MCAT > COMLEX for me.

 

[Raryn]

Is the high school exit exam a common thing? I did not have this in 2013.

State dependent. CA made it required around the time I graduated high school (2005). I can't remember if I was in the first class where it mattered or in the last class of the pilot program where they were testing it out - it didn't matter, because the test was an absolute joke. Pretty sure most 9th graders could have passed the "exit" exam.

 

[efle]

Is the high school exit exam a common thing? I did not have this in 2013.

I assumed it was a national thing but maybe it's only certain states. I know CA had it. It's a joke, questions like "x + 2 = 4, solve for x"

 

[AnatomyGrey12]

Take the suggestion in context, though. Imagine switching the USMLE, a knowledge-check exam, back to its original purpose as a pass/fail licensure minimum.

Then introduce a different exam, one built to assess clinical reasoning with a tight confidence interval, instead of assessing flashcard recalls with a huge sloppy CI. Let that new test be the percentile based report that residencies screen with.

This isn't some kind of outlandish novel system, either. For heck's sake this is what we do with high schoolers, give them a GED/exit exam to prove they got the minimum, but then use the SAT to actually assess reasoning and aptitude. We even do it on the level of individual courses, having high schoolers take in-class exams to get credit on the transcript, but then offering national AP tests to show mastery on a curve.

I honestly used to be fine with the idea of the current system until I actually got to dedicated. Shocking amounts of the material I was supposed to learn were clinically useless, most of the questions hinged on recall of some esoteric fact, there are wild swings in score between practice tests due to the massive CI, and at least for the upper half of the curve theres absolutely no link I can see between step performance and who functions best on the wards. Hitting my target score didn't even feel good, it doesnt boost your ego, it just gets a big sigh of relief that this stupid-ass hurdle wont get in the way of my match. I get that people want a way to identify themselves as outliers for PDs, but were deluding ourselves if we think this is a good option to do so.

No, the idea of changing USMLE scoring purely to "disrupt the system" is a straight faceplant into horse feces. If the new test has been studied, determined what the outcomes will be and those changes are clearly explained to medical students and applicants years in advance then sure, but doing it just to do it is ridiculous.

I have yet to hear an argument for this that wasn't from someone at one of the elite schools. You guys simply have blinders on with how such a sudden change in USMLE scoring would negatively impact a massive portion of medical students.

You want a sudden change in the USMLE exams? Get rid of Step 2 CS. That test is as much of a money making scam as essential oils.

 

[efle]

No, the idea of changing USMLE scoring purely to "disrupt the system" is a straight faceplant into horse feces. If the new test has been studied, determined what the outcomes will be and those changes are clearly explained to medical students and applicants years in advance then sure, but doing it just to do it is ridiculous.

I have yet to hear an argument for this that wasn't from someone at one of the elite schools. You guys simply have blinders on with how such a sudden change in USMLE scoring would negatively impact a massive portion of medical students.

You want a sudden change in the USMLE exams? Get rid of Step 2 CS. That test is as much of a money making scam as essential oils.

Aren't the people at elite schools the ones most objective in this discussion? Someone that doesn't really need to score well on it anyways to match well, someone that didn't have to sink years into anki and buy into the Step1 as #1 priority?

Like I said earlier, my school sends a whopping 2-3 people into stuff like ortho and derm every year, and half of them just stick around and home match. Plus I hit 250s myself. None of my views are motivated by a desire to replace the Step with name branding/prestige.

 

[Alkain]

Aren't the people at elite schools the ones most objective in this discussion?

They are the ones with the most to gain.

Someone with a 199 Step 1 probably isn't going to match derm/ortho/plastics regardless of if they're from generic MD school #104 or a top 20.

Get rid of the score and instead of
199 step 1 student from Yale vs 260 step student from OSU
its
Yale graduate who passed boards vs OSU graduate who passed boards.

Instead of needing to score well on step to match well, it becomes needing to do well in undergrad/MCAT to get into a t20 school to match well which is going the opposite direction in terms of selecting residents.

 

[efle]

They are the ones with the most to gain.

Someone with a 199 Step 1 probably isn't going to match derm/ortho/plastics regardless of if they're from generic MD school #104 or a top 20.

Get rid of the score and instead of
199 step 1 student from Yale vs 260 step student from OSU
its
Yale graduate who passed boards vs OSU graduate who passed boards.

Instead of needing to score well on step to match well, it becomes needing to do well in undergrad/MCAT to get into a t20 school to match well which is going the opposite direction in terms of selecting residents.

I mean, sure, someone at a big name with an atrocious step score would benefit. But that's quite an extraordinary case, since the most competitive schools boast step1 averages in the 240s these days. The far more common injustice is that a perfectly capable and hardworking OSU student with a longstanding dream of [insert surgical specialty] has their career of choice stolen away because they get a 230.

 

[Alkain]

I mean, sure, someone at a big name with an atrocious step score would benefit. But that's quite an extraordinary case, since the most competitive schools boast step1 averages in the 240s these days. The far more common injustice is that a perfectly capable and hardworking OSU student with a longstanding dream of [insert surgical specialty] has their career of choice stolen away because they get a 230.

Alright, less extreme case then. T20 grad with a 230 vs low tier MD grad with a 230 vs DO grad with a 230 trying to match into Gen Surg. Strip away step scores. T20 grad now has a higher chance than before. Low tier MD grad now has a lower chance than before. DO grad has slim to no chance.

So rather than give the graduate of a low tier MD/any DO the opportunity to compete for a spot using a standardized exam, it would be more fair to immediately crush their chances at any competitive specialty/residency by virtue of which school they attend rather than personal ability?

The far more common injustice is that a perfectly capable and hardworking OSU student with a longstanding dream of [insert surgical specialty] has their career of choice stolen away because they get a 230.

So they did worse than their peers and should be able to take that residency spot from someone else because they are capable and really wanted it?

If you want to think of it another way. I got a 508 on my MCAT but I should really deserve that spot at Yale over that other kid who got a 512 because I went to Cornell and they went to a rural state school. Therefore we should make the MCAT pass fail.

 

[907914]

Alright, less extreme case then. T20 grad with a 230 vs low tier MD grad with a 230 vs DO grad with a 230 trying to match into Gen Surg. Strip away step scores. T20 grad now has a higher chance than before. Low tier MD grad now has a lower chance than before. DO grad has slim to no chance.

So rather than give the graduate of a low tier MD/any DO the opportunity to compete for a spot using a standardized exam, it would be more fair to immediately crush their chances at any competitive specialty/residency by virtue of which school they attend rather than personal ability?

Those same students who have the capacity to perform well on the step exam are also likely students who can perform well in other domains. If you get rid of scored step exams, all of the time and brainpower dedicated towards Step can instead go towards productive research, high pass/honors in all clinical rotations and so on. The effort would just be distributed elsewhere, it wouldn’t just go away. And then all will be exactly as it is now in terms of the benefit one gets from the name of the school, except the student focus can be on their schooling and enjoying being a medical student (as in, actually utilizing the resources available to them that they are pay $60K per year for) instead of their anki, B/B, whatever the hell...

 

[Alkain]

Those same students who have the capacity to perform well on the step exam are also likely students who can perform well in other domains. If you get rid of scored step exams, all of the time and brainpower dedicated towards Step can instead go towards productive research, high pass/honors in all clinical rotations and so on. The effort would just be distributed elsewhere, it wouldn’t just go away. And then all will be exactly as it is now in terms of the benefit one gets from the name of the school, except the student focus can be on their schooling and enjoying being a medical student (as in, actually utilizing the resources available to them that they are pay $60K per year for) instead of their anki, B/B, whatever the hell...

Except you don't have those research opportunities or standardized clinical rotation grades at all schools.

Doing this would immediately (again) put most DO schools as well as MD schools not attached to a University or with decent in house research at a severe disadvantage. You'd be forcing students who have no interest in research to jump through a different hoop for residency. A capable and hardworking OSU student with a longstanding dream of [insert surgical specialty] has their career of choice stolen away because their school didn't have a great research program and only gave out 15% honors.

How would you redistribute time in pre-clinical years 1&2 to clinical years 3&4?

 

[Med Ed]

Honestly, I feel the best way to deal with boards is to get rid of the first two years of medical school and then add a clinical base year where they teach you basic clinical skills so medical school winds up being 3 years total.

I don't see any reason why I had to pay $50k a year to sit at home watching b&b/pathoma/sketchy and doing anki.

Use board scores instead of MCAT/GPA for med school admissions.

I have come to love this idea. There currently aren't enough medical students from affluent families. Enriching the applicant pool with students who can afford to spend 1-2 years doing nothing but prepping for Step 1 would be a significant improvement.

 

[notretired]

I have come to love this idea. There currently aren't enough medical students from affluent families. Enriching the applicant pool with students who can afford to spend 1-2 years doing nothing but prepping for Step 1 would be a significant improvement.

This is a pretty ironic statement given the current emphasis on research/volunteering experiences in med-school admissions. Unlike the above, parental connections can't get one a high s1 score. I'd take the chance on out-grinding privileged students on a free app like anki anyday over not even getting to the starting line.

 

[AnatomyGrey12]

Aren't the people at elite schools the ones most objective in this discussion?

Lol are you serious?

If you get rid of scored step exams, all of the time and brainpower dedicated towards Step can instead go towards productive research, high pass/honors in all clinical rotations and so on.

Right, so you get rid of the standardized comparison and want to replace it with even more emphasis on arbitrary crap that is completely subjective? Most medical student research is garbage, essentially flinging low quality crap at the wall hoping it sticks, and clinical grades are some of the most variable and subjective items on an application.

The effort would just be distributed elsewhere, it wouldn’t just go away. And then all will be exactly as it is now in terms of the benefit one gets from the name of the school, except the student focus can be on their schooling and enjoying being a medical student (as in, actually utilizing the resources available to them that they are pay $60K per year for) instead of their anki, B/B, whatever the hell...

Oh my naive applicant friend....Serious question, what resources do you think you’re paying 60k for?

 

[Med Ed]

Unlike the above, parental connections can't get one a high s1 score.

Not directly, but parental income can secure everything needed to be successful, starting from birth. I envision a world where wealthy parents give their children room, board, and Step resources (tutors, courses, qbanks, etc.) for up to 3 years in order to condition them into MCQ-destroying machines. The less privileged won't stand a chance.

 

[efle]

Alright, less extreme case then. T20 grad with a 230 vs low tier MD grad with a 230 vs DO grad with a 230 trying to match into Gen Surg. Strip away step scores. T20 grad now has a higher chance than before. Low tier MD grad now has a lower chance than before. DO grad has slim to no chance.

So rather than give the graduate of a low tier MD/any DO the opportunity to compete for a spot using a standardized exam, it would be more fair to immediately crush their chances at any competitive specialty/residency by virtue of which school they attend rather than personal ability?



So they did worse than their peers and should be able to take that residency spot from someone else because they are capable and really wanted it?

If you want to think of it another way. I got a 508 on my MCAT but I should really deserve that spot at Yale over that other kid who got a 512 because I went to Cornell and they went to a rural state school. Therefore we should make the MCAT pass fail.

But think about that premise a little more. The t20 grad with a 230 is going to have a great surgical match. Take away the step scores and he's still going to have a great surgical match, nothing's changed for him. If he has the opinion that Step1 is a terrible metric to use as it is currently used, it makes no sense to claim it's because of where he's studying. Besides, when you look at the actual numbers of slots filled by Top 20 med school graduates, it's a teeny tiny sliver. If someone struggles to match general surgery, with or without step scores involved, they're barking up the wrong tree if they try and blame the t20s. The real role of the step1 right now is in allowing the latter groups (typical MD and DO schools' students) to differentiate themselves from one another, not to let them battle against the tiny handful of top-ranked grads in their match.

Someone should be able to compete for that ortho spot whether they got a 230 or a 250, yes. This is not a crazy idea, even the creators of the test itself have explicitly said it's not supposed to be used to screen out the former and favor the latter. This has literally only been a modern phenomenon in the most recent generation of docs, do you think that in 1995 when nobody gave a crap about USMLE scores, it was impossible to select for capable candidates?

I'd honestly be fine with the MCAT getting switched to something like quintiles too. I've looked over the data from the AAMC about how the MCAT correlates with graduation rates and USMLE performance, and there is massive diminishing returns beyond 510+. If you told me a 525 scorer and 515 scorer should have to compete more in other areas of their application, because that difference doesn't make you more likely to graduate, pass boards, or be a better doctor, I'd agree with you.

Lol are you serious?

Look dude I got my 250+, I don't have a dog in this race. You might be able to write off one of my classmates if they made this argument with a 225 and wanted to match neurosurgery. But that ain't me, you can't use it to dismiss me, it's intellectually lazy of you.

Give me an actual argument for why the USMLE writers are wrong, for why it's actually OK to use this test to screen out 230s and favor 250s for a surgical specialty. One that has more substance than "my classmates who bombed step are also dumb on the wards"

 

[Lucca]

No, the idea of changing USMLE scoring purely to "disrupt the system" is a straight faceplant into horse feces. If the new test has been studied, determined what the outcomes will be and those changes are clearly explained to medical students and applicants years in advance then sure, but doing it just to do it is ridiculous.

I have yet to hear an argument for this that wasn't from someone at one of the elite schools. You guys simply have blinders on with how such a sudden change in USMLE scoring would negatively impact a massive portion of medical students.

You want a sudden change in the USMLE exams? Get rid of Step 2 CS. That test is as much of a money making scam as essential oils.

We’ve been over this several times now and neither of us are budging so I’m loathe to reply but there is no change to the exam that comes with changing the scoring. What you are doing is changing the scoring of the exam to more accurately reflect the resolution the exam is actually able to test on.

What would happen after that needs careful thought and consideration. I’m not saying “make Step 1 p/f tomorrow”. What I’ve said is 1) change step 1 to quartiles, 2) develop new and better methods for increasing objective information available to PDs (e.g. SLOEs), 3) cap the number of residency apps you can send.

2 and 3 are debatable and just my knee jerk recommendations based on what I’ve thought about this issue. That said, I’m 10,000% confident precisely *nothing* will be done unless Step 1 scoring changes. All this talk about researching and developing the perfect exam for med students or whatever means absolutely nothing if there’s no impetus for anyone to change what they are already doing. That doesn’t mean that if scoring changes that it ought to go into effect tomorrow. But there needs to be a decision made to catalyze downstream changes in residency selection.

I can appreciate why this discussion has higher stakes for others but frankly I’ve done enough on this site to hopefully have a earned a little “good faith”. To be more specific I think the group most in danger are DO students, I actually think most if not all MD students would benefit from the change. I’ll do whatever I have to do, I just think the current meta screws everyone over for the most part.

 

[notretired]

Not directly, but parental income can secure everything needed to be successful, starting from birth. I envision a world where wealthy parents give their children room, board, and Step resources (tutors, courses, qbanks, etc.) for up to 3 years in order to condition them into MCQ-destroying machines. The less privileged won't stand a chance.

Is this not already the case for the mcat? If s1 becomes the new barrier to entry for med school I don't see how we would be aiding privileged students any more than we currently are. If students are already using free/low cost resources to obliterate step 1/mcat - I don't really see how any of the above would meaningfully shift the #'s at the top end of the curve.

I'd argue that subjective metrics such as free labor harm applicants far more; because one can absolutely crush a mc exam while working night shifts at McDonalds. The sad fact is that despite toeing the line in an academic sense, these applicants are often overlooked because they lack the "holistic" attributes only attainable to the wealthy.

 

[907914]

Oh my naive applicant friend....Serious question, what resources do you think you’re paying 60k for?

I genuinely don’t know. Top schools? All the resources in the world from Nobel laureates to top facilities. Everything else? Same-ole-same-ole

 

[Lawpy]

… I kinda forgot what we're talking about here

Someone said that people are top schools are most objective, which honestly I don't know if that's meant to be sarcastic because otherwise, that's one of the most elitist things I've seen on SDN by far.

 

[efle]

Is this not already the case for the mcat? If s1 becomes the new barrier to entry for med school I don't see how we would be aiding privileged students any more than we currently are. If students are already using free/low cost resources to obliterate step 1/mcat - I don't really see how any of the above would meaningfully shift the #'s at the top end of the curve.

I'd argue that subjective metrics such as free labor harm applicants far more; because one can absolutely crush a mc exam while working night shifts at McDonalds. The sad fact is that despite toeing the line in an academic sense, these applicants are often overlooked because they lack the "holistic" attributes only attainable to the wealthy.

The idea that $$$ can buy you a top-end MCAT or Step score is a myth that non-wealthy people tell themselves to feel less upset about their performance on the curve. I say this as someone who didn't have money for prep classes or whatever else. I paid my registration fee and Uworld subscription with some of my loan money, bought a cheap copy of first aid and an old copy of Pathoma. Anki is free.

 

[efle]

… I kinda forgot what we're talking about here

Someone said that people are top schools are most objective, which honestly I don't know if that's meant to be sarcastic because otherwise, that's one of the most elitist things I've seen on SDN by far.

Idk man, if an HMS student with a 260 says they'd like to see the exam changed to quartiles? That would hurt more than help them, if it even affects them at all. Seems pretty stupid to tell them they can't have an opinion because they go to HMS.