USMLE versus MCAT difficulty?

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

tinyhandsbob

Full Member
2+ Year Member
Joined
Mar 16, 2019
Messages
166
Reaction score
59
To those who have taken both, which one did you find more difficult/stressful and why? I'm particularly interested in hearing from people who had a more average MCAT score (not >515+)

Members don't see this ad.
 
MCAT by far was more stressful. You need to do well in order to have a reasonable chance at med school. In comparison, you are almost guaranteed a residency spot as long as you don't fail the USMLE. This may not be at the residency or specialty of your choice, but you'll at least have an almost guaranteed pathway to becoming a physician.
 
  • Like
Reactions: 16 users
The people that are scoring 90+ percentile on the USMLE and COMLEX at my school had average to below average MCAT scores. Totally 2 different exams.
 
  • Like
  • Wow
Reactions: 16 users
Members don't see this ad :)
Thank the Lord you said that Moko lol.

On the other hand, you do have to think about this. Anyone can technically take the MCAT, but you’re taking those top scorers (who get into medical school) and now competing against them for the best step scores. I’m not a med student, but that’s gotta be a source of stress in that regard
Not to say that simply passing is enough, but when >95% of medical students pass on their first try, I think the odds are very much in your favor ;) .

As a general rule, once you're in medical school, we will do everything we can to ensure that you succeed. For better or for worse, Step scores play a major role in matching, and we know how important match lists are to potential applicants..
 
  • Like
  • Love
  • Haha
Reactions: 11 users
Not to say that simply passing is enough, but when >95% of medical students pass on their first try, I think the odds are very much in your favor ;) .

As a general rule, once you're in medical school, we will do everything we can to ensure that you succeed. For better or for worse, Step scores play a major role in matching, and we know how important match lists are to potential applicants..

I have it on good authority that some medical schools even let you fail a pre-clinical class or two without any record of it whatsoever.
 
  • Like
Reactions: 6 users
I have it on good authority that some medical schools even let you fail a pre-clinical class or two without any record of it whatsoever.

My school does this provided you pass the remediation, but you have to borderline fail a course. If you outright bomb a class, it goes on your record.
 
  • Wow
  • Like
Reactions: 2 users
I got an average MCAT score.
I found step a million times more stressful. It was a lot harder and so much more material was on it. Your entire career rides on it, which really made it stressful for me. At least with the MCAT, you’re not already 2 years into it. If you do badly on the MCAT, you can retake it or even consider other careers. It is a lot harder once you’re in med school and get a bad or failing step 1 score
 
  • Like
  • Wow
Reactions: 10 users
Step 1 is harder due to the increased competition and the fact if you pass you can’t retake it. I think stakes are actually higher as well as you’ve invested so much time/money into medicine already and being relegated to a field you don’t want to work in is the pits
 
  • Like
  • Wow
Reactions: 7 users
I did well on both (equivalent to 524+ on MCAT and a 250+ on USMLE).

The difficulty of the MCAT comes from testing your academic aptitude, rather than your knowledge base. You could study half as much as someone else and beat them by a lot, because the upper end of the bell curve is mostly differentiated by critical thinking/reasoning about new information presented to you in science and verbal passages. An example would be a passage describing the results of an experiment and asking you to interpret them.

The USMLE is an entirely different beast that is first and foremost testing your knowledge base / how much you memorized. Someone that memorized twice as much of First Aid will always score higher than someone who only knows half of it. An example would be a question describing a specific virus, that then asks you whether that virus is a DNA virus, positive sense RNA, or negative sense RNA. Theres no way to really reason out a correct answer, you either know it or you dont.

There are minor exceptions on both exams - the MCAT has a few discrete questions that are pure recall and the USMLE does have a few experiment interpretation questions - but overall that's the clear difference in flavor.

So whether you think it's easy or hard depends on where your strength lies. If you got a top percentile MCAT the USMLE is probably going to feel much worse to prepare for because its largely stuffing huge volumes of rote recall into your head. On the flip side if the MCAT felt extremely hard to study for and score well on, you'll probably find the USMLE feels a lot more fair, because theres a much more direct connection between studying more hours --> learning more information --> scoring higher.

Overall the correlation between exams is about 0.60. You can find many posts online about people who struggled a lot with the MCAT, but that did hundreds of thousands of Anki flashcards to memorize the entirety of First Aid and easily hit 250+ on USMLE.
 
  • Like
  • Wow
Reactions: 15 users
Oh and I'd also say the stressfulness of the Step1/USMLE entirely depends on your specialty of interest. If you want to be a surgical sub specialist (e.g. orthopedics, neurosurg, plastics, ophthalmology, urology, ENT, etc) you need to beat 80-90% of US medical students, which is VERY hard and stressful. Whereas if you are aiming for less competitive/community practice like family med or pediatrics, you really just need to comfortably pass it and can match fine with a below average score.

Edit: One final thing I'll add since premeds might not be aware of it:

The Step1 has become such an overwhelming determinant of your residency match that it's becoming the norm to begin your Step1 studying a year or more ahead of time, usually via flashcards/anki, and many students are prioritizing it over their actual school curriculum. It's becoming such a source of stress and distraction that the major education stakeholder groups (e.g. AMA, AAMC, student representatives, etc) held a convention a few months ago to formally recommend that the NBME consider changing the USMLE grading to Pass/Fail or quartiles, among other recommendations. You can read about it on the INCUS pages here:


It's very possible by the time you guys are taking the Step 1, there wont be any such thing as a 250+ target for competitive specialties any more.
 
Last edited:
  • Like
  • Sad
Reactions: 12 users
As noted, Step 1 is a knowledge test and it frankly has more impact on the rest of your life than the MCAT. All you really need is a good enough MCAT to get into a medical school. I found Step 1 far more stressful.

I also got >90th percentile on the MCAT with minimal studying, but got barely above average on Step 1 with a 6 week study block.
 
Last edited:
  • Like
  • Wow
Reactions: 10 users
For me, they were equivalently stressful, but for different reasons. As noted above, for the MCAT, it's sort of like your entrance ticket (obviously not your only one, but you get the idea). You're not "in the system" yet, and you need to do well to continue on the path to becoming a doctor. Step 1 is stressful because it's a bigger/harder exam than the MCAT and plays a large role in your residency chances. However, I was FAR MORE prepared for Step 1 than the MCAT (because Step 1 theoretically tests what you've been doing for 2 years in medical school, whereas the MCAT tests only a part of what you've been doing in undergrad). For the record, I did about average on the MCAT, but scored significantly above average on Step 1.
 
  • Like
Reactions: 5 users
The USMLE is an entirely different beast that is first and foremost testing your knowledge base / how much you memorized. Someone that memorized twice as much of First Aid will always score higher than someone who only knows half of it. An example would be a question describing a specific virus, that then asks you whether that virus is a DNA virus, positive sense RNA, or negative sense RNA. Theres no way to really reason out a correct answer, you either know it or you dont.

I'm not sure when you took Step 1, but the NBME has overhauled its question-writing process to marginalize items that rely on straight memorization. Each book (section) of USMLE has fewer items than years past, and they are essentially all vignette- or experiment-based, meaning the questions are longer. This has led to an increase in time per question from 72 to 90 seconds. They are also flushing out the classic keywords that have historically made many items straightforward exercises in clue hunting.

Suffice to say that every year Step 1 will more closely resemble an aptitude test, with a large knowledge fund being necessary but not sufficient.
 
  • Like
  • Sad
Reactions: 12 users
Members don't see this ad :)
I have it on good authority that some medical schools even let you fail a pre-clinical class or two without any record of it whatsoever.
If you failed a class and remediated, it goes on your record, and it affects your class rank.
 
SAT is mostly thinking
LSAT is pure thinking
MCAT is in between USMLE and LSAT
USMLE is almost pure memorization

36 MCAT
247 step1
261 step2ck

I go down in percentile based on how memory heavy the exam is aka crystallized knowledge testing vs. fluid intelligence. Also, I liked old MCAT more since more quant thinking with physics. Now more verbal reasoning heavy. But I think hat is a good change. Med school has very little quant thinking
 
Oh and I'd also say the stressfulness of the Step1/USMLE entirely depends on your specialty of interest. If you want to be a surgical sub specialist (e.g. orthopedics, neurosurg, plastics, ophthalmology, urology, ENT, etc) you need to beat 80-90% of US medical students, which is VERY hard and stressful. Whereas if you are aiming for less competitive/community practice like family med or pediatrics, you really just need to comfortably pass it and can match fine with a below average score.

Edit: One final thing I'll add since premeds might not be aware of it:

The Step1 has become such an overwhelming determinant of your residency match that it's becoming the norm to begin your Step1 studying a year or more ahead of time, usually via flashcards/anki, and many students are prioritizing it over their actual school curriculum. It's becoming such a source of stress and distraction that the major education stakeholder groups (e.g. AMA, AAMC, student representatives, etc) held a convention a few months ago to formally recommend that the NBME consider changing the USMLE grading to Pass/Fail or quartiles, among other recommendations. You can read about it on the INCUS pages here:


It's very possible by the time you guys are taking the Step 1, there wont be any such thing as a 250+ target for competitive specialties any more.

Do they really think changing it to quartiles will make a difference? That is the equivalent of changing MCAT scoring from a 45 point scale to a 528 scale. It is arbitrary.
 
  • Hmm
Reactions: 1 user
I'm not sure when you took Step 1, but the NBME has overhauled its question-writing process to marginalize items that rely on straight memorization. Each book (section) of USMLE has fewer items than years past, and they are essentially all vignette- or experiment-based, meaning the questions are longer. This has led to an increase in time per question from 72 to 90 seconds. They are also flushing out the classic keywords that have historically made many items straightforward exercises in clue hunting.

Suffice to say that every year Step 1 will more closely resemble an aptitude test, with a large knowledge fund being necessary but not sufficient.
What initiated the motive to make these changes to the exam? Why is it necessary to make it more of an aptitude exam and less what it was (still mostly is?)?
 
The MCAT was a pain in the (you know)...
 
Thank the Lord you said that Moko lol.

On the other hand, you do have to think about this. Anyone can technically take the MCAT, but you’re taking those top scorers (who get into medical school) and now competing against them for the best step scores. I’m not a med student, but that’s gotta be a source of stress in that regard

But I think the idea is that even if you do OK son the USMLE, you still WILL be a doctor. You WILL be an MD/DO. If you don’t do well on The MCAT, you may not be able to become a physician
 
  • Like
Reactions: 2 users
But I think the idea is that even if you do OK son the USMLE, you still WILL be a doctor. You WILL be an MD/DO. If you don’t do well on The MCAT, you may not be able to become a physician

I’d rather be a bio major working at Starbucks than an in debt surgical prelim resident in rural Nebraska any day
 
  • Wow
  • Like
  • Hmm
Reactions: 4 users
What initiated the motive to make these changes to the exam? Why is it necessary to make it more of an aptitude exam and less what it was (still mostly is?)?
this is a pipe dream of med ed to feel like they test like true thinking subjects like physics,math, quant econ, philosophy, logic, comp sci

wow dark circular lesion instead of escher for anthrax so much thinking.... try to learn only concepts and you will goof so hard. the base is still all heavy heavy memorization

it is still a memorization contest. btw I just took STEP2CK in May.
 
Last edited:
  • Like
Reactions: 5 users
I'm not sure when you took Step 1, but the NBME has overhauled its question-writing process to marginalize items that rely on straight memorization. Each book (section) of USMLE has fewer items than years past, and they are essentially all vignette- or experiment-based, meaning the questions are longer. This has led to an increase in time per question from 72 to 90 seconds. They are also flushing out the classic keywords that have historically made many items straightforward exercises in clue hunting.

Suffice to say that every year Step 1 will more closely resemble an aptitude test, with a large knowledge fund being necessary but not sufficient.
I took it in May. I'll say it had a lot less buzzwords and pure regurgitation than older practice NBMEs, for sure. But it was still much, much more a test of knowledge than reasoning.

When you get a question wrong in something like physics or organic chemistry, it's "oh yeah, that makes sense, I see why I'm wrong and that's right."

With the usmle, even this year's latest edition of new practice NBMEs, it's always: "Well that's a factoid I'd just never learned before. Guess I should spend an hour memorizing which CYP subtypes metabolize each major drug class."
 
  • Like
Reactions: 4 users
this is a pipe dream of med ed to feel like they test like true thinking subjects like physics,math, quant econ, philosophy, logic, comp sci

wow dark circular lesion instead of escher for anthrax so much thinking.... try to learn only concepts and you will goof so hard. the base is still all heavy heavy memorization

it is still a memorization contest. btw I just took STEP2CK in May.
There definitely were a lot more experiment interpretation / reasoning questions on my actual Step1 than any of the old NBMEs, and a lot less buzzwords.

But yeah in stark contrast to tests like SAT or MCAT, or even college coursework in prereqs, it's definitely still about who crams in the most knowledge.
 
  • Like
Reactions: 1 users
Do they really think changing it to quartiles will make a difference? That is the equivalent of changing MCAT scoring from a 45 point scale to a 528 scale. It is arbitrary.
It changes what the upper end target will be. Right now the typical goal for a competitive specialty in a desirable location is top 10-15% of test takers. Changing the system to quartiles would make it so "Quartile 1" was the norm to match these, literally twice as easy a target to hit.
 
  • Like
Reactions: 5 users
If you failed a class and remediated, it goes on your record, and it affects your class rank.

Not at every school.
But I think the idea is that even if you do OK son the USMLE, you still WILL be a doctor. You WILL be an MD/DO. If you don’t do well on The MCAT, you may not be able to become a physician

You'll realize later on that simply becoming a doctor isn't good enough. There are honestly fields of medicine I would rather leave medicine all together than have to do on a daily basis. Your USMLE is vital to being able to have options, and you are taking the test against probably the smartest overall testing pool in the world. Even hitting the median is honestly much harder than most pre-meds realize. The MCAT is largely a reading comprehension/problem solving test, whereas USMLE is a knowledge based test except the pool of knowledge they can use to design questions is essentially endless.

90-95 Percentile MCAT
Mid 240s Step 1.
 
  • Like
Reactions: 2 users
Not at every school.


You'll realize later on that simply becoming a doctor isn't good enough. There are honestly fields of medicine I would rather leave medicine all together than have to do on a daily basis. Your USMLE is vital to being able to have options, and you are taking the test against probably the smartest overall testing pool in the world. Even hitting the median is honestly much harder than most pre-meds realize. The MCAT is largely a reading comprehension/problem solving test, whereas USMLE is a knowledge based test except the pool of knowledge they can use to design questions is essentially endless.

90-95 Percentile MCAT
Mid 240s Step 1.
Something else to highlight is that the Step1 has a massive confidence interval and you aren't allowed to retake it. More specifically, its +/- EIGHT POINTS just to have 66% confidence.

That's insane when you consider the stakes. Someone who is a true 245 and interested in a surgical subspecialty can be highly competitive or nearly out of the running depending on whether they have a good luck and hit mid 250s, versus bad luck and hit mid 230s.

Swings that big happening to 1/3rd of the pool is NUTS.
 
  • Wow
Reactions: 1 users
More specifically, its +/- EIGHT POINTS just to have 66% confidence.
Instead of P/F, lets just have students take Step 1 twice and then average the scores./s

Serious question, how can a test be valid when the confidence interval can have you be anywhere across an entire 20-30 percentile range?
 
It changes what the upper end target will be. Right now the typical goal for a competitive specialty in a desirable location is top 10-15% of test takers. Changing the system to quartiles would make it so "Quartile 1" was the norm to match these, literally twice as easy a target to hit.
Except a student who has studied 400 hours does not know whether their current level of knowledge puts them at 74% (second quartile) or 76% (first quartile), so the incentive will be to study an additional 100 hours (or however much more time they can squeak out) to ensure they haven't missed the 1st quartile by a few percent. The only way to really dial it back is to make it a true P/F. Even then being how important the test is to one's career, the incentive is to maximize studying....nobody wants to fail by a few percent so it's hard to justify leaving any amount of free time unstudied.

One way to do it would be to develop a blind composite score made up of class rank + USMLE scores. Residencies would be blind to the specific elements of the composite score so there would not be as much weight on the USMLE. However, that pressure would get distributed over to the grades/class rank a bit. But maybe that's more desirable because its more balanced.
 
  • Like
Reactions: 1 user
Instead of P/F, lets just have students take Step 1 twice and then average the scores./s

Serious question, how can a test be valid when the confidence interval can have you be anywhere across an entire 20-30 percentile range?
Yeah the honest answer is, it cant be. With a 66% CI of 16 points, that means it's a 95% CI of 32 points. My own 95% CI spanned the entire range from 230s to 270s, so all they can confidently say is that I'm somewhere between average and top 1%.

Pretty crazy to think 1 in 20 people is scoring wildly above or below by 15+ points. But that's inevitable when you try to assess years of learning with a single 7 hour exam. They'll often ask only a single question about an entire subject area (e.g. I had only one question about inherited metabolic disorders). Theres just no way for that to paint an accurate picture of the test takers knowledge on a reliable basis.

Gotta remember this was intended to be a licensure exam like the bar exam for attornies, with high discriminatory ability being most important around the pass/fail threshold. The way it now gets used to differentiate 230 vs 240 vs 250 was not what it was built for.

Med Ed I'm sure can comment much more usefully about it.
 
  • Like
Reactions: 4 users
One way to do it would be to develop a blind composite score made up of class rank + USMLE scores.
But maybe that's more desirable because its more balanced.


In every one of these threads a new idea gets proffered and they get worse and worse. Using grades/class rank is a horrible idea because of how unstandardized it is. Some schools don’t even rank at all.

I have made my stance on this matter very clear in multiple threads and I have yet to see any sort of change proposed that is better than the current system we have. I find the arguments for a change in USMLE scoring to be extremely uncompelling.
 
  • Like
Reactions: 1 users
Serious question, how can a test be valid when the confidence interval can have you be anywhere across an entire 20-30 percentile range?

I'm not a psychometrician, but I have learned enough to be dangerous. In a nutshell, Step 1 was designed to discriminate in the score ranges close to the passing threshold. This is because the exam's primary purpose is to determine whether an individual either has or has not demonstrate the minimal level of medical knowledge permissible for licensing. Hence its full name: The United States Medical Licensing Exam.

Using the scoring scale for other purposes, such as resident selection, are secondary uses for which the exam was not designed. This is why the confidence intervals are terrible: the test wasn't built to discriminate anything useful above passing. I do not know the ideal solution to the current mess, but the proponents of the status quo are effectively content with a dysfunction system built from jury-rigged parts. Many of them can be forgiven, as they do not have longitudinal experience with the education environment surrounding USMLE and the match.
 
  • Like
Reactions: 5 users
Except a student who has studied 400 hours does not know whether their current level of knowledge puts them at 74% (second quartile) or 76% (first quartile), so the incentive will be to study an additional 100 hours (or however much more time they can squeak out) to ensure they haven't missed the 1st quartile by a few percent. The only way to really dial it back is to make it a true P/F. Even then being how important the test is to one's career, the incentive is to maximize studying....nobody wants to fail by a few percent so it's hard to justify leaving any amount of free time unstudied.

One way to do it would be to develop a blind composite score made up of class rank + USMLE scores. Residencies would be blind to the specific elements of the composite score so there would not be as much weight on the USMLE. However, that pressure would get distributed over to the grades/class rank a bit. But maybe that's more desirable because its more balanced.
It's true you might get screwed by falling slightly short of a quartile cutoff, but honestly you're much more likely to be helped than hurt by it. When you look at the NRMP data, theres not a lot of sub-74th percentile scores applying for those specialties anyways. Especially the most competitive places where you really wanted a 250+, those are the only ones that might demand top quartile. The more accessible programs that were matching lots of lower scores will just switch to cutting off the bottom half and matching Quartile 1 and 2 folks.

I wouldn't get class rank involved because schools vary so much in student bodies and curriculums. Someone who is median at the Vandy 1 year program might be top quartile if they'd instead done a more traditional curriculum against the U Tennessee student body.
 
Last edited:
  • Like
Reactions: 3 users
In every one of these threads a new idea gets proffered and they get worse and worse. Using grades/class rank is a horrible idea because of how unstandardized it is. Some schools don’t even rank at all.

I have made my stance on this matter very clear in multiple threads and I have yet to see any sort of change proposed that is better than the current system we have. I find the arguments for a change in USMLE scoring to be extremely uncompelling.
On the flip side, I've yet to hear any good arguments in defense of the USMLE in it's current format and current role. It makes absolutely no sense that someone needs to beat 80% of others on that exam to become a specialized surgeon. And I've yet to see anything on rotations that would back up the idea that higher step 1 = better resident. If I had to try and pick who had the highest and lowest step1 scores out of the Hopkins residents Ive rotated with, I would be 100% guessing. Vast swaths of what I learned to score well was useless basic science background in biochem, immunology, drug mechanisms, microbio nitty gritty, etc and none of that came up ONCE on the medicine floor.
 
  • Like
  • Wow
Reactions: 3 users
On the flip side, I've yet to hear any good arguments in defense of the USMLE in it's current format and current role. It makes absolutely no sense that someone needs to beat 80% of others on that exam to become a specialized surgeon. And I've yet to see anything on rotations that would back up the idea that higher step 1 = better resident. If I had to try and pick who had the highest and lowest step1 scores out of the Hopkins residents Ive rotated with, I would be 100% guessing. Vast swaths of what I learned to score well was useless basic science background in biochem, immunology, drug mechanisms, microbio nitty gritty, etc and none of that came up ONCE on the medicine floor.

Maybe for surgical specialities, they should just do a lifelike version of Operation (the kid's game) and you are scored on how many times you get buzzed
 
  • Like
Reactions: 2 users
Since this thread was about MCAT vs USMLE, let me add the confidence interval information for the former:

"MCAT total scores are reported with a 68% confidence band of plus or minus two points, and MCAT section
scores are reported with 68% confidence bands of plus or minus one point. Adding and subtracting two
points to an MCAT total score of 500, for example, defines a confidence band that begins at 498 and goes
to 502. This means that in 68% of cases, the true score for an examinee with a reported score of 500 lies
within the band that goes from 498 to 502."

The 66% CI for the MCAT is plus or minus 2 points!!!

The same CI for the USMLE is plus or minus 8 points! With no ability to retake it.

So think about how often we see people retaking the MCAT to get a better score, and then think about how messed up it would be if it was 4x more volatile, and you had to live with the first one.

(The two are surprisingly comparable with both containing most of their curve in a 50-60 point range)
 
  • Like
  • Wow
Reactions: 5 users
On the flip side, I've yet to hear any good arguments in defense of the USMLE in it's current format and current role. It makes absolutely no sense that someone needs to beat 80% of others on that exam to become a specialized surgeon. And I've yet to see anything on rotations that would back up the idea that higher step 1 = better resident. If I had to try and pick who had the highest and lowest step1 scores out of the Hopkins residents Ive rotated with, I would be 100% guessing. Vast swaths of what I learned to score well was useless basic science background in biochem, immunology, drug mechanisms, microbio nitty gritty, etc and none of that came up ONCE on the medicine floor.

You need to beat 80% of others not because you need to score that high to be a surgeon, but because it’s the only standardized comparison PDs and use and the residency spots are so competitive. Same way high MCAT scores don’t mean you’ll be a great doctor but the elite schools sure seem to have really high MCAT averages. Step scores are literally the only standardized thing on a residency application.

You might not have seen it at Hopkins but students who score better, on average, are better students on the wards. I have seen this over and over again already in my first two months of rotations and it’s not just my anecdote, this sentiment is echoed time and time again by attendings and residents in multiple spheres. Good students are good students. This isn’t to say there aren’t people who do better once they hit the wards, but to deny there isn’t the general overall trend is disingenuous.

Arguments for the current model include it allows a direct comparison which allows students at schools not named Hopkins to show they can compete and perform just as well as the students at Hopkins, and Step 1 is taken early enough that if you don’t perform as well as you wanted you still have lots of time to consider your options. Getting rid of a scored Step 1 just means a different test will be created that will be taken much closer to applications, residency programs will select more for school prestige than they already do, and place more emphasis on things even more arbitrary and stupid than Step 1 like research, clinical grades, and EC junk.

Quintiles are the only reasonable option I’ve seen.
 
Wait is anyone taking Step 1 confidence intervals seriously? I thought they had little value from a practical sense because people are apparently ok in not being 95% confident when making decisions.
 
  • Like
Reactions: 2 users
Wait is anyone taking Step 1 confidence intervals seriously? I thought they had little value from a practical sense because people are apparently ok in not being 95% confident when making decisions.

Ya but this is precisely our conundrum tho. Everyone agrees on what the problems with step scoring are, the differences are in what to do about it. Broadly, one camp says you can’t do anything that won’t be harmful to the vast majority of students; another says something has to be done about step scoring or nothing will ever be done to improve residency selection. Also everyone is operating at maximum cynicism because the strongest predictors of what your position is in this debate are: 1) your step score 2) your specialty of interest and 3) your med school in that order

It’s like the “is this your wallet?” Spongebob meme.
 
  • Like
Reactions: 1 users
Since this thread was about MCAT vs USMLE, let me add the confidence interval information for the former:

"MCAT total scores are reported with a 68% confidence band of plus or minus two points, and MCAT section
scores are reported with 68% confidence bands of plus or minus one point. Adding and subtracting two
points to an MCAT total score of 500, for example, defines a confidence band that begins at 498 and goes
to 502. This means that in 68% of cases, the true score for an examinee with a reported score of 500 lies
within the band that goes from 498 to 502."

The 66% CI for the MCAT is plus or minus 2 points!!!

The same CI for the USMLE is plus or minus 8 points! With no ability to retake it.

So think about how often we see people retaking the MCAT to get a better score, and then think about how messed up it would be if it was 4x more volatile, and you had to live with the first one.

(The two are surprisingly comparable with both containing most of their curve in a 50-60 point range)

Is there a way to reduce the Step 1 CI and if so how?
 
SAT is mostly thinking
LSAT is pure thinking
MCAT is in between USMLE and LSAT
USMLE is almost pure memorization

36 MCAT
247 step1
261 step2ck

I go down in percentile based on how memory heavy the exam is aka crystallized knowledge testing vs. fluid intelligence. Also, I liked old MCAT more since more quant thinking with physics. Now more verbal reasoning heavy. But I think hat is a good change. Med school has very little quant thinking
I agree with this assessment. I did better than you on the MCAT and worse on the USMLEs. The MCAT, particularly VR, had a lot of aptitude component you could reason your way through. There's plenty of people that, without dedicated studying, could pull off a 30 (~508) on the MCAT having just taken the prerequisites but not really having any dedicated study time. It's been 11 years, and I'm fairly certain if you gave me a couple weeks to review my physics formulas and my ochem I could probably pull off a respectable MCAT score even today.

There's no one that could ever pull off an equivalent percentile on the USMLE (240+) without study resources explicitly dedicated to the task. The knowledge base is just too large and the specific recall component is too strong. They're trying to make it more reasoning based but it's reasoning based on nit picked facts. It's only been 8 years, but I'm pretty sure if you were to give me a few weeks to review, *might* barely pass Step 1. Hell, I might fail without a month or more dedicated time. Step 2 and Step 3 are much more clinically relevant and less focused on the minutia, so I'd probably do very well on that if you gave me a few weeks to review my peds and obstetrics (I'm an adult subspecialist).

(Same thing with the SAT - I'm fairly certain I could get a similar score today to what I got 15 years ago - might need to spend a few days just dusting out cobwebs for algebra, but I remember that better than I remember specific enzymes for biochem)
 
You need to beat 80% of others not because you need to score that high to be a surgeon, but because it’s the only standardized comparison PDs and use and the residency spots are so competitive. Same way high MCAT scores don’t mean you’ll be a great doctor but the elite schools sure seem to have really high MCAT averages. Step scores are literally the only standardized thing on a residency application.

You might not have seen it at Hopkins but students who score better, on average, are better students on the wards. I have seen this over and over again already in my first two months of rotations and it’s not just my anecdote, this sentiment is echoed time and time again by attendings and residents in multiple spheres. Good students are good students. This isn’t to say there aren’t people who do better once they hit the wards, but to deny there isn’t the general overall trend is disingenuous.

Arguments for the current model include it allows a direct comparison which allows students at schools not named Hopkins to show they can compete and perform just as well as the students at Hopkins, and Step 1 is taken early enough that if you don’t perform as well as you wanted you still have lots of time to consider your options. Getting rid of a scored Step 1 just means a different test will be created that will be taken much closer to applications, residency programs will select more for school prestige than they already do, and place more emphasis on things even more arbitrary and stupid than Step 1 like research, clinical grades, and EC junk.

Quintiles are the only reasonable option I’ve seen.
I suppose I just havent rotated with people at the bottom end of the curve for comparison, the recent average here was high 240s with a pretty narrow std dev. Maybe a 250 looks a lot better next to a 210. Similarly to match Big 4 you cant be at the bottom end.

So really what I know is that among my friends and peers and the people training me, suggesting a 255 will do better for their patients than a 240 is absolutely laughable. Yet, that has a huge impact on matching competitive specialties. Add into this that they could both be true 248s and one got a lot luckier, which happens 1/3rd of the time, and it looks to me like a system that absolutely cannot be allowed to continue in it's current form.

Agree that my favorite fix is a switch to big bins like quartiles.
 
  • Like
Reactions: 1 user
Medicine will always be that rat race since around 70-80% of us are type A people... And the people in charge do not try to make things any better.
 
  • Wow
Reactions: 1 user
Medicine will always be that rat race since around 70-80% of us are type A people... And the people in charge do not try to make things any better.
Credit where its due: the dozens of schools that have been switching preclinical to Pass/Fail are sparing thousands of students from years of unnecessary stress, gunning and misery. That was probably considered an outlandish change at the time it was first being proposed, too.
 
  • Like
Reactions: 1 user
You need to beat 80% of others not because you need to score that high to be a surgeon, but because it’s the only standardized comparison PDs and use and the residency spots are so competitive. Same way high MCAT scores don’t mean you’ll be a great doctor but the elite schools sure seem to have really high MCAT averages. Step scores are literally the only standardized thing on a residency application.

You might not have seen it at Hopkins but students who score better, on average, are better students on the wards. I have seen this over and over again already in my first two months of rotations and it’s not just my anecdote, this sentiment is echoed time and time again by attendings and residents in multiple spheres. Good students are good students. This isn’t to say there aren’t people who do better once they hit the wards, but to deny there isn’t the general overall trend is disingenuous.

Arguments for the current model include it allows a direct comparison which allows students at schools not named Hopkins to show they can compete and perform just as well as the students at Hopkins, and Step 1 is taken early enough that if you don’t perform as well as you wanted you still have lots of time to consider your options. Getting rid of a scored Step 1 just means a different test will be created that will be taken much closer to applications, residency programs will select more for school prestige than they already do, and place more emphasis on things even more arbitrary and stupid than Step 1 like research, clinical grades, and EC junk.

Quintiles are the only reasonable option I’ve seen.

What is your metric for "better" on wards? If it's regurgitation of factoids to answer pimp questions then I would imagine there is a heavy correlation btw step scores and performance in this area. I'm vehemently against step going p/f for most of the reasons you've identified; it screws over students from lower tier schools (along with students who chose their state school for financial reasons). With that said, much like the mcat I don't think Step1 meaningfully identifies who will be a good doctor vs. not - there's just too many subjective qualities that factor in - but there's no way to rely solely on these qualities without leaving the door open to nepotism etc.

URM stats basically prove that beyond a certain point mcat/gpa can't identify meaningful differences in student performance in medical school and beyond. I imagine the same holds true for step; might be able to see it in 210 vs. 250 student, but can't imagine the you'd see it in 230-240-250 folk - especially once they have moved on to clinical practice.
 
  • Like
Reactions: 1 user
I suppose I just havent rotated with people at the bottom end of the curve for comparison, the recent average here was high 240s with a pretty narrow std dev. Maybe a 250 looks a lot better next to a 210. Similarly to match Big 4 you cant be at the bottom end.

So really what I know is that among my friends and peers and the people training me, suggesting a 255 will do better for their patients than a 240 is absolutely laughable. Yet, that has a huge impact on matching competitive specialties. Add into this that they could both be true 248s and one got a lot luckier, which happens 1/3rd of the time, and it looks to me like a system that absolutely cannot be allowed to continue in it's current form.

Agree that my favorite fix is a switch to big bins like quartiles.

Agree with all of these points, but it's just much easier to say this coming from a top-tier school. The accuracy of the test needs to be changed to fit what it is being used for; even if that means a multiple day exam.
 
  • Like
Reactions: 1 user
What is your metric for "better" on wards? If it's regurgitation of factoids to answer pimp questions then I would imagine there is a heavy correlation btw step scores and performance in this area. I'm vehemently against step going p/f for most of the reasons you've identified; it screws over students from lower tier schools (along with students who chose their state school for financial reasons). With that said, much like the mcat I don't think Step1 meaningfully identifies who will be a good doctor vs. not - there's just too many subjective qualities that factor in - but there's no way to rely solely on these qualities without leaving the door open to nepotism etc.

URM stats basically prove that beyond a certain point mcat/gpa can't identify meaningful differences in student performance in medical school and beyond. I imagine the same holds true for step; might be able to see it in 210 vs. 250 student, but can't imagine the you'd see it in 230-240-250 folk - especially once they have moved on to clinical practice.
Actually one of the major points brought up by the INCUS convention is that URM differences do continue to exist in the Step exams and residency match. One of the positive side effects of a pass/fail switch would be that way, way more minority med students would now stand a chance at surgical specialty matches.

Also, why does this conversation always pretend the USMLE has always been like this??? Go back to the 90s and nobody cared about your score, go back any further and the score literally doesn't exist. It's not like they had a hard time identifying highly competent students to fill desirable residency slots twenty years ago.
 
  • Like
Reactions: 2 users
Actually one of the major points brought up by the INCUS convention is that URM differences do continue to exist in the Step exams and residency match. One of the positive side effects of a pass/fail switch would be that way, way more minority med students would now stand a chance at surgical specialty matches.

Also, why does this conversation always pretend the USMLE has always been like this??? Go back to the 90s and nobody cared about your score, go back any further and the score literally doesn't exist. It's not like they had a hard time identifying highly competent students to fill desirable residency slots twenty years ago.

I think a major problem is the fact that rotations and the amount of responsibility granted to students has changed significantly between the 90's and today. It's a bit easier to identify the good from bad apples in a subjective sense when students are actually granted the opportunity to prove themselves on the day to day.

Edit: also didn't know that about URM's. If the theory holds true that socioeconomic status/background is attributable to URM differences on the mcat/gpa front, I can't see why this trend would continue into medical school - where all students are essentially exposed to the same curriculum.
 
Last edited:
  • Like
Reactions: 1 user
Top