Is the USMLE a reflection of intelligence?

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

PremedSurvivor

Full Member
10+ Year Member
Joined
Jun 13, 2011
Messages
145
Reaction score
40
So bit of a philosophical question here. I understand residency programs need a way of sifting through tons of applications. But just from an objective standpoint, is the USMLE a reflection of intelligence?

I recently took Step 1 (~8 weeks of preparation) and got a 250. There's a guy in our class (amazingly clever, invented something pretty remarkable and got it patented in undergrad) who surprised me by scoring a 240; nothing wrong with it, I just anticipated he would have gotten a 260 easy. Then, someone else in our class who can barely hold a conversation and looks perpetually stoned, got a 270.

Am I bitter? Haha probably :p! But actually, do you guys think the USMLE is a measure of intelligence? Or a predictor of a good clinician?

Members don't see this ad.
 
It's a snapshot of academic performance and medical knowledge. A lot of factors go into performance on test day. Some extremely smart people can just have a bad day and not score to their highest potential. Some unassuming people may surprise you with a solid performance. Some people can be incredibly book smart and then crash and burn in a clinical setting.
 
  • Like
Reactions: 3 users
Am I bitter? Haha probably :p! But actually, do you guys think the USMLE is a measure of intelligence? Or a predictor of a good clinician?

It is obviously a reflection of one aspect of intelligence. But it's sure as hell not an IQ test.

And it absolutely is not a predictor of a good clinician. There are far too many variables it doesn't measure that make a good clinician.
 
  • Like
Reactions: 1 users
Members don't see this ad :)
It's a memorization test with a bit of reading comprehension thrown in. Doesn't require much critical thinking, just knowledge.
 
  • Like
Reactions: 1 user
It tells me how well you can do on a multiple choice test. Which turns out to be a useful skill, as I need my residents to pass the boards.
 
  • Like
Reactions: 20 users
100% not a predictor of good clinical performance.

There's some baseline intelligence level needed to score well on standardized tests. There's also probably a cap each person has on scoring well on standardized tests.

It's not the be-all, end-all but it certainly gives you some idea about a candidate's intelligence and preparation (i.e. effort, ability to develop test taking skills, gross time studying).
 
It tells me how well you can do on a multiple choice test. Which turns out to be a useful skill, as I need my residents to pass the boards.
Fair enough! But if it's just about passing, does a 240 vs 250 vs 260 vs 270 really tell you anything about intelligence, if they all pass?
 
Only on SDN would someone bitch about getting an 250 on Step 1....
dat humble brag tho.

No don't get me wrong! I'm very happy with my score, and I absolutely should be grateful. I was just surprised at how people ended up. Like I said, I imagined the first person with the invention would get something crazy high.

Maybe it comes down to endurance? It IS an 8-hour exam. Or maybe I'm just really bad at reading people :p
 
No don't get me wrong! I'm very happy with my score, and I absolutely should be grateful. I was just surprised at how people ended up. Like I said, I imagined the first person with the invention would get something crazy high.

Maybe it comes down to endurance? It IS an 8-hour exam. Or maybe I'm just really bad at reading people :p

It's likely the latter. The sooner you stop analyzing other people and their performance and focus on your own, the happier and more productive you will be.
 
  • Like
Reactions: 1 users
100% not a predictor of good clinical performance.

There's some baseline intelligence level needed to score well on standardized tests. There's also probably a cap each person has on scoring well on standardized tests.

It's not the be-all, end-all but it certainly gives you some idea about a candidate's intelligence and preparation (i.e. effort, ability to develop test taking skills, gross time studying).

Interesting! I'm curious, what would you consider a good predictor of clinical performance then? Or is there one at all? Because third year can also have a lot of variability in terms of graders and or shy/ confident students.
 
It's likely the latter. The sooner you stop analyzing other people and their performance and focus on your own, the happier and more productive you will be.

Haha ouch! Very little faith in my ability to read people, huh? Not without good reason, I guess. But good advice, duly noted.
 
Members don't see this ad :)
>255 and I'll be the first to say it doesn't measure intelligence for ****. MCAT was much closer to an IQ test in my opinion; Step 1 tested pure memorization -- I wouldn't even say it required good reading comprehension skills. Even the hardest of the 3rd order questions usually only get missed because of silly stuff, like knowing the gene but forgetting which protein it encodes.
 
  • Like
Reactions: 6 users
No way, and this is coming from someone who got very good scores #humblebrag

The test is info regurg. Tests your memory and work ethic.
 
  • Like
Reactions: 2 users
Step 1 was intended to provide a test of basic science knowledge competency originally, e.g. will this medical student become an intern and resident and be able to care for patients? Now it's used to stratify applicants, because there aren't a lot of objective ways of comparing students from different schools, different curricula, and different grading schemes.
 
  • Like
Reactions: 2 users
Nope.
Many who failed Step 1 the first time turned out to be excellent doctors.
 
  • Like
Reactions: 1 users
So bit of a philosophical question here. I understand residency programs need a way of sifting through tons of applications. But just from an objective standpoint, is the USMLE a reflection of intelligence?

I recently took Step 1 (~8 weeks of preparation) and got a 250. There's a guy in our class (amazingly clever, invented something pretty remarkable and got it patented in undergrad) who surprised me by scoring a 240; nothing wrong with it, I just anticipated he would have gotten a 260 easy. Then, someone else in our class who can barely hold a conversation and looks perpetually stoned, got a 270.

Am I bitter? Haha probably :p! But actually, do you guys think the USMLE is a measure of intelligence? Or a predictor of a good clinician?

No.
 
  • Like
Reactions: 1 user
So bit of a philosophical question here. I understand residency programs need a way of sifting through tons of applications. But just from an objective standpoint, is the USMLE a reflection of intelligence?

I recently took Step 1 (~8 weeks of preparation) and got a 250. There's a guy in our class (amazingly clever, invented something pretty remarkable and got it patented in undergrad) who surprised me by scoring a 240; nothing wrong with it, I just anticipated he would have gotten a 260 easy. Then, someone else in our class who can barely hold a conversation and looks perpetually stoned, got a 270.

Am I bitter? Haha probably :p! But actually, do you guys think the USMLE is a measure of intelligence? Or a predictor of a good clinician?
Intelligence helps, but it's a multifactorial exam. Memory is a major component of the USMLE, but not a factor in general intelligence. The USMLE certainly doesn't determine who is a good clinician though, just how good of a baseline a clinician is building from.
 
  • Like
Reactions: 1 users
So bit of a philosophical question here. I understand residency programs need a way of sifting through tons of applications. But just from an objective standpoint, is the USMLE a reflection of intelligence?

I recently took Step 1 (~8 weeks of preparation) and got a 250. There's a guy in our class (amazingly clever, invented something pretty remarkable and got it patented in undergrad) who surprised me by scoring a 240; nothing wrong with it, I just anticipated he would have gotten a 260 easy. Then, someone else in our class who can barely hold a conversation and looks perpetually stoned, got a 270.

Am I bitter? Haha probably :p! But actually, do you guys think the USMLE is a measure of intelligence? Or a predictor of a good clinician?
It's aptitude test, not intelligence test. You gotta work for it.
 
  • Like
Reactions: 1 users
>255 and I'll be the first to say it doesn't measure intelligence for ****. MCAT was much closer to an IQ test in my opinion; Step 1 tested pure memorization -- I wouldn't even say it required good reading comprehension skills. Even the hardest of the 3rd order questions usually only get missed because of silly stuff, like knowing the gene but forgetting which protein it encodes.
I've actually heard that about the MCAT before... why do you think that is? Perhaps because it involved more critical thinking (verbal, science, math etc) ?
 
Nope.
Many who failed Step 1 the first time turned out to be excellent doctors.

I doubt that this is the case unless you're talking about FM in which being a good listener and communicator play a much bigger role in term of pt care at the primary level. It doesn't take much to pass Step 1; you just need to get like 60-64% of the quests corrected.
 
Intelligence helps, but it's a multifactorial exam. Memory is a major component of the USMLE, but not a factor in general intelligence. The USMLE certainly doesn't determine who is a good clinician though, just how good of a baseline a clinician is building from.
Memory is not a factor in general intelligence? Where'd you get that idea from? Most of those intelligence tests that psychologists perform have significant memory components to them, and are correlated with IQ.
 
Step 1 was intended to provide a test of basic science knowledge competency originally, e.g. will this medical student become an intern and resident and be able to care for patients? Now it's used to stratify applicants, because there aren't a lot of objective ways of comparing students from different schools, different curricula, and different grading schemes.
As @akinetopsia was suggesting the exam was designed to meet the needs of state licensure boards in determining competence as a physician. It was not initially intended as a numerical indicator to be used for resident selection. USMLE scores do have a low to moderate correlation with future clinical and examination performance, though there is little evidence that small differences in scores sometimes used to distinguish among students or residency applicants relate to subsequent performance (though this data was from 1993).

So, I would argue that the difference between a 260 and 270 (and even 240-270) means little about competence or performance as a physician.
 
  • Like
Reactions: 1 users
Memory is not a factor in general intelligence? Where'd you get that idea from? Most of those intelligence tests that psychologists perform have significant memory components to them, and are correlated with IQ.

Different types of memory.

Working memory (for computer analogy, RAM) very correlated with IQ.

Long term memory (hard drive) is not.

The latter is more important for USMLE.

Another way to think about it is that if it were open book, almost everyone would do well on the USMLE. That is not true for the MCAT.
 
  • Like
Reactions: 3 users
to some degree. more about how well you memorize.
 
Memory is not a factor in general intelligence? Where'd you get that idea from? Most of those intelligence tests that psychologists perform have significant memory components to them, and are correlated with IQ.
Not long-term memory. Working and short-term memory is what is primarily tested in IQ tests. It's a very different thing.
 
I've actually heard that about the MCAT before... why do you think that is? Perhaps because it involved more critical thinking (verbal, science, math etc) ?
There's little critical thinking in MCAT. It's test-taking skills + reading comprehension skills + some knowledge.
 
USMLE is a good test of memory, endurance in an 8 hour exam and ability to work hard. It will also punish you if you can't get your brain in order for a marathon i.e little things like getting enough sleep the night before, taking required breaks etc. You need some intelligence but not an IQ of 160 to do well on the test. None of the absolute *****s in my class did well but some of the supergeniuses did poorly because all they did was to party with the ladies and spend their medical school getting stoned.
What USMLE does not judge well is your presentation skills or the ability to personally wipe the professor/attendings a.. . Some students/ residents are really great at brown nosing and they do shine all the way through residency despite meh scores although they may not be very good physicians afterwards.
I haven't seen too many students who absolutely crushed the boards and were terrible physicians.
 
  • Like
Reactions: 1 users
Different types of memory.

Working memory (for computer analogy, RAM) very correlated with IQ.

Long term memory (hard drive) is not.

The latter is more important for USMLE.

Another way to think about it is that if it were open book, almost everyone would do well on the USMLE. That is not true for the MCAT.
Not long-term memory. Working and short-term memory is what is primarily tested in IQ tests. It's a very different thing.
Working memory is not completely independent of long-term memory, rather they work in concert with one another. Working memory is about the interaction b/w already stored info (long term memory) with the new sensory information you are being presented. So with working memory, you are selecting new sensory info, aligning it with your previous knowledge from long-term memory, and then making decisions based on that. It is also the gate-keeper to long term memory.

Someone with a great long-term memory will be more likely have a great working memory. For example: I tell you to memorize the number set 91119893101990. Someone with vast knowledge base will be able to recognize the dates as the day of the fall of the Berlin Wall: 9 November 1989 and the day of reunification: 3 October 1990. Those tables in FA and Pathoma are infinitely easier to memorize if you have a strong pre-existing knowledge base.

I believe this to be the reason why a high vocabulary is correlated with high IQ. People who memorize quickly may only need to be exposed to a new word 3-4 times to commit it to memory. Those people have high IQs. People that need to be exposed to the word 30-40 times to remember it will not have high vocab, and may have lower IQ.

I'm not saying USMLE is a good measure of IQ. I think MCAT was better at that. And I think SAT was even more strongly correlated with IQ than MCAT. But I do think there is some correlation with all of these tests and IQ. Does it make a significant difference in your ability to treat patients? No.
 
Last edited:
Working memory is not completely independent of long-term memory, rather they work in concert with one another. Working memory is about the interaction b/w already stored info (long term memory) with the new sensory information you are being presented. So with working memory, you are selecting new sensory info, aligning it with your previous knowledge from long-term memory, and then making decisions based on that. It is also the gate-keeper to long term memory.

Someone with a great long-term memory will be more likely have a great working memory. For example: I tell you to memorize the number set 91119893101990. Someone with vast knowledge base will be able to recognize the dates as the day of the fall of the Berlin Wall: 9 November 1989 and the day of reunification: 3 October 1990. Those tables in FA and Pathoma are infinitely easier to memorize if you have a strong pre-existing knowledge base.

I believe this to be the reason why a high vocabulary is correlated with high IQ. People who memorize quickly may only need to be exposed to a new word 3-4 times to commit it to memory. Those people have high IQs. People that need to be exposed to the word 30-40 times to remember it will not have high vocab, and may have lower IQ.

I'm not saying USMLE is a good measure of IQ. I think MCAT was better at that. And I think SAT was even more strongly correlated with IQ than MCAT. But I do think there is some correlation with all of these tests and IQ. Does it make a significant difference in your ability to treat patients? No.

Right but that's the problem with IQ exams there is an element of prior knowledge which can boost your score a lot. You can definitely isolate working memory from long term by throwing a situation where the common sense (or schema built from long term knowledge) cannot be used, which is what fluid intelligence is. Verbal does a rather good job with this. You can study for verbal and build a schema (i.e. common sense) of how to handle it. However, practicing verbal can only take you so far and if you don't have a good working memory then long term memory cannot save you.
 
Last edited:
  • Like
Reactions: 1 user
It is obviously a reflection of one aspect of intelligence. But it's sure as hell not an IQ test.

And it absolutely is not a predictor of a good clinician. There are far too many variables it doesn't measure that make a good clinician.
There is a plethora of studies which demonstrate that Step 1 score is not a predictor of resident performance.
 
  • Like
Reactions: 1 user
Right but that's the problem with IQ exams there is an element of prior knowledge which can boost your score a lot. You can definitely isolate working memory from long term by throwing a situation where the common sense (or schema built from long term knowledge) cannot be used, which is what fluid intelligence is. Verbal does a rather good job with this. You can study for verbal and build a schema (i.e. common sense) of how to handle it. However, practicing verbal can only take you so far and if you don't have a good working memory then long term memory cannot save you.
To which IQ test(s) are you referring?
 
I love these conversations. Always boils down to definitions. Then existentialism. Eventually we all become nihilists.
 
  • Like
Reactions: 1 users
I love these conversations. Always boils down to definitions. Then existentialism. Eventually we all become nihilists.
watchmenchapter12pg27panel.jpg
 
  • Like
Reactions: 2 users
Different types of memory.

Working memory (for computer analogy, RAM) very correlated with IQ.

Long term memory (hard drive) is not.

The latter is more important for USMLE.

Another way to think about it is that if it were open book, almost everyone would do well on the USMLE. That is not true for the MCAT.
Interesting! I agree with this. Definitely thought the MCAT was more challenging conceptually, vs Step 1 was brutal due to the sheer volume.
 
I've actually heard that about the MCAT before... why do you think that is? Perhaps because it involved more critical thinking (verbal, science, math etc) ?

The way the questions are worded and what they actually ask. Especially for the verbal reasoning section. Sure some of it is read and regurg the subject matter, but at least the verbal section requires some critical and abstract thinking skills. Idk about the CARS section (didn't exist when I took it). Part of why the reading section is actually relevant to future doctors, it's probably the best section of standardized tests in terms of examining critical thinking that all physicians are required to take.
 
People who suck at standardized exams /reading through an extensive syllabus are the ones who whine about them the most. I liked the standardized exam because I really suck at kissing butt but do really well on a piece of paper or in front of a computer/keyboard when butt kissing is not a factor. The standardized exam is an equaling factor in front of the brownnosers. There is probably a reason why medical schools/ residencies give so much weightage to the MCAT/USMLE.
Yes not all USMLE top takers are excellent physicians but then there are terrible physicians who just know how to talk and their patients love them a lot.
 
  • Like
Reactions: 1 user
I'm one of the controversial believers on here that believes the Mcat and the USMLE are designed to reflect intelligence. I've heard it argued back and forth over and over. It seems to be that those with high intelligence don't want to admit that they're pseudo-masked IQ tests because it would make them feel like their study habits and work they put in were pointless. And thus we'll continue to see this discussed year after year without ever a conclusion. I mean it makes sense. Programs don't just care if you're knowledgable. They also care if you're intelligent for on the fly high pressure situations. They also don't want to be sued by patients from bad decision making.
 
HS - does ACT/SAT show how smart you are?
Premed - does the MCAT show you how intellegent you are?
Med - Is the USMLE score surrogate for intelligence?
Post-med - Whomst does the testing... doth performance represent efflugence of intellect?
 
Top