Intelligence and the MCAT

[gettheleadout]

---

Members do not see this ad.

Continuing a discussion between @Ace-Co-A, @sakabato93, and myself elsewhere on SDN, I'd like to consider the concept of intelligence and its relation to the MCAT. I can identify a number of core issues to be addressed:

A) What is intelligence? How can it be meaningfully defined?

B) What is the purpose of the MCAT? Does the test succeed in fulfilling its objectives?

C) What is the relationship between intelligence and performance on the MCAT?

D) What problems can be identified with the conclusions we might draw from answering the above?

Here's my take on these issues.

A) The concept of intelligence is complex, and often it can be useful to operationalize intelligence in the context with which one is concerned. For example, it is my opinion that intelligence in the most broad sense can be operationalized at least in part in terms of several characteristics, which fall into categories related to (1) memory and (2) processing.

(1) Characteristics of memory relevant to the concept of intelligence: memory capacity (long-term, short-term, and working memory stores; how much information one is able to retain in memory), memory encoding (how easily/quickly one can add new information to each of the memory stores), and memory persistence (the duration of time for which information can be retained in memory stores without retrieval or reinforcement).

(2) Characteristics of processing relevant to the concept of intelligence: processing speed (the rate at which manipulation of information can be carried out), and parallel processing (the extent to which distinct processes can be carried out simultaneously).

All of that is to say that I find it reasonable to say that between two individuals, the one who can retain more information in memory, add new information more quickly and easily into memory, retain it the longest, think the fastest, and think at the most complex level is (ignoring other differences) the more intelligent one. This approach of course leaves out important facets such as characteristics associated with attention and perception, but I'm really not qualified to flesh out the entire concept of intelligence here. Add to those such macro-scale abilities like creativity, social behavior, and others and you get a seriously complicated situation. The characteristics I've presented, however, I do feel are pretty core (though not comprehensive) to the idea of intelligence at least the way I think of it.

Open questions: What might intelligence mean in the context of physician performance? Which of my core characteristics do you agree or disagree with? What might you add? Would you approach defining the term differently, and if so, how?

B) According to the AAMC…

The MCAT tests for mastery of basic concepts in biology, general chemistry, organic chemistry, and physics. Additionally, it serves to assess examinees’ capacity for problem solving and critical thinking.

I'm not surprised that the word "intelligence" is nowhere to be found. From this statement, however, we can identify that the MCAT is designed to measure the following:

1) The amount of information retained in memory (this is the science content test takers are responsible for knowing), and by necessity the accessibility of this information as well.

2) The degree to which two behaviors can be performed: problem-solving and critical thinking.I'm not going to get into defining those behaviors, but anyone familiar with the MCAT would agree that the passage-style format involves, at least to some extent, the assimilation of new information into memory and the processing of that information in the context of background knowledge.

Further, because the MCAT is a timed test, I feel I can reasonably argue that the exam measures processing speed (it's not like the time limit is 3 hours per section; a decent pace is required and is frequently a limiting factor is test taker performance).

I'm actually going to steer clear of addressing whether the test succeeds or not. The MCAT has respectable validity with regard to predicting USMLE Step 1 performance, and is the product of decades of development by professionals with background in test design. Overall, it almost certainly works pretty well in measuring what it's trying to measure.

Open questions: Are problem-solving and critical thinking necessarily representative of intelligence? Given that the test explicitly attempts to measure test taker knowledge, does this necessarily represent intelligence? Why or why not?

C) One of the points @Ace-Co-A raised in prior discussion examined the relationship between MCAT performance and intelligence:

My principal objection to equating or strongly relating MCAT score with 'intelligence' will be seen in the following inference: if it is always true that a higher MCAT score corresponds to a keener intellect, then we must say that someone with a higher MCAT score (let's say a 34) is smarter than someone with a lower MCAT score (let's say a 31), right?

It's an interesting issue; to what extend can we say that a higher score represents greater intelligence? How much of one's score is attributable to the MCAT's measurement of factors that we would agree represent aspects of intelligence? How certain can be we that someone who scores a 40 is at least somewhat more intelligence that someone who scores a 24? Absolutely? Not at all? What influences our answer?

D) @Ace-Co-A further noted problems with the strength of an MCAT score-intelligence relationship:

(1) MCAT scores vary from day-to-day. For example, Kim might score 34 one day, 33 the next, and 32 the day after. Which score is most representative of her intelligence? Is she smarter the first day (with her 34) than the third (with her 32)?

(2) Test scores vary depending on uncontrollable variables (the person might have a fever, her father might have died the night before, her untimely boyfriend proposed that morning, etc.)

(3) Test scores depend on the resources available to the individual (practice tests, books/information, problem sets, previous coursework, prep class, etc.)

These are legitimate issues that affect both the reliability and validity of the test itself. Standardization as a process acts to mitigate the various confounds in (2) to some extent, but the acknowledgement that they cannot be eliminated completely is part of why we have classical test theory to describe scores: one's score is always the sum of one's true, perfectly representative score and some unknown amount of error (X = T + E).

As an excellently reliable exam (r > 0.9) the MCAT has a standard error of measurement (SEM) of 2 composite points, which represents the typical degree to which one's reported score differs from one's true score. Examining the confidence intervals obtained when the SEM is known demonstrates that we should not distinguish between three scores of 34, 33, and 32, for example. All may be significantly representative of the test taker's true characteristics (as problematic as that may be for admissions purposes.)

My comment on (3) is simply that, depending on how relevant we think content knowledge is in terms of representing one's intelligence, the contribution of these external factors to the MCAT's measurement of one's intelligence may be smaller than would appear.

As a random thought, the MCAT is a multiple-choice test with no guessing penalty, so luck/chance/whatever you want to call it could account for significant reported score variance in some instances as well.

Conclusions: I personally think that the characteristics measured by the MCAT both directly and indirectly attempt to capture the test taker's intelligence (broadly defined), but acknowledge that there are certainly external factors that can affect both the ability of the exam to accurately measure what it attempts to and the representativeness of one's score. In general, however, I do think the exam does a pretty good job of measuring these intelligence-relevant characteristics. I would also emphasize that caution should be used in drawing strict conclusions and comparisons regarding intelligence when using MCAT scores as a measure, especially when scores are close or there are known external variables that may affect the reliability of the exam as a metric.

 

[bambam92]

Disparity related to culture/community of origin:

There's a lot of good to the MCAT.

One place it chronically fails, especially in verbal reasoning, is that it favors a certain "type of thinking" that has nothing to do with intelligence. It has to do with where the test-taker grew up, and how people communicate in that community.

One example of what I mean by, "type of thinking," on the MCAT is a passage/question stem/answer choice that requires the test taker to interpret EVERYTHING 100% literally, or to choose the proper English interpretation. I believe that this comes naturally for some people who are from educated communities. On the other hand, a person/test-taker from other communities (especially under educated ones), will be inclined to pick the more commonly used interpretation. In real life, for example, when I hear some people use a double negative, I know what they mean (and "what they mean" is NOT the literal proper English meaning). A test-taker from certain communities will be tempted to think that the MCAT is testing people skills (i.e. wants them to interpret "not never" as emphasizing the "not" part, making it a stronger statement than a single negative). Sure, they learned otherwise in English class, but a "people person" with a certain background will wonder what the test-maker really wants. They will wonder if the MCAT wants the "people person" answer (i.e. test-taker can relate to those without perfect grammar), or the literal hard-headed smart-ass robotic answer (is what they'll think). Some will just go with his/her instinct/habits and check the wrong box. Double negatives are NOT the only example of this, but on the test, I've noticed several scenarios where one's background will influence their answers.

To solve this, I would suggest that the AAMC run a focus group, using people from different backgrounds, to study this disparity.

(While I suspect some people might be able to overcome this. I believe it presents an additional setback relative other pre-meds from more educated communities.)

Another example of background influencing answer choice, is when a passage is about something other than science, and the person taking the test happens to be an expert (or just knowledgeable in that non-science field). Occasionally, there is a situation where the passage DOES provide evidence for answer choices that the test-maker seemed unaware of, due to the test-maker's lack of expertise in the field the passage is about. An Examkrackers example would be page 178 in the 2008 EK VR 101 book, #4, if you consider media research, a phenomenon EK didn't seem to consider. (SDN: I don't believe that I made a spoiler. If you disagree, feel free to erase that last line.) I know there's a couple AAMC examples but they're not coming to mind right now.

One place it chronically fails, especially in verbal reasoning, is that it favors a certain "type of thinking" that has nothing to do with intelligence."

Disparity related to culture/community of origin:

There's a lot of good to the MCAT.

One place it chronically fails, especially in verbal reasoning, is that it favors a certain "type of thinking" that has nothing to do with intelligence. It has to do with where the test-taker grew up, and how people communicate in that community.

One example of what I mean by, "type of thinking," on the MCAT is a passage/question stem/answer choice that requires the test taker to interpret EVERYTHING 100% literally, or to choose the proper English interpretation. I believe that this comes naturally for some people who are from educated communities. On the other hand, a person/test-taker from other communities (especially under educated ones), will be inclined to pick the more commonly used interpretation. In real life, for example, when I hear some people use a double negative, I know what they mean (and "what they mean" is NOT the literal proper English meaning). A test-taker from certain communities will be tempted to think that the MCAT is testing people skills (i.e. wants them to interpret "not never" as emphasizing the "not" part, making it a stronger statement than a single negative). Sure, they learned otherwise in English class, but a "people person" with a certain background will wonder what the test-maker really wants. They will wonder if the MCAT wants the "people person" answer (i.e. test-taker can relate to those without perfect grammar), or the literal hard-headed smart-ass robotic answer (is what they'll think). Some will just go with his/her instinct/habits and check the wrong box. Double negatives are NOT the only example of this, but on the test, I've noticed several scenarios where one's background will influence their answers.

To solve this, I would suggest that the AAMC run a focus group, using people from different backgrounds, to study this disparity.

(While I suspect some people might be able to overcome this. I believe it presents an additional setback relative other pre-meds from more educated communities.)

Another example of background influencing answer choice, is when a passage is about something other than science, and the person taking the test happens to be an expert (or just knowledgeable in that non-science field). Occasionally, there is a situation where the passage DOES provide evidence for answer choices that the test-maker seemed unaware of, due to the test-maker's lack of expertise in the field the passage is about. An Examkrackers example would be page 178 in the 2008 EK VR 101 book, #4, if you consider media research, a phenomenon EK didn't seem to consider. (SDN: I don't believe that I made a spoiler. If you disagree, feel free to erase that last line.) I know there's a couple AAMC examples but they're not coming to mind right now.

Good post. But, I completely disagree with your assumption that, "Theres a lot of good to the MCAT, but, one place it chronically fails, especially in verbal reasoning, is that it favors a certain "type of thinking" that has nothing to do with intelligence."
I feel VR is the most unlearnable section of the MCAT by far--it is critical reading/critical thinking. This is something that people have/ or haven't acquired over their life time. If there is any part of the MCAT that correlates with "innate intelligence" (and I'm not sure there is honestly) which it seems like you guys are going for in this thread.. 😵 it is definitely verbal. The other two sections can be studied via review books. Granted people can improve verbal with practice too, just not the margin that most can improve PS and BS. I think that a 30+ MCAT is doable for most who make it thru the pre-reqs successfully and prepare right. Granted I haven't taken yet, but will at end of summer--so will have to follow up on that notion lol.

 

[Rhino1000]

One place it chronically fails, especially in verbal reasoning, is that it favors a certain "type of thinking" that has nothing to do with intelligence."


Good post. But, I completely disagree with your assumption that, "Theres a lot of good to the MCAT, but, one place it chronically fails, especially in verbal reasoning, is that it favors a certain "type of thinking" that has nothing to do with intelligence."
I feel VR is the most unlearnable section of the MCAT by far--it is critical reading/critical thinking. This is something that people have/ or haven't acquired over their life time. If there is any part of the MCAT that correlates with "innate intelligence" (and I'm not sure there is honestly) which it seems like you guys are going for in this thread.. 😵 it is definitely verbal. The other two sections can be studied via review books. Granted people can improve verbal with practice too, just not the margin that most can improve PS and BS. I think that a 30+ MCAT is doable for most who make it thru the pre-reqs successfully and prepare right. Granted I haven't taken yet, but will at end of summer--so will have to follow up on that notion lol.

I'm surprised that you aren't considering the whole 15 to 20-year practice period that people undertake for verbal reasoning - i.e. reading in one's spare time during the years preceding the MCAT is arguably a huge factor in verbal scores; many of the top scorers attest to this avidity as their "secret". I would be likely to support an argument that suggests that verbal reasoning is largely a product of such long-term allocations of spare time and/or diligence when it comes to reading texts for class, and is therefore not a greater test of intelligence than interpreting complex physical/biological data and applying relatively simple concepts in a conceptual and logical manner in order to make sense out of it.

 

[Ace-Co-A]

Hello again. I think your last two blocks of text are the most interesting ones.

I'd feel confident in assuming that you recognize that in every field there will be variation between professionals; for better or worse almost every population is a bell curve on some characteristic(s). With regard to the impact recognition of this might have (internally within the profession, publicly, whatever), do you think medicine is unique in some way when compared to other professions?

I appreciate that variation between individuals in a given profession exists. I suppose it isn't unique in that there are other professions in which the impact recognition would be similar (perhaps there are 'bad doctors' just as there are 'bad judges'). I do think that a level of equality is maintained between colleagues. Other posts in this thread have cited the studies that show that a threshold MCAT score (~ 27) exists for predictability of success in (i.e., graduation from) medical school. Obtaining the MD/DO is the fundamental point of equality between physicians, even if there is - as you say - a bell curve on performance/intelligence/etc.

At this point, I want to go back to something I mentioned previously (i.e., mutability of MCAT performance) -- this paragraph will also touch upon my answer to the second part of your quote. Extending this idea to say that intelligence can vary over time, I'll attempt again to show something else I wonder about when considering connections between test (MCAT) performance and intelligence. I have to ask some clarifying questions, though. When we talk about 'intelligence', are we talking about something that can change over time (thinking here of a time frame of years rather than weeks)? Or is intelligence something with which you are endowed at birth / in early childhood? My end game here is to say that I think that the idea that intelligence can change (i.e., someone can become smarter by working hard at it) is important and true. In this way, although a threshold performance (established empirically in the studies quoted in this thread to establish what caliber of student is capable of completing medical school) is necessary, it should be possible for medical students to continue to work hard to try to become more intelligent (possibly demonstrated by a higher percentile Step 1 score than their MCAT score - after controlling for the different level of competition in those two groups).

You raise a great question about how multiple scores should be interpreted, which is obviously relevant to med school admissions. Issues with practicality aside, in an ideal world what could we to get around this issue? Require three administrations of the exam and average all scores? Forbid retesting at all and just go with one shot (possibly as long as you pass, like the USMLE?)

Honestly, I don't know. I've always wondered why - when we all know that there variation almost always exists between any number of trials - we consider only one test administration to be sufficient? But who knows the answer to this?

 

[mk04447]

Well.. who knows, I think that behavioral tests designed by psychologists may be a better option. It's not going to be cheap obviously, but chances are that a timed thought experiment will give more insight.

But, that's the point and what those tests aren't necessarily better. The MCAT is cheap, requires no effort on the part of the college, and can quickly sort applicants. Police departments send every applicant to a psychiatrist in the last stages of the hiring process, this is the most expensive part of the hiring process, but is also the least likely to disqualify an applicant; the applicants are vetted by the standardized testing at this point. Which begs the question, if psychological testing is better, how much, and is that increase worth the cost. I'd argue it isn't better and isn't worth the expense, the fact that law enforcement agencies use it is a product of litigation not quality.

So, wrapping this back to our discussion, the MCATs ability to pick successful applicants must be due to something other than intelligence. Otherwise, the test would resemble an IQ test. Note, I said resemble, because there will/would always be an MCAT, it is simply too damn profitable. We look to AT&T and cable providers to describe monopolies, we overlook smaller entities like the AAMC. But, try getting into medical school without paying them money in one form or another, even if you are given a waiver for economic reasons you will still trickle money to them via licensing rights purchased by someone else... but this is another discussion all together.

 

[RC4L]

oo

 

[RC4L]

Interesting post!

A) I think the most meaningful way to define intelligence is having the ability to assess and adjust to the best method of approach in various situations. By this definition I believe the best way to assess intelligence is to test the fluidity/diversity of cognitive processes as a unit and it seems to me that standardize tests do a fairly good job of this.

B) The dean of a local medical school came to my university about a year ago to give a speech on admissions and he described the purpose of the mcat in a way that made a lot of sense. He said classroom grades no longer speak for the student in a meaningful way because there is too much predictability in the classroom. The mcat has become a way for adcoms to see if a student's ability lies beyond memorization in a controlled environment which makes a lot of sense to me. In my experience, the mcat was far more challenging in terms of what I could do with what I learned versus what I have learned so I do feel like it serves its purpose.

C) I don't believe there is a ceiling for each test taker based on their innate abilities but I do believe the MCAT tests intelligence in the form of demanding a fluid thought process and an efficient method of thinking-something that can be learned. When students restrict their preparation to practice without reflection or simply learning the content, the MCAT places its limits. There are plenty of people who are more than capable of scoring well on the mcat or who are very intelligent yet do not score competitively and from what I've observed it usually boils down to lack of prep or failure to prepare correctly so I don't think the mcat fails to do its job often. I really disagree with the statement regarding those with better memories do best on the test because content really only goes so far at a certain point and everyone has the capacity to remember all the info with enough prep. Even if it takes one tester a lot longer to learn all that info than another, a medical school applicant should be prepared for the challenge.

 

[Drrrrrr. Celty]

But, that's the point and what those tests aren't necessarily better. The MCAT is cheap, requires no effort on the part of the college, and can quickly sort applicants. Police departments send every applicant to a psychiatrist in the last stages of the hiring process, this is the most expensive part of the hiring process, but is also the least likely to disqualify an applicant; the applicants are vetted by the standardized testing at this point. Which begs the question, if psychological testing is better, how much, and is that increase worth the cost. I'd argue it isn't better and isn't worth the expense, the fact that law enforcement agencies use it is a product of litigation not quality.

So, wrapping this back to our discussion, the MCATs ability to pick successful applicants must be due to something other than intelligence. Otherwise, the test would resemble an IQ test. Note, I said resemble, because there will/would always be an MCAT, it is simply too damn profitable. We look to AT&T and cable providers to describe monopolies, we overlook smaller entities like the AAMC. But, try getting into medical school without paying them money in one form or another, even if you are given a waiver for economic reasons you will still trickle money to them via licensing rights purchased by someone else... but this is another discussion all together.

A psychiatric evaluation is not a behavioral test. A behavioral test is essentially having a person in an environment preform tasks and testing for factors like latency or etc. They tend to have in research higher validity and statistical power in explaining or correlating phenomena.

 

[mk04447]

A psychiatric evaluation is not a behavioral test. A behavioral test is essentially having a person in an environment preform tasks and testing for factors like latency or etc. They tend to have in research higher validity and statistical power in explaining or correlating phenomena.

The effect is the same, higher cost and questionable value.

 

[mk04447]

Interesting post!

A) I think the most meaningful way to define intelligence is having the ability to assess and adjust to the best method of approach in various situations. By this definition I believe the best way to assess intelligence is to test the fluidity/diversity of cognitive processes as a unit and it seems to me that standardize tests do a fairly good job of this.

B) The dean of a local medical school came to my university about a year ago to give a speech on admissions and he described the purpose of the mcat in a way that made a lot of sense. He said classroom grades no longer speak for the student in a meaningful way because there is too much predictability in the classroom. The mcat has become a way for adcoms to see if a student's ability lies beyond memorization in a controlled environment which makes a lot of sense to me. In my experience, the mcat was far more challenging in terms of what I could do with what I learned versus what I have learned so I do feel like it serves its purpose.

C) I don't believe there is a ceiling for each test taker based on their innate abilities but I do believe the MCAT tests intelligence in the form of demanding a fluid thought process and an efficient method of thinking-something that can be learned. When students restrict their preparation to practice without reflection or simply learning the content, the MCAT places its limits. There are plenty of people who are more than capable of scoring well on the mcat or who are very intelligent yet do not score competitively and from what I've observed it usually boils down to lack of prep or failure to prepare correctly so I don't think the mcat fails to do its job often. I really disagree with the statement regarding those with better memories do best on the test because content really only goes so far at a certain point and everyone has the capacity to remember all the info with enough prep. Even if it takes one tester a lot longer to learn all that info than another, a medical school applicant should be prepared for the challenge.

Responding to point 'C'... I don't know why this portion of your post made me thing about the disability act. Why is additional time considered the equalizer? Is this because of a lack of options to level the playing field? Would someone with the best memory do better than a history major who had unlimited time? Frankly, I think time management is evaluated on the MCAT. With enough time I think almost anyone would score high 30's consistently.

 

[immolard]

I'm not sure if it has been said, but I think that the mcat fails to assess a persons ability to look at a concept and rethink of it in a different way in order to gain new insight. These types of things take time. Obviously the mcat is timed and doesn't allow for this. It's a processing game like an IQ test. For this reason I think that people like Einstein who was certainly strong in this area might not be properly assessed by an mcat or an IQ test because his true ability shined when contemplating concepts for long periods of time. I do think that this ability correlates with other intelligence though and so I'm sure he'd do well on most tests given the proper preparation.

 

[Drrrrrr. Celty]

I'm not sure if it has been said, but I think that the mcat fails to assess a persons ability to look at a concept and rethink of it in a different way in order to gain new insight. These types of things take time. Obviously the mcat is timed and doesn't allow for this. It's a processing game like an IQ test. For this reason I think that people like Einstein who was certainly strong in this area might not be properly assessed by an mcat or an IQ test because his true ability shined when contemplating concepts for long periods of time. I do think that this ability correlates with other intelligence though and so I'm sure he'd do well on most tests given the proper preparation.

The mcat in truth is a different exam than any other standardized exam. Most aren't designed to outright trick you or to construct a world scenario that outright invalidates your personal experience and senses.

Truth is that the biggest part about scoring well on the Mcat is about telling your personal sense and persuasions to be quiet and to do it the way they want it and in the world they have established.

I mean truth of the matter is that most people are smart enough to get through medical school and pass the boards even if they had a 24 or 25 on their Mcat. Hell three DO schools have about those averages and graduate a high amount of their students and match them.

It's why I continue to say that as a measure the Mcat enormously fails. If people are passing at a similar rate and becoming competent physicians then what are we really selecting for? Higher board scores for our schools so we can say hey, we match all of our students to MGH?

 

[RC4L]

Responding to point 'C'... I don't know why this portion of your post made me thing about the disability act. Why is additional time considered the equalizer? Is this because of a lack of options to level the playing field? Would someone with the best memory do better than a history major who had unlimited time? Frankly, I think time management is evaluated on the MCAT. With enough time I think almost anyone would score high 30's consistently.

That is a really good point. It is incredibly easy to waste time while studying for the MCAT and I think a lot of people study in a manner that enables them to remember more effectively than others rather than some having more fit brain.

 

[thestrugglingtraveler]

The mcat in truth is a different exam than any other standardized exam. Most aren't designed to outright trick you or to construct a world scenario that outright invalidates your personal experience and senses.

Truth is that the biggest part about scoring well on the Mcat is about telling your personal sense and persuasions to be quiet and to do it the way they want it and in the world they have established.

I mean truth of the matter is that most people are smart enough to get through medical school and pass the boards even if they had a 24 or 25 on their Mcat. Hell three DO schools have about those averages and graduate a high amount of their students and match them.

It's why I continue to say that as a measure the Mcat enormously fails. If people are passing at a similar rate and becoming competent physicians then what are we really selecting for? Higher board scores for our schools so we can say hey, we match all of our students to MGH?

High MCAT scores people do more research. People with higher MCAT scores tend to go into research (academic) medicine in high numbers. They are capable of more research, as research required reasoning and critical thinking. People with lower MCAT scores can pass the USMLE, etc and become good doctors as they can memorize. But they have limited critical thinking skills (as opposed to someone with high MCAT) and thus cannot perform original research.

i read somewhere that high MCAT correlated with more research publications in residency and beyond, which makes sense.

 

[Drrrrrr. Celty]

High MCAT scores people do more research ( Citation needed; Low mcat ppl usually end up in PhD programs btw so... you know...). People with higher MCAT scores tend to go into research (academic) medicine in high numbers ( Probably, mostly because high stat applicants can be MD/PhDs). They are capable of more research, as research required reasoning and critical thinking ( Probably not. Unless you're an MD/PhD chances are you don't know how to do research just coming from medical school as you're not trained for that. So the point is moot. On the topic of critical thinking, Ill touch on it later). People with lower MCAT scores can pass the USMLE, etc and become good doctors as they can memorize (So they can do their job? Ok.). But they have limited critical thinking skills (as opposed to someone with high MCAT) and thus cannot perform original research ( Citation needed).

i read somewhere that high MCAT correlated with more research publications in residency and beyond, which makes sense.[/QUOTE]

There was a point when I agreed that research demands an enormous ability to think critically and uniquely to pursue original and new research. But after actually working in research and being published, I don't believe it anymore.

Sorry, but if your argument is supposed to be salient then you probably have a low mcat score with this train of thought...

 

[thestrugglingtraveler]

High MCAT scores people do more research ( Citation needed; Low mcat ppl usually end up in PhD programs btw so... you know...). People with higher MCAT scores tend to go into research (academic) medicine in high numbers ( Probably, mostly because high stat applicants can be MD/PhDs). They are capable of more research, as research required reasoning and critical thinking ( Probably not. Unless you're an MD/PhD chances are you don't know how to do research just coming from medical school as you're not trained for that. So the point is moot. On the topic of critical thinking, Ill touch on it later). People with lower MCAT scores can pass the USMLE, etc and become good doctors as they can memorize (So they can do their job? Ok.). But they have limited critical thinking skills (as opposed to someone with high MCAT) and thus cannot perform original research ( Citation needed).

i read somewhere that high MCAT correlated with more research publications in residency and beyond, which makes sense.

I am not sure why you are so angry, but you still make some good points.

About PhD, I don't know about that. Certainly, at the bottom programs, a lot of the people were med-school rejects with low MCAT scores. But these people produce very poor research.

I have read research--this is well known now--that virtually all of the important research comes from labs at top programs. You can see if you read nature or science, that a few top schools are represented.

Now, these top researchers, are these low MCAT scorers, med-school rejects? I really doubt it.

About your own research. If it did not require critical thinking skills, I suspect that it was of low-quality. The sort that people in low rank schools publish, that goes on posters on some obscure journal.

Do you really think that the top works in Nature come from people with poor reasoning ability?

Lets think. Suppose if you pay someone to do good on the MCAT. $20,000 for each point.

I am certain that 99.9% of high IQ people (good critical thinking, good reasoning) with get a very high score. (assuming they are not rich).

Do you agree?

So a low score would mean that you are not of great intelligence. If someone is motivated and studies hard, and still gets a low score, that really means that he is dull.

Now, true, this dull man can get a PhD in sociology from a third-rate university, and publish papers in fourth-rate journals, as most such people tend to do. But he will not be able to publish the real research as he does not have critical thinking or reason ability, as measured by the MCAT. Do you not agree with me?

 

[thestrugglingtraveler]

Sorry, but if your argument is supposed to be salient then you probably have a low mcat score with this train of thought...

In other words, you are saying that people who have poor reasoning skills (like myself) are not able to get high MCAT scores? That is what I was saying, in fact.

 

[Doug Underhill]

The MCAT is only a measure of standardized test-taking ability. The BS section requires you to memorize information and regurgitate it hours to weeks later. This is important for medical school because you have to be pretty good at these skills to pass the USMLE exams and become a licensed physician. I think it is vastly overvalued in the admissions process, but the adcoms may see something I don't.

High MCAT scores people do more research ( Citation needed; Low mcat ppl usually end up in PhD programs btw so... you know...).

Ph.D programs are not looking for failed pre-meds. They're looking for people who want to spend the rest of their lives doing research. If you really, really want to be a doctor but your MCAT is too low for MD or DO, most Ph.D programs do not want you, because the things you actually do as a Ph.D are very different from medicine. For the low MCAT people there's nurse practioner, physician's assistant, podiatrist, or lots of other great healthcare jobs.

 

[LuluLovesMe]

A) What is intelligence? How can it be meaningfully defined?
Intelligence as a concept has no practical use so I see no point in defining it. If you want to determine if someone will succeed in med school, you test people on the metrics that correlate highly with success. The AAMC has deemed this to be Verbal Reasoning, Biological Sciences and Physical Sciences.

B) What is the purpose of the MCAT? Does the test succeed in fulfilling its objectives?
To test if students possess traits that the AAMC thinks reflects on their ability to succeed in med school.

C) What is the relationship between intelligence and performance on the MCAT?
There is a correlation. Nobody reasonable would argue against that. If you had reliable data and a large enough sample size, you could find the degree of correlation. What else is there to discuss here?

D) What problems can be identified with the conclusions we might draw from answering the above?
No problems.

 

[mehc012]

A) What is intelligence? How can it be meaningfully defined?
Intelligence as a concept has no practical use so I see no point in defining it. If you want to determine if someone will succeed in med school, you test people on the metrics that correlate highly with success. The AAMC has deemed this to be Verbal Reasoning, Biological Sciences and Physical Sciences.

B) What is the purpose of the MCAT? Does the test succeed in fulfilling its objectives?
To test if students possess traits that the AAMC thinks reflects on their ability to succeed in med school.

C) What is the relationship between intelligence and performance on the MCAT?
There is a correlation. Nobody reasonable would argue against that. If you had reliable data and a large enough sample size, you could find the degree of correlation. What else is there to discuss here?

D) What problems can be identified with the conclusions we might draw from answering the above?
No problems.

The problems discussed previously in this thread include factors which cause variation in MCAT scores which are not one of the things purported to be tested: content knowledge and critical thinking (call it intelligence if you wish). Once you introduce enough of those factors, the correlation breaks down.

 

[LuluLovesMe]

The problems discussed previously in this thread include factors which cause variation in MCAT scores which are not one of the things purported to be tested: content knowledge and critical thinking (call it intelligence if you wish). Once you introduce enough of those factors, the correlation breaks down.

Obviously the correlation cannot be 1, but the AAMC is constantly working to improve that and it's high enough to be significant. If there was little correlation, med schools wouldn't use MCAT scores.

 

[mehc012]

Obviously the correlation cannot be 1, but the AAMC is constantly working to improve that and it's high enough to be significant. If there was little correlation, med schools wouldn't use MCAT scores.

That assumes that med schools care about 'intelligence'.
I'd wager that the correlation between MCAT score and 'standardized testing skill' is far greater than that to 'intelligence'...and since med school reputations hinge on testing outcomes, that seems like a great reason for them to use it.

 

[LuluLovesMe]

That assumes that med schools care about 'intelligence'.
I'd wager that the correlation between MCAT score and 'standardized testing skill' is far greater than that to 'intelligence'...and since med school reputations hinge on testing outcomes, that seems like a great reason for them to use it.

I'm talking about the correlation between what the traits that AAMC tests and success in med school. I would argue the MCAT doesn't test intelligence. This has nothing to do with intelligence.

 

[mehc012]

I'm talking about the correlation between what the traits that AAMC tests and success in med school. I would argue the MCAT doesn't test intelligence. This has nothing to do with intelligence.

Uh, OK. Then I'm out, as that was kinda the premise of the thread and the interesting part of the ensuing discussion.

 

[LuluLovesMe]

Uh, OK. Then I'm out, as that was kinda the premise of the thread and the interesting part of the ensuing discussion.

Any my whole point was that this thread is really not necessary or useful.

 

[mehc012]

Any my whole point was that this thread is really not necessary or useful.

Dude, NO thread is necessary.
Very few are useful.
It was an interesting point of discussion when it was on-topic, and largely revolved around how we define intelligence and the effects of approaching it from only one cultural background.

 

[LuluLovesMe]

Dude, NO thread is necessary.
Very few are useful.
It was an interesting point of discussion when it was on-topic, and largely revolved around how we define intelligence and the effects of approaching it from only one cultural background.

Sure you can talk about intelligence and it may be interesting. But the context of the MCAT isn't needed in that case.

 

[mehc012]

Sure you can talk about intelligence and it may be interesting. But the context of the MCAT isn't needed in that case.

It's needed when the discussion turns to 'does basing the MCAT on one culturally biased measure of intelligence lead to a less diverse population of physicians?' and similar such things.
I mean, come on, did you read the thread or just the title?

 

[LuluLovesMe]

It's needed when the discussion turns to 'does basing the MCAT on one culturally biased measure of intelligence lead to a less diverse population of physicians?' and similar such things.
I mean, come on, did you read the thread or just the title?

But the MCAT isn't based on one culturally biased measure of intelligence.

 

[OneMan98]

This is a very interesting thread and I only had the opportunity to skim through the posts as I'm currently in the process of prepping for a big school/college project. That being said, while I like the idea of the thread I think the scope of the topic is too narrow because we're trying to find some association between the MCAT and intelligence (as per the title of this thread). However, from the few posts I read in detail, no one has mentioned the fact that the MCAT is not comprehensive and is, to a greater degree, subjective and/or based on luck. Now, I could make a lengthy post providing examples for these ideas but I think they're quite evident when you go into any thread discussing a particular MCAT. You constantly see people saying "I thought PS was hard" and others say "I thought PS was easy but BS was hard". I could go on and on. This occurs for the simple reason that the MCAT rarely ever tests on a multitude of concepts and often focuses in on a few for any given exam (you're almost guaranteed a genetics passage in BS, and an acid/base passage in PS). Thus, if a student is adequately prepared for those specific topics, they can answer questions with relative ease. Those that are not prepared OR do not enjoy said topic(s) will fail and do poorly, yet they may be brilliant in another topic that wasn't even tested on. This is particularly true of the VR section. Thus, the MCAT is a poor predictor of overall intelligence, let alone one's success in medical school. Just google "MCAT vs USMLE" and you see people with MCAT scores <30 with USMLE (step 1) scores >250, where one's "success" in medical school is based off their USMLE score.

 

[mehc012]

This is a very interesting thread and I only had the opportunity to skim through the posts as I'm currently in the process of prepping for a big school/college project. That being said, while I like the idea of the thread I think the scope of the topic is too narrow because we're trying to find some association between the MCAT and intelligence (as per the title of this thread). However, from the few posts I read in detail, no one has mentioned the fact that the MCAT is not comprehensive and is, to a greater degree, subjective and/or based on luck. Now, I could make a lengthy post providing examples for these ideas but I think they're quite evident when you go into any thread discussing a particular MCAT. You constantly see people saying "I thought PS was hard" and others say "I thought PS was easy but BS was hard". I could go on and on. This occurs for the simple reason that the MCAT rarely ever tests on a multitude of concepts and often focuses in on a few for any given exam (you're almost guaranteed a genetics passage in BS, and an acid/base passage in PS). Thus, if a student is adequately prepared for those specific topics, they can answer questions with relative ease. Those that are not prepared OR do not enjoy said topic(s) will fail and do poorly, yet they may be brilliant in another topic that wasn't even tested on. This is particularly true of the VR section. Thus, the MCAT is a poor predictor of overall intelligence, let alone one's success in medical school. Just google "MCAT vs USMLE" and you see people with MCAT scores <30 with USMLE (step 1) scores >250, where one's "success" in medical school is based off their USMLE score.

The MCAT is only based on luck if you don't bother to cover all of the content...in order to actually do well on the MCAT, you can't rely on the cards all falling your way. It may be a small-scale factor, but I would argue that content knowledge and critical thinking skills are far more important for doing well on the exam, more than enough so to overshadow luck as a determinant of score.

 

[OneMan98]

The MCAT is only based on luck if you don't bother to cover all of the content...in order to actually do well on the MCAT, you can't rely on the cards all falling your way. It may be a small-scale factor, but I would argue that content knowledge and critical thinking skills are far more important for doing well on the exam, more than enough so to overshadow luck as a determinant of score.

That may be so, but once again, it's not the bread of butter of this discussion. Sure, if someone is more familiar with each topic to be tested, they will have a higher chance of succeeding come test day. This is of course assuming they practiced most of the different "permutations" for each topic - which to you may translate as critical thinking skills, but to me translates to simply being well-prepared.

An example: Student A goes to calculus class, does not take notes, but goes home every night and does practice problems from the day's lesson. Student B goes to calculus class, takes rigorous notes, but does not practice any problems. Who has a higher chance of succeeding on a pop quiz next class?

Of course, this doesn't necessarily translate to VR. However, that is where the subjective component of my argument comes into play. If I find all 7 passages boring but another person finds all 7 passages interesting, who is more likely to do better on the VR section? Yet, subjective could also refer to the cultural argument made above by someone else. It is well-known that Caucasian students do better (on average) than any other race when it comes to the VR section.

While I might be willing to concede that luck plays a smaller role, I stand firm on the idea that this exam, like many others, is more about being well-prepared (or well-practiced to put it another way) than some broad sense of intelligence.

 

[mehc012]

That may be so, but once again, it's not the bread of butter of this discussion. Sure, if someone is more familiar with each topic to be tested, they will have a higher chance of succeeding come test day. This is of course assuming they practiced most of the different "permutations" for each topic - which to you may translate as critical thinking skills, but to me translates to simply being well-prepared.

Just for the record, that is not at all what I consider critical thinking skills. To me, critical thinking skills are used in order to puzzle out the answer when exposed to a new, unfamiliar problem.

Of course, this doesn't necessarily translate to VR. However, that is where the subjective component of my argument comes into play. If I find all 7 passages boring but another person finds all 7 passages interesting, who is more likely to do better on the VR section? Yet, subjective could also refer to the cultural argument made above by someone else. It is well-known that Caucasian students do better (on average) than any other race when it comes to the VR section.

Learning how to deal with boring, unengaging material is an important skill. However, I definitely agree that there is a cultural aspect at play, and if I remember my own response from page 1 of this thread correctly, I think that the VR section would be a better test if it involved a greater variety of cultural framings, not only because this would 'diversify' the test, but because those who truly have the ability to approach unfamiliar situations well would score more highly in that setting than those who are particularly good at one limited approach. As of now, the exam favors those who happen to be comfortable with one predominant reasoning style more than those who can take anything thrown at them.

While I might be willing to concede that luck plays a smaller role, I stand firm on the idea that this exam, like many others, is more about being well-prepared (or well-practiced to put it another way) than some broad sense of intelligence.

I am not willing to call it 'intelligence', but I believe that some skill at test-taking, or 'critical thinking' as I put it above, is a huge factor on the MCAT.
Quoting myself from another thread:

Honestly, I think there is a plateau to how much studying helps your score. You need a certain level of base knowledge to do reasonably well, then once you have that there is a large range where test-taking and critical thinking skills are the driving factors in score differences (these are harder to practice and thus why many people have difficulty raising their scores regardless of study time), and then finally, once you've nailed those bits, cramming in the final bits of minutiae can add the cherry-on-top last few points.

and more explanation on that:
There are some straight-up discretes, as well as some straightforward questions which rely on basic knowledge of concepts with little reasoning. Mastering these gets you a low-reasonable score.
There are then a bunch of questions where yeah, you need to have a grasp of the basics, but you also need to extend your reasoning and critical thinking beyond that, and remember to draw information from the passages and connect the dots. If you didn't get the straightforward Qs, odds are you won't get these.
Finally, there are some questions which require you to apply some critical thinking and basic concepts, and then bring in some outside details. Again, if you struggled with the basics, OR with applying critical thinking, you'll miss these anyway.

Also, while you can get to most answers eventually using critical thinking and passage information, this takes time. Knowing a ton of details and being familiar with a variety of situations allows you to get through it all more quickly. So you'll do fine if you can think critically, but you'll top out at some point if you don't have familiarity with various problem approaches and some specific content.

The point being, if you don't have a grasp of the basics (and I mean basic concepts here, not complex equations), you're going to score in the low range, period.
If you know the concepts, but can't think critically, you'll hit low-mid (probably just shy of 30).
The better you are at critical thinking, the higher you can score.
When you get to the range where each individual question starts mattering, you have to hit the tricky ones consistently to score well...this means knowing fine details and trivia.


So, basically, I believe that while studying is clearly beneficial, the meat of a strong MCAT score is in critical thinking skills...however, I would caution against correlating that trait with 'intelligence' specifically, in that I believe that you require intelligence in order to demonstrate critical thinking skills, but that a lack of success in one metric for evaluating 'critical thinking' does not by itself imply a lack of intelligence. There are simply too many confounding factors preventing the former from being a sound analogue for the latter.

 

[OneMan98]

Just for the record, that is not at all what I consider critical thinking skills. To me, critical thinking skills are used in order to puzzle out the answer when exposed to a new, unfamiliar problem.

That's the thing though, the more someone practices concepts they are unfamiliar with, the better they'll do, period. This is then perceived as superior critical thinking skills when it was simply the fact that one student took the time to strengthen their weaknesses (a practice that many pre-meds don't do). This does not necessarily translate to VR, but we'll get to that.

Learning how to deal with boring, unengaging material is an important skill.

How so? While I can see the application of it in some specialties of medicine, why should a student be judged on his/her ability to "critically think" on vague passages about Shakespeare or the art movements of the 14th century? I imagine you'll make the argument that it is testing one's ability to think critically about something they are unfamiliar with. Yet, passages that are so far out from the field simply test comprehension and recall. These traits are unique for each individual and depend on a variety of factors (some mentioned previously).

I am not willing to call it 'intelligence', but I believe that some skill at test-taking, or 'critical thinking' as I put it above, is a huge factor on the MCAT.

Exactly, in my opinion, test-taking skills and being "well prepared" play a major role on the MCAT.

Knowing a ton of details and being familiar with a variety of situations allows you to get through it all more quickly. So you'll do fine if you can think critically, but you'll top out at some point if you don't have familiarity with various problem approaches and some specific content.

Sounds like we're saying the same thing here, particularly that last sentence. It makes sense too. If you've seen almost every "permutation" possible you will do better on the exam. It is merely a matter of pattern-recognition and test-taking ability. For example, a passage on acid/base chemistry will almost always be paired up with one or two of the following: equilibrium, electrochemistry, or electrodynamics (physics - circuits). Just knowing this can give a student a huge boost because when he/she see's a passage on acid/base chemistry, they know what to expect at the very least.

So, basically, I believe that while studying is clearly beneficial, the meat of a strong MCAT score is in critical thinking skills...

I'm curious though, how would you advise someone who wants to improve on their MCAT score?


I personally don't follow the idea of "critical thinking". I do agree that it is important to have the ability to reason with the proper steps in logic, but I think this is something that has to be learned through more exposure to both the familiar and unfamiliar. Therefore, what is intelligence? If an unintelligent person followed the same patterns as an intelligent person, could they too become "intelligent"? As far as this discussion goes, I don't find a strong relationship between intelligence and the MCAT. I don't deny it's existence, but rather find the implications of it bogus.

 

[mehc012]

That's the thing though, the more someone practices concepts they are unfamiliar with, the better they'll do, period. This is then perceived as superior critical thinking skills when it was simply the fact that one student took the time to strengthen their weaknesses (a practice that many pre-meds don't do). This does not necessarily translate to VR, but we'll get to that.

Now you're just talking in circles. Yes, if they do not-critical-thinking stuff, that doesn't count as critical thinking. But the fact that that is possible in no way diminishes the fact that critical thinking skills are essential to strong MCAT performance.
Regardless of how well somebody covers the content, between the passages, integrating multiple concepts, and the infinite variability of possible questions, there will be some level of unfamiliarity on almost every question they encounter. Solving problems often comes down to sifting through the information, seeing what you're given, and finding a way to turn it into problems which you are familiar with and know how to find the answer. That process - recognition of disparate parts, connecting multiple steps, (I can use A to find C, then combine B+C to find D, then find E from D), finding similarity to familiar problems, etc - that is critical reasoning. If you have more tools in your toolbox (aka memorized more concepts or practiced more methods), you will be able to solve more problems, or to find quicker, more efficient ways to convert the MCAT question into a familiar process but you still have to do that, and those who are better at it will fare better, period.

How so? While I can see the application of it in some specialties of medicine, why should a student be judged on his/her ability to "critically think" on vague passages about Shakespeare or the art movements of the 14th century? I imagine you'll make the argument that it is testing one's ability to think critically about something they are unfamiliar with. Yet, passages that are so far out from the field simply test comprehension and recall. These traits are unique for each individual and depend on a variety of factors (some mentioned previously).

Yes, I absolutely argue that it is testing one's ability to find solutions to unfamiliar problems. Comprehension is a definite part of that, so I don't understand how that is a counter-argument. And I don't see how 'passages that are so far out from the field' could be testing recall...they're either from something the tester is familiar with and can recall, or they're from far out in the field, they can't be both. As for the boring, unengaging parts...I hate to break it to you, but even if you love medicine and love a lot of the material, at some point you will run into a subject that you hate, or find boring, and yet still have to approach, learn, or at least read through. For example, I find a lot of hematology mind-numbingly boring.

Exactly, in my opinion, test-taking skills and being "well prepared" play a major role on the MCAT.

OK, so we agree, except that you're lumping 'critical thinking skills' under 'well prepared' even though that's completely contrary to the definition of 'critical thinking skills', and I'm lumping them in as a subset of 'test taking skills'.

Sounds like we're saying the same thing here, particularly that last sentence. It makes sense too. If you've seen almost every "permutation" possible you will do better on the exam. It is merely a matter of pattern-recognition and test-taking ability. For example, a passage on acid/base chemistry will almost always be paired up with one or two of the following: equilibrium, electrochemistry, or electrodynamics (physics - circuits). Just knowing this can give a student a huge boost because when he/she see's a passage on acid/base chemistry, they know what to expect at the very least.

No, I'm saying that you need critical thinking and strong preparation on all topics, while you're saying that you need only the latter and that it can stand in for or be mistaken for critical thinking.

I'm curious though, how would you advise someone who wants to improve on their MCAT score?

It depends on the person and where their score is sitting. As I described above, if they're sitting in the 20s, they likely need a better grasp of the basic concepts. If they're in the 30s, but low, I'm not sure what should be done, because their limiting factor is the test-taking or 'critical thinking' bits. Perhaps very extensive practice of all topics and as many possible permutations/approaches as possible will broaden their toolkit enough that they need to make fewer connections to connect all of the dots, but that can only go so far. I'm sure there's a way to practice test-taking skills or critical thinking, but I'm no expert on that...I'd imagine that it'd just involve doing as much of it as possible (not even necessarily MCAT-based) and discussing thought processes with someone who is either good at it, or at the least has a tendency to take very different approaches than you.

I personally don't follow the idea of "critical thinking". I do agree that it is important to have the ability to reason with the proper steps in logic, but I think this is something that has to be learned through more exposure to both the familiar and unfamiliar. Therefore, what is intelligence? If an unintelligent person followed the same patterns as an intelligent person, could they too become "intelligent"? As far as this discussion goes, I don't find a strong relationship between intelligence and the MCAT. I don't deny it's existence, but rather find the implications of it bogus.

I never implied that test-taking skills or critical thinking skills were direct analogues for intelligence. In fact, I explicitly denied it. I think that you can easily lack those skills, or not be able to demonstrate them due to other barriers, and yet still be intelligent; I'm less certain about the opposite.

 

[StudyLater]

gettheleadout would most likely agree with which of the following?
A. The MCAT accurately reflects the intelligence of the test taker
B. Individuals who experience an emotionally-charged event immediately prior to their test time do more poorly than they otherwise would have
C. One should be wary that the MCAT attempts to predict intelligence, but does so fallibly
D. Confidence intervals determine a test taker's true score

 

[LuluLovesMe]

gettheleadout would most likely agree with which of the following?
A. The MCAT accurately reflects the intelligence of the test taker
B. Individuals who experience an emotionally-charged event immediately prior to their test time do more poorly than they otherwise would have
C. One should be wary that the MCAT attempts to predict intelligence, but does so fallibly
D. Confidence intervals determine a test taker's true score

This is a very well written MCAT question. Well done!

 

[Rhino1000]

gettheleadout would most likely agree with which of the following?
A. The MCAT accurately reflects the intelligence of the test taker
B. Individuals who experience an emotionally-charged event immediately prior to their test time do more poorly than they otherwise would have
C. One should be wary that the MCAT attempts to predict intelligence, but does so fallibly
D. Confidence intervals determine a test taker's true score

Intelligence tests require the same objections. Does that make intelligence tests (full length) not great measures of intelligence? The discussion includes the implicit consideration of the points that you bring up. If it makes you more comfortable with the discussion, you could explicitly define the conversation to be talking about trends in a population's MCAT confidence intervals rather than trends in actual MCAT score and it would change the conversation little. No one would suggest that MCAT scores are perfectly correlated with intelligence; i.e. there is a reason correlations between tests are more often around .6 rather than .95. The imperfect correlation is an all-encompassing thing that includes these factors as well as others, including different cognitive abilities being measured (e.g. possibly VR vs PS). Still, a worthy post.