Intelligence and the MCAT

[gettheleadout]

Continuing a discussion between @Ace-Co-A, @sakabato93, and myself elsewhere on SDN, I'd like to consider the concept of intelligence and its relation to the MCAT. I can identify a number of core issues to be addressed:

A) What is intelligence? How can it be meaningfully defined?

B) What is the purpose of the MCAT? Does the test succeed in fulfilling its objectives?

C) What is the relationship between intelligence and performance on the MCAT?

D) What problems can be identified with the conclusions we might draw from answering the above?

Here's my take on these issues.

A) The concept of intelligence is complex, and often it can be useful to operationalize intelligence in the context with which one is concerned. For example, it is my opinion that intelligence in the most broad sense can be operationalized at least in part in terms of several characteristics, which fall into categories related to (1) memory and (2) processing.

(1) Characteristics of memory relevant to the concept of intelligence: memory capacity (long-term, short-term, and working memory stores; how much information one is able to retain in memory), memory encoding (how easily/quickly one can add new information to each of the memory stores), and memory persistence (the duration of time for which information can be retained in memory stores without retrieval or reinforcement).

(2) Characteristics of processing relevant to the concept of intelligence: processing speed (the rate at which manipulation of information can be carried out), and parallel processing (the extent to which distinct processes can be carried out simultaneously).

All of that is to say that I find it reasonable to say that between two individuals, the one who can retain more information in memory, add new information more quickly and easily into memory, retain it the longest, think the fastest, and think at the most complex level is (ignoring other differences) the more intelligent one. This approach of course leaves out important facets such as characteristics associated with attention and perception, but I'm really not qualified to flesh out the entire concept of intelligence here. Add to those such macro-scale abilities like creativity, social behavior, and others and you get a seriously complicated situation. The characteristics I've presented, however, I do feel are pretty core (though not comprehensive) to the idea of intelligence at least the way I think of it.

Open questions: What might intelligence mean in the context of physician performance? Which of my core characteristics do you agree or disagree with? What might you add? Would you approach defining the term differently, and if so, how?

B) According to the AAMC…

The MCAT tests for mastery of basic concepts in biology, general chemistry, organic chemistry, and physics. Additionally, it serves to assess examinees’ capacity for problem solving and critical thinking.

I'm not surprised that the word "intelligence" is nowhere to be found. From this statement, however, we can identify that the MCAT is designed to measure the following:

1) The amount of information retained in memory (this is the science content test takers are responsible for knowing), and by necessity the accessibility of this information as well.

2) The degree to which two behaviors can be performed: problem-solving and critical thinking.I'm not going to get into defining those behaviors, but anyone familiar with the MCAT would agree that the passage-style format involves, at least to some extent, the assimilation of new information into memory and the processing of that information in the context of background knowledge.

Further, because the MCAT is a timed test, I feel I can reasonably argue that the exam measures processing speed (it's not like the time limit is 3 hours per section; a decent pace is required and is frequently a limiting factor is test taker performance).

I'm actually going to steer clear of addressing whether the test succeeds or not. The MCAT has respectable validity with regard to predicting USMLE Step 1 performance, and is the product of decades of development by professionals with background in test design. Overall, it almost certainly works pretty well in measuring what it's trying to measure.

Open questions: Are problem-solving and critical thinking necessarily representative of intelligence? Given that the test explicitly attempts to measure test taker knowledge, does this necessarily represent intelligence? Why or why not?

C) One of the points @Ace-Co-A raised in prior discussion examined the relationship between MCAT performance and intelligence:

My principal objection to equating or strongly relating MCAT score with 'intelligence' will be seen in the following inference: if it is always true that a higher MCAT score corresponds to a keener intellect, then we must say that someone with a higher MCAT score (let's say a 34) is smarter than someone with a lower MCAT score (let's say a 31), right?

It's an interesting issue; to what extend can we say that a higher score represents greater intelligence? How much of one's score is attributable to the MCAT's measurement of factors that we would agree represent aspects of intelligence? How certain can be we that someone who scores a 40 is at least somewhat more intelligence that someone who scores a 24? Absolutely? Not at all? What influences our answer?

D) @Ace-Co-A further noted problems with the strength of an MCAT score-intelligence relationship:

(1) MCAT scores vary from day-to-day. For example, Kim might score 34 one day, 33 the next, and 32 the day after. Which score is most representative of her intelligence? Is she smarter the first day (with her 34) than the third (with her 32)?

(2) Test scores vary depending on uncontrollable variables (the person might have a fever, her father might have died the night before, her untimely boyfriend proposed that morning, etc.)

(3) Test scores depend on the resources available to the individual (practice tests, books/information, problem sets, previous coursework, prep class, etc.)

These are legitimate issues that affect both the reliability and validity of the test itself. Standardization as a process acts to mitigate the various confounds in (2) to some extent, but the acknowledgement that they cannot be eliminated completely is part of why we have classical test theory to describe scores: one's score is always the sum of one's true, perfectly representative score and some unknown amount of error (X = T + E).

As an excellently reliable exam (r > 0.9) the MCAT has a standard error of measurement (SEM) of 2 composite points, which represents the typical degree to which one's reported score differs from one's true score. Examining the confidence intervals obtained when the SEM is known demonstrates that we should not distinguish between three scores of 34, 33, and 32, for example. All may be significantly representative of the test taker's true characteristics (as problematic as that may be for admissions purposes.)

My comment on (3) is simply that, depending on how relevant we think content knowledge is in terms of representing one's intelligence, the contribution of these external factors to the MCAT's measurement of one's intelligence may be smaller than would appear.

As a random thought, the MCAT is a multiple-choice test with no guessing penalty, so luck/chance/whatever you want to call it could account for significant reported score variance in some instances as well.

Conclusions: I personally think that the characteristics measured by the MCAT both directly and indirectly attempt to capture the test taker's intelligence (broadly defined), but acknowledge that there are certainly external factors that can affect both the ability of the exam to accurately measure what it attempts to and the representativeness of one's score. In general, however, I do think the exam does a pretty good job of measuring these intelligence-relevant characteristics. I would also emphasize that caution should be used in drawing strict conclusions and comparisons regarding intelligence when using MCAT scores as a measure, especially when scores are close or there are known external variables that may affect the reliability of the exam as a metric.

---

Members don't see this ad.

 

[Ace-Co-A]

Continuing a discussion between @Ace-Co-A, @sakabato93, and myself elsewhere on SDN, I'd like to consider the concept of intelligence and its relation to the MCAT. I can identify a number of core issues to be addressed:

A) What is intelligence? How can it be meaningfully defined?

B) What is the purpose of the MCAT? Does the test succeed in fulfilling its objectives?

C) What is the relationship between intelligence and performance on the MCAT?

D) What problems can be identified with the conclusions we might draw from answering the above?

Here's my take on these issues.

A) The concept of intelligence is complex, and often it can be useful to operationalize intelligence in the context with which one is concerned. For example, it is my opinion that intelligence in the most broad sense can be operationalized at least in part in terms of several characteristics, which fall into categories related to (1) memory and (2) processing.

(1) Characteristics of memory relevant to the concept of intelligence: memory capacity (long-term, short-term, and working memory stores; how much information one is able to retain in memory), memory encoding (how easily/quickly one can add new information to each of the memory stores), and memory persistence (the duration of time for which information can be retained in memory stores without retrieval or reinforcement).

(2) Characteristics of processing relevant to the concept of intelligence: processing speed (the rate at which manipulation of information can be carried out), and parallel processing (the extent to which distinct processes can be carried out simultaneously).

All of that is to say that I find it reasonable to say that between two individuals, the one who can retain more information in memory, add new information more quickly and easily into memory, retain it the longest, think the fastest, and think at the most complex level is (ignoring other differences) the more intelligent one. This approach of course leaves out important facets such as characteristics associated with attention and perception, but I'm really not qualified to flesh out the entire concept of intelligence here. Add to those such macro-scale abilities like creativity, social behavior, and others and you get a seriously complicated situation. The characteristics I've presented, however, I do feel are pretty core (though not comprehensive) to the idea of intelligence at least the way I think of it.

Open questions: What might intelligence mean in the context of physician performance? Which of my core characteristics do you agree or disagree with? What might you add? Would you approach defining the term differently, and if so, how?

B) According to the AAMC…



I'm not surprised that the word "intelligence" is nowhere to be found. From this statement, however, we can identify that the MCAT is designed to measure the following:

1) The amount of information retained in memory (this is the science content test takers are responsible for knowing), and by necessity the accessibility of this information as well.

2) The degree to which two behaviors can be performed: problem-solving and critical thinking.I'm not going to get into defining those behaviors, but anyone familiar with the MCAT would agree that the passage-style format involves, at least to some extent, the assimilation of new information into memory and the processing of that information in the context of background knowledge.

Further, because the MCAT is a timed test, I feel I can reasonably argue that the exam measures processing speed (it's not like the time limit is 3 hours per section; a decent pace is required and is frequently a limiting factor is test taker performance).

I'm actually going to steer clear of addressing whether the test succeeds or not. The MCAT has respectable validity with regard to predicting USMLE Step 1 performance, and is the product of decades of development by professionals with background in test design. Overall, it almost certainly works pretty well in measuring what it's trying to measure.

Open questions: Are problem-solving and critical thinking necessarily representative of intelligence? Given that the test explicitly attempts to measure test taker knowledge, does this necessarily represent intelligence? Why or why not?

C) One of the points @Ace-Co-A raised in prior discussion examined the relationship between MCAT performance and intelligence:



It's an interesting issue; to what extend can we say that a higher score represents greater intelligence? How much of one's score is attributable to the MCAT's measurement of factors that we would agree represent aspects of intelligence? How certain can be we that someone who scores a 40 is at least somewhat more intelligence that someone who scores a 24? Absolutely? Not at all? What influences our answer?

D) @Ace-Co-A further noted problems with the strength of an MCAT score-intelligence relationship:



These are legitimate issues that affect both the reliability and validity of the test itself. Standardization as a process acts to mitigate the various confounds in (2) to some extent, but the acknowledgement that they cannot be eliminated completely is part of why we have classical test theory to describe scores: one's score is always the sum of one's true, perfectly representative score and some unknown amount of error (X = T + E).

As an excellently reliable exam (r > 0.9) the MCAT has a standard error of measurement (SEM) of 2 composite points, which represents the typical degree to which one's reported score differs from one's true score. Examining the confidence intervals obtained when the SEM is known demonstrates that we should not distinguish between three scores of 34, 33, and 32, for example. All may be significantly representative of the test taker's true characteristics (as problematic as that may be for admissions purposes.)

My comment on (3) is simply that, depending on how relevant we think content knowledge is in terms of representing one's intelligence, the contribution of these external factors to the MCAT's measurement of one's intelligence may be smaller than would appear.

As a random thought, the MCAT is a multiple-choice test with no guessing penalty, so luck/chance/whatever you want to call it could account for significant reported score variance in some instances as well.

Conclusions: I personally think that the characteristics measured by the MCAT both directly and indirectly attempt to capture the test taker's intelligence (broadly defined), but acknowledge that there are certainly external factors that can affect both the ability of the exam to accurately measure what it attempts to and the representativeness of one's score. In general, however, I do think the exam does a pretty good job of measuring these intelligence-relevant characteristics. I would also emphasize that caution should be used in drawing strict conclusions and comparisons regarding intelligence when using MCAT scores as a measure, especially when scores are close or there are known external variables that may affect the reliability of the exam as a metric.

Thanks for this. I'll edit this response with some comments on your post tonight after work.

Edit: See post below.

 

[sakabato93]

I did not expect this. Normally, I would be ecstatic to be part of this conversation. Unfortunately, I'm studying abroad (finally after 3 years of gleeful anticipation; ) with a very intensive program, so I probably won't be able to dedicate to this thread as much as I would like. However, I am interested in this conversation, especially with the theoretical constructs that GTLO has created, so I'll keep tabs and attempt to give the occasional input.

At first, I was hesitant to participate because I thought that @Ace-Co-A was simply attacking me at first for an off-the-cuff comment, but I'm glad to see that the conversation is civil, intellectual and constructive

Faith in SDN restored.

 

[STLMako91]

I think the MCAT does not do a good job measuring intelligence. Is there a correlation? Sure, but there are too many external factors that have to be taken into account.

First of all, I think intelligence is just the speed at which an individual can fully understand new information. So I disagree with your definition of intelligence. Also, I would argue that the verbal reasoning section does have critical thinking, but it's 1/3 of an overall MCAT score and is the least reliable when it comes to measuring intelligence. The main reason is concerned with semantics and reading speed. A lot of intelligent people who don't spend a lot of time reading during childhood are behind in vocabulary and reading speed, even though their ability to grasp new information is just as fast as those who have grown up reading.

Therefore, the MCAT which is based on content, reading speed, semantics, and attention span does not do a sufficient job measuring intelligence. A good intelligence test has no content review and is based on pure logic and reasoning. The LSAT would be more representative but not fully since it too has a verbal reasoning section (I believe). Anyhow, the best tests for measuring IQ are…IQ tests!

 

[gettheleadout]

I think the MCAT does not do a good job measuring intelligence. Is there a correlation? Sure, but there are too many external factors that have to be taken into account.

First of all, I think intelligence is just the speed at which an individual can fully understand new information. So I disagree with your definition of intelligence.

How would you operationalize your definition? That is, how can we measure "full" understanding of new information?

Also, I would argue that the verbal reasoning section does have critical thinking, but it's 1/3 of an overall MCAT score and is the least reliable when it comes to measuring intelligence.

Do you believe there is no significant critical thinking aspect to the PS and BS sections?

The main reason is concerned with semantics and reading speed. A lot of intelligent people who don't spend a lot of time reading during childhood are behind in vocabulary and reading speed, even though their ability to grasp new information is just as fast as those who have grown up reading.

Therefore, the MCAT which is based on content, reading speed, semantics, and attention span does not do a sufficient job measuring intelligence. A good intelligence test has no content review and is based on pure logic and reasoning. The LSAT would be more representative but not fully since it too has a verbal reasoning section (I believe). Anyhow, the best tests for measuring IQ are…IQ tests!

Given that you believe the MCAT is a poor measure of intelligence, do you think it should be made a better measure? That is, does the MCAT need to be a good measure of intelligence to be a useful exam?

 

[STLMako91]

How would you operationalize your definition? That is, how can we measure "full" understanding of new information?


Do you believe there is no significant critical thinking aspect to the PS and BS sections?


Given that you believe the MCAT is a poor measure of intelligence, do you think it should be made a better measure? That is, does the MCAT need to be a good measure of intelligence to be a useful exam?

Well, I think there may be an inherent incapability of measuring a full understanding of new information since no one can fully understanding anything. It would be a lot easier if someone knew, with full certainty, everything about something and assess someone's knowledge base, but we only operate based on simple models of reality. However, you can get a good feel of someone's intelligence based on how quickly they seem to understand newly presented information just by honest observation; it would be even better if we could somehow get in that person's head and see for ourselves, but that's beyond the pale of reality.

I do think there is a lot of critical and logical thinking on the PS and BS sections, and I do think those tend to measure intelligence the best, but I noted the verbal section because the verbal still contributes to a whopping 1/3 of the overall score, and that is still a really large margin of inefficiency in the test itself. I definitely favor the PS and BS sections as measures of intelligence, but I still can't give it full credit because it is still knowledge based. A real IQ test assesses no prior knowledge or experience whatsoever; it is simply based on someone's raw ability to grasp newly presented information.

I think the MCAT should do a better job of measuring IQ. I think it should introduce more logic games and no verbal, but that's just my opinion. Other people favor the verbal because it measures other attributes such as communication ability, and I can respect that. Ultimately, I wish medical schools just took IQ scores and based GPAs as a way to assess someone's willingness to study vast amounts of information.

 

[Rhino1000]

I definitely favor the PS and BS sections as measures of intelligence, but I still can't give it full credit because it is still knowledge based. A real IQ test assesses no prior knowledge or experience whatsoever; it is simply based on someone's raw ability to grasp newly presented information.

Are you aware that currently accepted IQ tests (WAIS, Stanford-Binet) do in fact consider knowledge to be a great indicator of intellect? For example, vocabulary has a very high (I've often seen figures around r=.8) correlation to general intelligence compared to the other subsections of the major IQ tests, if not the greatest; it is also weighted as such. Perhaps it is because it takes intelligence to develop semantic memory of difficult words that one rarely comes across. Perhaps intelligent people tend to be more interested in concepts/knowledge in general, and so are able to develop their knowledge to a higher degree. It's possible that intelligent people tend to be introverted, which inspires them to spend more time reading or studying. Whatever be the reason for the high correlations between vocabulary (and knowledge) and intelligence, it is a fact that they correlate well, and therefore are good predictors of general intelligence. If we are trying to create a test that is most likely to distinguish intellects, I think it would be foolish to not give knowledge its due credit. So the fact that the MCAT tests knowledge may not be, in itself, a detriment to distinction. Ideally, we would be able to directly measure intelligence like you imagine, but as it is, correlations are some of the most useful tools we have. Just like most American (for example) people who take IQ tests have been around English words their entire life and so vocabulary is a viable way to predict intellectual force, most people who have taken the MCAT have at least been exposed to the material at some point in their lives, and so the relative acquisitions of the relevant material by students may be a satisfactory distinguishing factor to compare them.

 

[thatfeelwhen79]

zzzzzzzzzzzzzzzzzzzzzzzzzzz

 

[Rhino1000]

I am going into fourth year of theoretical mathematics. I'm beyond gifted in abstract math. My IQ is ~126-127 (99.7th percentile), with one subcategory of "cognitive speed" being extremely below average, 87 (~15 below average). I am struggling to get a 9 in verbal because I'm "slow". So in a way, I guess mcat verbal tests a subcategory of intelligence, but not the logic/spatial reasoning in most IQ tests.

Edit: Letting it go.

 

[thatfeelwhen79]

zzzzzzzzzzzzzzzzzzz

 

[EMDO2018]

Yes a better score on the MCAT correlates with better memory, and better reasoning ability to a lesser extent . Its unfair, but it is what it is. I know people who took the MCAT 3-4 times and couldn't get over 25, and a person who took it once and got 40. The amount of material tested in the MCAT is too much for anybody to remember all of the concepts, the person who gets the higher score remembers more.

 

[STLMako91]

Are you aware that currently accepted IQ tests (WAIS, Stanford-Binet) do in fact consider knowledge to be a great indicator of intellect? For example, vocabulary has a very high (I've often seen figures around r=.8) correlation to general intelligence compared to the other subsections of the major IQ tests, if not the greatest; it is also weighted as such. Perhaps it is because it takes intelligence to develop semantic memory of difficult words that one rarely comes across. Perhaps intelligent people tend to be more interested in concepts/knowledge in general, and so are able to develop their knowledge to a higher degree. It's possible that intelligent people tend to be introverted, which inspires them to spend more time reading or studying. Whatever be the reason for the high correlations between vocabulary (and knowledge) and intelligence, it is a fact that they correlate well, and therefore are good predictors of general intelligence. If we are trying to create a test that is most likely to distinguish intellects, I think it would be foolish to not give knowledge its due credit. So the fact that the MCAT tests knowledge may not be, in itself, a detriment to distinction. Ideally, we would be able to directly measure intelligence like you imagine, but as it is, correlations are some of the most useful tools we have. Just like most American (for example) people who take IQ tests have been around English words their entire life and so vocabulary is a viable way to predict intellectual force, most people who have taken the MCAT have at least been exposed to the material at some point in their lives, and so the relative acquisitions of the relevant material by students may be a satisfactory distinguishing factor to compare them.

Your method of measuring intelligence is doing so in an indirect fashion by measuring some of the byproducts of intelligence opposed to intelligence itself. The problem with that is that you are exposing yourself to a multitude of external variables that can create too much variance and contamination towards your results. The better approach would be to directly measure intelligence by providing new material and seeing how quickly participants can understand and potentially answer questions about it (I know there is no perfect way to to this). This approach takes away from many of the external factors that create a less accurate and representative test. Is there a correlation between MCAT score and IQ? Of course. However, if you want a more realistic test, stay away from correlations as much as possible and make a test that assesses how exactly intelligence is defined, which is the speed at which an individual can grasp new information. This is a more effective approach than to make measurements indirectly by attempting to measure correlated byproducts of intelligence. The best approach is to measure intelligence itself, not what tends to result from it.

 

[orangetea]

I haven't really read through some of the responses as much as I would have liked to but I just wanted to share my 2 cents as a part of my MCAT study break.

Personally, I think it's reasonable to ask individuals to have the basic knowledge of how the human body works, critical thinking skills, and a good intuition especially if they are pursing a career that requires quit a bit of responsibility(that is totally an understatement). But with that said it doesn't measure intelligence.. I mean what is really intelligence anyway? Is supposed to be a number?!? Because if that's the case why don't they just distribute an IQ test and allow certain brackets in!?!? My mind is kind of not working at the moment .. to many physics equations but I always wondering if not the MCAT what other factor(s) could they use besides the typical GPA extracurricular.. ??

Anyway great thread I am excited to read some of the responses

 

[deleted439911]

So I'm gonna post this first without reading any replies. But just so I don't get influenced by prior replies. And I respect if people have a different opinion or idea, these are just mines for now, as I do think my idea will change after reading the top posts.

1. Intelligence is the ability to learn, apply knowledge, and adapt to a given field. That is I think intelligence should be specified to certain things.

I stated intelligence as ability to learn, apply, and adapt, so it'll be measured based on these three. To measure learning, we might want to see how fast/slow it takes someone to learn something to a proficient level (e.g. how long it takes one to learn conversational Spanish). I think grading someone like giving them an A or B gives a rough measure of learning. But I don't think "apply" and "adapt" criteria are so easily measured as these are really determined on chance occurrence. That being said, I think intelligence even when specified to a field can only be measured up to a point.

2. I'm not a Med student or Adcom. So I will just say what I think the MCAT is...
-It assures the University a student has a proficient level of understanding of the sciences.
-Helps University know whether the student have good enough knowledge for med school. Someone with <XX will probably not pass their courses.
-By that, it helps University do a quick cut off, e.g. don't bother with people <YY.

But here's what I think the MCAT isn't
-Doesn't let the University know the student will be a "Good" Doctor.
-Doesn't assure the University that the student can actually adapt their scientific knowledge to chance situation.
-I think this is the case even with people having 40+
-Measures general intelligence. I think anyone having spent enough time studying the MCAT can get 40+ maybe some just longer than others.

3. I assume you're talking about "general intelligence" smart people who seems to learn things fast in pretty much anything. But, I'm gonna stick with talking about "specific intelligence". In that case, MCAT measures specifically how well you know your sciences in PS and BS, and by verbal they measure whether you can comprehend an article and limitations of the article.

4. I think the main problem is what I stated in what the MCAT doesn't tell you someone will be a good doctor, or even a good medical researcher. There may be some statistics say... higher MCAT tend to lead to better researchers, but that's likely to be an association not causation. Plus, many great discoveries were done by somewhat eccentric people (Endosymbiosis) and some great doctors were thought to be too radical (handwashing!). I'm confident that at least in the USA we don't rely too much on scoring such that East Asian countries might, as med schools seek other qualifying things such as experience, community involvement, and even hobbies. But I am worried that some med schools who have too high of a MCAT score might cut off potentially great candidates.

 

[Rhino1000]

Your method of measuring intelligence is doing so in an indirect fashion by measuring some of the byproducts of intelligence opposed to intelligence itself. The problem with that is that you are exposing yourself to a multitude of external variables that can create too much variance and contamination towards your results. The better approach would be to directly measure intelligence by providing new material and seeing how quickly participants can understand and potentially answer questions about it (I know there is no perfect way to to this). This approach takes away from many of the external factors that create a less accurate and representative test. Is there a correlation between MCAT score and IQ? Of course. However, if you want a more realistic test, stay away from correlations as much as possible and make a test that assesses how exactly intelligence is defined, which is the speed at which an individual can grasp new information. This is a more effective approach than to make measurements indirectly by attempting to measure correlated byproducts of intelligence. The best approach is to measure intelligence itself, not what tends to result from it.

I addressed measuring intelligence directly already, and I see that you seem to have noticed that. The problem is, intelligence is a very broad concept that may have multiple components, such as long-term semantic memory, working memory, pfft, arithmetic ability or what have you. If you define intelligence as working memory and test working memory as such, then many experts on the subject would object saying, "hey guy, intelligence isn't synonymous with working memory, you totally forgot about processing speed, so your test is invalid" (etc.); multiple tests are necessary or are at least useful, which is why all the major IQ tests right now don't only measure one specific skill. In other words, there isn't one specific test that is considered to be able to capture the thing deemed as general intelligence better than multiple tests would be able to do. When you try to account for each of the different components of intelligence (even as you deem fit), and let's say you test a group of subjects with respect to each of them and analyze the results, you realize that they have positive intercorrelations with eachother. With these positive correlations, you are able to create a statistical construct that is able to better describe the person's overall intelligence than any single ability test is able to. The empirical fact is, things like vocabulary match up sometimes very accurately with these statistical constructs, rendering it possible for things like vocabulary to be accurate approximations of them, or at least be components of some additional statistical construct that is able to predict that other aforementioned statistical construct accurately.

Just because we estimate your future SAT score from your ACT score using statistical methods, that doesn't mean that because we are using statistical correlations to interconvert the two, that we are adding all kinds of contamination and variance compared to predicting your future ACT score from a former ACT score. The reason for this is that your ACT score itself can have variance with scores from future administrations, rendering any inconsistencies between the two more or less negligible. Certainly a strong correlation would exist, and one could accurately predict the one from the other.

 

[Gauss44]

Disparity related to culture/community of origin:

There's a lot of good to the MCAT.

One place it chronically fails, especially in verbal reasoning, is that it favors a certain "type of thinking" that has nothing to do with intelligence. It has to do with where the test-taker grew up, and how people communicate in that community.

One example of what I mean by, "type of thinking," on the MCAT is a passage/question stem/answer choice that requires the test taker to interpret EVERYTHING 100% literally, or to choose the proper English interpretation. I believe that this comes naturally for some people who are from educated communities. On the other hand, a person/test-taker from other communities (especially under educated ones), will be inclined to pick the more commonly used interpretation. In real life, for example, when I hear some people use a double negative, I know what they mean (and "what they mean" is NOT the literal proper English meaning). A test-taker from certain communities will be tempted to think that the MCAT is testing people skills (i.e. wants them to interpret "not never" as emphasizing the "not" part, making it a stronger statement than a single negative). Sure, they learned otherwise in English class, but a "people person" with a certain background will wonder what the test-maker really wants. They will wonder if the MCAT wants the "people person" answer (i.e. test-taker can relate to those without perfect grammar), or the literal hard-headed smart-ass robotic answer (is what they'll think). Some will just go with his/her instinct/habits and check the wrong box. Double negatives are NOT the only example of this, but on the test, I've noticed several scenarios where one's background will influence their answers.

To solve this, I would suggest that the AAMC run a focus group, using people from different backgrounds, to study this disparity.

(While I suspect some people might be able to overcome this. I believe it presents an additional setback relative other pre-meds from more educated communities.)

Another example of background influencing answer choice, is when a passage is about something other than science, and the person taking the test happens to be an expert (or just knowledgeable in that non-science field). Occasionally, there is a situation where the passage DOES provide evidence for answer choices that the test-maker seemed unaware of, due to the test-maker's lack of expertise in the field the passage is about. An Examkrackers example would be page 178 in the 2008 EK VR 101 book, #4, if you consider media research, a phenomenon EK didn't seem to consider. (SDN: I don't believe that I made a spoiler. If you disagree, feel free to erase that last line.) I know there's a couple AAMC examples but they're not coming to mind right now.

 

[mehc012]

Disparity related to culture/community of origin:

There's a lot of good to the MCAT.

One place it chronically fails, especially in verbal reasoning, is that it favors a certain "type of thinking" that has nothing to do with intelligence. It has to do with where the test-taker grew up, and how people communicate in that community.

One example of what I mean by, "type of thinking," on the MCAT is a passage/question stem/answer choice that requires the test taker to interpret EVERYTHING 100% literally, or to choose the proper English interpretation. I believe that this comes naturally for some people who are from educated communities. On the other hand, a person/test-taker from other communities (especially under educated ones), will be inclined to pick the more commonly used interpretation. In real life, for example, when I hear some people use a double negative, I know what they mean (and "what they mean" is NOT the literal proper English meaning). A test-taker from certain communities will be tempted to think that the MCAT is testing people skills (i.e. wants them to interpret "not never" as emphasizing the "not" part, making it a stronger statement than a single negative). Sure, they learned otherwise in English class, but a "people person" with a certain background will wonder what the test-maker really wants. They will wonder if the MCAT wants the "people person" answer (i.e. test-taker can relate to those without perfect grammar), or the literal hard-headed smart-ass robotic answer (is what they'll think). Some will just go with his/her instinct/habits and check the wrong box. Double negatives are NOT the only example of this, but on the test, I've noticed several scenarios where one's background will influence their answers.

To solve this, I would suggest that the AAMC run a focus group, using people from different backgrounds, to study this disparity.

(While I suspect some people might be able to overcome this. I believe it presents an additional setback relative other pre-meds from more educated communities.)

Another example of background influencing answer choice, is when a passage is about something other than science, and the person taking the test happens to be an expert (or just knowledgeable in that non-science field). Occasionally, there is a situation where the passage DOES provide evidence for answer choices that the test-maker seemed unaware of, due to the test-maker's lack of expertise in the field the passage is about. An Examkrackers example would be page 178 in the 2008 EK VR 101 book, #4, if you consider media research, a phenomenon EK didn't seem to consider. (SDN: I don't believe that I made a spoiler. If you disagree, feel free to erase that last line.) I know there's a couple AAMC examples but they're not coming to mind right now.

Furthermore, I would argue that a VR which draws from a more diverse set of cultural perspectives would
a) go a lot further into promoting the diversity which schools claim to desire, as well as social/empathy skills which the AAMC claims to be trying to target with its new curriculum.
b) actually allow students to demonstrate more critical thinking and ability to adapt their thought patterns to fit a situation. If all passages adhere to one cultural norm, those who think only that way will have the advantage. If there is variety in the test, those who are able to consider things from different angles would do better.

 

[Gauss44]

Intelligence: Overall aptitude of the mind. Ability of the mind to function in any or every context. How good of a "rig" your brain.
Knowledge: Information that can be stored in a mind. Knowledge =/= (is unequal to) intelligence.
MCAT Purpose: To test one's skill in physics, chemistry, biology, and verbal reasoning. That's ALL the MCAT does. To say it does anything else, IMO, would require making ASSUMPTIONS that may be true in some situations and false in others.
Is there a relationship between intelligence and MCAT performance?: You need a certain amount of minimal intelligence to do well on the MCAT. Other factors influence MCAT performance as well such as how long you were able to study, if you could afford prep material, tutors, etc.
What problems can be identified?: I don't have enough time to answer that right now.
What might intelligence mean in the context of physician performance? A certain level of "intelligence" or cognitive function, IMO, should be a minimum prerequisite for becoming a physician, but much more is to be desired: Emotional and social intelligence, character, moral compass, and people skills to name a few.
Which of my core characteristics do you agree or disagree with? They all sound pretty good, but I think there's more to it as you mentioned.

 

[mk04447]

You're all dorks!

Having said that I will chime in LOL.

The MCAT measures nothing but your relative position among other applicants. It is surprising to me that nobody pays particular attention to the fact that the test is curved over 30 days. That is why the time limit exists, they can't have anyone finish the damn thing. The AAMC is given the statistical thumbs up by so many but they never mention it being designed that way. It is a perfect scam, the questions don't even have to be that similar, just throw them out. It needs to be variable enough to fashion a bell, that's it.

So what is it measuring? I would argue hard work. The fact is most premeds are for lack of another word, kids, without other obligations. Who put in the time and who didn't. Of course a genius will have an easier time of anything requiring aptitude. Similarly, someone who reads at twice the normal rate will have an easier time with VR, they can read it twice. These anomalies don't impact anything, the sample size is too large, 80-100k each year.

Hence the correlation with med school and the USMLE. Unfortunately despite many people's belief to the contrary, humans are really pretty comparable. The differences between us are small on the whole. You don't have to be an anomaly or a prodigy to succeed in med school you just have to be willing to work at it... Most folks aren't. Sometimes the simple answer is the best answer.

 

[gettheleadout]

The MCAT measures nothing but your relative position among other applicants. It is surprising to me that nobody pays particular attention to the fact that the test is curved over 30 days. That is why the time limit exists, they can't have anyone finish the damn thing. The AAMC is given the statistical thumbs up by so many but they never mention it being designed that way. It is a perfect scam, the questions don't even have to be that similar, just throw them out. It needs to be variable enough to fashion a bell, that's it.

This is absolutely false.

 

[Shams al Deen]

The biggest problem to me is that studying for the test will always raise your score, and hours spent studying has little to do with "intelligence." Even for the 35+ scoring folks, three months before their test date they were likely scoring high teens or low 20s. Did they miraculously increase their intelligence in 3 months? So when comparing someone with a 25 versus someone with a 37, you cannot say the difference in their scores is due to intelligence. It is much more likely that the difference in scores is due to the fact that one student studied a lot more than the other (either throughout undergrad or for the test specifically). Something like an IQ test takes away a lot of that studying bias that the MCAT has. Also, I hear the new SAT is being designed so that studying for it won't help as much. The new MCAT seems to being going the other way on that, probably adding another 4 weeks to the normal 3 month study plan.

And I agree with OP's description of intelligence. I think it definitely should encompass things like how quickly you memorize new material, how long you keep the content knowledge, etc. But what if the 37 scorer spent 3 months doing content review, and the 25 scorer spent a week?

 

[Mad Jack]

The biggest problem to me is that studying for the test will always raise your score, and hours spent studying has little to do with "intelligence." Even for the 35+ scoring folks, three months before their test date they were likely scoring high teens or low 20s. Did they miraculously increase their intelligence in 3 months? So when comparing someone with a 25 versus someone with a 37, you cannot say the difference in their scores is due to intelligence. It is much more likely that the difference in scores is due to the fact that one student studied a lot more than the other (either throughout undergrad or for the test specifically). Something like an IQ test takes away a lot of that studying bias that the MCAT has. Also, I hear the new SAT is being designed so that studying for it won't help as much. The new MCAT seems to being going the other way on that, probably adding another 4 weeks to the normal 3 month study plan.

And I agree with OP's description of intelligence. I think it definitely should encompass things like how quickly you memorize new material, how long you keep the content knowledge, etc. But what if the 37 scorer spent 3 months doing content review, and the 25 scorer spent a week?

First AAMC practice test was a 36, last practice test was a 36, actual score 35. I've met plenty of people that started in the teens and ended in the mid 20s after many months of rigorous study. I think studying helps you insofar as you can't answer questions that cover material you are unfamiliar with, but once you are familiar with all of the material, your maximum score is limited by your intelligence, as the MCAT is more of a critical thinking test than anything. The answers are hidden in the questions more often than not.

What separates those who do well from those with exceptional scores is the ability to make oftentimes extremely abstract connections between the passage data and questions. I had a couple passages where I hardly remembered the material, but was just able to logically piece together "given the three questions provided and what I know about general physics principles, the only possible series of answers is X, Y, and Z." It's a hard process to describe, but basically is like you're staring at a puzzle and the longer you stare the more you see how everything fits together until it can only fit in one possible way. This is the sort of thinking the MCAT is looking for, and it requires a great deal of fluid intelligence with a foundation built upon at least average crystallized intelligence. You can study for a year, but it doesn't help if you can't draw the right connections quickly, or, at the very least, accurately retain the information.

 

[UoCfin]

@gettheleadout
@Shams al Deen
@Gauss44

Concerning 1) memory:

Where am from we have a thing (not to get too involved here-it is a religious thing). Kids on the onset, say about 7+ either take 2-3 years off or concurrently with other secular subjects memorize a 600 page religious book. For those kids who continue to pursue that route of *knowledge* some, but usually few comparatively go on to 'memorize' bigger and i mean BIGGER volumes of books [ big= 8 volumes of narrations *each volume containing approximately 1000-2000 narrations]. But not only the 8 volumes of 1 book, but say about 6 different books each having about 4-8 volumes. These select people turn out to be scholarly and rightfully so.

Concerning 2) processing:

As for the many kids and youth who memorize the one religious book, it is of no surprise that all you need to do is open up the book *about 6000+ verses* and start reading from which the kid/youth who has memorized will pick up where u are and keep going.

As for the later, the scholarly class, even more surprising is their ability to answer questions with relevant quotations that support the premise of your dilemma or inquiry from any of those 6 books [4-8 volumes here and there].

Conclusion/Query: We can all say that perhaps the 1 book (600 pages) having been memorized by about 1 million so far is a feat many of us can attain, but the inquiry here is with that scholarly class that compounds volumes and volumes of material from memory and stores it in their long-term memory only to retrieve it so quickly with a simple question. Knowledge/Intelligence...? Just something I can't define yet properly and it does propose an intelligent debate like the one above.

 

[orangeblue]

We ought to also keep in mind that there are various types of intelligence: emotional, physical, mental (and even that in creative pursits such as a film, book or standarized test etc).

 

[gettheleadout]

We ought to also keep in mind that there are various types of intelligence: emotional, physical, mental (and even that in creative pursits such as a film, book or standarized test etc).

I'm familiar with Gardner's multiple intelligences, but no doubt some would be more relevant to the practice of medicine than others.

 

[Ace-Co-A]

Sorry I haven't responded to this yet. I've been randomly wicked busy the past couple days, but I certainly don't want to develop a reputation for running away from a fight.

I'll post soon (hopefully tomorrow).

 

[gettheleadout]

Sorry I haven't responded to this yet. I've been randomly wicked busy the past couple days, but I certainly don't want to develop a reputation for running away from a fight.

I'll post soon (hopefully tomorrow).

Haha no fighting here, Ace.

 

[Ace-Co-A]

Very well. I've read through the responses so far, and I think the one that most interests me is @gettheleadout 's first post:

A) What is intelligence? How can it be meaningfully defined?

A) The concept of intelligence is complex, and often it can be useful to operationalize intelligence in the context with which one is concerned. For example, it is my opinion that intelligence in the most broad sense can be operationalized at least in part in terms of several characteristics, which fall into categories related to (1) memory and (2) processing.

(1) Characteristics of memory relevant to the concept of intelligence: memory capacity (long-term, short-term, and working memory stores; how much information one is able to retain in memory), memory encoding (how easily/quickly one can add new information to each of the memory stores), and memory persistence (the duration of time for which information can be retained in memory stores without retrieval or reinforcement).

(2) Characteristics of processing relevant to the concept of intelligence: processing speed (the rate at which manipulation of information can be carried out), and parallel processing (the extent to which distinct processes can be carried out simultaneously).

All of that is to say that I find it reasonable to say that between two individuals, the one who can retain more information in memory, add new information more quickly and easily into memory, retain it the longest, think the fastest, and think at the most complex level is (ignoring other differences) the more intelligent one. This approach of course leaves out important facets such as characteristics associated with attention and perception, but I'm really not qualified to flesh out the entire concept of intelligence here. Add to those such macro-scale abilities like creativity, social behavior, and others and you get a seriously complicated situation. The characteristics I've presented, however, I do feel are pretty core (though not comprehensive) to the idea of intelligence at least the way I think of it.

Open questions: What might intelligence mean in the context of physician performance? Which of my core characteristics do you agree or disagree with? What might you add? Would you approach defining the term differently, and if so, how?

I have not studied the psychology of intelligence in detail, and so it would be inappropriate for me to criticize your outline of the core characteristics of intelligence. Furthermore, these characteristics (memory and processing) are very reasonable, and will likely suffice for present purposes.

Your first question here ("What might intelligence mean in the context of physician performance?") deserves some attention. It seems to me that the two characteristics (memory and processing) account for the skills required of a physician. Any challenges to this?

B) What is the purpose of the MCAT? Does the test succeed in fulfilling its objectives?

B) According to the AAMC…

I'm not surprised that the word "intelligence" is nowhere to be found. From this statement, however, we can identify that the MCAT is designed to measure the following:

1) The amount of information retained in memory (this is the science content test takers are responsible for knowing), and by necessity the accessibility of this information as well.

2) The degree to which two behaviors can be performed: problem-solving and critical thinking.I'm not going to get into defining those behaviors, but anyone familiar with the MCAT would agree that the passage-style format involves, at least to some extent, the assimilation of new information into memory and the processing of that information in the context of background knowledge.

Further, because the MCAT is a timed test, I feel I can reasonably argue that the exam measures processing speed (it's not like the time limit is 3 hours per section; a decent pace is required and is frequently a limiting factor is test taker performance).

I'm actually going to steer clear of addressing whether the test succeeds or not. The MCAT has respectable validity with regard to predicting USMLE Step 1 performance, and is the product of decades of development by professionals with background in test design. Overall, it almost certainly works pretty well in measuring what it's trying to measure.

Open questions: (1) Are problem-solving and critical thinking necessarily representative of intelligence? (2) Given that the test explicitly attempts to measure test taker knowledge, does this necessarily represent intelligence? Why or why not?

To question (1), I emphatically affirm this. As justification, I offer a question: What use would there be for memorization of a vast amount of information if the person were unable to apply this 'knowledge'? I assume that 'problem-solving' and 'critical thinking' are other ways of saying 'processing', one of the core characteristics of intelligence named above. This is especially clear when thinking about medicine, where it is imperative that a physician both retains a large quantity of information and applies this knowledge to particular situations.

To question (2), although wary of the rhetorical games that might be played here, yes, I believe that a measurement of knowledge should represent that person's intelligence.

C) What is the relationship between intelligence and performance on the MCAT?

D) What problems can be identified with the conclusions we might draw from answering the above?

C) One of the points @Ace-Co-A raised in prior discussion examined the relationship between MCAT performance and intelligence:

It's an interesting issue; to what extend can we say that a higher score represents greater intelligence? How much of one's score is attributable to the MCAT's measurement of factors that we would agree represent aspects of intelligence? How certain can be we that someone who scores a 40 is at least somewhat more intelligence that someone who scores a 24? Absolutely? Not at all? What influences our answer?

This is the most interesting part of your post to my eye, and I'll conclude with this. [I note that my comments quoted in Part D are plausible reasons underlying this overarching concern (i.e., reasons that the MCAT score can vary from day-to-day and depending on external conditions).] It's funny because my reaction very much depends on the comparison being made. That is, I tend to agree with statements like "Someone who scored a 40 is smarter than someone who scored a 24." However, I am hesitant to state that someone with a 34 is smarter than someone with a 31. You also appreciate this hesitation in your own concluding statement:

Conclusions: ... I would also emphasize that caution should be used in drawing strict conclusions and comparisons regarding intelligence when using MCAT scores as a measure, especially when scores are close or there are known external variables that may affect the reliability of the exam as a metric. (Italics mine)

Frankly, I don't have an answer to this. I also admit that some of my hesitation here is not defensible: I believe that medicine is a field in which collegiality and solidarity are important - to believe that some physicians are smarter than others seems to undermine this belief. My uncomfortable feeling here, however, has no bearing on the truth of the thing.

Nonetheless, I can elaborate on some of the reasons I think that comparisons between people with different MCAT scores show the ridiculousness of trying to use MCAT score as an indication of intelligence. You ask "How certain can be we that someone who scores a 40 is at least somewhat more intelligence that someone who scores a 24? Absolutely? Not at all? What influences our answer?" I'll continue this discussion to highlight my reasons for hesitation. We agree that the MCAT does relate to intelligence. However, our answer is influenced by whether we accept or deny the mutability of the MCAT score of either individual. If the person with the 24 tested again and scored a 35, we would have to decrease our certainty that the person with the 40 (let's say she equals her score of 40 on the second administration) is the same amount more intelligent than the other individual. Our suspicion that external factors were involved in this difference in performance is therefore increased [assuming two categories of factors that affect performance: (1) internal (intelligence) and (2) external (see Part C)].

I guess I've given enough of a response here to continue the conversation. You will, of course, direct my attention to parts of my response or previous parts that require clarification or elaboration.

A pleasant weekend to all readers.

 

[Purplownz]

Another serious (but obvious) confounder in attempting to relate MCAT scores to intelligence is the inherent preparation that goes into achieving good results. I know several of my classmates who scored high twenties/low thirties on diagnostics but ended up with 40+ on the real thing a few months later. Neither their processing speed nor memory capacity likely changed significantly in that time frame. Had they been more lax with their preparation (like some of my other friends) they might have ended up with scores in the mid thirties. Just knowing someone's name and the associated MCAT score leaves out so much information that few reasonable conclusions can be drawn. Maybe they only used Kaplan verbal without ever hearing about the AAMCs and failed to "get used to" the style of the test makers, resulting in a much lower verbal score. Is their processing speed/reading comprehension any different in this scenario? Doubtful. Wildly differing access to study resources/information + test day catastrophes (anxiety?) that people mentioned need to be taken into account.

The other significant question is whether the MCAT *should* test intelligence. Do the most intelligent people make the best doctors? Maybe, maybe not. However, one could argue that in the context of med school admissions the MCAT should test intelligence because determination, work ethic, compassion, and personability (other important characteristics) may be assessed via other means such as GPA, school prestige, ECs, and the interview. In reality, I'm guessing that the most successful MS1 and MS2 students are at least fairly intelligent and are motivated enough to assimilate large amounts of information over short periods of time. To that end, the MCAT can serve as a useful tool for measuring not only intelligence, but drive and determination (who was motivated enough to master all 4 sciences, do tons of verbal passages, and a gazillion practice tests in order to do well). However, I'd agree that being exceptional in the former (intelligence) can mask deficiencies in the latter (but that both are represented by one's MCAT score).

 

[mk04447]

Luck has its place in MCAT scoring... the poster who said friends who consistently scored high 20's and low 30's yet achieved 40+ on the real test demonstrate this. But, I don't believe this is "luck" in the sense that test takers are consistently choosing a, b, c, or d correctly with more frequency. Rather, test takers can choose a lucky test date. Because of the vast range of material with the potential to appear on an MCAT, someone could simply perform well on a test that plays to their strong areas. I think we can all agree there are subjects we'd prefer to see over others.

"Luck" alone is enough to say that someone who scores a 40 is not necessarily more intelligent than someone with a 20. Further, going down this road makes me continue to assert that the MCAT doesn't test anything but the relative study practices and background science of the test takers themselves. Clearly, Newton would score less than 20 on an MCAT because necessary information wasn't available during his time, but you wouldn't say he was anything other than a genius.

This thread continually over complicates simple matters... Of course a more intelligent person is likely to ultimately test better for a variety or reasons, from vocabulary to instinct gained through successful life experience, but that is it. The MCAT is no more capable than the stupid quizzes my girlfriend likes to take in her Cosmo magazine at predicting intelligence.

 

[Purplownz]

Luck has its place in MCAT scoring... the poster who said friends who consistently scored high 20's and low 30's yet achieved 40+ on the real test demonstrate this. But, I don't believe this is "luck" in the sense that test takers are consistently choosing a, b, c, or d correctly with more frequency. Rather, test takers can choose a lucky test date. Because of the vast range of material with the potential to appear on an MCAT, someone could simply perform well on a test that plays to their strong areas. I think we can all agree there are subjects we'd prefer to see over others.

"Luck" alone is enough to say that someone who scores a 40 is not necessarily more intelligent than someone with a 20. Further, going down this road makes me continue to assert that the MCAT doesn't test anything but the relative study practices and background science of the test takers themselves. Clearly, Newton would score less than 20 on an MCAT because necessary information wasn't available during his time, but you wouldn't say he was anything other than a genius.

This thread continually over complicates simple matters... Of course a more intelligent person is likely to ultimately test better for a variety or reasons, from vocabulary to instinct gained through successful life experience, but that is it. The MCAT is no more capable than the stupid quizzes my girlfriend likes to take in her Cosmo magazine at predicting intelligence.

Excellent point. I'm willing to bet that most SDNers could beat Einstein on the MCAT. Now if he were to do a post-bac (post PhD haha?), take all of the pre-reqs again, and study seriously with SN2ed's schedule could he break 40? Probably (assuming that he doesn't get destroyed by verbal, which by the way is another issue: how do you account for geniuses whose native language is not English?) But his eventual 40+ would be the result of his willingness to do all of the aforementioned things, rather than his innate intellect.

 

[Drrrrrr. Celty]

Intelligence in my opinion has to do with your capacity and ability to learn. If you can learn to tie your shoes you're intelligent for a 2 year old, etc.

The mcat obviously does test something we may consider calling intelligence or factors that pertain to it. That being said I don't think the PS or BS do a significantly effective job at probing intelligence. PS & BS questions frequently range from minute and random to apply and acknowledge. I believe that in truth, the PS section's questions such as rate laws or energy test intelligence the best, where as questions such as which of the following are an example of a metathesis reaction ( A reaction that you've never learned about reasonable or if you've read in your studying it was mentioned once) is not.

Verbal can however probe into intelligence and one's ability to collect and apply information obtained in a passage. Verbal is unique in that it doesn't require any outside information other than a general grasp of basic paradigms of literature and philosophy. The answer is in the passage, you just need to read it and then sow it together in a neat and concise sentence.

But honestly, I just think it's a really crappy test as a whole. And any individual measure within it of intelligence can be better obtained from other measures.

 

[Drrrrrr. Celty]

Excellent point. I'm willing to bet that most SDNers could beat Einstein on the MCAT. Now if he were to do a post-bac (post PhD haha?), take all of the pre-reqs again, and study seriously with SN2ed's schedule could he break 40? Probably (assuming that he doesn't get destroyed by verbal, which by the way is another issue: how do you account for geniuses whose native language is not English?) But his eventual 40+ would be the result of his willingness to do all of the aforementioned things, rather than his innate intellect.

Tbh, I doubt Einstein would have accomplished half the things he did if we had our obsession on 'cheap' standardized tests as we do today. I'm not big on the whole, my child isn't good at tests thing. But standardized tests are a different beast entirely.

And essentially you cannot. A person who's native language is not English will spend time converting the passages into his own preferred language, then converting the answer back into English. It essentially means that as a non-native speaker you're hemorrhaging a lot of time just purely on making sense of things.

 

[gettheleadout]

Very well. I've read through the responses so far, and I think the one that most interests me is @gettheleadout 's first post:

I have not studied the psychology of intelligence in detail, and so it would be inappropriate for me to criticize your outline of the core characteristics of intelligence. Furthermore, these characteristics (memory and processing) are very reasonable, and will likely suffice for present purposes.

I am no expert on intelligence research either, so if you or anyone else would like to propose alternative approaches for defining core aspects of intelligence I invite them!

Your first question here ("What might intelligence mean in the context of physician performance?") deserves some attention. It seems to me that the two characteristics (memory and processing) account for the skills required of a physician. Any challenges to this?

I agree that they account for much of technical knowledge and thought required to practice medicine, but I think it would be interesting to explore some of less "typical" ways in which these capabilities must be applied; say, while recognizing that this complicates things, with social intelligence (whatever that encompasses), which I think you would agree is particularly important for physicians to be efficacious in the contexts of teamwork and patient interaction/education.

To question (1), I emphatically affirm this. As justification, I offer a question: What use would there be for memorization of a vast amount of information if the person were unable to apply this 'knowledge'? I assume that 'problem-solving' and 'critical thinking' are other ways of saying 'processing', one of the core characteristics of intelligence named above. This is especially clear when thinking about medicine, where it is imperative that a physician both retains a large quantity of information and applies this knowledge to particular situations.

I wholly agree.

To question (2), although wary of the rhetorical games that might be played here, yes, I believe that a measurement of knowledge should represent that person's intelligence.

This is one area where I'm a bit wary of drawing conclusions. Surely we cannot hope to rely upon much else besides the measurement of knowledge as a proxy for capacity for knowledge, but we must be careful to avoid associating the possession of knowledge itself with the possession of intelligence. Of course as you said, this is partly rhetorical.

This is the most interesting part of your post to my eye, and I'll conclude with this. [I note that my comments quoted in Part D are plausible reasons underlying this overarching concern (i.e., reasons that the MCAT score can vary from day-to-day and depending on external conditions).] It's funny because my reaction very much depends on the comparison being made. That is, I tend to agree with statements like "Someone who scored a 40 is smarter than someone who scored a 24." However, I am hesitant to state that someone with a 34 is smarter than someone with a 31. You also appreciate this hesitation in your own concluding statement:

Frankly, I don't have an answer to this. I also admit that some of my hesitation here is not defensible: I believe that medicine is a field in which collegiality and solidarity are important - to believe that some physicians are smarter than others seems to undermine this belief. My uncomfortable feeling here, however, has no bearing on the truth of the thing.

I'd feel confident in assuming that you recognize that in every field there will be variation between professionals; for better or worse almost every population is a bell curve on some characteristic(s). With regard to the impact recognition of this might have (internally within the profession, publicly, whatever), do you think medicine is unique in some way when compared to other professions?

Nonetheless, I can elaborate on some of the reasons I think that comparisons between people with different MCAT scores show the ridiculousness of trying to use MCAT score as an indication of intelligence. You ask "How certain can be we that someone who scores a 40 is at least somewhat more intelligence that someone who scores a 24? Absolutely? Not at all? What influences our answer?" I'll continue this discussion to highlight my reasons for hesitation. We agree that the MCAT does relate to intelligence. However, our answer is influenced by whether we accept or deny the mutability of the MCAT score of either individual. If the person with the 24 tested again and scored a 35, we would have to decrease our certainty that the person with the 40 (let's say she equals her score of 40 on the second administration) is the same amount more intelligent than the other individual. Our suspicion that external factors were involved in this difference in performance is therefore increased [assuming two categories of factors that affect performance: (1) internal (intelligence) and (2) external (see Part C)].

You raise a great question about how multiple scores should be interpreted, which is obviously relevant to med school admissions. Issues with practicality aside, in an ideal world what could we to get around this issue? Require three administrations of the exam and average all scores? Forbid retesting at all and just go with one shot (possibly as long as you pass, like the USMLE?)

 

[gettheleadout]

Another serious (but obvious) confounder in attempting to relate MCAT scores to intelligence is the inherent preparation that goes into achieving good results. I know several of my classmates who scored high twenties/low thirties on diagnostics but ended up with 40+ on the real thing a few months later. Neither their processing speed nor memory capacity likely changed significantly in that time frame. Had they been more lax with their preparation (like some of my other friends) they might have ended up with scores in the mid thirties.

I completely agree with this. The MCAT is clearly not only a measure of intelligence one of its major purposes is to test for a specific knowledge base, which can only be gained through study (though the extent of that study may vary between individuals, maybe due to differences in intelligence, but also other factors).

Just knowing someone's name and the associated MCAT score leaves out so much information that few reasonable conclusions can be drawn.

With regard to intelligence alone? Maybe. In general though I disagree, I think even looking at just a score can justify drawing conclusions about that test taker's background and problem-solving ability in the context of the testable content.

Maybe they only used Kaplan verbal without ever hearing about the AAMCs and failed to "get used to" the style of the test makers, resulting in a much lower verbal score. Is their processing speed/reading comprehension any different in this scenario? Doubtful.

An interesting example. Surely we all recognize that preparation accounts for much of one's performance on a test of particular knowledge and particular skills, though in general I don't think awareness of this precludes drawing conclusions about the test takers. Case in point: I would argue that a test taker who scores 15's in PS and BS has greater knowledge of and/or ability to apply scientific information/principles than a test taker who scores 9's. The difference may be attributable to differences in prep, and not representative of intelligence, but I think the conclusion that the scores are representative of a difference in that regard remains valid.

Wildly differing access to study resources/information + test day catastrophes (anxiety?) that people mentioned need to be taken into account.

As I said above I agree with the first issue here, but on the subject of test day catastrophes I think such occurrences are uncommon enough that they should be the exception to convention in drawing conclusions from scores, and further that they would hopefully be remedied by retesting and provision of explanation of the catastrophe to admissions officers so that interpretation of the first score could be done properly.

The other significant question is whether the MCAT *should* test intelligence. Do the most intelligent people make the best doctors? Maybe, maybe not. However, one could argue that in the context of med school admissions the MCAT should test intelligence because determination, work ethic, compassion, and personability (other important characteristics) may be assessed via other means such as GPA, school prestige, ECs, and the interview. In reality, I'm guessing that the most successful MS1 and MS2 students are at least fairly intelligent and are motivated enough to assimilate large amounts of information over short periods of time. To that end, the MCAT can serve as a useful tool for measuring not only intelligence, but drive and determination (who was motivated enough to master all 4 sciences, do tons of verbal passages, and a gazillion practice tests in order to do well). However, I'd agree that being exceptional in the former (intelligence) can mask deficiencies in the latter (but that both are represented by one's MCAT score).

Agreed, and on the subject the intelligence-doctoring skill correlation I would step back and ask first, is there a minimum intelligence requirement to practice medicine well? I would argue there must be.

 

[mk04447]

Tbh, I doubt Einstein would have accomplished half the things he did if we had our obsession on 'cheap' standardized tests as we do today. I'm not big on the whole, my child isn't good at tests thing. But standardized tests are a different beast entirely.

And essentially you cannot. A person who's native language is not English will spend time converting the passages into his own preferred language, then converting the answer back into English. It essentially means that as a non-native speaker you're hemorrhaging a lot of time just purely on making sense of things.

Talk about an entirely different discussion... ****ty as they are, this shrinking world necessitates an equalizer. I would suggest there are better options, but in this newly politically correct world where everyone is afraid of their shadow and the impending frivolous lawsuit, standardized testing is the best alibi to have.

 

[gettheleadout]

Luck has its place in MCAT scoring... the poster who said friends who consistently scored high 20's and low 30's yet achieved 40+ on the real test demonstrate this.

Just to be clear that's not what @Purplownz said:

I know several of my classmates who scored high twenties/low thirties on diagnostics but ended up with 40+ on the real thing a few months later. Neither their processing speed nor memory capacity likely changed significantly in that time frame. Had they been more lax with their preparation (like some of my other friends) they might have ended up with scores in the mid thirties.

Scoring low on a diagnostic is not the same as having half a dozen data points consistently fall in a particular range. Certainly preparation makes a difference (I scored 26 on a Kaplan diagnostic as a college freshman. I clearly lacked the knowledge base to score highly, yet after taking the prereq's and preparing for the exam through content review and practice, I scored above 99th percentile on all 8 AAMC FL's and on the real deal), but the jump you're referring to being attributable to luck, even mostly, must be beyond rare in my opinion.

But, I don't believe this is "luck" in the sense that test takers are consistently choosing a, b, c, or d correctly with more frequency. Rather, test takers can choose a lucky test date. Because of the vast range of material with the potential to appear on an MCAT, someone could simply perform well on a test that plays to their strong areas. I think we can all agree there are subjects we'd prefer to see over others.

"Luck" alone is enough to say that someone who scores a 40 is not necessarily more intelligent than someone with a 20.[/quote]
"Not necessarily" may be the key here. Blindly guessing, there is a technically non-zero possibility that two test takers could achieve those scores, with no difference in test taker characteristic at all. Obviously that possibility is statistically improbable, as the chance of occurrence is negligible, but as we consider how much we think intelligence contributes to score we get a greater probability that the score represents a difference in intelligence. An individual who scores 150 on an IQ test may not necessarily be more intelligent than one who scores 90, but it is highly probable that he/she is (assuming IQ represents intelligence, of course).

Further, going down this road makes me continue to assert that the MCAT doesn't test anything but the relative study practices and background science of the test takers themselves.

I will strongly disagree with this for the reason explained below:

Clearly, Newton would score less than 20 on an MCAT because necessary information wasn't available during his time, but you wouldn't say he was anything other than a genius.

Pick at random a single premedical college student and given he/she and Isaac Newton access to the same MCAT prep materials, methods, and time. If the MCAT tests nothing beyond "the relative study practices and background science of the test takers themselves," you should see little difference in their scores, should you not? Personally, I'd be putting all my money on Newton, and I bet you can guess why.

This thread continually over complicates simple matters... Of course a more intelligent person is likely to ultimately test better for a variety or reasons, from vocabulary to instinct gained through successful life experience, but that is it.

While it may seem like overcomplication, I think your reference to "instinct" is a great example of why purposed, in-depth discussion of this issue can be productive, if not simply intellectually stimulating.

The MCAT is no more capable than the stupid quizzes my girlfriend likes to take in her Cosmo magazine at predicting intelligence.

We'll have to agree to disagree on this, then.

 

[gettheleadout]

Excellent point. I'm willing to bet that most SDNers could beat Einstein on the MCAT. Now if he were to do a post-bac (post PhD haha?), take all of the pre-reqs again, and study seriously with SN2ed's schedule could he break 40? Probably (assuming that he doesn't get destroyed by verbal, which by the way is another issue: how do you account for geniuses whose native language is not English?) But his eventual 40+ would be the result of his willingness to do all of the aforementioned things, rather than his innate intellect.

On the contrary, I would assert it would be the result of the two in concert.

 

[gettheleadout]

Intelligence in my opinion has to do with your capacity and ability to learn. If you can learn to tie your shoes you're intelligent for a 2 year old, etc.

The mcat obviously does test something we may consider calling intelligence or factors that pertain to it. That being said I don't think the PS or BS do a significantly effective job at probing intelligence. PS & BS questions frequently range from minute and random to apply and acknowledge. I believe that in truth, the PS section's questions such as rate laws or energy test intelligence the best, where as questions such as which of the following are an example of a metathesis reaction ( A reaction that you've never learned about reasonable or if you've read in your studying it was mentioned once) is not.

Definitely an appropriate distinction to note, in my opinion. No doubt some questions are more geared toward the simple measurement of content knowledge (admittedly one explicit aim of the test) while others required more problem-solving and critical thinking.

Verbal can however probe into intelligence and one's ability to collect and apply information obtained in a passage. Verbal is unique in that it doesn't require any outside information other than a general grasp of basic paradigms of literature and philosophy. The answer is in the passage, you just need to read it and then sow it together in a neat and concise sentence.

Verbal opens its own can of worms, doesn't it? You touched on the issue below:

A person who's native language is not English will spend time converting the passages into his own preferred language, then converting the answer back into English. It essentially means that as a non-native speaker you're hemorrhaging a lot of time just purely on making sense of things.

Although personally I think VR has a higher error rate anyway...

But honestly, I just think it's a really crappy test as a whole. And any individual measure within it of intelligence can be better obtained from other measures.

Mind elaborating? By most common metrics I'm aware of the MCAT is a solid psychological test (reliability coefficient > 0.9, validity coefficient vs. Step 1 > 0.4).

 

[Drrrrrr. Celty]

Talk about an entirely different discussion... ****ty as they are, this shrinking world necessitates an equalizer. I would suggest there are better options, but in this newly politically correct world where everyone is afraid of their shadow and the impending frivolous lawsuit, standardized testing is the best alibi to have.

Well.. who knows, I think that behavioral tests designed by psychologists may be a better option. It's not going to be cheap obviously, but chances are that a timed thought experiment will give more insight.

 

[Drrrrrr. Celty]

Definitely an appropriate distinction to note, in my opinion. No doubt some questions are more geared toward the simple measurement of content knowledge (admittedly one explicit aim of the test) while others required more problem-solving and critical thinking.


Verbal opens its own can of worms, doesn't it? You touched on the issue below:

Although personally I think VR has a higher error rate anyway...

Mind elaborating? By most common metrics I'm aware of the MCAT is a solid psychological test (reliability coefficient > 0.9, validity coefficient vs. Step 1 > 0.4).

I think it's much lower than that. I remember reading the meta and it's like R =.44. Which is a reasonable correlation. ( Albeit the tests they were correlating with from the AAMC 3 to 8 days are so much easier that the statistic might be worthless and honestly may prove to be increasingly less and less).

I'm not sure if I would label the MCAT a solid psychological test nor that the metric is all that clear. Point is that people graduate from Carib and DO schools with < 24s at almost the same rate as those who enter with >30s and likewise those who enter as low stat students in MD schools fair similarly well to their in-class students. Maybe they don't score 90th percentile on their step 1, but they graduate and they match into their top 3.

So you tell me if you think it's all that solid or useful.

 

[gettheleadout]

I think it's much lower than that. I remember reading the meta and it's like R =.44. Which is a reasonable correlation. ( Albeit the tests they were correlating with from the AAMC 3 to 8 days are so much easier that the statistic might be worthless and honestly may prove to be increasingly less and less).

What's lower than what? The r value you're citing is the one I said was >0.4. Although actually, for MCAT total vs Step 1, r = 0.66 (Donnon, Paolucci, and Violato, 2007), we're in agreement.

I'm not sure if I would label the MCAT a solid psychological test nor that the metric is all that clear. Point is that people graduate from Carib and DO schools with < 24s at almost the same rate as those who enter with >30s and likewise those who enter as low stat students in MD schools fair similarly well to their in-class students. Maybe they don't score 90th percentile on their step 1, but they graduate and they match into their top 3.

So you tell me if you think it's all that solid or useful.

So the issue with validity is that it's contextual. A test itself is neither valid nor invalid; a test has validity as a function of the appropriateness of the sort of conclusions that can be drawn from it. I'd offer the following:

In this study, we found that the combination of UGPAs and MCAT total scores predicts UP [unimpeded progress toward graduation], an academic outcome that relies on data beyond standardized test scores and occurs about six years after application to medical school. MCAT total scores, however, contribute more to the prediction of UP than do UGPAs.

...

Our findings indicate that UGPAs and MCAT total scores are both strong predictors of the extent to which matriculants will experience UP; this is important because it shows that UGPAs and MCAT total scores predict academic performance in medical school well beyond the first two years. Our findings are also consistent with research showing that MCAT scores predict IP [impeded progress toward graduation] in medical school as well as academic outcomes beyond test scores, such as grades in basic sciences courses, clerkship performance, and academic difficulty or distinction.

Sounds useful to me.

 

[Drrrrrr. Celty]

What's lower than what? The r value you're citing is the one I said was >0.4. Although actually, for MCAT total vs Step 1, r = 0.66 (Donnon, Paolucci, and Violato, 2007), we're in agreement.


So the issue with validity is that it's contextual. A test itself is neither valid nor invalid; a test has validity as a function of the appropriateness of the sort of conclusions that can be drawn from it. I'd offer the following:


Sounds useful to me.

I'm saying that the meta analysis which trumps the 2007 article that you posted because it also includes it in it's data is a .44. But I made mentioned that the mcat has since the days of the AAMC 3 to AAMC 8 changed enormously. It's a far harder and much more bat **** crazy test. As such I think that the statistic likely now can be imagined to be less than .44.

I didn't claim that it wasn't contextual. And yes, a test can be invalid because a test is designed FOR a certain role.

Again, I'm not surprised that they are predictive. But you're not going to mega-graduate so if the difference between the low mcat group and the high mcat group in terms of graduation rate is less than 5% then what exactly are we looking at that is significant in anyway? And even then if it's predicting that the correlation plateaus around 26-27 as most research has shown, then they're using the wrong statistical test to begin with and need to be using a 2-way ( P.s reason they're not is because if they two-tail it it'll be non-significant). So truth of the matter is that it's predictive power stops after a certain relatively attainable range.

And not to beat the horse into mush. But the DO pathway hasn't combusted into flames yet. Hence the mcat's predictive power and use seems to be either only applicable to the allopathic context ( Which is an unreasonable assumption) or is as I said, predictive only at a certain range.


All I'm saying is that it doesn't seem all that useful to me.

 

[gettheleadout]

I'm saying that the meta analysis which trumps the 2007 article that you posted because it also includes it in it's data is a .44. But I made mentioned that the mcat has since the days of the AAMC 3 to AAMC 8 changed enormously. It's a far harder and much more bat **** crazy test. As such I think that the statistic likely now can be imagined to be less than .44.

Would you mind linking to the more recent meta-analysis you're referring to?

I didn't claim that it wasn't contextual. And yes, a test can be invalid because a test is designed FOR a certain role.

A test cannot be judged invalid absent context. The purpose for which a test is designed is irrelevant, what matters is the purpose for which the test is being used. This is a basic tenet of evaluating test validity; a test designed to predict X can be useless in predicting X but very useful in predicting Y.

Again, I'm not surprised that they are predictive. But you're not going to mega-graduate so if the difference between the low mcat group and the high mcat group in terms of graduation rate is less than 5% then what exactly are we looking at that is significant in anyway? And even then if it's predicting that the correlation plateaus around 26-27 as most research has shown, then they're using the wrong statistical test to begin with and need to be using a 2-way ( P.s reason they're not is because if they two-tail it it'll be non-significant). So truth of the matter is that it's predictive power stops after a certain relatively attainable range.

Dunleavy et al. puts it at 30:

The likelihood of a matriculant experiencing UP increases consistently as his or her MCAT score increases until MCAT total score exceeds 30, at which point it tends to level off.

And not to beat the horse into mush. But the DO pathway hasn't combusted into flames yet. Hence the mcat's predictive power and use seems to be either only applicable to the allopathic context ( Which is an unreasonable assumption) or is as I said, predictive only at a certain range.

This is really a separate issue I'm not knowledgeable enough to take on (e.g. issues of COMLEX vs USMLE, DO stats vs graduation rate, AOA vs LCME residency outcomes, etc.)

All I'm saying is that it doesn't seem all that useful to me.

Fair enough.

 

[Drrrrrr. Celty]

Would you mind linking to the more recent meta-analysis you're referring to?


A test cannot be judged invalid absent context. The purpose for which a test is designed is irrelevant, what matters is the purpose for which the test is being used. This is a basic tenet of evaluating test validity; a test designed to predict X can be useless in predicting X but very useful in predicting Y.


Dunleavy et al. puts it at 30:



This is really a separate issue I'm not knowledgeable enough to take on (e.g. issues of COMLEX vs USMLE, DO stats vs graduation rate, AOA vs LCME residency outcomes, etc.)



Fair enough.

I don't have it. All I know is it's 2008 I believe.

Uh, not at all. An IQ test is designed to test nominally whether someone is disabled or normal. Any outside extrapolation is borderline meaningless. This is a common example.

The meta found that 26-27 is where USMLE pass rate and other factors become utterly insignificant.

You can take it on my knowledge. 99% of DOs matched last year into something. 70 or 76% were into ACGME residencies. Graduation rate is 92-94%. USMLE pass rate is 92% ( Relatively well proportionated between the stat ranges i.e it's not just 3.5/30s who are taking it). Average stats = 3.5/26.5.


The point is that given that the osteopathic side has no combusted and fallen apart. I can't help but think that probably it's not all that predictive.

 

[gettheleadout]

I don't have it. All I know is it's 2008 I believe.

I'll take a look.

Uh, not at all. An IQ test is designed to test nominally whether someone is disabled or normal. Any outside extrapolation is borderline meaningless. This is a common example.

In fact, yes. The Standards for Educational and Psychological Testing states that validity represents “the degree to which evidence and theory support the interpretation of test scores entailed by proposed uses of tests.” (APA, 1999) The purpose for which a test was developed is irrelevant outside of that context, the proposed uses for a test can vary widely and validity for them the same, and it is inappropriate to say blanket "a test is valid/invalid" without giving care to context. For further clarification on this accepted, modern view of validity in psychological testing I encourage you to consult a psychologist with publishing background in testing and assessment theory, as I have.

The meta found that 26-27 is where USMLE pass rate and other factors become utterly insignificant.

You can take it on my knowledge. 99% of DOs matched last year into something. 70 or 76% were into ACGME residencies. Graduation rate is 92-94%. USMLE pass rate is 92% ( Relatively well proportionated between the stat ranges i.e it's not just 3.5/30s who are taking it). Average stats = 3.5/26.5.

The point is that given that the osteopathic side has no combusted and fallen apart. I can't help but think that probably it's not all that predictive.

I do feel we're abandoning the original issue here. The MCAT clearly has predictive validity at least across a portion of the scoring spectrum when considering success in medical school. Whether that predictive value is appropriately applied in the context of admissions is a separate issue.

 

[Purplownz]

I completely agree with this. The MCAT is clearly not only a measure of intelligence one of its major purposes is to test for a specific knowledge base, which can only be gained through study (though the extent of that study may vary between individuals, maybe due to differences in intelligence, but also other factors).


With regard to intelligence alone? Maybe. In general though I disagree, I think even looking at just a score can justify drawing conclusions about that test taker's background and problem-solving ability in the context of the testable content.

Absolutely. However, the more variables we are unable to hold constant, the less predictive these scores are of the specific component we are examining (i.e intelligence). If we extend our analysis of the MCAT to include factors other than intelligence, we can certainly draw clear-cut distinctions between individuals with different scores.

An interesting example. Surely we all recognize that preparation accounts for much of one's performance on a test of particular knowledge and particular skills, though in general I don't think awareness of this precludes drawing conclusions about the test takers. Case in point: I would argue that a test taker who scores 15's in PS and BS has greater knowledge of and/or ability to apply scientific information/principles than a test taker who scores 9's. The difference may be attributable to differences in prep, and not representative of intelligence, but I think the conclusion that the scores are representative of a difference in that regard remains valid.

I definitely agree. In light of this, we should all be able to agree with the following statement: An individual with a very high MCAT score is LIKELY to possess qualities typically associated with intelligent people (memory capacity, processing speed, and recall). However, the inverse of this is NOT true. Those who do not achieve high scores on the MCAT exam cannot necessarily be associated with low intelligence.

Although perhaps dubious, the first analogy that came to my head involves the 4th year match process. I believe that we can gauge an individual's intelligence from their MCAT scores to the same extent that a medical student's academic profile can be inferred from their match result. For instance, somebody who scores 15's in PS and BS is likely, like you said, to have "[great] knowledge of and/or ability to apply scientific information/principles" as well as possess some significant level of intelligence. Similarly, someone who matches Harvard dermatology or NYU plastics is likely to have crushed the step, honored all of their rotations, and have multiple pubs.

HOWEVER, somebody who scores a 28 on their MCAT may still possess incredible skill, perhaps even be a genius w/ photographic memory. We simply have no idea just by looking at their scores. Similarly, somebody who matches family medicine in North Dakota may still go to a top med school, have a 250+ step I score, and have all clinical honors. Perhaps they absolutely love FM and their fiance has a job there. Again, we simply do not know based on a single result (the match) without examining the background story that led to that end result. I personally know a girl who was one of the best students at Columbia P&S with incredible stats who matched psych at a community program in NY. Nobody would have guessed that it was her first choice. Obviously, nobody chooses to receive low MCAT scores (although length of study, study materials, and motivation are chosen), but the analogy does demonstrate the difficulties of assessing one or multiple complex parameters based on an outcome that is decided by so many different things.

As I said above I agree with the first issue here, but on the subject of test day catastrophes I think such occurrences are uncommon enough that they should be the exception to convention in drawing conclusions from scores, and further that they would hopefully be remedied by retesting and provision of explanation of the catastrophe to admissions officers so that interpretation of the first score could be done properly.

Agreed, I was just throwing around ideas for discussion.

Agreed, and on the subject the intelligence-doctoring skill correlation I would step back and ask first, is there a minimum intelligence requirement to practice medicine well? I would argue there must be.

I agree. The next questions are:
1. What is this level of intelligence that can be reasonably expected of all medical students (if this can even be quantified)?
2. Is this level adequately accounted for by current admissions standards? If so, via what mechanisms?Based on everything we have discussed, it would seem that the current standards of 3.65+/32+ for MD schools seem satisfactory.

 

[Purplownz]

On the contrary, I would assert it would be the result of the two in concert.

Yes. This is the foundation of my original argument. I should have written: rather than *changes* in intelligence.

 

[Drrrrrr. Celty]

I'll take a look.


In fact, yes. The Standards for Educational and Psychological Testing states that validity represents “the degree to which evidence and theory support the interpretation of test scores entailed by proposed uses of tests.” (APA, 1999) The purpose for which a test was developed is irrelevant outside of that context, the proposed uses for a test can vary widely and validity for them the same, and it is inappropriate to say blanket "a test is valid/invalid" without giving care to context. For further clarification on this accepted, modern view of validity in psychological testing I encourage you to consult a psychologist with publishing background in testing and assessment theory, as I have.


I do feel we're abandoning the original issue here. The MCAT clearly has predictive validity at least across a portion of the scoring spectrum when considering success in medical school. Whether that predictive value is appropriately applied in the context of admissions is a separate issue.

Eh, maybe. I have a poor view of the IQ test just purely because I find the justification and the literature supporting it poorly defensible considering it's application being one of mental health and largely avoided by the larger clinician population.

I don't think we're abandoning the original issue. And agreed, I think across low scoring populations i.e under 27 it becomes useful. Above? Not so much.
All I can say is that the example of osteopathic medicine in this conversation is useful in simply providing a large population that seems to largely be in exception to the predictive capabilities.

 

[gettheleadout]

Yes. This is the foundation of my original argument. I should have written: rather than *changes* in intelligence.

Sorry, didn't mean to misinterpret your post.