FYI - For Applicants - MCAT score inflation

[exPCM]

---

Members don't see this ad.

FYI - for applicants - looks to me like there is score inflation occurring in the MCAT -
mean of 26.7 in 1998 vs. mean of 27.9 in 2009 for all applicants and matriculants.

Not sure if/why AAMC is increasing the mean score. Looks like they are keeping the standard deviation steady.

http://www.aamc.org/data/facts/applicantmatriculant/table17-fact2009mcatgpa98-09-web.pdf

 

[deleted307836]

It's not that the mean score is increasing, it's just that people are retaking the test more and using their best score. It's easier to retake now that the test is computerized.

Edit... Right?

 

[apumic]

Probably not "MCAT inflation." In reality, this is probably simply the Flynn Effect at work. It's well-documented for a variety of cognitive skills tests, so I would not be the least bit surprised to see it with the MCAT data as you have shown.

 

[apumic]

It's not that the mean score is increasing, it's just that people are retaking the test more and using their best score. It's easier to retake now that the test is computerized.

Edit... Right?

The trend is over time. It's not a 1-time change as would be expected by the factor you are describing. Instead, we see a gradual trend over the past decade or so. IMO, the most probable explanation would seem to be a case of the Flynn Effect.

 

[deleted307836]

The trend is over time. It's not a 1-time change as would be expected by the factor you are describing. Instead, we see a gradual trend over the past decade or so. IMO, the most probable explanation would seem to be a case of the Flynn Effect.

Interesting. I always thought that because of the Flynn Effect, the IQ scale was often recalibrated so that the median is always 100. Wouldn't the AAMC do the same then -- keep the mean (weighted) score the same as average performance increases?

 

[jm192]

I'm under the impression there's just alot more available test prep. People have more study aids, and classes are everywhere now.

 

[deleted307836]

.

 

[apumic]

Interesting. I always thought that because of the Flynn Effect, the IQ scale was often recalibrated so that the median is always 100. Wouldn't the AAMC do the same then -- keep the mean (weighted) score the same as average performance increases?

Sure, to a degree. The Flynn effect does not occur evenly over the distribution, however, which may make calibration more difficult. Since the Flynn Effect tends to taper at higher achievement levels, we should be seeing a gradual change in the skew of the scores. Unfortunately, the data above assumes a perfectly normal distribution and so we cannot know for sure what is going on based upon the data above.

 

[deleted307836]

Sure, to a degree. The Flynn effect does not occur evenly over the distribution, however, which may make calibration more difficult. Since the Flynn Effect tends to taper at higher achievement levels, we should be seeing a gradual change in the skew of the scores. Unfortunately, the data above assumes a perfectly normal distribution and so we cannot know for sure what is going on based upon the data above.

Very good explanation 👍.

I need to take a stats class...

 

[deleted307836]

I still don't understand though how we can make assumptions about the grading scale just based on APPLICANTS... wouldn't we need the info for all test takers then?

 

[unhappytnt]

I still don't understand though how we can make assumptions about the grading scale just based on APPLICANTS... wouldn't we need the info for all test takers then?

I agree with this. I think the sole reason why the average score is increasing is because this date is based on applicants & matriculants. I would say that a significant amount of test takers who score in the < 27 range simply choose not to apply, thus not appear on this list, and this increases the average.

 

[Slowpoke]

The data set is quite ambiguous. Is it pooled from applicants who matriculated? or a mixture of applicants & matriculants?

edit: Looking at the total applicants it appears that it's a mixture of both. Looking at the averages of GPA + MCAT it gives quite a different picture then what I normally would have assumed.

 

[NickNaylor]

Regardless of the data, why would the AAMC even want to inflate the numbers? That makes absolutely no sense. They have no skin in the game. It's not like they can brag about applicants having higher MCAT scores like an undergrad institution can.

 

[phltz]

Yeah, the most likely explanation is that med school applications are becoming more competitive, and everyone knows it. Maybe 10 years ago people with scores in the 20s were still applying to med school in decent numbers (whether or not they got in, I don't know), but now they don't even try.

 

[pennstate4life]

The rise in scores is due to the one thing and one thing only: wikipedia

 

[Elpenor]

This data alone means nothing. One would need the data of the TEST TAKERS. Not just the matriculants / applicants.

 

[exPCM]

This data alone means nothing. One would need the data of the TEST TAKERS. Not just the matriculants / applicants.

Okay. Sorry, maybe my theory was wrong. Maybe more people are just taking the MCAT and then not applying to medical school. Maybe the mean score for all takers hasn't changed. It does look like from the table above that the number of applicants has increased over the past few years.

 

[apumic]

I still don't understand though how we can make assumptions about the grading scale just based on APPLICANTS... wouldn't we need the info for all test takers then?

Yes, we would. Nice catch! It still could show a general increase in test takers' scores IF we knew that the correlation was perfect (or near-perfect) between test taker scores and applicant scores; however, we do not know this for sure, so no conclusions can really be made for the data. I glanced at the data w/o looking at the population -- I just assumed based on what the OP wrote that the population was test takers.

 

[RogueUnicorn]

remember also, the mcat is scaled at the subscore level, not the total score

 

[DirtyJustice]

To Apumic, I doubt this is the Flynn effect. The MCAT is a percentile based test, so no matter how many more questions people get right the score should always be the same.

 

[apumic]

To Apumic, I doubt this is the Flynn effect. The MCAT is a percentile based test, so no matter how many more questions people get right the score should always be the same.

According to the AAMC, scale scores are assigned PRIOR to percentile ranking. It's technically criterion scored with those criteria being taken from prior percentile ranking studies. In other words, percentile ranks are applied post-test (i.e., once everyone has taken the test) while scales are applied pre-test.

 

[Siggy]

I still don't understand though how we can make assumptions about the grading scale just based on APPLICANTS... wouldn't we need the info for all test takers then?

This. The data is for applicants. It's very much possible that as the average matriculant scores increase, the applicants with weaker MCAT scores are deciding to forgo applying to AAMC institutions.

 

[fishbait63]

MCAT isn't just for MD schools so have the mean go up might mean there are more people who are scoring lower and applying to physician therapy, or podiatry school (no offense to those institution but their required scores are lower)

 

[phltz]

A quick look at the AAMC website suggests that all test takers may have seen a bit of an upward drift over the past decade. The 2000 average for examinees is 24.3 <http://www.aamc.org/students/mcat/admissionsadvisors/examstatistics/scaledscores/combined00.pdf>, for 2001 is 24.5, for 2002 is 24.1, for 2003 is 24.7, for 2004 is 24.6, for 2005 is 24.7, for 2006 is 25.1, for 2007 is 25.1, for 2008 is 24.9, for 2009 is 25.1.

Note that 2003 is the first year that Verbal Reasoning scores could go all the way up to 15, which makes comparisons before and after slightly off, although given the small portion of examinees who score this highly, I think it has a minimal impact on the average.

The number of examinees has also climbed from 55k in 2000 to 79k in 2009, which is a substantial increase.

It appears that average scores have, in fact, increased over the past decade, by a bit less than a point. Extrapolating the trendline into the future, in 2210, the average MCAT score should exceed a 45, and they'll need to come up with a new system of scoring. Maybe they'll add another section or two.

 

[wanderer]

A quick look at the AAMC website suggests that all test takers may have seen a bit of an upward drift over the past decade. The 2000 average for examinees is 24.3 <http://www.aamc.org/students/mcat/admissionsadvisors/examstatistics/scaledscores/combined00.pdf>, for 2001 is 24.5, for 2002 is 24.1, for 2003 is 24.7, for 2004 is 24.6, for 2005 is 24.7, for 2006 is 25.1, for 2007 is 25.1, for 2008 is 24.9, for 2009 is 25.1.

Note that 2003 is the first year that Verbal Reasoning scores could go all the way up to 15, which makes comparisons before and after slightly off, although given the small portion of examinees who score this highly, I think it has a minimal impact on the average.

The number of examinees has also climbed from 55k in 2000 to 79k in 2009, which is a substantial increase.

It appears that average scores have, in fact, increased over the past decade, by a bit less than a point. Extrapolating the trendline into the future, in 2210, the average MCAT score should exceed a 45, and they'll need to come up with a new system of scoring. Maybe they'll add another section or two.

You seem to be selecting the data that is conforming to your viewpoint. If you compare the data from 2006-2009 there is no increase at all.
However, I wouldn't be surprised if there was a subtle increase over the last X number of years as people are now more aware of the scores required and the studying required to get those scores. (If this is true the skewness should become more positive). There's also more study material out there. We don't really have nearly enough information (in terms of distribution, skewness, kurtosis etc.) to really know what's going on.

 

[addo]

What I am most surprised by is the SD on GPA, especially cGPA.
in 1998 the mean was a 3.4 with almost a .4 SD! If currently, that was the case I could sleep a lot better.

Edit- nevermind, the table shown was for applicants.

 

[lstone13]

I still don't understand though how we can make assumptions about the grading scale just based on APPLICANTS... wouldn't we need the info for all test takers then?

I agree with this. I think the sole reason why the average score is increasing is because this date is based on applicants & matriculants. I would say that a significant amount of test takers who score in the < 27 range simply choose not to apply, thus not appear on this list, and this increases the average.

🙂👍👍 My thinking exactly.

 

[htg2010]

anyone know why the applicant numbers decreased from 98-02?

 

[Baron Samedi]

anyone know why the applicant numbers decreased from 98-02?

Dotcom business boom and the well-being of finance at that point. A lot of the bright minds were going into those fields rather than medicine.

 

[manraj76]

The Flynn effect doesn't make sense because the test is curved so if everyone is smarter then getting a higher score would be harder, and the mean should still remain the same. My guess is that people retake it more and they get a better score the second time around and use that score.

 

[apumic]

The Flynn effect doesn't make sense because the test is curved so if everyone is smarter then getting a higher score would be harder, and the mean should still remain the same. My guess is that people retake it more and they get a better score the second time around and use that score.

Did you read earlier in the thread? The curve has nothing to do w/ it as the scale is applied via criterion, not percentile rank.