MCAT Scoring

[aldol16]

Hey guys,

I've done a lot of searching on online forums - here and Reddit, among others - and it seems that nobody is entirely sure how the MCAT percentiles are calculated. Are they based on previous administrations of the exam, only on others who took the exam that day, only on those current and past test-takers who took the exam in a similar time of year, etc.? It seems to me that these factors could introduce a lot of discrepancies into the results. For instance, people who take the exam in the summer or early fall would have a huge advantage over people who take it in January, April or May if everybody is compared to everybody else.

Can anybody offer definitive insight (no guessing) into this?

---

Members don't see this ad.

 

[theonlytycrane]

I'm gonna take it in July to avoid the gunners. jk But also interested in this question!

 

[Xenith]

Pretty sure they are established based on previous years. They have nothing to do with the cohort you take the exam with... So no point in finding the strategic time when less gunners take it. There was a brief period of time (April and May 2015) when this did not apply because the new MCAT was super new and they didn't have precious data. But now they do.

 

[aldol16]

Pretty sure they are established based on previous years. They have nothing to do with the cohort you take the exam with... So no point in finding the strategic time when less gunners take it. There was a brief period of time (April and May 2015) when this did not apply because the new MCAT was super new and they didn't have precious data. But now they do.

Do you mean percentiles are calculated based on the bulk scores of everybody who has taken it previously? I ask specifically about that because people who have had the summer to study would obviously be at an advantage relative to those who take it during the school year - generally speaking.

 

[Xenith]

Do you mean percentiles are calculated based on the bulk scores of everybody who has taken it previously? I ask specifically about that because people who have had the summer to study would obviously be at an advantage relative to those who take it during the school year - generally speaking.

Yes. You are compared to everyone who has or will take the new mcat. That is why it's a standardized test. They need to be able to compare you to someone who took it last year and someone who will take it in the future.

 

[el_duderino]

Does it actually matter? Presumably you're not going to game the system by taking it in the summer vs winter or whatever, right? Take the exam when you're well-prepared to take the exam. If you're prepared, you'll do fine.

 

[aldol16]

Yes. You are compared to everyone who has or will take the new mcat. That is why it's a standardized test. They need to be able to compare you to someone who took it last year and someone who will take it in the future.

I understand what a standardized test is. I was just wondering if they took into account when students take it. There are many ways of standardizing - you can do a crude standardization, e.g. calculate acceptable BMI based off of the national average, or match as many variables as possible to increase external validity, e.g. calculate acceptable BMI based off of the averages for an individual's age range.

From your response, it sounds like they're doing the former rather than the latter. Thank you for your response.

 

[aldol16]

Does it actually matter? Presumably you're not going to game the system by taking it in the summer vs winter or whatever, right? Take the exam when you're well-prepared to take the exam. If you're prepared, you'll do fine.

No, it doesn't matter. I was just curious as to how they standardize it because it seemed to me like such a bad way to do it.

 

[dempty]

Realistically, the exam and scoring process is extensively validated to ensure that a standardized test is both valid and reliable. If you could take the exam at a different time and score better due to a different test-taker pool, then the exam wouldn't be a good exam. This isn't a half-baked idea run out of someone's garage-- they very likely have hired the appropriate people to aid in the process.

 

[aldol16]

Realistically, the exam and scoring process is extensively validated to ensure that a standardized test is both valid and reliable. If you could take the exam at a different time and score better due to a different test-taker pool, then the exam wouldn't be a good exam. This isn't a half-baked idea run out of someone's garage-- they very likely have hired the appropriate people to aid in the process.

Which is exactly why I'm asking if anybody has valid knowledge about how they do the corrections. It's a very simple argument to make that somebody taking the exam in July would have an advantage over someone taking it in January, for example. Which is why I'm confident that they have accounted for it. And because I'm curious, I'm asking if anybody knows how they account for it.

 

[el_duderino]

I think the AAMC is pretty bright and trust that they're not idiots. Maybe they are, but probably not. It doesn't matter to us anyway.

 

[dempty]

Which is exactly why I'm asking if anybody has valid knowledge about how they do the corrections. It's a very simple argument to make that somebody taking the exam in July would have an advantage over someone taking it in January, for example.

I agree it's a very simple argument, but it's also a highly unlikely scenario. Employing a bit of an equilibrium thought experiment would lead you to conclude that the opportunity probably doesn't exist. Let's assume the AAMC doesn't correct for "seasonality" in the exams. Even if this opportunity did exist, people would take the exam at the best times, and the opportunity would eventually dissipate (because who wouldn't want the advantage?).

Which is why I'm confident that they have accounted for it. And because I'm curious, I'm asking if anybody knows how they account for it.

I suspect the easiest and most reliable source of information is the AAMC. Examining bodies often don't volunteer all the information regarding a process, but rather the most pertinent material is shared (for practical reasons). I can't see how sending them a well-written email could do any harm. This is your best bet for actually making progress with your questions.

 

[aldol16]

Thank you! I didn't think of that - I'll definitely see what I can find out.

 

[dempty]

Thank you! I didn't think of that - I'll definitely see what I can find out.

No problem--please share what you find. This is likely a question that people will have from time to time.

 

[theonlytycrane]

@aldol16 did you ever learn any more about this, out of curiosity? I was wondering how exam scores would be kept comparable even between different years.

 

[Panteleimon]

https://students-residents.aamc.org/applying-medical-school/article/how-new-mcat-exam-scored/

It doesn't matter when you take it. Your raw score is modeled into a scaled score which "remains constant". Scaled score is the same if you take it in the spring, summer, fall, or winter. Scaled score is the same if its raining, snowing, or sunny outside. I think the only time it matters is during the third full moon of the lunar year, when neurotic premeds prowl the forums...

Not that it's the third full moon of the lunar year.

 

[aldol16]

@aldol16 did you ever learn any more about this, out of curiosity? I was wondering how exam scores would be kept comparable even between different years.

Yeah, so your scores are curved only based on other students who took the same test as you (not the same test day, but the same version of the test). There's some secret formula they use to make sure your scaled score is really representative of how you do relative to other students. However, time of year is not a factor they take into account. I guess it's up to the student to take it when he or she is the most ready for it and so the AAMC assumes all test dates are equivalent.