MCAT Curved or Pre-scaled?

[jared_the_great]

This is a topic with much confusion surrounding it. And I blame that mostly on AAMC's perpetually vague explanations.

It seems the prevailing wisdom here on SDN is that the scale for any given MCAT is determined prior to the test date based on the difficulty of the questions included on it (difficulty assessed by including them as experimental questions/passages on previous exams). This seems like a pretty fair, logical way to do it.

However ... does the following from this FAQ perhaps negate that explanation and indicate that it is, indeed, curved based on the performance of all test-takers on that particular day?

Why does it still take so long to turnaround MCAT scores now that the writing section is gone?

The analyses required to score the multiple-choice sections of the MCAT exam take about the same time as those conducted for the writing sample. Previously, scoring for these sections occurred simultaneously. Therefore, students should not expect to receive scores any earlier with the writing sample removed than when it was included in the exam.

---

Members don't see this ad.

 

[TexasSurgeon]

Maybe its because you don't know how to use Google

Examinees often ask if earning a high score or higher percentile is easier or harder at different times of the testing year. They ask whether they have a better chance of earning a higher score in April or in August, for example. The question is based on an assumption that the exam is scored on a curve, and that a final score is dependent on how an individual performed in comparison to other examinees from the same test day or same time of year.



While there may be small differences in the MCAT exam you took compared to another examinee, the scoring process accounts for these differences so that an 8 earned on physical sciences on one exam means the same thing as an 8 earned on any other exam. The percentile provided on your score report simply indicates what percentage of examinees from the previous testing year scored the same as you did on the MCAT exam.



How you score on the MCAT exam, therefore, is not reflective of the particular exam you took—including the time of day, the test date, or the time of year—since any difference in difficulty level is accounted for when calculating your scale scores (see above for information about scaling).

https://www.aamc.org/students/applying/mcat/preparing/85436/preparing_understandingscores.html

 

[gettheleadout]

I am 100% serious I opened the MCAT Disc. forum just now thinking to myself how often this question is asked and wondering if I should put something in my signature about it.

The raw-to-scaled score conversion scale is set before the exam is administered. The information you've present in the OP only contradicts the info in other AAMC materials if you choose to interpret it to mean that the analysis of raw and scaled scores is specifically being conducted using examinee data from that test date. It does not say this.

I really, sincerely hope this issue can be put to rest on SDN.

 

[jared_the_great]

Maybe its because you don't know how to use Google

Except... nowhere in that section does it say the scale is set prior to the exam being administered. It says: "Just trust us. Our scaling is totally, completely fair & perfect, and you definitely deserve exactly the score we give you."

SDN knows for a fact that they're not figuring the scale after the fact. SDN knows for a fact that your score is determined the instant you hit "submit." Even though AAMC says it takes them 30 days of hard work after the exam to figure it out.

 

[IslandStyle808]

Except... nowhere in that section does it say the scale is set prior to the exam being administered. It says: "Just trust us. Our scaling is totally, completely fair & perfect, and you definitely deserve exactly the score we give you."

SDN knows for a fact that they're not figuring the scale after the fact. SDN knows for a fact that your score is determined the instant you hit "submit." Even though AAMC says it takes them 30 days of hard work after the exam to figure it out.

Here is the section stating whether the test is curved or scaled "again." Choose what you want to believe. There was a written section before and of course that was the reason before 2012 that they needed time. Maybe it is easier for them to keep the 30 day time frame and that way it is easier to double check things, who knows.

https://www.aamc.org/students/applying/mcat/preparing/85436/preparing_understandingscores.html

 

[jared_the_great]

To me, a new statement from the AAMC saying explicitly that scores are not scaled prior to the exam, but rather in the 30 days afterward, was interesting and surprising.

I assumed others might find it interesting as well, and might be willing to discuss the implications.

Sorry if I misjudged the climate here, or upset anyone by challenging their deeply cherished beliefs about a pre-set scale.

 

[gettheleadout]

Sorry if I misjudged the climate here, or upset anyone by challenging their deeply cherished beliefs about a pre-set scale.

Before you go insulting people, let's get something straight. This:

To me, a new statement from the AAMC saying explicitly that scores are not scaled prior to the exam, but rather in the 30 days afterward, was interesting and surprising.

Is not what the AAMC statement you quoted in the OP says. You're referring to the establishment of the scale itself (when scores are converted is irrelevant, it could be 5 seconds before score release and it wouldn't matter, we care about when and how the scale itself is created), and the AAMC statement does not, explicitly or otherwise, say that the scale for the particular exam form is created after the administration of the exam. Furthermore, the AAMC statement in no way supports a curve, which your thread title misleadingly presents as the alternative to the already known truth that it is not a curved exam, but scaled based on the test content.

What it says is the analyses required to score it are performed after the administration. All this means is they're converting scores from raw to scaled and analyzing the bulk data as they go. For each administration they will of course perform data analysis to look for abnormalities that would reflect systematic computer scoring error, cheating patterns, etc. Their statement does not imply that they are establishing the conversion scale post-exam nor that they are curving scaled scores based on test-taker performance. The conclusions you're drawing from the AAMC statement are erroneous.

In any case, I find it interesting that you're lamenting the lack of discussion when I already raised these problems with your interpretation in my first response (quoted below), and you had no reply.

The information you've present in the OP only contradicts the info in other AAMC materials if you choose to interpret it to mean that the analysis of raw and scaled scores is specifically being conducted using examinee data from that test date. It does not say this.

In this post I hope to have clarified that I'm referring to an establishment of a conversion scale based on test-day examinee performance, something that, I reiterate, is not even implied by the AAMC statement in question.

 

[shh-pthway]

... how many experimental questions do you think verbal has? Because it is 40-q section, I cannot imagine that it would have many. It's an interesting section because each question can count heavily (especially when in double digits)

Also for science sections, I noticed on the MCAT that there would sometimes be questions that would be very related in what the q's were asking (as in, if you get one wrong, you are probably getting the other wrong too). I wonder if in these circumstances 1 out of the 2 is experimental... (of course, at the end of the day, who knows...)

 

[BerkReviewTeach]

The interesting thing as I see it is that whether or not some or part of the exam is based on previous data points (test takers who had the same passage on a previous exam), it doesn't change the fact that you need to get as many questions right as possible to get a good score on the MCAT. The time and energy spent pondering the details of the curve and then stressing if it's fair is wasted time and energy that would be better spent preparing for the MCAT. If you had a tangible answer, which is highly unlikely to ever come, would it change how you studied for the MCAT?

 

[jared_the_great]

Before you go insulting people

Was I the one being insulting here? What I perceived was that I posted what I thought was interesting, new info on a much-discussed topic, and got only exasperation, dismissal and condescension. It genuinely surprised me.

This:
Is not what the AAMC statement you quoted in the OP says.

Ok, you got me. I was exaggerating a bit. I do think it's a short and pretty reasonable inference, though.

What it says is the analyses required to score it are performed after the administration. All this means is they're converting scores from raw to scaled and analyzing the bulk data as they go.

Really? You think it takes them 30 days to generate scores when the scale is already pre-determined? That would be the process that the AAMC system performs in seconds for practice exams.

If they're insisting it takes them 30 days to do the analysis, I find it far more plausible that they're referring to the difficult and time-consuming part of scoring --- setting the scale.

You seem to be extraordinarily certain that the scale is set in stone prior to the exam, so maybe you've read something I haven't. I don't see anything in this thread that says that. I certainly don't see it in the FAQ that's been reposted here more than once for benefit of dumb old me. I haven't seen it in the other AAMC material I've read. But it's entirely possible there is other official info of which I'm unaware.

 

[jared_the_great]

Did you ever consider the fact that the AAMC knows all your practice test scores, because they administered them?

What if the scale isn't based on data from test-day performance, or from prior experimental passages. What if the scale is set by comparing everyone's real exam performance with their practice exam performance?!?

dun dun DUN!

Ha ha...

 

[TBRBiosadist]

Most threads on sdn are enlightening I think I lost a few brain cells in this one

 

[DoctorLacrosse]

Did you ever consider the fact that the AAMC knows all your practice test scores, because they administered them?

What if the scale isn't based on data from test-day performance, or from prior experimental passages. What if the scale is set by comparing everyone's real exam performance with their practice exam performance?!?

dun dun DUN!

Ha ha...

haha I have to admit, I've actually thought about this before. crazy to imagine lol #AAMCconspiracy