The Grading of Experimental Passages?

Boost your MCAT score by 12 points or more.
This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
Fivo

Fivo

Membership Revoked
Removed
10+ Year Member
Joined
May 22, 2012
Messages
1,438
Reaction score
13
Can someone fill me in if they know?

I've read a lot that everyone says an experimental passage/question isn't factored into your mcat score. But that just doesn't make sense. If a tester gets that question/passage right, then they would have to be rewarded for it, right? Seeing as how amcas doesn't advertise to the thousands of test takers each year, that there's experimental questions, then they have to assume that some test takers will spend more time on more 'difficult' problems. It would seem ridiculous not to give someone points if they got the right answer.

Does my post make sense? Someone please let me know!
Members don't see this ad.
 
Sort by date Sort by votes
Fivo said:
If a tester gets that question/passage right, then they would have to be rewarded for it, right?
Click to expand...
Not necessarily. It's their rodeo - they can do what they want. This is another reason not to speculate on how well you did on the MCAT after you take it. You could bomb a passage or do extremely well on a passage, but if it's an experimental one, it doesn't count. It might not be fair but it's how they improve the test.
 
Upvote 0 Downvote
i agree, because you could spend a good portion of your time on the "experimental" passage not really knowing its that one.
 
Upvote 0 Downvote
Fivo said:
Can someone fill me in if they know?

I've read a lot that everyone says an experimental passage/question isn't factored into your mcat score. But that just doesn't make sense. If a tester gets that question/passage right, then they would have to be rewarded for it, right? Seeing as how amcas doesn't advertise to the thousands of test takers each year, that there's experimental questions, then they have to assume that some test takers will spend more time on more 'difficult' problems. It would seem ridiculous not to give someone points if they got the right answer.

Does my post make sense? Someone please let me know!
Click to expand...
Just a few thoughts:

Experimental questions aren't necessarily the hardest questions on the exam, they're just the ones that don't have any/sufficient test history use to have a scale established. Given that, it's just as likely that any given experimental question or passage on the exam is of whatever difficulty compared to the questions which actually will contribute to your score.

The MCAT Essentials very explicitly says that experimental items do not count toward your score.

The reason it makes sense to not give a test taker points for getting an experimental question right is that you don't know how to scale the difficulty of the question. As the AAMC told kasho, which he related to us in another thread, the questions are not all of equal value.
 
Upvote 0 Downvote
Members don't see this ad :)
gettheleadout said:
Just a few thoughts:

Experimental questions aren't necessarily the hardest questions on the exam, they're just the ones that don't have any/sufficient test history use to have a scale established. Given that, it's just as likely that any given experimental question or passage on the exam is of whatever difficulty compared to the questions which actually will contribute to your score.

The MCAT Essentials very explicitly says that experimental items do not count toward your score.

The reason it makes sense to not give a test taker points for getting an experimental question right is that you don't know how to scale the difficulty of the question. As the AAMC told kasho, which he related to us in another thread, the questions are not all of equal value.
Click to expand...

I saw this the other day and did not post, but this seriously does not make sense to me. Kaplan explicitly states that all question are of equal value. It seems like the AAMC has changed something recently or Kaplan just finds this easier to say when teaching their strategies. I'm just so confused. It seems like with this, Kaplan would say to move past the hard questions quickly, do the easier ones first, then come back to the hard ones (within one passage) and they do not teach that at all.

Sent from my iPhone.
 
Upvote 0 Downvote
DAPI said:
I saw this the other day and did not post, but this seriously does not make sense to me. Kaplan explicitly states that all question are of equal value. It seems like the AAMC has changed something recently or Kaplan just finds this easier to say when teaching their strategies. I'm just so confused. It seems like with this, Kaplan would say to move past the hard questions quickly, do the easier ones first, then come back to the hard ones (within one passage) and they do not teach that at all.

Sent from my iPhone.
Click to expand...

I'll email the AAMC myself again and get a specific answer in writing if you want, but Kaplan doesn't make, administer, and grade the test...


Sent from my iPhone using SDN Mobile
 
Upvote 0 Downvote
Fivo said:
Can someone fill me in if they know?

I've read a lot that everyone says an experimental passage/question isn't factored into your mcat score. But that just doesn't make sense. If a tester gets that question/passage right, then they would have to be rewarded for it, right? Seeing as how amcas doesn't advertise to the thousands of test takers each year, that there's experimental questions, then they have to assume that some test takers will spend more time on more 'difficult' problems. It would seem ridiculous not to give someone points if they got the right answer.

Does my post make sense? Someone please let me know!
Click to expand...

1. Experimental passages aren't necessarily difficult.
2. It's the test taker's decision what they spend more or less time on.
 
Upvote 0 Downvote
gettheleadout said:
I'll email the AAMC myself again and get a specific answer in writing if you want, but Kaplan doesn't make, administer, and grade the test...


Sent from my iPhone using SDN Mobile
Click to expand...

Yeah, I realize that, it just seems Kaplan would know and be able to inform their students accordingly. Just seems strange to me.

You can email them if you like, but I already took the MCAT so it does not really affect me.

Sent from my iPhone.
 
Upvote 0 Downvote
I mean, it also seems like the AAMC practice test would have the same difficulty scoring system if they did it that way as the questions are rated easy, medium, and difficult already.

Sent from my iPhone.
 
Upvote 0 Downvote
DAPI said:
Yeah, I realize that, it just seems Kaplan would know and be able to inform their students accordingly. Just seems strange to me.

You can email them if you like, but I already took the MCAT so it does not really affect me.

Sent from my iPhone.
Click to expand...
Oh I know, congrats on your score and your overall application by the way, I think you're in for a successful cycle. :) I want to know for my own purposes haha.
DAPI said:
I mean, it also seems like the AAMC practice test would have the same difficulty scoring system if they did it that way as the questions are rated easy, medium, and difficult already.

Sent from my iPhone.
Click to expand...
I've thought about this as well, and I'm not sure what the explanation is. Maybe there's only very slight variation in question values and so the scales are still accurate? I don't know.
 
Upvote 0 Downvote
Sending them an email would be a good way to have an official answer from the AAMC. The person who originally called them before me said that the AAMC representative they spoke with gave them, as an example, that a hard question would be worth 1 point and an easy question would be worth 0.25 points. The lady I spoke with wasn't sure at first, spoke with a supervisor, then came back and confirmed that harder questions are worth more.

Kaplan either doesn't know, doesn't want to bother their students with such specifics, or they want their students to study everything equally and not over analyze the test too much.

I will be applying to Kaplan (hopefully) soon so I should be able to find out directly from their management what the deal is.
 
  • Like
Reactions: 1 user
Upvote 0 Downvote
kasho11 said:
Sending them an email would be a good way to have an official answer from the AAMC. The person who originally called them before me said that the AAMC representative they spoke with gave them, as an example, that a hard question would be worth 1 point and an easy question would be worth 0.25 points. The lady I spoke with wasn't sure at first, spoke with a supervisor, then came back and confirmed that harder questions are worth more.

Kaplan either doesn't know, doesn't want to bother their students with such specifics, or they want their students to study everything equally and not over analyze the test too much.

I will be applying to Kaplan (hopefully) soon so I should be able to find out directly from their management what the deal is.
Click to expand...
I'll email them then and get an answer in writing. I like that the AAMC is response to inquiries, it's nice to have decent customer service from a company with a monopoly. :laugh:

Also, I read the last part and it took me a second to realize that you meant apply to as in for a job haha... I could ask my campus Kaplan manager as well, since they've offered me a job before. I'm interested to see what they say either way.
 
Upvote 0 Downvote
Yeah, I would seriously love to know what Kaplan, and other test prep companies, have to say about this because it seems wrong for them to say over and over that each question is worth the same when that may not be the case.... Sounds like a class action law suit :D
 
Upvote 0 Downvote
gettheleadout said:
I'll email them then and get an answer in writing. I like that the AAMC is response to inquiries, it's nice to have decent customer service from a company with a monopoly. :laugh:

Also, I read the last part and it took me a second to realize that you meant apply to as in for a job haha... I could ask my campus Kaplan manager as well, since they've offered me a job before. I'm interested to see what they say either way.
Click to expand...

Definitely let us know. My other reason for calling them was to find out a legitimate answer as to why it takes them a month to report our grades. It became clear that the essays were not holding them up and she just kept saying "the system" takes that long to do it. I would really love to speak to an IT guy there and see what he has to say about it because with the computing power that we have available to us now, I can't for the life of me imagine why their computers couldn't calculate the scores within at most a weeks time... something fishy is going on. We will see what happens next year when they remove the writing section, but the representative I spoke with didn't mention that as the issue.
 
Upvote 0 Downvote
Members don't see this ad :)
gettheleadout said:
Oh I know, congrats on your score and your overall application by the way, I think you're in for a successful cycle. :)
Click to expand...

Missed this comment earlier. Thanks!

I have a feeling the AAMC is going to get a bunch of calls on this on Monday :laugh:

I also just posted this question in the Kaplan forum as their books explicitly say all questions are equal whereas the AAMC does not really do that in the MCAT essentials. It will be interesting to see if we get an answer from Kaplan.

See thread there: http://forums.studentdoctor.net/showthread.php?p=12729652#post12729652
 
Upvote 0 Downvote
Just to update everyone, here's the response I got from the AAMC with my initial question included:

Dear ______:

Thank you for contacting the Medical College Admission Test (MCAT) program.

We regret that we do not provide specific details about the scoring process for security and validity purposes. However, we recommend that you visit https://www.aamc.org/students/applying/mcat/preparing/85436/preparing_understandingscores.html to learn more about how to interpret your scores.

Please let us know if you have questions or are in need of any additional assistance.

Warm Regards,
Tiondra

The MCAT Resource Center
Association of American Medical Colleges 2450 N. St. NW Washington DC 20037

General Inquiries: 202-828-0690
www.aamc.org/mcat

Customer Issue: Subject:Scoring Question
Date Sent: 2012/07/10
Time Sent: 15:37
Sender ID: _____________
Subject: Scoring Question
Message Text: MCAT Resource Center,

According to the MCAT Essentials:

"Because each form has the potential to be slightly more difficult or easy than another, raw scores are converted to a scale that takes into consideration the level of difficulty of test questions in each testing form ensuring equal scoring."

I'm wondering if I could have a detailed explanation of how this affects the scoring of the exam?

Are individual questions given different amounts of raw weight? (i.e. one question could be worth 0.5 relative points and another worth 1.0) Or does this mean that the questions are all equally "worth" the same amount and that an exam form which includes more difficult questions will merely have a more lenient scale for the raw scores to be converted? (i.e. on a given form of the exam the number of questions correct needed to achieve a score of 10 would be greater than that required on an exam form with more difficult questions.)

I would really appreciate a detailed enough answer to clarify this.

Thank you!

______________
______________<mailto:____________>
Attachments:
CC Recipients:MCAT Open/Close
Click to expand...
As you can see, this isn't something that anyone can actually know for sure.
 
Last edited:
Upvote 0 Downvote
epic bump, interesting email you received, seems like we are totally kept in the dark about the scoring of the exam.
 
Upvote 0 Downvote
YouNeverKnow22 said:
epic bump, interesting email you received, seems like we are totally kept in the dark about the scoring of the exam.
Click to expand...

It does seem that way. Still, I think I would most likely bet (and it seems easier to assume) that the difficulty of the questions on a particular exam is factored in simply by adjusting the scale and that all questions are worth equal points.
 
Upvote 0 Downvote
gettheleadout said:
It does seem that way. Still, I think I would most likely bet (and it seems easier to assume) that the difficulty of the questions on a particular exam is factored in simply by adjusting the scale and that all questions are worth equal points.
Click to expand...

that's what I think, props to you for sending that email, it really highlights the ambiguity that the AAMC likes to hide under :laugh:
 
Upvote 0 Downvote
I think I know where the different grading of points come in.

On a easy test each question could be worth a small amount of points but on a hard test a question could be worth a larger amount of points. For example, on the easy test a 28/40 is an 8 but on the hard test a 24/40 is an 8, because on the harder test each question is worth more points than on the easier test.
 
Upvote 0 Downvote
gettheleadout said:
It does seem that way. Still, I think I would most likely bet (and it seems easier to assume) that the difficulty of the questions on a particular exam is factored in simply by adjusting the scale and that all questions are worth equal points.
Click to expand...

Agreed. The whole scaling individual questions to different values just goes against a lot of the information they do release on how they score tests. See here:

"How are the multiple-choice sections of the MCAT exam scored?
Each score that you achieve on the three scored multiple-choice sections is based on the number of questions you answer correctly. This raw score is a reflection of your correct answers only. This means that a wrong answer will be scored exactly the same as an unanswered question; there is no additional penalty for wrong answers. Therefore, even if you are unsure of the correct answer to a question, you should make your best guess.

The scores from each of these three sections will be converted to a scale ranging from 1 (lowest) to 15 (high). For example, if your raw score on one of the sections is between 40 and 43, your converted score might be 11. Scores ranging from 44 to 46 might have a converted score of 12, and so forth. The voluntary Trial Section is not scored."(https://www.aamc.org/students/applying/mcat/preparing/85436/preparing_understandingscores.html)
 
Upvote 0 Downvote
maybemed2013 said:
I think I know where the different grading of points come in.

On a easy test each question could be worth a small amount of points but on a hard test a question could be worth a larger amount of points. For example, on the easy test a 28/40 is an 8 but on the hard test a 24/40 is an 8, because on the harder test each question is worth more points than on the easier test.
Click to expand...

That doesn't make any sense. Both tests would have the same number of questions, so the total possible "raw" points would be different between the two in this case, but each question would still contribute 1/40 or 1/52 of the total "raw" points for that section regardless of whether it's an "easy" test or a "hard" test.

If you instead mean to say that easy questions within a test are worth fewer "raw" points than hard questions, then that is one possible interpretation of the scaling. Another is that the difference in overall difficulty of the test form just alters the scaling, so questions are equal amounts of "raw" points in every test but the number correct needed for a certain scaled score changes.
 
Upvote 0 Downvote
Tots said:
Agreed. The whole scaling individual questions to different values just goes against a lot of the information they do release on how they score tests. See here:

"How are the multiple-choice sections of the MCAT exam scored?
Each score that you achieve on the three scored multiple-choice sections is based on the number of questions you answer correctly. This raw score is a reflection of your correct answers only. This means that a wrong answer will be scored exactly the same as an unanswered question; there is no additional penalty for wrong answers. Therefore, even if you are unsure of the correct answer to a question, you should make your best guess.

The scores from each of these three sections will be converted to a scale ranging from 1 (lowest) to 15 (high). For example, if your raw score on one of the sections is between 40 and 43, your converted score might be 11. Scores ranging from 44 to 46 might have a converted score of 12, and so forth. The voluntary Trial Section is not scored."(https://www.aamc.org/students/applying/mcat/preparing/85436/preparing_understandingscores.html)
Click to expand...

Yeah, the confusion just comes about because they don't explicitly say that each question is worth the same number of raw points (though this does seem more likely), since we know that they internally rate questions based on relative difficulty (i.e. the point of giving experimental questions on every MCAT).
 
Upvote 0 Downvote
what the bloody hell is going on. So it's possible that each question isn't worth the same amount??
 
Upvote 0 Downvote
Platonic said:
what the bloody hell is going on. So it's possible that each question isn't worth the same amount??
Click to expand...

This was the question we had a while back (and a while back in this thread), but A) the AAMC won't explicitly comment on it, and B) it seems unlikely it's actually the case that question value varies. It's more likely, simply logistically, that the scale is just adjusted based on difficulty distribution of questions within an exam form.

Basically it's possible, but I'd just assume they're all the worth the same.
 
Upvote 0 Downvote
You must log in or register to reply here.
Next unread thread

Similar threads

M
Are JW passages for B/B CP... etc representative of the actual exam?
Replies
0
Views
915
mdx5152
M
W
What is your thought process when reading CARS passages and questions? Step by step process
Replies
3
Views
2K
JAK2-STAT3
J
M
  • Question
Jack Westin Duplicate CARS Passages?
Replies
1
Views
879
zorcandpals
zorcandpals
Lakersoverclippers
MCAT CARS Matching Passage with Answer Choices
Replies
0
Views
904
Lakersoverclippers
Lakersoverclippers
L
  • Question
Running out of time on passages
Replies
16
Views
1K
Lolaaa
L
Top