how does the curve work ?
- Thread starter thinkpositive
- Start date
- Joined
- Jun 27, 2004
- Messages
- 8,508
- Reaction score
- 7
- Points
- 4,591
- Medical Student
yea i'd like to know that oo...there are so many {false} rumors. Is each version scaled differently?
- Joined
- Jun 27, 2004
- Messages
- 8,508
- Reaction score
- 7
- Points
- 4,591
- Medical Student
that's what i heard too but i don't believe it. how could a curve be set when it measures specific people's performance on a specific test?????
- Joined
- Jan 7, 2004
- Messages
- 416
- Reaction score
- 6
- Points
- 4,551
- Resident [Any Field]
I'm guessing that all of the questions that were on our exams have shown up on multiple administrations before, so they have a good set of datapoints from which to draw up an 'expected' bell curve (ie: this is where the experimental passages come in)..
And after each administration, they run the stats (either by form # or by passage.. (since it seems multiple forms had the same passages/questions.. doing it by passages would give them a larger sample size and thus a more accurate representation).. compare it to the 'expected' bell curve, and if there arent any anomolies, create a new bell curve, with which they will use to do the 'scaled' scores. Either that or they combine the data points from this test with the data points from the older administrations to come up with an updated bell curve.. (which means they would need to do it by passage b/c i assume they wouldnt ever have the same forms between adminstrations)...
i took a couple of stats courses in undergrad.. so this makes the most sense to me and would seem to be the most fair way to standardize the grading scale...
And after each administration, they run the stats (either by form # or by passage.. (since it seems multiple forms had the same passages/questions.. doing it by passages would give them a larger sample size and thus a more accurate representation).. compare it to the 'expected' bell curve, and if there arent any anomolies, create a new bell curve, with which they will use to do the 'scaled' scores. Either that or they combine the data points from this test with the data points from the older administrations to come up with an updated bell curve.. (which means they would need to do it by passage b/c i assume they wouldnt ever have the same forms between adminstrations)...
i took a couple of stats courses in undergrad.. so this makes the most sense to me and would seem to be the most fair way to standardize the grading scale...
- Joined
- Aug 9, 2002
- Messages
- 11
- Reaction score
- 0
- Points
- 0
This is from e-mcat.com, posted after the April exam:
"The curve is not "predetermined" because we don't know ahead of time what the raw->scale score conversion table will be. The field testing (use as experimentals) of all items gives us enough information to make sure that the item is working as intended (not miskeyed, not of an extreme difficulty, and high scorers generally answer it right more often than low scorers). It is not enough to give us the precise difficulty estimate that would let us know ahead of time what the conversions will be.
Through the judicious reuse of items across years, we are able to compare every group's performance to that of the group who took the test in 1991, when the "new" MCAT began. For example, the increasing average BS scores across the years tells us that applicants today know more biology and organic than did those ten years ago. If items were not reused, you'd just be out of luck if you happened to get a hard form, or had some really smart people also take your form.
That's how we keep April and August on the same scale, too. By mixing of items from previous Aprils and Augusts, we can compare every exam with earlier exams from both time periods. Our goal is to make it where, if the same person, with the same level of knowledge, took both April and August (of any year), they would earn the same score both times.
For this reason, April is no easier than August. August scores do tend to be a little lower, partly because about 40% of the examinees are repeating because they were not satisfied with their April scores. This group does not (usually) include the high scorers, so their average is lower. August first-time takers also score a little lower on average. Is it because they are taking it early, before their Junior year? Is it because they weren't ready for April and delayed until August? We don't know for sure, but it's been consistent across the years.
I know this is more than you asked for, but I hope it is helpful. You've just taken your first class in psychometrics."
And then, in response to a request for clarification:
"After we do our equating of the difficulty of the forms, it is as if everyone took exactly the same set of items. Try thinking of the averages in terms of people, not of scores. If the average person taking the MCAT in April knows a little more than the average person who takes it in August, then the average score will be a little higher in April.
Here's real data:
www.aamc.org/students/mcat/examineedata/august03tbl1r.pdf
shows that the average VR score in Aug, 2003 was 8.1, while the average VR score in April, 2003 was 8.2. (www.aamc.org/students/mcat/examineedata/table0403.pdf)
This is not a difference in the exams themselves, this is a difference between the groups of people who choose to take the exam at the different times. If a person, without either learning or forgetting anything, took the MCAT both times, they should get the same score. But they might be below average in April, and above average in August, because the averages are different. (Ok, I realize that's kind of hard when the averages are 8.1 and 8.2 and people only get whole numbers as scores, but I hope you see what I mean.)
You have the same chance of earning your 10 or 12 or whatever, regardless of whether you take it the MCAT in April or August."
"The curve is not "predetermined" because we don't know ahead of time what the raw->scale score conversion table will be. The field testing (use as experimentals) of all items gives us enough information to make sure that the item is working as intended (not miskeyed, not of an extreme difficulty, and high scorers generally answer it right more often than low scorers). It is not enough to give us the precise difficulty estimate that would let us know ahead of time what the conversions will be.
Through the judicious reuse of items across years, we are able to compare every group's performance to that of the group who took the test in 1991, when the "new" MCAT began. For example, the increasing average BS scores across the years tells us that applicants today know more biology and organic than did those ten years ago. If items were not reused, you'd just be out of luck if you happened to get a hard form, or had some really smart people also take your form.
That's how we keep April and August on the same scale, too. By mixing of items from previous Aprils and Augusts, we can compare every exam with earlier exams from both time periods. Our goal is to make it where, if the same person, with the same level of knowledge, took both April and August (of any year), they would earn the same score both times.
For this reason, April is no easier than August. August scores do tend to be a little lower, partly because about 40% of the examinees are repeating because they were not satisfied with their April scores. This group does not (usually) include the high scorers, so their average is lower. August first-time takers also score a little lower on average. Is it because they are taking it early, before their Junior year? Is it because they weren't ready for April and delayed until August? We don't know for sure, but it's been consistent across the years.
I know this is more than you asked for, but I hope it is helpful. You've just taken your first class in psychometrics."
And then, in response to a request for clarification:
"After we do our equating of the difficulty of the forms, it is as if everyone took exactly the same set of items. Try thinking of the averages in terms of people, not of scores. If the average person taking the MCAT in April knows a little more than the average person who takes it in August, then the average score will be a little higher in April.
Here's real data:
www.aamc.org/students/mcat/examineedata/august03tbl1r.pdf
shows that the average VR score in Aug, 2003 was 8.1, while the average VR score in April, 2003 was 8.2. (www.aamc.org/students/mcat/examineedata/table0403.pdf)
This is not a difference in the exams themselves, this is a difference between the groups of people who choose to take the exam at the different times. If a person, without either learning or forgetting anything, took the MCAT both times, they should get the same score. But they might be below average in April, and above average in August, because the averages are different. (Ok, I realize that's kind of hard when the averages are 8.1 and 8.2 and people only get whole numbers as scores, but I hope you see what I mean.)
You have the same chance of earning your 10 or 12 or whatever, regardless of whether you take it the MCAT in April or August."
- Joined
- Jun 27, 2004
- Messages
- 8,508
- Reaction score
- 7
- Points
- 4,591
- Medical Student
thanks, that was a very comprehensive and informative description of the process
and i hope that's true that traditionally August people do worse
and i hope that's true that traditionally August people do worse
- Joined
- May 17, 2004
- Messages
- 304
- Reaction score
- 0
- Points
- 0
Psycho Doctor said:
I hope thats indicative of the other test-takers and not us 😉
- Joined
- May 29, 2004
- Messages
- 463
- Reaction score
- 1
- Points
- 4,551
- Location
- Bardo
- Pre-Health (Field Undecided)
Psycho Doctor said:
Why? This wouldn't benefit any of the August test-takers, since most people would do a little worse compared to the April test-takers.
- Joined
- Jun 27, 2004
- Messages
- 8,508
- Reaction score
- 7
- Points
- 4,591
- Medical Student
well yea ...true..sorry for not being clear on that very important point! 😀
- Joined
- Jun 27, 2004
- Messages
- 8,508
- Reaction score
- 7
- Points
- 4,591
- Medical Student
yea but we would do better compared to them and thus get a higher % and score
- Joined
- Jun 27, 2004
- Messages
- 8,508
- Reaction score
- 7
- Points
- 4,591
- Medical Student
some passages are thrown in randomly to "test" how people do on them; they don't count and you don't know which ones they are
- Joined
- Apr 28, 2004
- Messages
- 41
- Reaction score
- 0
- Points
- 0
I really do not fully understand that aamc quote! From what I got out of it, the curve really is set before we take the exam, but there is a separate curve for each passage, not the entire exam. i guess the overall curve for your form is actually a composite of the individual passage curves that comprise it. Thus the curve for form AK (my ridiculously hard form) should be a more generous curve based on the individual passages that all happened to be hard passages, and happened to find themselves together on my particular form, but the curve was set by prior examinees over the years who took those passages-- passages which were scattered over many different forms at the time-- and not the other people who were taking the exam the same time I was. I guess this would make sense. I just hope I'm right that form AK was very hard and has a more generous curve. Of course, this begs the question why AAMC would allow one form to be comprised of harder passages than another form. Even though there might be a more generous curve, that is not taking into account the psychological effects of having one hard pasage after another. That really deflates one's confidence and could ruin their focus and resolve during the exam. Such happened to me.
- Joined
- Jun 27, 2004
- Messages
- 8,508
- Reaction score
- 7
- Points
- 4,591
- Medical Student
no wonder it takes so long to grade it...they need complex math to figure it all out and none of the graders are trained properly...it's a two month training course
- Joined
- May 26, 2004
- Messages
- 7
- Reaction score
- 0
- Points
- 0
i definitely agree. BN BS was a beast. but i am sure they all have their tough passages.
- Joined
- Aug 16, 2004
- Messages
- 2
- Reaction score
- 0
- Points
- 0
Granted I'm only a HS student... I have a friend interested in the MCAT and I took the free AAMC practice test to starve off boredom (I know, I'm weird)... but if it's scaled like the SAT (I think it is) it goes like this:
1. Each form has a few (maybe 10%) of the questions repeated. That is, there are a few questions on each test edition that have appeared on the MCAT before.
2. The MCAT scorers use those questions for reference. Those use this in conjunction with the curve of the test to scale it.
For example: Let's say there are editions BN and AK, with AK being very hard and BN being very easy. Both have the same reference questions, of varied difficulty.
Let's say now that both editions get the same result on the reference questions. Seeing that the test groups are equal, AAMC will give BN a mathetmatically correct harsher curve, 'cause the questions are easier, and AK will get an easier curve.
Now let's say AK was in August and BN was in April, and in reality the questions were of very similar difficulty -- only, AK test-takers happened to be stupid and BN ones were brighter. AAMC will note the difference in the reference questions and give them an appropriate curve.
As an added example, if 100,000 preschool students take your form of the MCAT, the scale will NOT be adjusted. A college curve would be adjusted, but not the scale: AAMC would conclude from the reference questions that the group is just stupider.
1. Each form has a few (maybe 10%) of the questions repeated. That is, there are a few questions on each test edition that have appeared on the MCAT before.
2. The MCAT scorers use those questions for reference. Those use this in conjunction with the curve of the test to scale it.
For example: Let's say there are editions BN and AK, with AK being very hard and BN being very easy. Both have the same reference questions, of varied difficulty.
Let's say now that both editions get the same result on the reference questions. Seeing that the test groups are equal, AAMC will give BN a mathetmatically correct harsher curve, 'cause the questions are easier, and AK will get an easier curve.
Now let's say AK was in August and BN was in April, and in reality the questions were of very similar difficulty -- only, AK test-takers happened to be stupid and BN ones were brighter. AAMC will note the difference in the reference questions and give them an appropriate curve.
As an added example, if 100,000 preschool students take your form of the MCAT, the scale will NOT be adjusted. A college curve would be adjusted, but not the scale: AAMC would conclude from the reference questions that the group is just stupider.
- Joined
- Jun 23, 2003
- Messages
- 5,786
- Reaction score
- 913
- Points
- 5,301
- Attending Physician
tweaks said:
What was the point of this? Someone posted how it is actually figured 10 posts ago.
Those of us who took the MCAT for the first time in August have a somewhat better advantage over those who took it first in April because, as the quote stated, 40% of August takers are re-takers from April. Combining this with the known fact that most people don't improve, those of us taking it for the first time in August (if well prepared) should do a little better on average than everyone else.
Let's find out in October.
- Joined
- Aug 16, 2004
- Messages
- 2
- Reaction score
- 0
- Points
- 0
JKDMed,
I was trying to reiterate it for people who didn't understand it. The point I was making was that it *doesn't matter* if you take it August or April because of the scaling system. Yes, if you take it in August your score will be in a higher percentile amongst your group, because the "average student" is worse, BUT your scaled score -- a 28, 31, etc -- will remain the same, again because the average score is worse.
I was trying to reiterate it for people who didn't understand it. The point I was making was that it *doesn't matter* if you take it August or April because of the scaling system. Yes, if you take it in August your score will be in a higher percentile amongst your group, because the "average student" is worse, BUT your scaled score -- a 28, 31, etc -- will remain the same, again because the average score is worse.
thanks for explaining guys ! 🙂
