Proposal for a better MCAT

[wizzed101]

I know you AAMC people read this forums to ensure that no questions are leaked.

Problem: the current test does not cover enough topics and may not adequately demonstrate the strengths of each candidate.
Solution: Obviously, more testing are required. But how? Requiring people to take multiple tests and average them do nothing to alleviate the outliners and since it is a requirement, some adcoms may still be swayed by the high score/low score.

I propose a competition system similar to one used in olympic swimming. Basically, test takers have to participate in rounds. Later round corresponds to higher score bracket.

Round 1: 472-500. If you score higher than 499, you move to the next round. But you don't have to if you believed that you have reached your max potential. Your official MCAT score will be 500.

If you score below that, your official score will be whatever it is. You cannot progress further/retake the test within 12 months.

Q: What if I score like 523 on my first attempt and decide to stop?
A: Your official score will be 500.

Round 2: 500-510
Same rules

Q: What if I score 498?
A: Your official score will be 500 since you have passed the first round.
Round 3: 510-515.

And so on: 515 -519, 519- 523, 523-528

Q: Are the tests on later rounds harder?
A: Not at all. Just like in swimming, you don't change pools in later rounds.

The tests are available 10 times a year. And since they are of the same quality, you can take any exams available and progress on your own: a person at the 500-510 bracket can take the same test as another person in the 520-528 bracket.

Q: Do I have to pay for multiple test?
A: You pay the normal fee to start. If you decide to stop, that's all you have to pay. If you fail to progress, you pay for your failure.

Q: I am confident that I can score 526 every time and want to progress faster. What are my options?
A: 526 is within the 523-528 bracket. If you score within that range, it will be your official MCAT score.
If you fail to do so and only get, for example, 522, then your official score will be determined by your current bracket.

Example: You are in 510-515 and score a 522 and thus failed the 523-528 bracket, your official score will be 515. But if you have gotten 509 instead, your official score will be 510.

Be careful, if you start a high gambit at a very low bracket can end very badly. Use your judgments.

Q: How long can I wait before I start the next round?
A: 6 months

Q: If I fail a round and wait for a year, how will my score report look like?
A: You will have 2 score.

The score report will only contain passed/failed bracket, gambit and the final official scores of each tournament.
-----------------------------------------------------------
What do you guys think? I think future generation would love it. Very fair!

---

Members don't see this ad.

 

[787593]

Bad idea

 

[wizzed101]

Yea. I would hate it too. But it's for the future generation. Something gotta give.

Besides, just by looking at the journey of a candidate to reach his final score, one can tell a lot about that candidate: are they prudent? reckless? confident? overly conservative? arrogant? etc..

 

[SwordOfTheMorning]

So you want to have multiple rounds, 3+, with 6 month waits in between? That's 2 or more years of testing. That sounds like a horrible idea, even ignoring all the other problems.

I'd probably die if I had to take the mcat that many times and over that long of a period. I took it twice and still hated my life for doing so.

 

[wizzed101]

Q: I am confident that I can score 526 every time and want to progress faster. What are my options?
A: 526 is within the 523-528 bracket. If you score within that range, it will be your official MCAT score.
If you fail to do so and only get, for example, 522, then your official score will be determined by your current bracket.

Example: You are in 510-515 and score a 522 and thus failed the 523-528 bracket, your official score will be 515. But if you have gotten 509 instead, your official score will be 510.

Be careful, if you start a high gambit at a very low bracket it can end very badly. Use your judgments.

Q: How long can I wait before I start the next round?
A: 6 months

If 6 months have passed, you will be stuck with the current score and have to wait another 6 months for the next cycle.

 

[Dr. Jan Itor]

The MCAT doesn't test enough topics?

Which topics, exactly, do you feel that it neglects that would be vital for med school? And how does this round system "adequately demonstrate the strengths of each candidate?"


Sent from my iPhone using SDN mobile

 

[kb1900]

This is a terrible idea. What problem are you trying to solve exactly rofl?

 

[spartawdw]

But haven't there been findings that if you score at least a 26 you will be successful in medical school?

I have no idea what years of nonstop testing would do to positively affect the quality of matriculants

 

[Ismet]

LOL heck no

 

[WedgeDawg]

I don't think this would benefit anyone at all other than the people who make money off administering the MCAT.

 

[Lawpy]

Problem: the current test does not cover enough topics and may not adequately demonstrate the strengths of each candidate.

This isn't true so I don't know what you're solving.

 

[787593]

Essentially taking an already stressful and time-consuming process and making it worse...

 

[mcloaf]

The OP reads like you got some questions you didn't know the answer to on your MCAT and thought that it was unfair.

"May not adequately demonstrate the strengths of each candidate" AKA if I'd gotten a testing form which just hit my strengths I wouldn't have any problems with the MCAT.

 

[Iz_FtB]

I think the MCAT is already very good at doing what it sets out to do: testing a broad span of knowledge in a way that often requires more than rote memorization. The content list is out there for everyone to see, the questions are fair, the answers are thoughtfully written, and "surprises" on the test aren't usually surprises but a gap in being able to apply things we already know. If we are talking about future improvements, I would focus on making cheap study resources more easily available to everyone.

 

[Horse Apiece]

Only a good idea if the goal was to try and wring every penny out of premed students. The AAMC charges ~$350 per MCAT session, and that is bad enough.

 

[aldol16]

I think the MCAT is already very good at doing what it sets out to do: testing a broad span of knowledge in a way that often requires more than rote memorization. The content list is out there for everyone to see, the questions are fair, the answers are thoughtfully written, and "surprises" on the test aren't usually surprises but a gap in being able to apply things we already know. If we are talking about future improvements, I would focus on making cheap study resources more easily available to everyone.

Couldn't agree with the bolded part more. OP and I had quite a discussion on another thread about randomness involved in achieving a certain score range, which I won't recapitulate here. He/she obviously believes that there is a lot of randomness involved and thus a lot of people scoring in that range aren't really in that range and just got lucky. Hence the whole system here. What he/she refuses to acknowledge is that if one is adequately prepared for the MCAT and takes it multiple times under similar conditions, there's no reason for a huge discrepancy in score at all. Somebody scoring a 523 the first time is unlikely to score a 498 the second time around unless there is some very different condition, like computer failure or something. Somebody who scores a 523 the first time will score very close to that same score the second time around and so on. So the whole system is quite useless.

 

[AnatomyGrey12]

Hell no. Taking the thing once was one time too many.

 

[When the Moon Forgot]

(Content redacted to preserve anonymity)

 

[Hospitalized]

How about this:

508 (equivalent to the old 30) or above gets honors
500 or above gets pass
499 or below gets fail

Simple enough?

With this H/P/F grading scheme, we can finally stop obsessing over this idiotic test.

Actually seems the most logical. Maybe a 520+ can have a highest honors or something because those people are usually intrinsically good test takers.

 

[Donald Juan]

The MCAT is one test. It is not meant to be a fully comprehensive measure of everything everyone can and has done.

 

[RogueBanana]

So.... you're suggesting we take the most stressful exam of our lives over 5 times for a grand total of 35 hours of continuous testing. Only to prove that we can consistently score within the same range? Or just line the pockets of testing centers & the AAMC?

 

[wizzed101]

No not at all.
If you are confident that you can score 526 then take the highest gambit, pay once, test once and be done with it. If you are good, it will not affect you!
Besides, it is actually beneficial for a lot of folks. You take physics, bio, chem in the first year, then you take the first round to see where you are. Study for 6 months and do another round. It's a great boon for a lot of people.
Read again.

Couldn't agree with the bolded part more. OP and I had quite a discussion on another thread about randomness involved in achieving a certain score range, which I won't recapitulate here. He/she obviously believes that there is a lot of randomness involved and thus a lot of people scoring in that range aren't really in that range and just got lucky. Hence the whole system here. What he/she refuses to acknowledge is that if one is adequately prepared for the MCAT and takes it multiple times under similar conditions, there's no reason for a huge discrepancy in score at all. Somebody scoring a 523 the first time is unlikely to score a 498 the second time around unless there is some very different condition, like computer failure or something. Somebody who scores a 523 the first time will score very close to that same score the second time around and so on. So the whole system is quite useless.

Good point. Then this system should not affect genius you! Unless you think that your critical thinking skill hinges on whether or not you know your range. That's an odd sentiment And fyi, yes I can score 520+ consistently. I had a self-inflicted condition after the C/P section so I only score 518. I was quite upset about that. I don't think you even realize what I was arguing in the other thread. I never said that you cannot score 520+ consistently. I believe that you can based on the way you explained things in the MCAT sub-forums.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Back on track. In this system, the only score you know is the final score. Even if you score 526 in the first round, you wouldn't know. You only know that you passed and if you stop then, you get a 500.

And actually the AAMC may make less money! In my proposal, if you pass a round you don't have to pay anything save the initial fee. If you decide to stop, you pay nothing.

Who benefits most? Adcoms.

 

[baratheonfire]

I've never read such brilliant satire in my life

 

[lpp06]

 

[onceawolverine]

I'm traumatized just after reading this idea.

 

[FindMeOnTheLinks]

just when I thought I'd heard it all


Sent from my iPhone using SDN mobile

 

[Ismet]

Only a good idea if the goal was to try and wring every penny out of premed students. The AAMC charges ~$350 per MCAT session, and that is bad enough.

It's only $350? Dang that seems so cheap after going through the USMLEs!

 

[AnatomyGrey12]

Who benefits most? Adcoms.

Oh yes, please explain this.... how does it help any more than the current system?

 

[mmm_cats]

How about this:

508 (equivalent to the old 30) or above gets honors
500 or above gets pass
499 or below gets fail

Simple enough?

With this H/P/F grading scheme, we can finally stop obsessing over this idiotic test.

Or just stick to percentiles. 99th percentile and whatever a 508 is are very different.

 

[mcloaf]

It's only $350? Dang that seems so cheap after going through the USMLEs!

Seriously, still looking at the $1900 hole in my wallet from Step 2 CK and CS a couple months ago.

 

[Horse Apiece]

Seriously, still looking at the $1900 hole in my wallet from Step 2 CK and CS a couple months ago.

So when the school factors in aid for that year, do they account for the Step exams adding a hefty sum?

 

[mcloaf]

So when the school factors in aid for that year, do they account for the Step exams adding a hefty sum?

I had enough wiggle room that I didn't have to go above what I had already borrowed, but presumably they'd help you out if needed. Generally speaking they estimate pretty high for COL so I don't think most students max out their loans and borrow the full amount every year.

Also should be noted that the above are just fees for the testing itself, not the travel and hotel required to get to one of the 5 cities in the US where they administer Step 2 CS.

 

[Ismet]

So when the school factors in aid for that year, do they account for the Step exams adding a hefty sum?

I think my school did, it increased the cost of attendance so you could take out more loans. Makes the hefty sum even heftier.

 

[lpp06]

In all seriousness - this idea adds complexity to a process that already has enough steps, screens and applications. Additionally, why would you want to add more material? What is the point? The MCAT isn't primarily a test of your knowledge of the basic sciences, it is a test to see how well you can apply your knowledge in difficult settings.

The swimming analogy is moot here. The best swimmers are the ones who can swim the fastest, not the ones who understand the intricacies of their stroke and can apply that in different scenarios. The MCAT is more akin to being able to be the best swimmer in a pool, ocean, lake, harbor, bay, marina, kiddie pool, infinity pool and any derivative body of water.

The length of time it took you to explain the idea will certainly translate into Adcoms having to spend more time comparing multiple applicants. Imagine the discussions:

"John scored a 510 while testing in 520-525 but Mary scored a 510 while testing in a 510 - 515. What if Mary actually scored 508? Who's better? Who cares - reject"

 

[When the Moon Forgot]

(Content redacted to preserve anonymity)

 

[wizzed101]

The length of time it took you to explain the idea will certainly translate into Adcoms having to spend more time comparing multiple applicants. Imagine the discussions:

"John scored a 510 while testing in 520-525 but Mary scored a 510 while testing in a 510 - 515. What if Mary actually scored 508? Who's better? Who cares - reject"

No. The report card will look something like this:

John: Passed 500-510. Failed 523-526. Final Score: 510
Mary: Passed 475-500. Failed 500-510. Final Score 508.
Kayla: Passed 475-500. Passed 500-510. Not in progress. Final Score: 510
Kathleen: Passed 475-500. Passed 500-510. Failed 510-515. Final Score: 513
Jim: Passed 475-500. Passed 500-510. Failed 510-515. Final Score: 510.

And you have the date of each attempt.
If I were to pick 2, I'll pick Kayla and Kathleen.

 

[WedgeDawg]

No. The report card will look something like this:

John: Passed 500-510. Failed 523-526. Final Score: 510
Mary: Passed 475-500. Failed 500-510. Final Score 508.
Kayla: Passed 475-500. Passed 500-510. Not in progress. Final Score: 510
Kathleen: Passed 475-500. Passed 500-510. Failed 510-515. Final Score: 513
Jim: Passed 475-500. Passed 500-510. Failed 510-515. Final Score: 510.

And you have the date of each attempt.
If I were to pick 2, I'll pick Kayla and Kathleen.

Why do Kathleen and Jim have different final scores

 

[Goro]

And you know this how?


Problem: the current test does not cover enough topics and may not adequately demonstrate the strengths of each candidate.

 

[lpp06]

No. The report card will look something like this:

John: Passed 500-510. Failed 523-526. Final Score: 510
Mary: Passed 475-500. Failed 500-510. Final Score 508.
Kayla: Passed 475-500. Passed 500-510. Not in progress. Final Score: 510
Kathleen: Passed 475-500. Passed 500-510. Failed 510-515. Final Score: 513
Jim: Passed 475-500. Passed 500-510. Failed 510-515. Final Score: 510.

And you have the date of each attempt.
If I were to pick 2, I'll pick Kayla and Kathleen.

You put a lot of thought into this, so I applaud your creativity, but it offers no distinct advantage over the current system. It only injects complexity and subjective interpretation of something that's meant to be an objective measure. Have you taken the test yet? Also, I don't know if you've applied yet, but when that time comes you'll learn that there are an infinite number of opportunities to display the qualities you mentioned above: prudence, judgement, etc.

 

[kb1900]

Honestly op has provided little to no rationale in his post and thread. Think this is just a troll

 

[Gurkhali]

Why do Kathleen and Jim have different final scores

I believe since you have to get a 515 or above to move onto the next round, they both "failed". Kathleen got a 513 and Jim got a 510. This is starting to sound like a game. A nightmarish, twisted game only the most disturbed masochists would play. Seems like OP is trying to elaborately troll but to go through all this effort under the false premise: "Problem: the current test does not cover enough topics and may not adequately demonstrate the strengths of each candidate".......why?

 

[Lawpy]

No. The report card will look something like this:

John: Passed 500-510. Failed 523-526. Final Score: 510
Mary: Passed 475-500. Failed 500-510. Final Score 508.
Kayla: Passed 475-500. Passed 500-510. Not in progress. Final Score: 510
Kathleen: Passed 475-500. Passed 500-510. Failed 510-515. Final Score: 513
Jim: Passed 475-500. Passed 500-510. Failed 510-515. Final Score: 510.

And you have the date of each attempt.
If I were to pick 2, I'll pick Kayla and Kathleen.

Why do Kathleen and Jim have different final scores

You put a lot of thought into this, so I applaud your creativity, but it offers no distinct advantage over the current system. It only injects complexity and subjective interpretation of something that's meant to be an objective measure. Have you taken the test yet? Also, I don't know if you've applied yet, but when that time comes you'll learn that there are an infinite number of opportunities to display the qualities you mentioned above: prudence, judgement, etc.

I believe since you have to get a 515 or above to move onto the next round, they both "failed". Kathleen got a 513 and Jim got a 510. This is starting to sound like a game. A nightmarish, twisted game only the most disturbed masochists would play. Seems like OP is trying to elaborately troll but to go through all this effort under the false premise: "Problem: the current test does not cover enough topics and may not adequately demonstrate the strengths of each candidate".......why?

I honestly have no idea where OP intends to go with his plan. The examples listed left me more confused than before.

 

[onceawolverine]

Kathleen: Passed 475-500. Passed 500-510. Failed 510-515. Final Score: 513
Jim: Passed 475-500. Passed 500-510. Failed 510-515. Final Score: 510.
If I were to pick 2, I'll pick Kayla and Kathleen.

Why do Kathleen and Jim have different final scores

 

[wizzed101]

Alright. I will just ignore bad memes and personal attacks.

BTW, you cannot accuse me of being a masochist. At worst, it's sadism.
-----------------------------------------------------------------------------------------------------------------------------------

You put a lot of thought into this, so I applaud your creativity, but it offers no distinct advantage over the current system. It only injects complexity and subjective interpretation of something that's meant to be an objective measure. Have you taken the test yet? Also, I don't know if you've applied yet, but when that time comes you'll learn that there are an infinite number of opportunities to display the qualities you mentioned above: prudence, judgement, etc.

It's the same test. My proposal does not take away any objectivity. Plus, you agreed that it did help provide basis for judging other subjective qualities, which can only come closer to the objective one with more data.

And that is the crux of its benefits for adcoms: more data.

Why do Kathleen and Jim have different final scores

Because Kathleen failed the 510-515 bracket with 513 while Jim could've failed with a 510, a 507, a 495 etc... we don't know. It's not without risk just to simply move onto the next bracket. Adcoms may look at Jim and... have many thoughts...

And you know this how?


Problem: the current test does not cover enough topics and may not adequately demonstrate the strengths of each candidate.

I have no data. The AAMC tells us that all version of the tests are roughly the same, also with no data. Their words and my words.... It doesn't look good on my part.
I may be very wrong and they have some brilliant algorithm to evaluate test versions. I have my doubts due to my inability to come up with a way to do so.

You can try it yourself. Make 2 passages of vastly different subjects (biochem vs light, anatomy vs immunology etc...) that test the "same things." Actually I can, but it will be so easy that monkeys can do it because I have to design it to fit the lowest common denominator. Forget about passage, how do you compare these two questions:
What is the start codon?
A) AUG
B) UGG
C) AGC
D CCG

What factors initiate the intrinsic pathway of the blood clotting cascade?
A) IX
B) IV
C) II
D) XII

Which one is harder? Since I wrote those questions, for me they are both easy: either you know or you don't. Neither of them is explicitly stated in the content outline. But I bet you if I use the 2nd question, people will scream bloody murder.
So the only solution is to see the % of people answering it correctly. But that is also problematic due to 2 obvious reasons.

The first is a matter of principle. Consider these 2 questions:

Question 1: A patient with a history of cardiovascular disease is given heparin. After the injection of the drug, this patient, compared to the general population, most likely has an increased chance of:
I. Internal bleeding
II. Blockage of blood vessels
III. Blood in urine

A. I
B. II
C. I and III
D. I, II and III

Question 2: A patient with a history of cardiovascular disease is given a blood thinning drug. After the injection of the drug, this patient, compared to the general population, most likely has an increased chance of:
I. Internal bleeding
II. Blockage of blood vessels
III. Blood in urine

A. I
B. II
C. I and III
D. I, II and III

Intrinsically, question 1 is harder because you have to know what heparin is (AAMC outline- blood clotting cascade- please don't tell me it's reasonable learn the cascade without learning the regulatory mechanisms). But then since most people will not know it. They will guess and when they guess they have 25% chance of getting it correctly. If the question is evaluated by means of polling, then 1 is easier than 2.

The second problem is applicability. The AAMC cannot possibly poll every single question before every test is given. So what do they do? Either they do nothing: aka arbitrarily assign a difficulty or they modify the ones that they do have data for. The second is dubious because the wording of the questions and the answer choices affect the difficulty. When I took the test, I had like 20 mins left after reviewing everything in C/P and stuck with one question. I got it correctly because I finally figured out what the hell they were asking. It was very easy. But whoever wrote that made up a name of type of experiment (I looked that up after the test, there was no such thing - and honestly it sounded stupid) and asked me to "compare." Got 132.

There is so much you can dress up a question until it gets ridiculous.
--------------------------------------------------------------------------------------------------------------
So how do we solve this problem? One answer is to add more question: with more data, randomness vanishes! But I can't imagine it will fly if I propose to make each section 200 questions or so.
But the most obvious answer to the question how "difficult" and "fair" the test was is to ask the test each test taker. We can't show them the score because if they do well they will tend to say that it's fair and vice versa.

How does this help? At the moment, no matter how we slice it, data for each applicant regarding the MCAT is n=1. Making them taking the test again and again doesn't do anything because n=3 is equally useless statistically. Using my proposed system, we give an applicants a chance to evaluate their academic performance. If they think their true bracket is 515-519 and the final score agrees, it will speak volumes for their cases. And if there is a disagreement, the system provide ample opportunities to protect oneself from overestimation. Use your judgment wisely. Although, as I stated above, it is not without risk to just moving one bracket by one like in the case of Jim: he passed the 500-510 and attempted the next and still got 510 as the final score. This can be read flag akin to current situation of retaking the MCAT and doing worse.

And point is this will provide very valuable data! Let me give another example:

Consider two excellent candidate both scoring 521
James: 475-500 ---> 500-510 -----> 510-515 ----->515 - 519-----> 519 - 523. Final score: 521
Kayla: 500-510 ----> 515-519 -----> 519-523. Final Score: 521

They are not identical. Draw any conclusion you want. I personally will lean toward Kayla.

 

[lpp06]

Alright. I will just ignore bad memes and personal attacks.

BTW, you cannot accuse me of being a masochist. At worst, it's sadism.
-----------------------------------------------------------------------------------------------------------------------------------

It's the same test. My proposal does not take away any objectivity. Plus, you agreed that it did help provide basis for judging other subjective qualities, which can only come closer to the objective one with more data.

And that is the crux of its benefits for adcoms: more data.

Because Kathleen failed the 510-515 bracket with 513 while Jim could've failed with a 510, a 507, a 495 etc... we don't know. It's not without risk just to simply move onto the next bracket. Adcoms may look at Jim and... have many thoughts...



I have no data. The AAMC tells us that all version of the tests are roughly the same, also with no data. Their words and my words.... It doesn't look good on my part.
I may be very wrong and they have some brilliant algorithm to evaluate test versions. I have my doubts due to my inability to come up with a way to do so.

You can try it yourself. Make 2 passages of vastly different subjects (biochem vs light, anatomy vs immunology etc...) that test the "same things." Actually I can, but it will be so easy that monkeys can do it because I have to design it to fit the lowest common denominator. Forget about passage, how do you compare these two questions:
What is the start codon?
A) AUG
B) UGG
C) AGC
D CCG

What factors initiate the intrinsic pathway of the blood clotting cascade?
A) IX
B) IV
C) II
D) XII

Which one is harder? Since I wrote those questions, for me they are both easy: either you know or you don't. Neither of them is explicitly stated in the content outline. But I bet you if I use the 2nd question, people will scream bloody murder.
So the only solution is to see the % of people answering it correctly. But that is also problematic due to 2 obvious reasons.

The first is a matter of principle. Consider these 2 questions:

Question 1: A patient with a history of cardiovascular disease is given heparin. After the injection of the drug, this patient, compared to the general population, most likely has an increased chance of:
I. Internal bleeding
II. Blockage of blood vessels
III. Blood in urine

A. I
B. II
C. I and III
D. I, II and III

Question 2: A patient with a history of cardiovascular disease is given a blood thinning drug. After the injection of the drug, this patient, compared to the general population, most likely has an increased chance of:
I. Internal bleeding
II. Blockage of blood vessels
III. Blood in urine

A. I
B. II
C. I and III
D. I, II and III

Intrinsically, question 1 is harder because you have to know what heparin is (AAMC outline- blood clotting cascade- please don't tell me it's reasonable learn the cascade without learning the regulatory mechanisms). But then since most people will not know it. They will guess and when they guess they have 25% chance of getting it correctly. If the question is evaluated by means of polling, then 1 is easier than 2.

The second problem is applicability. The AAMC cannot possibly poll every single question before every test is given. So what do they do? Either they do nothing: aka arbitrarily assign a difficulty or they modify the ones that they do have data for. The second is dubious because the wording of the questions and the answer choices affect the difficulty. When I took the test, I had like 20 mins left after reviewing everything in C/P and stuck with one question. I got it correctly because I finally figured out what the hell they were asking. It was very easy. But whoever wrote that made up a name of type of experiment (I looked that up after the test, there was no such thing - and honestly it sounded stupid) and asked me to "compare." Got 132.

There is so much you can dress up a question until it gets ridiculous.
--------------------------------------------------------------------------------------------------------------
So how do we solve this problem? One answer is to add more question: with more data, randomness vanishes! But I can't imagine it will fly if I propose to make each section 200 questions or so.
But the most obvious answer to the question how "difficult" and "fair" the test was is to ask the test each test taker. We can't show them the score because if they do well they will tend to say that it's fair and vice versa.

How does this help? At the moment, no matter how we slice it, data for each applicant regarding the MCAT is n=1. Making them taking the test again and again doesn't do anything because n=3 is equally useless statistically. Using my proposed system, we give an applicants a chance to evaluate their academic performance. If they think their true bracket is 515-519 and the final score agrees, it will speak volumes for their cases. And if there is a disagreement, the system provide ample opportunities to protect oneself from overestimation. Use your judgment wisely. Although, as I stated above, it is not without risk to just moving one bracket by one like in the case of Jim: he passed the 500-510 and attempted the next and still got 510 as the final score. This can be read flag akin to current situation of retaking the MCAT and doing worse.

And point is this will provide very valuable data! Let me give another example:

Consider two excellent candidate both scoring 521
James: 475-500 ---> 500-510 -----> 510-515 ----->515 - 519-----> 519 - 523. Final score: 521
Kayla: 500-510 ----> 515-519 -----> 519-523. Final Score: 521

They are not identical. Draw any conclusion you want. I personally will lean toward Kayla.

Your inability to let go or acknowledge that your idea may be flawed is startling. To suggest that an adcom should be able to infer personal qualities from a candidate based on their MCAT is laughable. The test you've proposed doesn't improve our current system enough to justify any drastic increase in complexity. The uber-competitive nature of the premed population will select no other bracket than the highest one for their test in an effort to feel like they can be one step ahead of the curve.

The AAMC administers this test 60,000 times a year. Each question is studied rigorously to determine it's general level of difficulty. The AAMC nests "experimental" non-scored questions into every section to accumulate information of the question and its content's difficulty. Further, they gave test takers in 2014 sample sections of new content to start producing data for the 2015 version.

I'm recusing myself from any further responses.

 

[WedgeDawg]

This thread is now providing evidence that a 132 on the bio section is incompatible with intact cost:benefit analysis capability, and suggests that we need to drastically overhaul the MCAT testing system as has been proposed in the OP.

 

[aalamruad]

Couldn't agree with the bolded part more. OP and I had quite a discussion on another thread about randomness involved in achieving a certain score range, which I won't recapitulate here. He/she obviously believes that there is a lot of randomness involved and thus a lot of people scoring in that range aren't really in that range and just got lucky. Hence the whole system here. What he/she refuses to acknowledge is that if one is adequately prepared for the MCAT and takes it multiple times under similar conditions, there's no reason for a huge discrepancy in score at all. Somebody scoring a 523 the first time is unlikely to score a 498 the second time around unless there is some very different condition, like computer failure or something. Somebody who scores a 523 the first time will score very close to that same score the second time around and so on. So the whole system is quite useless.

(disclaimer: the following is not in any way a defense of the nonsensical original post)

I've seen you make this argument about consistency quite often, and I have to put in my two cents about it. In my experience prepping for the test, my initial practice tests were around 518 and steadily rose to 522-524 by the time I took the actual test, and then my actual score dropped to 516 (8 points lower than my most recent practice test, which was an official AAMC test). I'm virtually positive that test anxiety didn't play any significant role, and all my practice tests were conducted under extremely realistic testing conditions. With practice tests that were consistently in the high 99th percentile in the last couple months of my prep, and then such a significant drop on the actual test day to the 95th percentile (based on what I largely believe to be unlucky guesses on obscure trivia that I likely wouldn't have even covered if given an extra few hundred hours of study time), I don't think the test is quite as consistent as you believe it to be. I'm saying this because you've frequently claimed that multiple practice test scores specifically within the 520+ range should be repeatable with a high level of certainty, and I don't think I have to explain how significant a 4.8 percentile drop is at the far right end of the bell curve of test scores. (I've also seen plenty of people score 8 points higher than their practice tests, but that might be beside your point since you're mostly claiming that high scores should be consistent, not necessarily that low scores should be as consistent too.) Granted there don't tend to be many trivia questions, but there are just enough so that unlucky test-takers who usually score 520+ can drop 1-2 points on each section, and up to ~4 points on the psych/soc section which is more trivia-based.

I think you're coming at this issue with an uncommon and slightly unrealistic perspective, because you're not an average MCAT-taker by any stretch of the imagination. This has led you to make two claims, one of which is fair while the other is inaccurate. Your claim that it's possible to prep enough to guarantee a 520+ score is true for the most part (at least for a lot of people). I've seen your posts in the MCAT Q&A forum, and your level of understanding of all the material is unreal. You clearly understand virtually every concept so thoroughly that your 526 is no surprise and is certainly almost guaranteed to be repeated no matter how many times you take the test. What you don't account for is that it's very possible to score 520+ on a consistent basis without the level of understanding that you have. Since there are so few good practice tests available, a sample size of ~5 practice tests with scores of 520+ simply isn't large enough to guarantee a 520+ score on the real thing. I'm sure my situation was rare, but it certainly isn't unprecedented and probably not all that uncommon.

tl;dr if the legendary @aldol16 scored sub-520 on the real thing, it would be an utter fluke, but I don't think it's too unlikely for consistent 520+ scorers on practice tests to drop below 520 on the real thing just based on variations of the test and that person's strengths and weaknesses. You can have plenty of holes in your knowledge while consistently scoring very high on practice tests.

 

[baratheonfire]

None of this makes any sense

 

[Med Ed]

What do you guys think? I think future generation would love it. Very fair!

Congratulations, you have come up with a proposal that would both incite violence and induce vomiting. That said, the AAMC should be due for another revision by about 2035, so you will have plenty of time to lobby.

 

[wizzed101]

Your inability to let go or acknowledge that your idea may be flawed is startling. To suggest that an adcom should be able to infer personal qualities from a candidate based on their MCAT is laughable. The test you've proposed doesn't improve our current system enough to justify any drastic increase in complexity. The uber-competitive nature of the premed population will select no other bracket than the highest one for their test in an effort to feel like they can be one step ahead of the curve.

The AAMC administers this test 60,000 times a year. Each question is studied rigorously to determine it's general level of difficulty. The AAMC nests "experimental" non-scored questions into every section to accumulate information of the question and its content's difficulty. Further, they gave test takers in 2014 sample sections of new content to start producing data for the 2015 version.

I'm recusing myself from any further responses.

And that's why they will fail hard. There is a reason why I make a 519-523 and 523-528 bracket instead of just 520-528. Anyone that pick the 523-528 under any circumstance (even when they have passed the 519-523) is of questionable decision making capacity.

Each question is studies rigorously? Let say that your claim they administered this test 60,000 times a year. How many questions is that? "Study" rigorously how long? Multiply that up and see if it makes any sense. I mean, look at my examples. Those 2 simple questions! How do grade them?

And if you have actually read what I wrote, you should see that I have acknowledged my ignorance somewhere

This thread is now providing evidence that a 132 on the bio section is incompatible with intact cost:benefit analysis capability, and suggests that we need to drastically overhaul the MCAT testing system as has been proposed in the OP.

But it shouldn't cost the AAMC much! I mean the most expensive part is the test bank Just some change on the UI, and a tutorial to explain how things work and we are good to go! And Adcoms get more data without costing them a penny. They can look at the report card anyway they want.

Congratulations, you have come up with a proposal that would both incite violence and induce vomiting. That said, the AAMC should be due for another revision by about 2035, so you will have plenty of time to lobby.

I know that. And none of this will affect us and thus we will have less bias. It is our duty to decide for the future generation in this manner. They wouldn't know what is good for them. Hell, if you asked me before I took the test, I would think that it was a pretty stupid idea without giving it much thought. But now I have become a better person regarding the bias, I have a duty to think for the unfortunate who not yet.