It's lupus!

2+ Year Member
Mar 26, 2017
446
405
Texas
Status (Visible)
  1. Medical Student
So I did some number crunching in preparation for retaking my MCAT. I found a spread sheet of a total of 147 people who reported their full length practice exam scores and their actual MCAT scores. For each I chose to keep only their latest practice FL for comparison to their actual score.

I wanted to know which full lengths would be best to maximize my time practicing and reviewing. I make no conclusions on this information wholeheartedly, because well this "study" has some serious and obvious flaws that significantly reduce the ability to do so. See pic of the summary below.

Difference: Refers to the Real MCAT Score minus the particular practice exam being evaluated.

Number of test takers: Refers to the number of observations (people reporting their score for that particular exam)

Diagnostics were not included in this breakdown.

Warning: I would always recommend that you take everything you see from some random guy on SDN with a major grain of salt and definitely use some critical thinking skills when interpreting this stuff.

mcat numbers.JPG
 
  • Like
Reactions: 2 users
Nov 1, 2017
190
222
Status (Visible)
  1. Medical Student (Accepted)
So I did some number crunching in preparation for retaking my MCAT. I found a spread sheet of a total of 147 people who reported their full length practice exam scores and their actual MCAT scores. For each I chose to keep only their latest practice FL for comparison to their actual score.

I wanted to know which full lengths would be best to maximize my time practicing and reviewing. I make no conclusions on this information wholeheartedly, because well this "study" has some serious and obvious flaws that significantly reduce the ability to do so. See pic of the summary below.

Difference: Refers to the Real MCAT Score minus the particular practice exam being evaluated.

Number of test takers: Refers to the number of observations (people reporting their score for that particular exam)

Diagnostics were not included in this breakdown.

Warning: I would always recommend that you take everything you see from some random guy on SDN with a major grain of salt and definitely use some critical thinking skills when interpreting this stuff.

View attachment 225629
I do appreciate the time that you've taken to compile this data and while I do think that it has some validity in saying about how much Kaplan really deflates their scores, I'm skeptical about NS and AAMC being of the same quality, though I could be proven wrong on that. My biggest concern is the bias of who is reporting their scores in the first place. The average and median scores for the pool on the actual MCAT is 515 here, which if I recall correctly, is above the 90th percentile, which isn't exactly going to be an accurate representation of the population. Interesting results though nonetheless.
 
  • Like
Reactions: 1 user
D

deleted600623

Although there might be flaws, this does confirm the general notion from most people that Kaplan's tests are not representative at all and NS is better but still not representative of the real thing. All in all, the practice tests written by the actual test writers is going to be the best practice but if more practice is needed, NS is one of the best options out of the 3rd party companies.
 
About the Ads

It's lupus!

2+ Year Member
Mar 26, 2017
446
405
Texas
Status (Visible)
  1. Medical Student
I do appreciate the time that you've taken to compile this data and while I do think that it has some validity in saying about how much Kaplan really deflates their scores, I'm skeptical about NS and AAMC being of the same quality, though I could be proven wrong on that. My biggest concern is the bias of who is reporting their scores in the first place. The average and median scores for the pool on the actual MCAT is 515 here, which if I recall correctly, is above the 90th percentile, which isn't exactly going to be an accurate representation of the population. Interesting results though nonetheless.
Oh, yeah. Thats why I put the disclaimer out there. It is far from any conclusive information. One thing to consider is people who do well are more apt to submit scores than those who don't. Another consideration is that the observation sizes vary a bit. Putting all that aside though, I calculated the power of each and it is pretty much zero. This includes the AAMC practice test. This is largely due to such a small sample size for each. That being said, you were right. I went back through my data and found data errors. Next step difference increased dramatically but still remains below Kaplan's. After correcting for that this is what I ended up with. View attachment 225782 Power.JPG Power.JPG New MCAT.JPG
 

edgerock24

7+ Year Member
Jul 28, 2012
999
741
Status (Visible)
  1. Medical Student
Oh, yeah. Thats why I put the disclaimer out there. It is far from any conclusive information. One thing to consider is people who do well are more apt to submit scores than those who don't. Another consideration is that the observation sizes vary a bit. Putting all that aside though, I calculated the power of each and it is pretty much zero. This includes the AAMC practice test. This is largely due to such a small sample size for each. That being said, you were right. I went back through my data and found data errors. Next step difference increased dramatically but still remains below Kaplan's. After correcting for that this is what I ended up with. View attachment 225782 View attachment 225783 View attachment 225783 View attachment 225784
Good collection of data. Kaplan is terrible in my opinion. What are you scoring on your practice exams?
 

It's lupus!

2+ Year Member
Mar 26, 2017
446
405
Texas
Status (Visible)
  1. Medical Student
Good collection of data. Kaplan is terrible in my opinion. What are you scoring on your practice exams?
Personally I'd argue that the low scoring nature of Kaplan's exams are beneficial. The content matters more than the score so long as the score isn't being inflated. I've taken several practice exams and been studying in total since February. My Kaplan ones prior were between 490-494. Took my first MCAT in May, that was a 496. Since then I took a diagnostic a month ago through next step and scored a 499 and their full length a week ago and scored a 504. I'll take another practice exam in two weeks and one a week leading up to January 20th.
 
This thread is more than 3 years old.

Your message may be considered spam for the following reasons:

  1. Your new thread title is very short, and likely is unhelpful.
  2. Your reply is very short and likely does not add anything to the thread.
  3. Your reply is very long and likely does not add anything to the thread.
  4. It is very likely that it does not need any further discussion and thus bumping it serves no purpose.
  5. Your message is mostly quotes or spoilers.
  6. Your reply has occurred very quickly after a previous reply and likely does not add anything to the thread.
  7. This thread is locked.
About the Ads