Kaplan Full Length

[GrandMasterB]

---

Members do not see this ad.

Hey Guys, I was wondering what you all thought about the kaplan full length exam? The guy at the center said that above an 80% equates to like 95th%ile. Does that sound right? He said that it should refelect the same %iles as the actual test does- where 55-65% is passing- 182, and 80% should be around 235-244. This seems like a more optomistic estimate than I had heard elsewhere on this board. Also, someone from my school said that ppl who scored above an 80% all got above a 250. What have you guys heard?

 

[hj2005]

another question about the kaplan simulation for those who took it:

what did you get out of it? did you learn the material or was it more for the experience? should i use one of my last 9 days to do it?

 

[mikegoal]

im taking it to get used to sitting in front of a computer for 8 hours to take a test

 

[Jaded Soul]

Originally posted by GrandMasterB
Hey Guys, I was wondering what you all thought about the kaplan full length exam? The guy at the center said that above an 80% equates to like 95th%ile. Does that sound right? He said that it should refelect the same %iles as the actual test does- where 55-65% is passing- 182, and 80% should be around 235-244. This seems like a more optomistic estimate than I had heard elsewhere on this board. Also, someone from my school said that ppl who scored above an 80% all got above a 250. What have you guys heard?

That 80% = 95th percentile sounds really exaggerated. I can't figure out the math right now, but I'm sure there are stats on the percentage of people that passed and the average score for all the test takers that would allow you to figure out the percentile breakdown.

Everyone seems to be totally confused to the way your percentage score correlates to the 2-digit and 3-digit scores, so I don't have any reason to believe Kaplan would know any better.

 

[Jaded Soul]

Originally posted by hj2005
another question about the kaplan simulation for those who took it:

what did you get out of it? did you learn the material or was it more for the experience? should i use one of my last 9 days to do it?

It was good for the experience, not to learn the material. I didn't even bother looking up the explanations to the questions. The Kaplan where I took my test was really busy and I wasn't used to that kind of environment at all. People coming and going out of the room almost constantly. If the real thing is anything like that, I won't be taken by surprise, which is a good thing.

 

[doc05]

Originally posted by Jaded Soul
That 80% = 95th percentile sounds really exaggerated. I can't figure out the math right now, but I'm sure there are stats on the percentage of people that passed and the average score for all the test takers that would allow you to figure out the percentile breakdown.

Everyone seems to be totally confused to the way your percentage score correlates to the 2-digit and 3-digit scores, so I don't have any reason to believe Kaplan would know any better.

I have to disagree. The fact is that we (as medical students) don't know how the thing is scored. But if ~60% is a passing score, why isn't an 80%=95th percentile? Maybe the Kaplan guy was wrong, but the thing is that Kaplan knows the USMLE inside-out, almost as well as the people at the NBME.

 

[GrandMasterB]

Yeah, I am not sure how close the kaplan ppl are, but I don't think it is all that likely that you could predict the breakdown given the mean, sd and number of ppl getting each score- because the curve isnt bell shaped at all. the kaplan guy said basically that above a 240 or so- how much each question is worth to your final score depends on how many ppl with that score you are competing against. so basically the number of questions (or percent correct) does not affect your percentile/final score in a predictable manner beyond anecdotal evidence of what ppl got and info on their raw scores on practice exams- that is my reasoning to give the kaplan guy a little credit. 😱

 

[Jaded Soul]

Originally posted by doc05
I have to disagree. The fact is that we (as medical students) don't know how the thing is scored. But if ~60% is a passing score, why isn't an 80%=95th percentile? Maybe the Kaplan guy was wrong, but the thing is that Kaplan knows the USMLE inside-out, almost as well as the people at the NBME.

Okay, I did the math and Kaplan does overestimate a bit (provided I didn't bastardize some of the calculations). These are all estimations based on the 1999 data in the front of my edition of First Aid. I think that you can assume that it's a normal distribution.

Passing is a 3-digit score of 180. If we assume that 60% is passing (most cite 55-65% as passing), this makes the highest attainable 3-digit score 300. [180/0.6=300]

80% of 300 = 240

In 1999, the mean for Step 1 was 215 with a s.d. of 20. Assume a Gaussian distribution and that the mean score is the 50th percentile.

One s.d. above the mean is 235. This corresponds to the 84th percentile. [Scores of 185-235 encompass 68% of all the scores in a normal distribution (1 s.d below and above the mean). 100%-68%=32% "remaining" scores. That means 16% of the scores are higher 1 s.d. above the mean and 16% of the scores are lower than 1 s.d. below the mean. For a score of 235 (1 s.d. above the mean), 100%-16%=84%.]

Two s.d. above the mean is 255. Since 2 s.d. around the mean encompasses 95% of all the scores, this corresponds to the 97.5th percentile in a normal distribution. [Similar calculation as above.]

The difference between a score of 235 and 255 is 13.5 percentiles. [97.5-84=13.5] Within that range, that's a change of 0.675 percentiles per point. [13.5/20=0.675]

A score of 240 (which we determined above to be an 80%) is 5 points above 235. This is roughly the same as 3.375 percentiles above the 84th = 87.375th percentile. [5x0.675=3.375]

Thus, a score of 240 (80% correct) is approximately an 87th percentile score, not the 95th percentile that Kaplan claims.

If you assume a negative skew to the distribution (median > mean), it would tend to drive the percentiles even higher, requiring you to score even better to reach a certain percentile than in a normal distribution.

Okay, that's way too much statistics for my liking. Someone who knows what they are talking above can go ahead and rip apart the numbers now. 😎

 

[GrandMasterB]

I guess the assumption of a gaussian curve is where we differ. Here's my reasoning for not assuming a normal distribution: since the most tests (especially this one- because it is meant to assess a level of competence rather than splay ppl out along an even distribution) don't achieve the lofty goal of fairly assessing and stratifying ppl based on their results- a gaussian distirbution doesn't result without some "playing" with the numbers. Assuming that most ppl get in roughly the same % correct range ie 50-85-(probably with some lumping in there) you can't exactly give everyone the score "they deserve" because everyone would get the same score- ie a billion 200s and no 160s or 250s. I think that assuming a gaussian curve imposes the requirement of an equal distance between each %ile- ie the same number of questions wrong will affect your score the same way no matter what you have scored compared with the rest of the population. I think that at the bottom of the scale you have a lot more lee-way and can get a number of questions wrong without changing your score- where as at the top of the scale a few questions either way can really change your score. isn't this the way the mcat and the sat work too? i don't know, i gave a pretty hand-waving argument- but i think that is the best i can do for now- instead of studying of course.

 

[doc05]

I see what you are getting at, but I think you've confused 2 issues. The first is the Gaussian distribution of scores. This IS a normal distribution, based on the USMLE's scale (182=passing, 216=mean, etc.) The scale is an arbitrary construct mean to create a normal distribution. That is what the Kaplan guy meant (your score depends on how others do...)

Now you ARE correct in saying that the value of a question varies based on how many you've gotten correct. First Aid says that around the "passing" mark, each question ~0.67 point on the 3-digit scale. At the higher end of the scale, I'd suspect each question is worth more.

so.... 0.80*300=240. (+-) 10 points, since the scoring is pretty much a mystery.

and anecdotal evidence is good. standardized tests are meant to be reasonably reliable from one administration to the next. So if hypothetically, your friend managed an 85 and then got a 250 on the real thing, you can use that as a rough guide in approximating what you might score.