Is it weird that scores in the mid 230's are getting 99's?

[Taus]

---

Members do not see this ad.

I know it doesn't really matter...but just seems a little low for a 99 compared to previous years

 

[gunnison]

yeah, you think this could mean the distribution of scores is narrower this year?

 

[Scaredshizzles]

If I had to quess the range is narrower, although they'll list the statistics the same for mean/SD because that data is from over the course of the year or something. The reason is probably because the board wants to de-emphasize the use of step 1 in residency selection and so if most peoples scores look similar numerically, it can't really be used as much to differentiate students. However, it appears they only did this during the mass batch of tests that they get during the late May-June period, and so people who took it before were not as effected by it.

I think it is a good idea, because clerkship performance is probably a 10x better predictor of the sort of physician someone can be than step 1.

 

[Taus]

If I had to quess the range is narrower, although they'll list the statistics the same for mean/SD because that data is from over the course of the year or something. The reason is probably because the board wants to de-emphasize the use of step 1 in residency selection and so if most peoples scores look similar numerically, it can't really be used as much to differentiate students. However, it appears they only did this during the mass batch of tests that they get during the late May-June period, and so people who took it before were not as effected by it.

I think it is a good idea, because clerkship performance is probably a 10x better predictor of the sort of physician someone can be than step 1.

I see what you're saying...but having the 3 digit score right next to it kind of negates the idea. If you had a huge batch of 99's you could still easily differentiate b/w a 235/99 and 260/99 (though once your score is in that range....who cares...you should be set unless you'll die w/o derm..)

 

[Law2Doc]

The reason is probably because the board wants to de-emphasize the use of step 1 in residency selection and so if most peoples scores look similar numerically, it can't really be used as much to differentiate students.

Somehow I suspect residency directors will just adjust their gradient, and not be thrown by a tighter grouping - ie whereas they were perhaps picking 230 applicants over 220 before, they could just as easilly be picking 230 over 228. Board scores will be an important tool in residency selection as long as they exist, because there isn't really anything else standardized across schools to look at.

 

[exmike]

Somehow I suspect residency directors will just adjust their gradient, and not be thrown by a tighter grouping - ie whereas they were perhaps picking 230 applicants over 220 before, they could just as easilly be picking 230 over 228. Board scores will be an important tool in residency selection as long as they exist, because there isn't really anything else standardized across schools to look at.

the boards are scored so that a 230 in any given year is equivalent. this is stated explicitly on the USMLE website. I have never heard of anyone using the two digit number for residency selection. I personally would ignore it.

 

[Scaredshizzles]

245 was a 99 last year, 244 was a 98, for whatever it is worth.
235 is a 99 this year.

"the boards are scored so that a 230 in any given year is equivalent. this is stated explicitly on the USMLE website. I have never heard of anyone using the two digit number for residency selection. I personally would ignore it.:"

Yea, but obviously that's not something possible despite the claim. There is no way you can adjust scores to make it that way unless the exams are the same exact questions and no one has heard of any of the questions going into it. It would actually be closer to the same if they actually did curve exams because it is more likely that the test population from year to year, with the thousands of students, will be the same on the whole as opposed to the exam being the same.

 

[Scaredshizzles]

I'm not sure why y'all think that mid-230s hasn't always been a 99. I just looked up last year's score thread and mid-230s was a 99 last year. I specifically remember a 238 being a 99 in 2005 (I remember one person's score specifically) so it was probably the same then.

Evidence-based medicine, people. Evidence-based medicine!

http://forums.studentdoctor.net/showthread.php?t=265475&page=8

Last year's score thread is still up, you can see a 238 was a 96...230 was a 96 this year. I'm almost certain I remember 244 was a 98 last year. And I'm pretty sure it was 245 in 05 also. Before that, I wasn't in med school so I have no clue lol.

 

[Scaredshizzles]

In terms of getting really high 3 digit scores, those of us who took it early lucked out a little I think. And since we're going to get compared against the same group as who just took it, it is a tad bit of an advantage.

 

[lord_jeebus]

the boards are scored so that a 230 in any given year is equivalent. this is stated explicitly on the USMLE website. I have never heard of anyone using the two digit number for residency selection. I personally would ignore it.

Bingo

Yea, but obviously that's not something possible despite the claim. There is no way you can adjust scores to make it that way unless the exams are the same exact questions and no one has heard of any of the questions going into it. It would actually be closer to the same if they actually did curve exams because it is more likely that the test population from year to year, with the thousands of students, will be the same on the whole as opposed to the exam being the same.

Well, for the same reason that someone taking the test on different days is going to get a different score, you're right, but they have a lot of data on each question and are able to sufficiently mix old and new questions that within the span of a few years the same three-digit score should mean pretty much the same thing.

You're going with the much more unlikely theory that the NBME has chosen this summer to change its long-standing scoring methods to the disadvantage of anyone who took it after this May, yet can offer no real evidence to support this. Your argument is based entirely on two-digit scores, which have no published significance except 75 = pass (unlike the three-digit score which officially should allow valid comparisons between different years of test takers). Furthermore, you suggest that the NBME would cover this up by not changing their published statistics to reflect this.

I don't see why the NBME would go through the trouble of putting one group of students at a disadvantage (and keeping it a secret), when they could more easily keep everything the same.

If the NBME really wanted to do what you said, they'd make Step I a pass/fail test. The key difference between that and what you're saying is that people who are involved in the Step I design process have actually discussed doing this.

 

[Scaredshizzles]

It's not statistically possible to do it though lord jeebus, that is why exams are curved in the first place. I was a math major in undergrad, and I'm not sure whether or not what I'm saying makes sense to most, but in either case...

The NBME has and continues to be very vague about their scoring system. They don't even tell you what the exam is scored out of...300? 350? 290? No ones knows and they don't want anyone to know. To publish a mean and standard deviation in isolation is pretty much meaningless without knowing the range of possible values. If thousands of people take the exam, then someone must be in the 99.9%tile, no? What is the z score on that, well over 3.5...Does that mean 1 in every 1000 people score over a 290?? Not very likely...assuming there is a max score, there must be some sort of asymptote. The fact that they tell you they don't put these scores on a bell curve and then give you a standard deviation just tells you how silly the scoring system is. These are not mathematicians, that is for sure.


And it is not a real disadvantage because most people in the US take the exam during the May-June period anyway.

 

[lord_jeebus]

You still haven't given a reason why anyone should believe that there has been a secret narrowing of the score range.

Mean and SD don't tell you everything, but as a math major it should be obvious to you that for the comparison of scores for the majority of test takers, they should be sufficient for a program director to get a feel for how impressive or not a score is relative to others. I don't think they care if someone with a 280 is in a percentile that corresponds with what their score would be if the test were graded on a normal disribution or if the theoretical maximum was a 300 vs 350; they just care that it's really really high (or perhaps they just care that it's higher than their arbitrary interview cutoff).

 

[Scaredshizzles]

Oh, you're right in that case, I just tossed it out as a possibility that they might've purposefully narrowed the range. Based on a cursory look at the scores at my school, and the scores for the few of us who took it early, and knowing in general how competitive some of the students at my school are, I feel fairly strongly that the range was narrower this year. But the range might've been narrow just because a certain percentage of the questions on the exam in June were super hard. That it wasn't intentional, but that is just the way it worked out. Since the questions on the exam are very similar within a short time range (i.e. more than half the questions will be the same for people who took the exam within a few days of each other), that explanation might be more likely.

Furthermore the NBME claims that the 2 and 3 digit score is performance based. But it makes it very hard to say then for someone with a 238/99 this year and a 244/98 last yearwho did better, considering both scores are supposedly performance based...the two scores contradict each other. People seem to think only the 3 digit is performance based, but the NBME in fact does not make that distinction if you read their disclaimer. They only say that percentiles aren't performance based.....Which is really stupid because with such a large sample size, each year derived from students who go through the exam same medical courses at the exact same schools...the percentiles certainly are the best comparison between years....90th percentile from one year to the next should represent a student at the same level, etc....But the board doesn't want to deal with percentiles anymore. And I can make a guess on the reasons but you will say I don't have evidence and you would be correct.

 

[chandlerbing]

I got a 235 / 98 and I'm just wondering if it's possible that someone else with a 235 could have a 2 digit score of a 99??? Someone in another forum reported that they got a 235/99 and I'm wondering if it was just a typo or if it is possible for two people with the same three digit score to get a different two digit score???

 

[DOCTORSAIB]

I've been following Step 1 scores on prep4usmle.com over the past year, a website with mostly IMG's (they take the USMLE throughout the year, not just May-July like us). Anyways, a 236 seems to be the cut-off for a "99" this year. Correct me if I'm wrong guys.

Last year, a minimum score of 244 landed you a 99. I know that for a FACT..😉

 

[Scaredshizzles]

Who knows, maybe that person had a 235.4 and you had a 234.2....There is just no way of knowing, the board prefers to keep it vague.

 

[OHMAN0125]

If I had to quess the range is narrower, although they'll list the statistics the same for mean/SD because that data is from over the course of the year or something. The reason is probably because the board wants to de-emphasize the use of step 1 in residency selection and so if most peoples scores look similar numerically, it can't really be used as much to differentiate students. However, it appears they only did this during the mass batch of tests that they get during the late May-June period, and so people who took it before were not as effected by it.

I think it is a good idea, because clerkship performance is probably a 10x better predictor of the sort of physician someone can be than step 1.

Hmm... first of all I don't think the 2 digit score means much. It's always posted next to the 3 digit score anyways. So residency directors will most likely differentiate scores using the 3 digit scores as they have been doing.
The range is probably narrower at the top (>235) during May-June this year meaning less people performed as well as last yr during this time? Regardless, a 235 this yr is basically a 235 from 5 years ago. Everyone knows that. The 2 digit score is rubbish in my eyes. Directors will seek the 3 digit score. And if you got above a >235 it really doesnt matter since it will open all your doors (maybe not straight plastics). The rest depends on recs and phone calls to directors.

 

[Scaredshizzles]

You're right about that.

The only part I disagree with is people "performed worse"...we're talking very large sample size of very similar background people...Why in the world would performance decrease? On the other hand, the question bank of a few hundred questions used during a particular month can be somewhat harder...And there is no objective way to guage difficulty other than performance.. If people "performed worse" that just means that the test was harder, it doesn't say that the students weren't as good. There are more students total taking this test than there are questions in there question bank. Also, neuro was very heavily emphasized in a lot of the tests in June from what I heard, which might've made a difference this year.

 

[OHMAN0125]

You're right about that.

The only part I disagree with is people "performed worse"...we're talking very large sample size of very similar background people...Why in the world would performance decrease? On the other hand, the question bank of a few hundred questions used during a particular month can be somewhat harder...And there is no objective way to guage difficulty other than performance.. If people "performed worse" that just means that the test was harder, it doesn't say that the students weren't as good. There are more students total taking this test than there are questions in there question bank. Also, neuro was very heavily emphasized in a lot of the tests in June from what I heard, which might've made a difference this year.

I didnt mean to say that the students weren't as good this year. I agree with you that the exam was probably more tricky and neuro was huge which may have contributed to such a weird range. But again, the range actually works in favor for those with a 235 but not for those with say a 250 if directors look at the 2 digit score since they will see them as similar. However, we all know that's not what they really look at.

 

[Hallm_7]

This is confusing. We know that the two digits are on a different scale this year. However, what are the odds that a test with at least 20,000 test-takers every year would see its SD drop significantly enough to account for the change in the two digits? It seems that a sample size that large would see its mean and SD remain essentially constant year to year.

 

[Scaredshizzles]

Yep I agree. Although some places that aren't too concerned about step 1 scores will use a 2 digit cutoff for interviews and then toss it away as a criteria for selection.

What I really could use are some connections. It is like that kid on the Cosby show used to always say, "It's not what you know, it's who you know."

 

[logos]

My understanding was that the two digit scores only exist because some states require in their laws a score greater than 75% on the test to be licensed. Thus their reqs. would not be satisfied by a score on a 3 digit scale.

I was surprised in the letter that they dont tell you what percentile relative to other test takers your score represents.

 

[Scaredshizzles]

"It seems that a sample size that large would see its mean and SD remain essentially constant year to year."


That is assuming that the test scores are curved on a bell curve. But they aren't, not exactly anyway. Even when the USMLE reported percentiles, the scores were never on a bell curve....Less than +2 SDs was a 99th percentile for example, if you calculate 1 SD as the difference between 50th and 84th percentile. As I made the point above though, publishing SDs on a non normalized data set is pretty much useless.

 

[exmike]

My understanding was that the two digit scores only exist because some states require in their laws a score greater than 75% on the test to be licensed. Thus their reqs. would not be satisfied by a score on a 3 digit scale.

I was surprised in the letter that they dont tell you what percentile relative to other test takers your score represents.

Again if you read the USMLE website, it discusses why they dont use percentiles - its related to the reason that a 230 is identical from year to year. go educate yourselves!

http://www.usmle.org/FAQs/faqusmlescores.htm

 

[FDGB]

You still haven't given a reason why anyone should believe that there has been a secret narrowing of the score range.

Mean and SD don't tell you everything, but as a math major it should be obvious to you that for the comparison of scores for the majority of test takers, they should be sufficient for a program director to get a feel for how impressive or not a score is relative to others. I don't think they care if someone with a 280 is in a percentile that corresponds with what their score would be if the test were graded on a normal disribution or if the theoretical maximum was a 300 vs 350; they just care that it's really really high (or perhaps they just care that it's higher than their arbitrary interview cutoff).

Ok...I wasn't a Math major, but because the curve is skewed the mean and SD really don't help you too much in figuring out what percentile a score falls. You really need more info like median, range, etc.

EDIT: Ok I actually read the whole thread and this was mentioned...sorry! Also some people seem to think that the two digit score is a percentile. It is not.

 

[Miami_med]

My school's average went up 8 points this year (They are happy to admit this, though they won't give us our actual scores). This would be pretty unlikely if the whole scoring system was changed in a fashion that lowered scores.

 

[Hallm_7]

What's your school's average this year?

I have a friend that goes to school at UT Memphis and she said their average score went up by about that same amount from last year. That's a huge jump in average score from one year to another. Most schools typically fall in a 3-4 point range over the course of a few years and any gains are realized over time.

At the University of Toledo (my school) it seems like everyone has been getting more than 220, with many more than normal getting 230s, 240s, and 250s, than normal. Over the last three years we've had mean scores in the 216-219 range as a school.

Of course this only adds to the confusion since it seems that a jump in the mean score would increase the three digit score required to get a 99 for the two digit.

Does anyone know if there was a dramatic increase in the number of international medical students or osteopathic students taking Step I this year? Because those groups typically score less on Step I than US MD students, a large influx of test takers from these groups might alter the typical distrubution of scores. Purely speculation through.....I have no evidence that this has happened.

 

[Miami_med]

Over the last three years- 219, 217, and 225 now respectively.

 

[OHMAN0125]

What's your school's average this year?

I have a friend that goes to school at UT Memphis and she said their average score went up by about that same amount from last year. That's a huge jump in average score from one year to another. Most schools typically fall in a 3-4 point range over the course of a few years and any gains are realized over time.

At the University of Toledo (my school) it seems like everyone has been getting more than 220, with many more than normal getting 230s, 240s, and 250s, than normal. Over the last three years we've had mean scores in the 216-219 range as a school.

Of course this only adds to the confusion since it seems that a jump in the mean score would increase the three digit score required to get a 99 for the two digit.

Does anyone know if there was a dramatic increase in the number of international medical students or osteopathic students taking Step I this year? Because those groups typically score less on Step I than US MD students, a large influx of test takers from these groups might alter the typical distrubution of scores. Purely speculation through.....I have no evidence that this has happened.

Paranoid are we? The avg this yr for step 1 is a 218 according to the score transcript. That's about the same as previous years between a 215-220. Your school's class probably just did better. It happens. Rest assured, >235 still means you can do Derm. lol.

 

[Hallm_7]

You mean I shouldn't be worried about the average this year jumping to 230-something? 😀🙂😀