experimental questions

[Bandit]

---

Members don't see this ad.

anyone have a REAL idea of how many there are/where on step one? How do they work the scoring then?

 

[lord_jeebus]

Any answer to this question is pure speculation.

I imagine they have no impact on your score. How the remaining questions become a three-digit score is also pure speculation.

 

[mudphudwannabe]

Yes, it is veiled in mystery. I've heard there are about 50 scattered throughout the test, but that it varies from test to test. Somehow they magically compensate for it..

 

[anesthesiarocks]

Any answer to this question is pure speculation.

I imagine they have no impact on your score. How the remaining questions become a three-digit score is also pure speculation.

I think it is pretty much assumed that 1 real question = 1 pt on the three digit score and that there are about 290-300 or so "real" questions.

So someone who gets 70% right on the test scores about a 205. 60% is about a 175-180. 80% a 235 or more, etc.....

that matches up pretty well with what people actually score when they take the test. A lot of people who score 230 or so and "think" they only got 70% or so right actually probably got close to 80% right....

 

[lord_jeebus]

I think it is pretty much assumed that 1 real question = 1 pt on the three digit score and that there are about 290-300 or so "real" questions.

So someone who gets 70% right on the test scores about a 205. 60% is about a 175-180. 80% a 235 or more, etc.....

that matches up pretty well with what people actually score when they take the test. A lot of people who score 230 or so and "think" they only got 70% or so right actually probably got close to 80% right....

Again, pure speculation.

Steps II and III have more questions but are still scored on the same three-point scale with similar means.

 

[anesthesiarocks]

Again, pure speculation.

it sure seems to break down about that way though. So whether that's the way they really do it or not, it's almost certainly a good predictor of your score. Take 290 or so and multiple it by the percentage you actually got right and thats probably going to be about your score.

 

[lord_jeebus]

it sure seems to break down about that way though. So whether that's the way they really do it or not, it's almost certainly a good predictor of your score. Take 290 or so and multiple it by the percentage you actually got right and thats probably going to be about your score.

I agree, there's just no evidence to back that up.

 

[anesthesiarocks]

I agree, there's just no evidence to back that up.

sure there is....all those cases can be considered data points.

 

[lord_jeebus]

sure there is....all those cases can be considered data points.

What cases? Do you know anyone who knows their actual raw score?

 

[Bandit]

see now i think this is BS. If there are 50 "experimental" questions we should all get 50 free points. i know in reality this could not possible be the case, but wow--like this isnt tough enough!!!

 

[anesthesiarocks]

What cases? Do you know anyone who knows their actual raw score?

with a test consisting of 350 questions, people can "guess" their raw score to within 5-7% points most of the time.....based on how they felt taking the test relative to previous nbme forms and such. Now that represents a rather wide range(15 points or so in terms of raw score), but the n in this case is large enough for that to be overcome.

And yeah, I've heard of people saying that they think they got 65% or so right and ended up with a 240 or whatever, but then when you dig deeper and ask them what their qbank% leading up to the test was it was in the high 70s......and since the behavorial(just for starters) in the real test is a lot simpler than qbank behavorial, and perhaps less minute detail is required on the real test, people generally do about 5% higher on the real test from their random qbank towards the end. And high 70's + 5% = low 80s, which would correspond to 240 or so.

The real test also has more complete gimmes than qbank. even though the real test may require more non-linear thinking, the increased number of "what virus is associated with increased risk of cervical cancer" gimme type questions on the real thing is higher than in qbank(and certainly USMLEW)

 

[lord_jeebus]

But even such an estimate is invalid because no one knows how many questions are experimental (and thus not scored) and how difficult those questions are relative to the rest of the test.

The test has a lot of easy questions but I did note a number of seemingly easy questions that were actually quite nasty and could thus lead some to overestimate their scores.

A lot of your reasoning is based on further speculation involving NBME tests (which also do not give you a raw score) and the assumption that people beat QBank by about 5%, which is reasonable but again purely speculative. And again, who knows how many of the tough (or easy ones) were not scored.

Nevertheless I think your methodology for estimating what one will get post-test is a good one.

 

[blz]

there are times when i feel like i will totally fail a qbank session and then when i push submit at the end, i see an 86%. people definitely understestimate how well they guess and their performance. a lot of scoring well on this test is being able to make the best guess. of course, most type A med students can't fathom that just because they made an educated guess, it doens't mean they got it wrong. all this speculation anesthesiarocks is making is such bs. him and his ******ed theories are getting a little annoying. the worst part is he will defend them so adamantly (see post on friend who only studied biochem, lol). once you're done with the test, just live your life and forget about it.

 

[mudphudwannabe]

see now i think this is BS. If there are 50 "experimental" questions we should all get 50 free points. i know in reality this could not possible be the case, but wow--like this isnt tough enough!!!

From what I understand, the experimental questions (however many there are...) aren't counted in the score. But maybe you knew that.

I guess the point is, if you see some really random, off the wall question that you would never get right even if you had a million years to study..don't get too worked up. It might not count in the end. 🙄 The hoops we have to jump through...

 

[Bandit]

i guess what im asking is what if you get them right? does it help your score?

 

[lord_jeebus]

i guess what im asking is what if you get them right? does it help your score?

If getting them right helped your score, then missing them would mean not getting points. Which is would be the same as a regular question. So very likely no.

I had one question where there was an obvious typo that made 2 answers both correct.

 

[RockShox]

The experimental questions are NOT graded right or wrong they do nothing for your score. The are also not all "impossible" or "crazy" questions that everyone thinks they are like some professor who imagined a question during a tequila moment and said, "hey, lets try this one."

The purpose of the experimental questions is to do a statistical analysis of those students that year and control for the progression of medical knowledge on future tests. The tests have to evolve because medicine is ever changing. By putting those 50 questions in they can see what newer material students are learning and include them in future tests. There are statistical ways to see if students are guessing on questions and weed those ones out.

It is by this process that they can at least try to standardize the test. additionally you can see out of those questions how difficult they were and grade them to make a well rounded test.

Sometimes the experimentals are actually questions that have been used on the test for years but they want to see if they have become so standard for the rising classes that they no longer work on a test of that nature and then are retired.