i wonder if the experimental q's count toward your score? i wonder if there is any way under the sun to find out exactly how nbme scores these 🙁
I don't think so.
They haven't been 'field tested' and therefore lack validity. In essence, the NBME does not know what is measuring on an experimental question until they have developed a performance profile.
Then, later if it doesn't live up to that profile, it may get thrown out.
For instance, imagine a new experimental question on treatment of cardiac failure.
The NBME will place this experimental question out in the pool until they collect enough responses (for argument's sake, let's say at least a thousand).
They will then know, that of those thousand responses
Option A) was choosen 15% of the time
Option B) was choosen 8% of the time
Option C) was choosen 45% of the time (and is correct)
Option D) was choosen 20% of the time
Option E) was choosen 12% of the time
Then, they can assign a degree of difficulty to the question to weigh it appropriately in an exam.
However, if in live testing, the profile is not met within certain limits, it means that the question cannot be used as a "true" measurement of your abilities. (This is probably one of the reasons it takes at least three weeks to get your score back and longer if you take the exam during peak season. The NBME is still figuring out how appropriate the new questions are.)
Perhaps, because protocols have changed or because the initial experimental sample size was too small, etc... So, it is then thrown out and not counted either way (and perhaps sent back to the question writers to try again).
