practice NBME raw score conversion

[boardsbandit524]

I am a bit of a newbie here, can anyone tell me how to convert my NBME raw score (on the forms one through four) to a percentile and thus a STEP1 score? Please and thank you.

---

Members don't see this ad.

 

[zedx]

Uhm...the score report comes with a conversion sheet for NBME Score--> Step 1 3-digit score conversion. As for the 2-digit score, the debate is still unresolved as to what it represents though popular thought is that it is not a percentile score...

 

[Blesbok]

There is not conversion. You need to take the test online, not using the downloadable copies. Also, 50% of the answers in the downloadable tests are wrong.

 

[wacky96]

blesbok--dont be a jack*ss..everyone knows that these forms exist online..the reason people use them is bc the nbme doesnt allow you to effectively use your questions to learn from..i am sure there is someone who has a formula to assist in this process and it would be interesting to know.

 

[It'sElectric]

You gotta love future professionals already trying to scam the system. Pay the $45, check out your provided score, then feel free to find the "answer form"...which may or may not be accurate.

 

[wacky96]

Is the self righteousness always so thick around here..its this form of passivity thats the reason compensation drops a bit more each year

 

[It'sElectric]

Is the self righteousness always so thick around here..its this form of passivity thats the reason compensation drops a bit more each year

You're really comparing cuts in compensation to scamming the NBME? Classy.

 

[Pinkertinkle]

Seriously stop being a cheapskate.

 

[wacky96]

Does everyone find you this annoying or am I just getting a special vibe. Two points.. yes i think your mindset is part of the problem in medicine. Second if the NBME would just allow you see the questions with some form of explanation then people wouldnt be as frequently downloading them so they can look things up and try to learn the concepts..you really should get more than a number for your 45 bucks..and I am CERTAIN none of the critics have listened to PIRATED GOLJAN

 

[boardsbandit524]

anyone please? I'm a poor student and could really use the raw score conversion

 

[RussianJoo]

there is no conversion. no formulas work, i have tried a few of them and they work for one score but when you try to use them for a different score your numbers aren't right. I got a 230 on one of them and it ended up being 84% correct based on one of those answer keys. but i am sure those answer keys aren't 100% correct either.


and Wacky96, keep running your mouth like that and good luck getting any sort of help on this forum ever again.

 

[skm95]

Most of the answers to NBME questions out there are based on someone's guess work. There is no guarantee that those answers are right.

But if you think you have all the correct answers to the questions then here is formula to compute your score.

Cut&Paste following in a Spread-Sheet. Paste it in cell A2 and enter your score in cell A1 (raw score out of 200).

=IF(AND(50<=A1,A1<=200), INT(70.25-0.0000002068*(A1*4)^3+0.0000938*(A1*4)^2+0.3009*(A1*4)),"Enter a number between 50 and 200")

The formula is nothing more than a polynomial line-equation derived from the NBME provided chart.

Again, this is only useful if you think you have correct answers, Do you?

 

[westernmed007]

Yes, I believe I do. Do you need help with that? BTW how did you come across this?

 

[Cards21aceking]

Most of the answers to NBME questions out there are based on someone's guess work. There is no guarantee that those answers are right.

But if you think you have all the correct answers to the questions then here is formula to compute your score.

Cut&Paste following in a Spread-Sheet. Paste it in cell A2 and enter your score in cell A1 (raw score out of 200).

=IF(AND(50<=A1,A1<=200), INT(70.25-0.0000002068*(A1*4)^3+0.0000938*(A1*4)^2+0.3009*(A1*4)),"Enter a number between 50 and 200")

The formula is nothing more than a polynomial line-equation derived from the NBME provided chart.

Again, this is only useful if you think you have correct answers, Do you?

Not sure on the accuracy of this. It doesn't even correlate w/ the score posted above. Impressive effort though.

 

[skm95]

Yes, I believe I do. Do you need help with that? BTW how did you come across this?

As I said the equation is derived from NBME provided sample performance chart. The equation will never accurately represent your 3-digit score unless you have a perfect score 200/200 and you know answer to all the questions.
Why? Because NBME most likely assigns different points to each question. Some questions may be worth 2 points and other may be worth 6 points.
Since there are total 200 questions and maximum score is 800, therefore on average each question is worth 4 points.
Now you don't need any formula/equation; just take your raw score and multiply it by 4 and that will give you your score from 800. You can then lookup your approx. 3-digit score using NBME provided sample performance chart: https://apps.nbme.org/nsasweb/doc/sample_CBSSA.pdf

But if your score is 100/200, then multiplying with 4 will not give you accurate picture, because you don’t know the points for each question; you may have answered mostly 2-point questions correctly.

So what is this equation and where did I get it from? It is just for fun, really! You can derive it quickly yourself (if you know about some spread-sheet tricks). Just cut and paste all the data from NBME provided sample performance chart and paste it in any spreadsheet. Using chart wizard draw a scatter-point-chart. On the chart add a Trend-Line (pick polynomial option for trend). The Trend-line has an option to display equation. There, you have your equation now! To get more accurate equations (which hug the curve more accurately) google for "polynomial regressions" and you will find many online calculators to find the best possible mathematical function for curved lines.

The equation I posted had multiply by 4 added in there. If you want raw equation of NBME provided sample performance chart, then here it is:

=INT(70.7-0.0000002066*A1^3+0.0000937*A1^2+0.30096*A1)

Try it for any input from 200-800 and result will match 3-digit score provided in NBME provided sample performance chart.

 

[g6pddfishnc]

.

 

[skm95]

Is this how the actual exam is scored?

It is a little more complicated than that.
Let me try to explain how exams like USMLE are designed. What I am going to describe is based on general theory, and USMLE most likely does things differently.

Designing a good exam is a science by itself. A lot of statistical calculations go into finding the perfect question, answer choices, and score distribution. There are many statisticians who work along side field-professionals to design professional exams.

USMLE uses statistical normalization to compute and calibrate score for each question; yes! each and every question. That is also the reason, why you never know about maximum possible score. How does it work? I don't know how USMLE does it, but here is an example of how it can be done:

Suppose I am to start teaching a class in a college. To test my students I want to develop a question-bank from which I can give the final exam every semester. One of my goals is to keep adding new questions to this bank and keep removing the old ones. When adding the questions, I want to make sure that questions are not very-very easy and they are also not very-very difficult. My other goal is to keep the grading standard consistent. For this I define my own statistical distribution based on either some standard mathematical model or define a new one. My plan is to give 300 questions on final and for this I pick a distribution with mean M = 215 and standard-deviation SD = 20. I will call it SKM95 distribution.

Given all the parameters, now let's work on the problem in a simplest possible way.

I will start by giving a practice test to all my students every week. Let's assume that each question on the test is worth 10 points. After every practice test I will compute mean and standard-deviation for each question. So, let's say that for Question-1 (Q1) the M = 5.2 and SD = 1.5.

Based on M and SD of every practice test question, I can decide which questions are too difficult and which are too easy. I can then remove those from my final question-bank, e.g. if on particular practice test question all students get 10/10 or all get 0/10 then that question will not make to final, I will drop it from question-bank.

At the end of the semester I will have a full question-bank from which I can randomly pick 300 questions and give the final to my students. Let's assume Q1 is on the final. Now pick 3 students A,B, and C randomly. On the final A gets 10/10, B gets 0/10, and C gets 7/10 on Q1. Now I will compute z-value for these 3 students, for Q1.
Formula for z = (Score - M) / SD.
For A, z = (10 - 5.2) / 1.5 = 3.2
For B, z = ( 0 - 5.2) / 1.5 = -3.47
For C, z = ( 7 - 5.2) / 1.5 = 1.2

For a full 300 question final, the M and SD will be different per question and to compute final z-score I have to add z-values for all 300 questions and divide by 300.

For this example let's assume that final exam has only one question. So, now the next step is to compute final score by mapping this z-score to my SKM95 distribution, and that can be computed by using the same z-value formula, except in this case the unknown is Final Score, so the equation will be:
Final Score = M + (SD * z)
For A, Final = 215 + (20 * 3.2 ) = 279
For B, Final = 215 + (20 * -3.47) = 146
For C, Final = 215 + (20 * 1.2 ) = 239

You can see that even with 100% correct the Max score was 279 and not 300, and with 0% correct the minimum was 146 and not zero.

Now you know why no one knows the maximum score on USMLE, and since very question has different z-score therefore no one can answer the question, "how many questions I need to answer to get a 236/99?"

In this imaginary scenario I can also repeat some of the questions in next semester practice tests. For these questions the M and SD will have to be re-computed based on new sample size (previous semester class size plus next semester class size). It will skew the per question distribution curve slightly so before the next semester final I will have re-adjust my SKM95 mean and standard deviation (.e.g. I may have to move M from 215 to 214 and SD from 20 to 21, etc..), to keep the grading standard consistent.

Now you know why USMLE SD and M shifts over period of time.

In my make-believe world I can also add some 50 new practice test questions to final and increase the final to 350 questions and call these practice questions experimental questions. These 50 won't be counted towards final score and only I will know about these experimental questions (sound familiar!)

With this scoring system all the students can theoretically get a perfect 270+ on final exam. But probability of that happening is next to zero, why? Because in practice/experimental questions if all students get 10/10 or 0/10, then that question is thrown away. But on paper I can still claim that every one can get a perfect score ;-)

Now this was a very simple explanation of how a scoring system like SUMLE can be designed. In reality it is much more complex.

USMLE, most likely, repeats same experimental question for 1 year before converting it to a question that counts, therefore sample size is huge/10000+.
Each experimental question then goes through item-analysis. This is where some of the following functions are performed:

Calculate p-value. This is probability of getting this question right. On a five-response multiple choice question, optimum difficulty level is 0.50 for maximum discrimination between high and low achievers.

Calculate point-biserial correlation (PBC). This is how good students did on question compared to their over all test score. A highly discriminating question indicates that the students who had high tests scores got the question correct whereas students who had low test scores got it incorrect. Goal for USMLE like exam is to get PBC of 0.4 or more.

Calculate Reliability coefficient. Using Kuder-Richardson formula compute the degree to which a question measures a single cognitive construct. Goal is to get 0.9 or above for USMLE like exam.

Distracter Analysis. On a multiple choice question if A is correct answer then B,C,D, & E are distracters (wrong answers). Distracters should appeal to low scorers who have not mastered the material whereas high scorers should infrequently select the distracters.

Distribution skew. In most processional exams the distribution is always negatively skewed (to right), including USMLE exam.

So, based on item-analysis USMLE can design each and every test question with very well defined boundary conditions. Ever wondered why Kaplan or UW score estimators predict a wide ranging score compared to NBME tests? That's because NBME questions are taken from actual USMLE exams, and all of them have gone through rigorous item-analysis.

OK! that's enough statistics for the day! Back to studying for Step-1!!!!!

 

[g6pddfishnc]

.

 

[skm95]

One other mathematical fact:
The 2-digit score USMLE provides is not based on any statistical distribution. It is a simply a linear representation of 3-digit score on a different scale.
We know that minimum passing score is 185, and from score reporting forum threads we know that for last one year or so the minimum 99 is 236.

My assumption is that 185/236 also represent standard-deviation times some number on a 3-digit statistical distribution graph. For example if SD = 20 and mean = 215, then 185 = mean - 1.5 * SD, and 236 = mean + 1.05 * SD.

But regardless of what they represent, the 2-digit score can be calculated by following simple line-equation:

Y = (Y2 - Y1) / (X2 - X1) * (X - X1) - Y1

Where Y2 = 99, Y1 = 75, X2 = 236, X1 = 185,
So the equation can be simplified to:
Y = (99 - 75) / (236 - 185) * (X - 185) - 75
Y = (0.4715 * (X - 185)) - 75
Y = (0.4715 * X) - 87.5 - 75
Y = (0.4715 * X) - 12.5

So for 236/185 markers we can compute a complete 2-digit table using this equation.

 

[amic1283]

would someone mind telling me how to get to these answer keys for the nbme self-assessments? I've tried googling and havent had any luck. thanks in advance!

 

[RussianJoo]

they should come with your illegally downloaded copy of the nbme tests.

 

[amic1283]

im not illegally downloading the forms... im buying them from the site but wanted to know if just the answers were out there somewhere? if not, then where are these downloadable tests? if someone could message me i would really appreciate it, thanks.

 

[osli]

They are out there, but I have no idea where. Ares is usually good at finding stuff you aren't supposed to have.

 

[doc20]

you will hear lot of things
a good passing score around 200 coressponds to 60-70%

 

[ifu]

As I said the equation is derived from NBME provided sample performance chart. The equation will never accurately represent your 3-digit score unless you have a perfect score 200/200 and you know answer to all the questions.

Why? Because NBME most likely assigns different points to each question. Some questions may be worth 2 points and other may be worth 6 points.
Since there are total 200 questions and maximum score is 800, therefore on average each question is worth 4 points.

I don't think NBME assigns different points to each question on NBME forms it provides people with. That would be the case were the test adaptive, but with NBME forms everyone gets the same questions. I tend to believe the three digit score is the actual number of questions you get correct out of 300 (after they drop around 50 questions they decide to call experimental). Since every question is worth the same point, you should try to get as many as possible right without getting stuck with difficult questions. Here it is an interesting take on the issue:

http://www.meddean.luc.edu/LUMEN/MedEd/USMLE/USMLE-I-2004HandoutWebversion.ppt

 

[danaos]

Hahaha - I know what ya mean

 

[lord_jeebus]

Is this really the case? I have NBME Form 1 and I would be interested to know from people who took the form online if their first question on their 4 sections was the same one I got:

Posting NBME questions is a violation of copyright and not permitted on this forum.

The NBME tests (and USMLE) are not adaptive tests.