Problem with NBME Exams, they overestimate your score!

[automan2]

---

Members don't see this ad.

A lot of people are posting similar concerns (my nbme exam scores are way higher than my qbank/UW), so I thought I would throw out my theory.

The NBME exams have been around for a few years, so any recent edition of review books if obviously going to make sure they cover every single topic outlined in those tests. Goljan even says it in his lecures " I have studied released review questions for years"

This means that we are studying from sources that specifically focus on the content of the NBME exams, and that is why people's scores are so high.

I can remember dozens of questions that were mentioned almost word for word in previous study sources when i took the first two NBME exams.

Is anyone else concerned that we are studying "for" the NBME exams, and when we get to the actual Step 1, the questions will be foreign and much more difficult?

 

[Pianoboe01]

I purchased Forms 2 (670/252) and 3 (770/263), and I thought they were about the same style and difficulty as the actual Step 1. Personally, I thought my scores on the NBMEs seemed way too high, but I still have several weeks til I get my Step 1 score (took it on June 1).

 

[ORLHNS41]

I've definitely thought the same thing. I'm just putting faith in the other people who have posted on this forum and say there's a strong correlation. I've done better on my NBME's than I expected to, which helps with confidence, but seriously hinders my motivation. I take the exam on Monday and at this point i'm just trying not to forget too much. I definitely plan on posting my Step 1 score along with my NBME scores. It seems the higher your NBME score the higher the possibility it may fall a bit on step 1. I've also seen some people post that their Step 1 score was higher than their NBME scores.... If my score falls too much, i'll just sue the NBME for false confidence.... or a 265... HA!

 

[joe6102]

No, the test hasn't changed much since last year when the NBME forms were a great indicator. In 2006 they used the same review books as we did, and the NBME forms tended to underpredict.

 

[automan2]

No, the test hasn't changed much since last year when the NBME forms were a great indicator. In 2006 they used the same review books as we did, and the NBME forms tended to underpredict.

In 2006 there wasn't first aid 2007 and there wasn't rapid review part 2. In fact, the entire high yield series and the entire rapid review serious has been completely reworked. Also, we have usmleworld this year, which wasn't nearly as big last year. So almost every review book is different. Now that doesn't mean they changed to include these secret answers, but judging from the changes to FA and rapid review series only, there have been significant changes.

Do I think they did a good job at predicting last year? Yes, but I am not too sure about my chances this year....Don't think you are entitled to a 240, just because you got that on a practice test which you were practically directly studying for.

 

[surfgirl2]

Perhaps since people are using usmleworld which is such a thinker qbank, people are just more capable of answering the questions on the NBME exams.

I don't know about you but it is frightening to think that the NBME exams are "easy" and overprediting. They have been good predictors in the past and I think perhaps we should wait to see the correlations for people who took the exam this year before saying they are too easy. Just a thought.

 

[joe6102]

In 2006 there wasn't first aid 2007 and there wasn't rapid review part 2. In fact, the entire high yield series and the entire rapid review serious has been completely reworked. Also, we have usmleworld this year, which wasn't nearly as big last year. So almost every review book is different. Now that doesn't mean they changed to include these secret answers, but judging from the changes to FA and rapid review series only, there have been significant changes.

Do I think they did a good job at predicting last year? Yes, but I am not too sure about my chances this year....Don't think you are entitled to a 240, just because you got that on a practice test which you were practically directly studying for.

Don't do this. There are a ton of people taking the test in the next couple of weeks, and you are destroying their confidence based on pure speculation. It is much more likely that the NBME forms will predict as well as last year. If there is a difference, it won't be huge.

Maybe the review books and USMLEWorld are better written and teach the important points better. After all, they are written by the same kinds of people who write medical school exam questions. You are learning the concepts through review books, and these are the same concepts that are on Step 1.

 

[oms22]

Hopefully this will help your confidence: Just received Step 1 scores this week and it was 30 points highe than my Form 3 which I took one week prior to Step 1. So, you can see that yes, the NBME's can inflate your score but the sure can underestimate it as well. Just continue to keep plugging along and trust your studying. I think the best way to use the NBME's is to see areas that you may need improvement in.....bottom line. It seems as though (by readind multiple forums) that a score correlation is not strong enough to trust.

 

[blz]

lol standard automan controversy

 

[automan2]

Hopefully this will help your confidence: Just received Step 1 scores this week and it was 30 points highe than my Form 3 which I took one week prior to Step 1. So, you can see that yes, the NBME's can inflate your score but the sure can underestimate it as well. Just continue to keep plugging along and trust your studying. I think the best way to use the NBME's is to see areas that you may need improvement in.....bottom line. It seems as though (by readind multiple forums) that a score correlation is not strong enough to trust.

Well said, bottom line is "Do Your Best" as a funny lady with an accent once told me.

Don't freak out if your score is lower on the nbme than you would like it, just work harder.

But, at the same time, don't think that because you are getting a 240 on the nbme, that you will get a 240 on the real deal.

 

[automan2]

lol standard automan controversy

I figured since psycho doc had disappeared, it was time to stir the pot and get people thinking!

Not trying to piss people off, just a theory!

 

[blz]

I figured since psycho doc had disappeared, it was time to stir the pot and get people thinking!

Not trying to piss people off, just a theory!

but you're kind of right about it - if you listen to the goljan lectures he spits off questions directly from the nbme exams. regardless though, if you got your head together the day of the exam, there's no reason you shouldn't score similar to nbmes

 

[SoCuteMD]

This could go SOOO many different directions!

a) NBMEs underpredict because you have more adrenaline and are really "giving it your all" on test day

b) NBMEs overpredict because of the review book material bias suggested above

c) NBMEs underpredict because of the new availability of UWorld

d) NBMEs...


you get the point.

Bottom line - if you studied your @$$ off and know RR, First Aid, and a few other sources - YOU WILL BE OK.

Anybody taking the test soon - don't let this discussion freak you out.

 

[automan2]

I guess my point is why is everyone so "freaked out" by this thought.

It seems that everyone is so quick to post their high scores and their progress. They are proud of their scores, and they want people to know about it. You don't see the people with 195's posting on this board, but there are the approx same number of 195's as 245s. Maybe SDN is so full of great advice that people who visit this board do better. I would guess is just the gunner nature of people on this baord. If that is the case, Why would anyone be so easily shaken by this simple theory?

Why are med students (who I consider failry successful people) such headcases?

If a simple thread on an internet message board is making your doubt yourself and making you "freak out" then you need some serious help. It's just a discussion. (granted my awesome title probably adds to the nature of this problem! Ha)

Good day and get back to the books

 

[joe6102]

I guess my point is why is everyone so "freaked out" by this thought.

It seems that everyone is so quick to post their high scores and their progress. They are proud of their scores, and they want people to know about it. You don't see the people with 195's posting on this board, but there are the approx same number of 195's as 245s. Maybe SDN is so full of great advice that people who visit this board do better. I would guess is just the gunner nature of people on this baord. If that is the case, Why would anyone be so easily shaken by this simple theory?

Why are med students (who I consider failry successful people) such headcases?

If a simple thread on an internet message board is making your doubt yourself and making you "freak out" then you need some serious help. It's just a discussion. (granted my awesome title probably adds to the nature of this problem! Ha)

Good day and get back to the books

Who is freaked out? I was just pointing out that this is a stressful time for some, and making a statement like the title of this thread will only add to the stress. And since you have nothing to back it up besides pure conjecture, it is irresponsible. At best you are stirring the pot for ****s and giggles, at worst you are trolling and will be banned again.

 

[Pinkertinkle]

Problem with automan2, he always thinks the sky is falling!

 

[Pinkertinkle]

 

[winsicle]

i'm sure the average scores tend to increase a bit each year - as word gets out on heavily tested subject trends (e.g., molec bio), so there must be someway that they account for this, otherwise scores would be inflated and you couldn't truly compare students from year to year...

BUT UW and RR seem like they are vastly superior to previous study materials


SO I think the REAL issue this year is whether they adjust the scoring curve:
-will they be judging us primarily amongst ourselves (this class of Step 1ers) OR
-equally comparing us to past years takers...


compared to previous years, I think we are all gonna look like geniouses

 

[Law2Doc]

Bottom line - if you studied your @$$ off and know RR, First Aid, and a few other sources - YOU WILL BE OK.

Anybody taking the test soon - don't let this discussion freak you out.

Agreed. It's not like you are really going to do anything different based on whether these practice resources underestimate or overestimate, so who cares. To a large extent you should really worry more about it as a teaching tool experience -- to guage whether you know a lot or a little about various components (and direct your focus more heavilly on things you need work on), and worry less about the numerical scores. Just my 2 cents.

 

[wiryMD]

i'm sure the average scores tend to increase a bit each year - as word gets out on heavily tested subject trends (e.g., molec bio), so there must be someway that they account for this, otherwise scores would be inflated and you couldn't truly compare students from year to year...


SO I think the REAL issue this year is whether they adjust the scoring curve:
-will they be judging us primarily amongst ourselves (this class of Step 1ers) OR
-equally comparing us to past years takers...


compared to previous years, I think we are all gonna look like geniouses

There is a way they account for these things, i believe it's called statistics. And "word gets out on heavily tested material" is that a joke, the NBME, who write the USMLE have tens of thousands of questions, your exam is simply a random collection of these, so telling friends what was on your exam doesn't really help them out. And if you read the information posted by the NBME they don't give you a percentile because they don't norm the exam to a specific mean, so a 230 could be the 75%, the 90% the 60% depending on the year, but regardless of the percentile anyone who scored a 230 can be considered equivalent. The reason they do this is so you can compare students from year to year. Percentiles are dependent on the knowledge base of everyone else taking the exam, so by eliminating them the step 1 gives a more objective measure. Compared to previous years people have done much better, in fact the original step 1 was designed to have a average score around 200, but now the average is around 215.

 

[winsicle]

There is a way they account for these things, i believe it's called statistics. And "word gets out on heavily tested material" is that a joke, the NBME, who write the USMLE have tens of thousands of questions, your exam is simply a random collection of these, so telling friends what was on your exam doesn't really help them out.

Statistics, eh? never heard of it.

True, telling your friends what was on any given exam won't help, but you can't deny that there has been trends that develop - the # of Molec bio questions has increased in recent years. How many people in 2002 were reading HYCM?

And if you read the information posted by the NBME they don't give you a percentile because they don't norm the exam to a specific mean, so a 230 could be the 75%, the 90% the 60% depending on the year, but regardless of the percentile anyone who scored a 230 can be considered equivalent. The reason they do this is so you can compare students from year to year. Percentiles are dependent on the knowledge base of everyone else taking the exam, so by eliminating them the step 1 gives a more objective measure. Compared to previous years people have done much better, in fact the original step 1 was designed to have a average score around 200, but now the average is around 215.

Fair enough, but as I said THIS years testers seem to have a big advantage from prior years, SO do they account for this (e.g., comparing us internally to form a curve)?

If as you implied, we are compared equally to prior years (on the same curve, etc.) we are gonna smoke them.

 

[wild0ne]

..compared to previous years, I think we are all gonna look like geniouses

well, maybe not all of us.. lol. j/k.

I'm not sure if I understood wiryMD's post correctly. But, whatever statistics they use to scale the USMLE, I hope they're used to scale the NBME's.

I agree though... I highly doubt new test prep material can sway distributions of a standardized test in a year or two. Most of my class has never heard of UW. About half started using Goljan towards the end of the year. A vast majority are using Kaplan as their sole prep.

 

[wiryMD]

If as you implied, we are compared equally to prior years (on the same curve, etc.) we are gonna smoke them.[/QUOTE]

Yes you will do better than people in previous years, since the Step I was first implemented scores have gotten progressively better. But you are not graded on the same "curve," like I said before, a 230 today equals a 230 tomorrow. That is to say, the guy who scored a 230 in 1998 would score a 230 taking it today, but the percentile relative to all other test takers would be different in either year. Will you smoke them? well if by smoke you mean that 5% of the 1998 class scored >225 and 25% of your class scores >225, then yes, but what does that mean since it's still a 225, and nobody really knows what percentile that is anyway. No matter how much study material, review books, etc... you have available it doesn't change the fact that if you study a lot you will do well. And if you want to know exactly what is on the step I, go to their site and click on the step I information, it gives a percentage breakdown of what topics are covered for an average exam, so yes people were studying biochem/molecular back in 2002. It is likely that this years national average will be roughly the same as the past few years (214-216). As to how NBME actually comes up with the scores, it's anybodies guess, they don't release that information. Good luck on step I

 

[Miami_med]

Average MCAT scores are rising amongst matriculants, people who are good at tests tend to do well on tests, average USMLE scores rise. I don't really think that there's much else to it. Maybe the exception is the fact that the amount of study effort put forth by the average student has increased from the initial creation of the test (the doctors of old didn't spend a non-stop month-6 weeks studying for the boards) also raises the score. However, the notion that because we now have USMLE world we will smoke them, seems like a whole lot of nonsense.

 

[joe6102]

The bigger picture here is in terms of residencies. If a ton of people score 230, then 230 isn't as much of an advantage when it comes time to apply. If everyone scores higher than in years past, the only ones who really benefit are the ones who would have failed otherwise.

 

[drguy22]

NBME's tend to underpredict.

look, i sucked at my NBME exams..bigtime.

the last NBME i took was form 3, which people say should be your score. Well i got a 214!!!!🙁

i got my score back and i got a 230+/97!

so for anyone who is taking this test. Just study your A*S off the last 2 days. you WILL boost your score!! dont listen to anyone else. if u think u can do it, you will. Period!

 

[doctorp82]

NBME's tend to underpredict.

look, i sucked at my NBME exams..bigtime.

the last NBME i took was form 3, which people say should be your score. Well i got a 214!!!!🙁

i got my score back and i got a 230+/97!

so for anyone who is taking this test. Just study your A*S off the last 2 days. you WILL boost your score!! dont listen to anyone else. if u think u can do it, you will. Period!

A time frame between last NBME and when you take the test helps too.

"I got a 200 on my NBME and then got 230+ when I took the real thing! However, they were about 2 months apart..." <--- someone somewhere, but not me because I don't have my scores back yet.

 

[drguy22]

A time frame between last NBME and when you take the test helps too.

"I got a 200 on my NBME and then got 230+ when I took the real thing! However, they were about 2 months apart..." <--- someone somewhere, but not me because I don't have my scores back yet.

3 days before..sorry i forgot to put that.

 

[GOGETA]

3 days before..sorry i forgot to put that.

Dude you are an anomaly, in a good way for you and congrats

Most of the people score +or - 5 even 10 on there last NBME.
If they took it 1 week or two before there test.

Now if you are saying that underpredict that is just
If you say that overpredict maybe

 

[71263]

I purchased Forms 2 (670/252) and 3 (770/263), and I thought they were about the same style and difficulty as the actual Step 1. Personally, I thought my scores on the NBMEs seemed way too high, but I still have several weeks til I get my Step 1 score (took it on June 1).

I agree. I did think the real thing was slightly harder than the NBMEs, but not significantly harder to the point where I would say that the NBMEs are inaccurate. When my score comes out, well see...