Just reporting what I've read. I've heard that a number of times.
I've heard a lot of medical students say that on reddit and SDN. Reddit did a pretty big survery. And yeah it definitely doesn't underpredict for everyone!
Two things, 1. NBME 19 over-predicts for a few people, but I did not se anyone on this graph who scored a 250 on NBME 19 and then managed to get a 210 in the real thing. It looks like the majority of people who it over predicted for scored within an acceptable range of their score on NBME 19 (of course there will always be outliers). 2. I would not focus on the NBMEs as strongly as everyone does for multiple reasons. (side note, there is one person on these graphs who seems to perform extremely well on the NBMEs and then scores significantly lower on the real deal. Clearly something is going on there, so we can't base any theories on that, even though its really easy to focus on the person who keeps scoring in the 250s+ and then scores in the low 200s on the real deal.)
1. People take these NBMEs differently. Some people "cheat" on those NBMEs. There are numerous boards where people talk about looking up multiple answers while taking the practice NBMEs. I am sure they are aware, but Step 1 isn't open book or open Google. Additionally, there are some people who look up the NBME answers or go to quizlet and find decks for the NBMEs, study the decks and then take the practice exam. This is a very bad idea, don't do this. I saw one poster who did this and was scoring 260+ on the practice tests and then scored in the low 200's on the real deal (shocker). Then there are people like me, who genuinely took them in testing conditions, did not look up answers and did not study from decks made from the practice exams. The NBMEs should be most predictive for people who take the exams like that, but even still there is some luck involved.
2. People take these NBMEs at different times in their studying. That will throw off the curve. If everyone takes the first available NBME at the beginning of their studying and everyone takes the last NBME at the end, that is obviously going to sway the curve. For NBME 19 this is especially true because A LOT of people take it last as they believe it to be most predictive and have the strongest resemblance to the real exam. But, people also take NBME 19 first so that also messes with the curve or your personal score, because if you take NBME 19 first while the majority are taking it last, you are obviously going to score much lower. Maybe that is why it is believed to under-predict so much. My school made us take an NBME like eight months prior to our exam date as some sort of bizarre tactic to scare us into studying. So obviously that exam will under predict for me, I hope lol.
My suggestion with the NBMEs is to take no more than 1 per week, or every 2 weeks during dedicated, up to you. The questions on my exam were nothing like the NBME questions and I don't really feel as though they helped me much at all. I knew someone who only took one NBME and got a 240+ on the real deal. I know someone who took them all, never scored above a 250 and got a 265+ on the real deal. So there is no right answer, but I think it is safe to say if you are consistently scoring 230+ on the NBMEs while taking them under real conditions, you are in pretty good shape. If you are all over the place like 210, 250, 220, 265 you probably need to look at the types of questions you are missing or the topics that are consistently your trouble areas. Do not assume that since you got a 265 on one exam that you're prepared for a 265 on the real deal. I still don't have my score and felt pretty terrible after my exam, but I took all of the NBMEs and my scores were within 2 points of each other during dedicated. What I can say is that my NBME scores did not go up that much during dedicated, 11 points was about my max. But I went from scoring 75-77% correct on UW to scoring 85-90% consistently. I also took my consistency on the NBMEs to indicate that I was in a pretty solid place, because they obviously do not ask the same questions on each NBME, so I felt like I was prepared to handle any conglomeration of questions they handed me. Also be sure to look at your bar graphs on the NBMEs and number missed. At the end of dedicated I was missing 10-15 fewer questions and my bars had shifted to the far right of the graph from the middle even though the SD of my exams = 2 points. Theoretically, all of this pointed toward a pretty solid score for me regardless of which exam I was handed. And then they handed me an exam that was clearly written by satan himself and I freaked. This would be a perfect example of how you can consistently earn solid scores on the NBMEs and then find that they over predicted by quite a lot, because on test day you find yourself wishing you had an adult diaper and a change of pants.
🙂
