#1 thing of interest is this big recent push to switch USMLE to pass/fail, like the Bar exam works for law licensing. The idea is that it was never meant to be a selector for clinical residency training, and its abuse for this purpose has started causing problems. Namely, it's caused medical students to ignore valuable prep for the wards/their faculty led coursework, in favor of spending all their time on Anki flashcarding and Qbank questions. So there was a big convention in Philadelphia a few weeks ago where all the main players (NBME, AMA, AAMC, etc) got together to discuss. A summary recommendation draft is expected in the next couple months. You can check out their page on USMLE over here:
www.usmle.org/incus/?#timeline
In short, it might be that in a year or two the USMLE reports things differently, perhaps by quartile or even fully by Pass/Fail. On the other hand they might tell all these organizations of med students & faculty to go to heck, and instead keep the scaled system at the behest of residency directors.
#2 thing of interest is the recent retirement of the old practice NBMEs, with the rollout of a few new ones to replace them. This is supposedly because the old ones had lost their predictive validity. Of note surrounding all this is that there was new leadership as of a year or two ago, so there may be a shift in priority on how people are tested. For example, cutting back on rote recall questions about esoteric enzymes, and instead focusing on application of principles like reading basic imaging or EKGs.
#2.a - the NBMEs are, statistically speaking,
very differently composed than the actual Step 1. To get an
average predicted score (230) on an NBME you need to hit more than 80% correct. In other words, the tests are so easy, they compress the entire top half of the curve into a handful of questions, such that an average student (230) and a top end student (255+) are only differentiated by about 1/8th of questions. The real step 1 has a much lower average percent correct, allowing for a much better bell curve and many more differentiating questions between the low end, middle, and high end. In short, this means the NBMEs are not that useful for a lot of high performing students and
feel extremely different to take. One big impact this has is training people to take their sweet time because the NBME prompts are so short and straightforward and you can't afford to make any errors. Then people get to the actual step1 and feel like they got hit by a train and barely managed to finish (or didn't even finish some blocks), because of how much longer and more difficult the average questions are.
#2.b - Uworld lets you see the percent getting each question correct, and there's a really interesting trend. Questions that can be answered by having a flashcard memorized tend to get absolutely dominated by modern med students. E.g. I just had a question today that asked about which sub-protein of the histone complex is the one outside the core (answers were like Histone Protein 1, Histone Protein 2, Histone Protein 3...)
~80% of people got this correct. Absolutely blew my mind that such a vast majority knew this off the top of their head. What principle is this testing? Who cares. It's in zanki. Gotta know it.
Meanwhile, questions that can't really be flashcarded and require reasoning through a system - e.g. "which of these conditions would cause the following change in this Cardiac Output vs End Diastolic Volume plot" - will have a minority answering correctly.
It really seems like the rise of Anki/Sketchy/Pathoma/Boards&Beyond/obsessive step preparation from the first day of school, has fundamentally changed the game of preclinical medical education (arguably, for the worse). Students are cramming far, far more information into their heads than ever before, but potentially at a loss of time spent
understanding the underlying systems.
Ironically, the likely response from the USMLE is going to be to start testing more understanding and less knowledge...if they keep reporting a scaled score at all, that is. I've seen recent high scorers say that only half a dozen of their 280 questions were "first aid facts" and the other 98% was applying fundamentals. I believe them. The days of buzzwords and nice direct NBME type stuff is gone. You're no longer going to get a vignette that starts with "20 year old athlete collapses on the field..." where you can glance at the answers and smash Hypertrophic Cardio without bothering to read the rest. Instead, you're gonna get some long vignette based on a rare hereditary syndrome nobody has ever heard of, where you get a few key EKG findings and they ask you to derive from that which ion channel is mutated.
