The original thread (or whatever twitter calls it) seems to make three points. 1) The NBME charges too much money, pays people too much, and has developed "mission creep" and is bloated; 2) Step 2 CS is a waste, and 3) Maybe Steps should be P/F.
As for point 1, I think that seems somewhat true. The NBME started as producing the USMLE exams. They;ve now expanded to create the shelf exams. And of course they are happy to sell students practice questions for more income. How much does it actually cost to create the USMLE exams? I'm not certain, but exams are expensive to create and administer (esp when you;re keeping the contents secret). But it's true that the NBME could increase the price to $5000 per step and there's nothing anyone could do about it. Should the gov't regulate how much these exams cost, much like the way the gov't regulates utilities? That's a political question.
Point 2, CS. His point is really interesting and ironic. He argues that because the exam is pass/fail and 95% pass, it's useless. His solution is to remove it. Another solution is to score it -- then there would be "discriminating value" to the 95% who pass since you could score high or low. Also, if 5% fail AND if those 5% actually have a problem, then from my view point the exam has some value. It could be argued that schools can "evaluate student clinical skills", but schools have a huge incentive to pass their students. Does a fail in CS portend problems? I'm not certain, but early on in the CS experience I figured it was a silly waste of an exam and matched two people who failed the first time. Both ran into major problems. Both graduated, but I'm more convinced that the exam actually measures something. Like all exams it's not perfect -- someone who is fine will fail the exam.
Point 3 (and the major point for discusison on the thread), should S1/2 be turned to pass/fail? I refer you to point 2. Making it pass/fail will make it useless. Perhaps "useless" is better than the status quo.
What will absolutely happen, as operaman above me has just stated, is that each field will create their own exam. It wouldn't be very hard, we could actually just use the ITE exams that most fields already have. Or, the NBME will beef up it's shelf exams and use that -- then you'll have the pleasure of paying them again. So then students would need to take the steps and pay for them, and then pay for a new exam. And if you want to do Derm with an IM prelim, you might have to take both the Derm and IM exams. The exams might only be offered a few times a year. You might only be allowed to take it once. If the NBME ran the process, your transcript would likely include all exams you take, so if you dual apply everyone would see the two exams. And the NBME (or something similar) would need to run this, because IMG's need to be able to take the exams, perhaps in other countries (just like the USMLE is now). Is this better than where we currently are? Maybe, but all we've done is trade one high stakes exam for another.
If we make it P/F without another exam, how am I supposed to decide who / how to rank people? More medical schools are moving to P/F grading, even in the clerkships. if everyone passes their clerkship and passes the USMLE, how do I sort through the applications? LOR's where everyone says that this student is in the top 5% of all people they have ever worked with? Personal statements? Volunteer activities? The bottom line is that the residency application process is competitive (as is most of life), and I need some way to assess performance. I'm all for a "better" way, but I just don't know what it is.
