If Step I and II become pass fail, then there are a couple options.
A) The NBME shelf exams for third year become the de facto yardstick. The advantages are that they're multiple (so no "one and done" like Step 1) so you can improve and they're somewhat focused in areas you may be interested in so that may be appealing to residency directors (although someone applying for neurosurgery with, for example, a fantastic surgery shelf but a poor medicine shelf is going to raise eyebrows). Disadvantages are not all the schools use them currently, which would somewhat "force" all students to take them, perhaps paying out of pocket. Also, I think the NBME would not appreciate this and may do the same thing to shelfs that they did to Step 1 and get rid of "numerical" scores.
B) One or multiple "standardized" exams takes its place, either through the NBME or outside of it to be the objective grader. The advantage would be since it's BUILT to objectively quantify your performance, it'd probably have better discriminating power AND repeat validity than the current Step 1. The disadvantages to this model is clear: it's another test meaning more study time for students and more cash. Plus, it's material is likely to be redundant with Step 1. Plus, if multiple tests spring to fill it's place, it's inevitable that certain specialties will prefer certain tests, muddling the whole situation. For example, test A may be preferred by Medicine, Pediatrics, and Emergency medicine programs while test B is preferred by surgery, OB/GYN, and Radiology and Pschiatry has their own test they prefer on to themselves. And then you'll still end up with maverick program directors who go against hte grain and you'll have Pediatric residencies requesting version B...it'd be a huge headache.
