Another reason the AOA/NBOME are mentally challenged.
Take two students, for example, that both had mediocre performances probably worthy of passing but not by a lot. One has his two worst patients dropped, the other has her two best patients dropped. He passes, she fails, due solely to the flip of a coin.
You might think the written tests are no different since chance determines what form/question set you will get on any given day, but (1) there are hundreds of questions on the exam, and statistically the average question difficulty will be more similar than not, and (2) even if the question set just happens to not fit your forte, you are judged against all other students taking exactly that same set of questions under exactly the same conditions, which is as fair as you can ask for in a test.
With patient encounters, it is possible that a comparison is implemented such that you are judged against the average of all other students having exactly the same set of patients (after dropping 2) as you. But that isn't the point. It is student performances that aren't standardized, not the patients. A student doesn't necessarily do bad on patient A and good on patient B because of the quality of the patient, or fake illness presented, but possibly for a myriad of other reasons like being nervous for the first encounter or two, distracted by a full bladder before a pee break, thrown off by a random statement made by the actor, etc.
It would only make sense that they do something similar for all students, like drop your best and worst, or drop the worst two, or drop nothing. Or whatever. But not flip a coin and then say "damn, he would have passed but looks like we'll be dropping his best two, such a shame..."
If that is true, they are ******s. To such an extent that I can't believe it is true.