After finishing 3rd year and watching my classmates, it seems like it would be fairly easy to categorize students into 3 groups: 1. People who are trying hard, coming in on time/early, practicing presentations, basically doing the things Dr. Desai describes in his book. 2. People who are doing fine but for whatever reason, are not putting in the extra effort. 3. People who deserve to fail-- violate pt confidentiality, don't show up when they should, fail the shelf, etc.
I think the problem comes in taking the first 2 groups of students and trying to sort them into honors, high pass, and pass categories. I bet with only 1 or 2 evaluations you can't make a reasonably repeatable determination about whether someone deserves a H or HP. For instance, if you somehow designed a situation where the same student went through the same clerkship multiple times, they might get a H 60% of the time and NH 40% of the time. I would imagine that the repeatability would improve the more attendings are involved in the evaluation, as long as each has worked with the student long enough to get an accurate picture. The problem is that most of our grades are based on just a few evals, and so the personal scale of the individual evaluator matters as much as the student's performance.
