Bear with me while I try to make sense of this.
Shelf exams are purchased by medical schools and graded by those individual medical schools. How those grades are normalized is up to the school that purchased the particular exam. The raw scores are reported back to the NBME but not any percentiles or normalized scores.
This is definately not true. I've seen the professors take our exams and ship them directly back to the NBME in a well-sealed box. Then the NBME sends the score report back to the school a few days or weeks later. No grading of the exams are done at the school that purchased the exams. I KNOW this to be the case.
You have no way of knowing whether exactly how your score is normalized unless the faculty at your school is willing to share that information with you. Again, one course may normalize a "shelf" to one mean while another may use another score as a mean. Many schools, mine included, do not even administer these subject-based pre-clinical science exams because they are not very useful. Since most curricula are integrated, subject-based basic science shelf exams are obsolete.
It's normalized against the national mean that the NBME has on file because they are the ones that grade it.
I definately don't think the exams are obsolete for a couple of reasons. First off, they are retired board questions and the boards has plenty of subject-based questions on it. Sure there are plenty of interdisciplinary questions on the boards, but those questions end up on multiple subject-based exams. For instance, if a question is a mix of microbiology and pathology, it's fair game for the microbiology shelf and the pathology shelf. This can be aggravating if you haven't had that material yet, but you're in the same boat with the rest of your class, so it all works out.
Depending on where your school/department sets the mean, those "percentiles" can end up across a spectrum of grades that are only meaningful to your particular class (and definitely not a means of comparison).
Shelf exams for clinical sciences can be used in a similar manner but in general, a raw percentage of 50% is the mean across the U.S. Our school would normalize that percentage to a mean of 45% and would grade us from there. This meant that we had to score above the national mean in order to pass the shelf. We also had in-house exams for every clinical subject that we took at the end of rotations.
No, they are standardized by the NBME and sent to your school, which is discussed in the link that I gave above. It was only recently that med schools were even allowed to know what the scores the NBME sent them were equal to in terms of percent. Your second paragraph argues against everything else you've said. How would your school know what the national mean was if they graded your shelf exams in-house?
I don't believe for a second that students at your school had to score above the national mean in order to pass the shelf unless you had an unbelievably large failure rate. You make it sound as if your school's worst student is average at an average school. "Premier schools" would like use to believe that line of thinking, but it's crap. There's a lot more overlap between test scores at the #1 medical school in the country vs. the "worst" med school in the country than is given credit for.
