If LORs are so bad, then why do PD's rank them as so important (see NRMP stats I linked at the top of this page).
I'm the only medical doctor I know of that's actually looked at the science of the admission process.
The PD in my residency program, during my last year, for example, never touched anything from the field of industrial or consult liason psychology. The prior PD during my first 3 years did have a psychology degree and did incorporate what he learned from that field.
Most PDs are similar to the former, not the latter in their approach. They simply take in the data, not knowing how truly reliable and valid each method is. They believe that because they are psychiatrists, they already have a mastery of these types of things when in fact they do not. E.g. psychiatrists, unless specifically trained to do something, often are no better than a layman in many things such as detecting if someone is lying, marriage counseling, etc, because the overwhelming majority have no training specifically in that area. But since they are psychiatrists, many egocentrically and narcissistically start to think they are experts in those area.
This is an all too common phenomenon, e.g. psychiatrists rarely know how to do DBT, yet try to treat borderline PD patients with polypharmacy that usually does not work, and IMHO are really actually doing harm upon the patient that does not meet the standard of care, but I digress.
The science of the admission process has been well studied. Several corporations and universities have done several studies in this area because they want the best people. The overwhelming majority of psychiatrists do not study this in part of their curriculum.
I took an industrial psychology class in college, and during that time, was in an organization where I interviewed about 15 people a semester during an admission process. I actually started incorporating what I learned in that course into the admission process and found it very effective. I went through 6 semesters of this.
I then saw the PD during the MATCH process during my last year when I was actually a part of it because I was a chief resident, and I opined that her methods were far too personal and not evidenced-based. E.g. The PD during my last year put people at the top that she favored, on a friendship level, at the top of the MATCH lists, that IMHO was something of a boundary violation, not to mention some of them lied to her telling her they put our program at #1, yet they matched in other programs despite us giving them the top MATCH spots. (If you know the math of how the MATCH works, if they truly put us at #1, they would've matched at our program. It had to have happened that way by the algorithm). The PD even reacted in anger as if she was backstabbed after this occurred that only further reinforced my theory that some of the choices being put into the list were done for inappropriate reasons.
I do know for a fact that at Jefferson medical school, they employed an expert of sorts who actually studied the admission process. I've seen him give lectures on the science of the admission process and how best to incorporate psychology into making the educational curriculum better. So I wouldn't be surprised if Jefferson was actually using this data, but that's the only place I've seen that's done this as far as medical residencies go. Several corporations have people take psychological tests to try to put employees that match better together.