And Im on my phone again so I can't just edit to add... sorry.. but your 2nd link shows a really good correlation. Several people say they were accurate to within 10 points (including statements of "ot underestimated by 5" that is pretty good as far as predictive value goes)
EDIT: back on the computer now
Here is the thing - I dont see any reason why including other schools and increasing the sample size will automatically skew the USMLE correction upwards. 150 tests is a pretty good number of tests when they correlate so highly. If I were doing any other study and my data was correlated so well I wouldnt worry at all about sample size. In most cases increasing the sample size will increase significance and would be expected to increase R, but not necessarily affect the equation.
and because
r is only .85, it is entirely expected that there will be people who are not perfectly characterized. And there will be people who were entirely UNcharacterized by the equation (again... because R is not = 1.0). That is just a fact intrinsic to the numbers. If I wanted to rationalize things away as strongly as you are Id just say that such people are more likely to be vocal about the faults of the equation than these guys
.
but even if we accept the "SDN pre-osteo conversion factor" and just add 20 points to the equation to account for nearly everyone who says their score was underpredicted, the AOA match stats are still significantly behind the ACGME match stats by another 20-30 points in almost every specialty (ive got a spreadsheet, if you're curious)
and the only point that was made here (and the original question) is that AOA match seems to select lower scoring applicants as a whole. This isnt true for DOs entering ACGME which I stated earlier, so none of this should be misunderstood as DO bashing.