I surely don't want to pretend that I know Mil's practice and can speak for him. But I will give my impression.
Rip me Mil if I am out of line and I will apologize. But I think I have a reason for your "poor candidates". LOCATION.
I left my first job b/c of location. I wanted a lifestyle that appealed to me outside of work. I understand that different people have different needs but highly desirable locations attract highly desirable people.
I would bet these Drs. would have had trouble elsewhere as well. Mil may have high standards but they are not ones that can't be met by most well trained physicians.
And with this I have answered why I left me first job. Great job. Great Drs. Bad location for me.