I thought Step II scores are best to be taken late unless one did poorly on step I? I thought most LOR were more or less good so how do you differentiate a good letter from another good letter?
For most programs, that is correct. See:
http://forums.studentdoctor.net/showthread.php?t=829175
http://forums.studentdoctor.net/showthread.php?t=778027
This has been discussed at length in many SDN residency forums and the Aunt Minnie radiology forums and I can provide many more links as needed. Also, see:
Key criteria for selection of radiology residents: results of a national survey., Otero et al, Acad Radiol, 13 (2006):1155-64
Determinant Criteria for Considering Interview (64/70 programs responding to survey were academic) in order of most to least important:
1. USMLE scores
2. Dean's Letter
3. Class Rank
4. Recommendations
5. Honor society membership
6. Leadership experiences
7. Research experience
8. Grade in a specific rotation
9. Employment experience
10. Volunteer experience
The best LORs are LORs from people who got to know you beyond just a clinical rotation and who are big names in the field. Thus, doing research with big names in your future residency field can be
extremely helpful. This is one of many reasons to do your PhD in your field if interest. You should be able to obtain LORs from physicians or physician-scientists in your field beyond a letter from a rotation. If you don't do that, spending a few months in fourth year doing research in your field of interest may help your application significantly in this regard. More on this later.
Please read the OP questions carefully... The questions were not about the importance of USMLE Step I but about slight variations from year to year on the score of Step I (score creep).
The meaning of a 245 will vary from year to year, but is a top tier score every year because is more than one standard deviation from the mean and median scores.
When I was considering rads and rad onc initially as a second year, the average score for both was 235. When I scored a 240/98 on my step 1, I was very pleased. When I applied, the average score for both was 240. I was told by a home faculty memeber that a 98 two digit score could hurt me at programs ("most strong academic programs want all their residents to be 99th percentile").
When I found out a 240 step 1 score was merely average and the cutoff score at some programs, I took step 2 early and scored very highly on it. I don't think that really made any difference. But if I had scored the previous average of 235 on step 1, I don't know how many of my 11 out of 45 interviews would have cut me off for having less than a 240 (less than average).
Why do I care? Big name academic programs where I can get the research training I need want the top residents including the top step scores. Non-academic programs won't interview me at all because they don' have the research I need. This is called the MD/PhD trap and I'd written about it in the article I linked previously.
So does 5 points of creep in 5 years really make any difference? I'll leave that for the reader to decide.
Having said all of this, there are several papers that have been placing questions on the validity of using Step I and/or II as predictors of scores or passing rates of board examinations after residency. Clearly, USMLE scores do not capture the essence of predicting good doctors.
I 100% agree. The MCAT does not predict how good of a medical student or physician you will be either. But competition is what it is. Ideally, a successful MD/PhD with many publications would be a valued commodity as long as they were competent in medical school, but this is
not grounded in reality for competitive specialties. The 245 you posited as an example is somewhere around the 90th percentile for step 1 scores nationally. If you score at the 51st percentile (say a 221), it is likely you will not match at all in radiology or rad onc. 10 years ago when the average was in the low 230s, this likely would not have been the case (a little longer term score creep there...).
That said, the arguments I have heard from program faculty are as follows. What the USMLE tests is how good you are at passing board exams. Residency board exam pass rates are very important to residency programs. They want the least chance of a resident failing the boards, as this creates a lot of problems in the program. Similarly, a student's clinical grades predict, if nothing else, how amiable the student will be when dealing with staff, faculty, and scut work. Thus, taking a student who is highest in these arbitrary indicators is a soft predictor of how little of a problem that resident will be.
Conversely, I have been repeatedly told what a problem MD/PhD residents can be. First, the MD/PhD wants research time that takes them out of clinic. Second, they often don't focus on as much clinical trivia and want to focus on research or other big picture questions. Both of these can make them appear weaker than their co-residents, and puts them at a higher risk of failing their specialty boards. Given that there is no guarantee that an MD/PhD resident will go on to have an academic career, many faculty and PDs do not see the purpose of cutting them any slack clinically or when evaluating them on the MD side for the residency program.
Further, I have been repeatedly reminded that "not everyone in the field needs to be an academic" and that "we're happy if we take one or two MD/PhDs a year and everyone else canbe clinically focused" by top research programs. If you have a lot of MD/PhD competition, this further hurts you, as you then have to compete with other MD/PhDs if you are in a field that has many of them. Radiology is the 4th most popular specialty for MD/PhDs and there are few strongly academic programs. Radiation oncology has a ton of MD/PhDs. Even when being evaluated against MD/PhDs at the radiology or rad onc programs, I still think AOA and step 1 score are most important, because faculty simply don't know how to evaluate graduate school work. In this regard, doing your research in the field you wish to match in can help you because they get a better sense of your accomplishments and potential. But the benchmark at many programs is still: "you published right?", and thus the research work of a 4th year medical student who published a first author clinical paper or two becomes roughly the same as your 4 years of blood, sweat, and tears.
The interview and match process was incredibly disheartening for me, because I used to believe strongly in a physician-scientist career for myself. I remain passionate about this topic for the following reason. I was told repeatedly when I stated my program to pace myself and that I would be able to obtain any residecy I wanted as long as I passed step 1 and my coursework. As such, my first and second year performance was average to below average (at my top tier medical school) though I didn't fail anything. It was only when a faculty member at a home program told me I may not match at all coming back from my very successful PhD that I kicked it into high gear.
And while I honored just about everything when I returned to medical school, that was not enough to push my class rank where it needed to be, and a sky high step 2 doesn't replace an average step 1 score. So when I received no interviews from the region of the country where I wanted to do residency, was shut out by most of the big academic programs, and then began to fear whether I'd match at all as I got the fewest interviews of all the MD-onlies applying from my medical school, I wished I could go back in time and change things.
Hopefully my warning can serve to help those junior students from making the same mistakes. Even if you don't think you want a competitive specialty or region now, things may change when you meet someone, or you realize rad onc is the coolest specialty in medicine, or that your collaborator happens to be at UCSF and that would be the best place to further your career.
