What residency directors look for

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

El Curandero

Full Member
10+ Year Member
Joined
Dec 26, 2011
Messages
91
Reaction score
19
Hey all,

I was talking to my buddy (he's a fourth year in the same DO program that I'm in) that matched into psychiatry this year and he seemed a little disappointed in where he matched. He said that psychiatry programs look less at step scores like other specialties do and give a lot of weight to which med school they attended/who wrote your letters of recommendation.

I'm honestly not trying to cause any ill feelings to ya'll as a community but if this is true then why? In a way it seems kind of elitist, which strikes me as strange coming from the psych community because you are working with a lot of patients who most people consider to be part of the lowest rank in society (I don't but a lot of people do sadly). This would make it seem like most psychiatrists would hesitate to judge someone based on prestige or societal standards.

I would appreciate opinions from people in the know how thanks.

Members don't see this ad.
 
Well again, I think it's kind of judgmental to assume that someone working at their community hospital is a terrible practitioner because they are not well known or w/e. I think there are bad MDs and DOs but there are plenty of great practitioners coming from both sides. There are two types of elitism in my mind. One is trying to get the best candidates through examining more objective measures and another is a boy's club in a certain way where the people high up on the food chain stay there ect.

Also I don't think you can compare law and business to psychiatry because they are so inherently different in the people that they serve on a daily basis. You often find people in top tier business and law who are judgmental of people that they view as "lesser." In a field like psychiatry (or medicine in general) this is really important to avoid.

I'm not saying it's not ok to take the best students into an elite program but are the metrics that they are basing med students on really result in the selection of the best future practitioners? Maybe you are right and medical students from ivy league schools would really make better practitioners but somehow I find that skeptical.
 
Well again, I think it's kind of judgmental to assume that someone working at their community hospital is a terrible practitioner because they are not well known or w/e. I think there are bad MDs and DOs but there are plenty of great practitioners coming from both sides. There are two types of elitism in my mind. One is trying to get the best candidates through examining more objective measures and another is a boy's club in a certain way where the people high up on the food chain stay there ect.

Also I don't think you can compare law and business to psychiatry because they are so inherently different in the people that they serve on a daily basis. You often find people in top tier business and law who are judgmental of people that they view as "lesser." In a field like psychiatry (or medicine in general) this is really important to avoid.

I'm not saying it's not ok to take the best students into an elite program but are the metrics that they are basing med students on really result in the selection of the best future practitioners? Maybe you are right and medical students from ivy league schools would really make better practitioners but somehow I find that skeptical.

Oh sweet summer child, who still believes physicians aren't routinely incredibly sh*tty to poor people.
 
  • Like
Reactions: 2 users
Members don't see this ad :)
he seemed a little disappointed in where he matched. He said that psychiatry programs look less at step scores like other specialties do and give a lot of weight to which med school they attended/who wrote your letters of recommendation.
I think your friend is upset with not matching where he wanted and so is saying mean things as an excuse for why he couldn't go somewhere awesomer. Your friend is not a resident yet; he doesn't know what residencies are looking for.
 
  • Like
Reactions: 1 user
I think your friend is upset with not matching where he wanted and so is saying mean things as an excuse for why he couldn't go somewhere awesomer. Your friend is not a resident yet; he doesn't know what residencies are looking for.


Maybe you're right about being mad about not matching to where he wanted to go but I don't think him stating his experience through the residency selection process is mean it's just an observation. That observation may or may not be true so that's why I came to this forum to see if it's true.
 
There are no hard rules on any of this. You'll see a lot of the SDN gospel truisms come up quite flat when you sit through your first rank meeting. There are general guidelines about what programs value, but the range in these is huge. At our program we don't care much about board scores beyond being reassured you're not going to have passing issues. Fit trumps everything here. School is fairly irrelevant unless you're not anywhere remotely connected to the greater area, and that's only because we want to make sure you're serious about coming here and not just tossing out an application. We're not a top program but we've filled all our spots without having to go any farther down our list than the amount of spots we have available 2 out of the last 3 years.
 
  • Like
Reactions: 2 users
There are no hard rules on any of this. You'll see a lot of the SDN gospel truisms come up quite flat when you sit through your first rank meeting. There are general guidelines about what programs value, but the range in these is huge. At our program we don't care much about board scores beyond being reassured you're not going to have passing issues. Fit trumps everything here. School is fairly irrelevant unless you're not anywhere remotely connected to the greater area, and that's only because we want to make sure you're serious about coming here and not just tossing out an application. We're not a top program but we've filled all our spots without having to go any farther down our list than the amount of spots we have available 2 out of the last 3 years.


really? eg. you have 5 spots and matched the top 5 on your ROL 2/3 year?.....
 
really? eg. you have 5 spots and matched the top 5 on your ROL 2/3 year?.....
Yes, and in no small part due to the "rules" that you hear on SDN not being remotely true everywhere.
 
Yup. Good but not top programs tend to get their most desired ROLs. Lower end programs and top end programs tend to have to go deeper on their lists.

Bottom end program have problems since few people want to go there.

Good programs tend to be highly desired by applicants and programs have a large pool to select from.

Top programs are all gunning for the same 30-40 people.
 
My advisor told me that programs (the ones I was talking about with him were considered "top" programs) go down roughly half of their ROLs? Is that not true?
 
There are two types of elitism in my mind. One is trying to get the best candidates through examining more objective measures and another is a boy's club in a certain way where the people high up on the food chain stay there ect.
Oi. I get rankled by how people throw around terms like "old boy's club." When it's used haphazardly, it draws attention away from the "old boy's club" that actually exists.

The reason that a lot of programs don't use the "more objective measures" is that a lot of us have seen that it doesn't correlate to performance in residency. I had a colleague that tracked 10 years of residents Step 1 and 2 scores to performance and the result was scatterplot. Granted, that was for EM, but I have a hunch the same would be true of psychiatry. Step scores successfully evaluate how good students are at taking standardized multiple choice tests related to medicine. Full stop.

I believe (and I'm far from alone on this) that the best prediction of what kind of resident clinician an applicant will be is what kind of student clinician an applicant was. This is based on the quality of their clinical experience and their performance during those experiences.

So how do we determine that? The more familiar I am with the clinical experiences an individual has the better. This means I have a preference for the better medical schools and quality medical schools near my program. This is based on the fact that this is where I have the most experience with the work product. This also means that I like to see good performance in their core rotations (I don't care about MS-4 electives). Lots of honors helps, although I don't compare one applicant to another this way, as I know which programs' rotations give honors to 60% of their students (looking at you, Ivy Psych!) and which give it to 15%.

The best assurance is good LORs. The challenge is that the effectiveness is directly proportional to how familiar I am with the source. I don't mean that I know their research, I mean that I know them personally or (more importantly) I am familiar with their LORs. I've read what comes across as a pretty ho-hum reference from a certain C/L dinosaur that I know means they thought this applicant was the best thing since sliced bread. I've also read absolutely stunning raves from famous people in the field whose language I recognize as being largely templates from year to year, so it doesn't mean as much. Given this familiarity, I can be more confident about how a student will perform as a resident if they come from schools I know (and faculty I know) which tends to be better programs and closer ones (see above).

Now, the "old boy's club" does exist. This club is a group of those in power who seek to attract and promote those like them. Traditionally in medicine, this has been Judeo-Christian White Men from great schools (medical, undergraduate and [ideally] boarding schools) who tried to recruit from the same. This is not the case in my program and from networking with what I would consider top programs in psychiatry, it is not the norm at most of the great programs. If anything, appreciation and value of diversity is a huge push and at many programs this is actively sought out. Some of the most sought after applicants by top programs are individuals from immigrant backgrounds, people of color, or individuals from walks of life under-represented in medicine. I'm sure "old boy's clubs" exist in psychiatry residency programs. I wouldn't work at one. And I wouldn't want to train at one, so if you are passed over by one, consider yourself lucky.
I'm not saying it's not ok to take the best students into an elite program but are the metrics that they are basing med students on really result in the selection of the best future practitioners? Maybe you are right and medical students from ivy league schools would really make better practitioners but somehow I find that skeptical.
I do too. Most of us would rather have one of the top students at a good regional medical school than someone who floated through an ivy (though the ivy kid will find a happy home at some program desperate to have that school on their "current residents" letterhead).

What metrics makes the best practitioner is always going to be argued about back and forth in the community as well as in each program. There is much better data for what isn't effective for predicting residency performance than what is effective. Step scores don't seem to correlate. Interview performance also doesn't seem to, but imho, residencies tend to over-value that as well. Fits into the narcissistic tendencies of doctors in general and psychiatrists specifically.
 
My advisor told me that programs (the ones I was talking about with him were considered "top" programs) go down roughly half of their ROLs? Is that not true?
Very roughly. It's hard to really brag about where you ended up on your rank list, as it really says more about who you're interviewing than anything. It feels good to end up higher on your list, as you're matching the applicants you wanted more, but the key is to selectively interview only folks that you'd be delighted to have at your program.

Look at it this way: assume AOA is a measure of the best applicants in the field (it isn't as a rule, necessarily, but it's handy shorthand for now). I don't know how many we had this past year, but in the year I matched, there were 32 AOA folks who matched in psych. So figure that among the top 10 programs, there are maybe 100 slots. Assume each of the programs ranked these 32 folks in the top 32 slots. If they were evenly distributed among these top programs, each would get about 3 matches and be 32 down on their list with 7 slots to go.

This is all scratch, but you can see how it's very easy for the top programs to dip pretty low because I reckon the top half of their rank list is going to look pretty similar to the top half of most of the best program's rank lists. So it's very easy to dip well below the half-way mark. The better programs don't interview nearly the quantity some the lesser known community programs do.
 
. Step scores successfully evaluate how good students are at taking standardized multiple choice tests related to medicine. Full stop.

I get that step scores aren't the alpha and the omega of clinical practice, but statements like the above are oversimplifications and are not exactly true either. Step scores do tell a lot about work ethic, time management, endurance, intelligence and most importantly level of medical knowledge and ability to integrate medical concepts. Thank god for standardized testing because they kinda level off the playing field.
 
Step scores do tell a lot about work ethic, time management, endurance, intelligence and most importantly level of medical knowledge and ability to integrate medical concepts.
Yeah, I'd agree with you to a point. With the exception of integration issue, I would hold that all the other features are within the "student" domain and not the "clinician" domain. I haven't seen any correlation between residents step scores and their work ethic, time management, endurance, and intelligence on clinical rotations.
Thank god for standardized testing because they kinda level off the playing field.
yup. I just wish the standardized metric we had was something of more utility. If LORs and grades on MS-3 rotations could be made level, that would be something particularly useful. Steps, less so.

But it's certainly program specific. Steps can be the end-all/be-all at some places and at others they're looked at as "did great," "did good," "passed okay," and "I'm worried about them failing Step 3 and mucking up our call coverage..."
 
Yeah, I'd agree with you to a point. With the exception of integration issue, I would hold that all the other features are within the "student" domain and not the "clinician" domain. I haven't seen any correlation between residents step scores and their work ethic, time management, endurance, and intelligence on clinical rotations.

yup. I just wish the standardized metric we had was something of more utility. If LORs and grades on MS-3 rotations could be made level, that would be something particularly useful. Steps, less so.

But it's certainly program specific. Steps can be the end-all/be-all at some places and at others they're looked at as "did great," "did good," "passed okay," and "I'm worried about them failing Step 3 and mucking up our call coverage..."

Actually the only meta-analysis that I'm aware of did end up finding that objective measures of performance in med school, including Step scores, had the best correlation with subjective measures of residency performance (i.e evaluations) with Step 2 having the highest correlation. The effects were modest but were better than all subjective measures (including interviews and LORs which had pretty bad numbers. Clerkship grades were also well correlated but for some reason they were considered 'objective' in this study). (Associations between residency selection strategies and doctor performance: a meta‐analysis) .
 
Last edited by a moderator:
  • Like
Reactions: 1 user
Thanks, fpsychdoc. I can't follow the link right now where I'm at, but will be sure to check out the meta analysis. Thanks for providing.
 
Just on the surface of things. If there are two types of schools, type A and type B for undergraduate medical education. The question can be asked and accurately answered, are type A rejections getting into type B schools, or are type B rejections getting int0 type A schools? You may call it elitism, but clearly post graduate institutions will be looking at A over B in the former and B over A in the latter because that is were the more impressive students went (or at least perceived more impressive as determined by the state of the art at looking at these things). Clearly there are other factors and some overlap in the quality of education offered, but sometimes perception dictates reality more than reality dictates impression. Please note that I have left the categories generic. I haven't said Ivy vs non-Ivy or AMG vs IMG, or DO vs MD, or universities with four legged mascots vs two legged mascots.... Schools can decide what qualities they value and I'm sure there are ranges for that.
 
The truth is that match results depend on a whole host of things that you have no control over, and the same applicant could match at their top or their bottom just depending on what applicant year they happen to be in. Maybe you happen to get an interview with someone who just found out their spouse is cheating on them or whose mom just died. Maybe the PD happens to be best friends with your research mentor. Maybe you live in an area with many big name schools who will give you preference just on geographical alone. As someone mentioned above, we know what doesn't predict residency performance (interviews, step scores), but until someone comes up with a better method of evaluation, we're stuck with the current paradigm where a lot of it is up to chance. It is not a fair system, it's just the least unfair system we've been able to develop. At the end of the day, I think where you match has very little to do with whether or not you are going to be happy or be a good physician. Work hard wherever you are and in the great words of Beyoncé's grandmother, make lemonade out of the lemons.
 
  • Like
Reactions: 2 users
Top