What has more impact: Clerkship evaluation comments or the actual grade?

[PokerDoc]

A residency director looks at two applications and for the most part they read like:


Application 1 is a Pass and has comments including "stays late, does that little extra to make the day easier for the whole staff, would make a steller resident in X, etc etc" great comments

Application 2 is a High Pass (lets leave honors out of this): Comments are incredibly generic "student did well, good job"


which of these applications does the average residency director look more favorably on? Does the grade hold more weight even if the comments are better with the worse grade?

---

Members don't see this ad.

 

[MDpride]

A residency director looks at two applications and for the most part they read like:


Application 1 is a Pass and has comments including "stays late, does that little extra to make the day easier for the whole staff, would make a steller resident in X, etc etc" great comments

Application 2 is a High Pass (lets leave honors out of this): Comments are incredibly generic "student did well, good job"


which of these applications does the average residency director look more favorably on? Does the grade hold more weight even if the comments are better with the worse grade?

I'd like to know that too

 

[2012mdc]

I'd like to know that too

Same here. At my school the shelf is a big part and people can easily end up with an A/honors with generic comments.

 

[vicinihil]

I feel like grades are easier to glance at and assess other than actual comments. I'm not even sure if the comments make it on the application other than in the Dean's letter which stands for little.

 

[PokerDoc]

I feel like grades are easier to glance at and assess other than actual comments. I'm not even sure if the comments make it on the application other than in the Dean's letter which stands for little.

The dean's letter stands for little being your opinion or fact?

 

[newbie04]

The dean's letter stands for little being your opinion or fact?

It looks like it counts for something:

 

[Droopy Snoopy]

It looks like it counts for something:

What's the source of the chart? My program gives out a handout to assist evaluators with residency apps (as a senior resident I got to do this some during an elective month in the fall). Our instructions went along pretty well with those priorities. Things are different obviously, depending on specialty and location, but I'd be willing to bet that detailed comments that can be tied to a real attending-student relationship will generally outweigh generic anything. Ranking applicants is a gamble; you want to careful when considering somebody whose dean's letter has a bunch of disembodied numbers without real substance.

 

[vicinihil]

The chart is misleading. The headline is for whether or not programs use the MSPE as a deciding factor. The only thing we can derive from it is that about 30% of programs never glance at it.

A better chart is as attached...it shows what the importance of the MSPE is. For those that don't want to download it, a quick synopsis.

Across all specialties with 5.0 = most important

MSPE = 3.9

---as comparison-----

LOR = 4.2
Grades in clerkship in desired specialty = 4.2
Honors in that clerkship = 4.1
Grades in core clerkships = 4.0
Honors in core clerkships = 4.0
Step 1 = 4.1
Step 2 = 4.0
MD Degree (vs DO) = 4.1
AOA = 3.6

--------------------------

In conclusion: Grades > MSPE

 

[Way2Swanky]

Comments are worthless. Only the grade matters, and if it's not Honors, it's not going to help you. High pass and pass will buy you the same ticket to disappointment.

A residency director looks at two applications and for the most part they read like:


Application 1 is a Pass and has comments including "stays late, does that little extra to make the day easier for the whole staff, would make a steller resident in X, etc etc" great comments

Application 2 is a High Pass (lets leave honors out of this): Comments are incredibly generic "student did well, good job"


which of these applications does the average residency director look more favorably on? Does the grade hold more weight even if the comments are better with the worse grade?

 

[studentologist]

Comments are worthless. Only the grade matters, and if it's not Honors, it's not going to help you. High pass and pass will buy you the same ticket to disappointment.

Do you think comments factor in when determining AOA?

 

[MilkmanAl]

In conclusion: Grades > MSPE

I don't know what the deviations on those things are, but I'd be willing to bet that the only thing that even has a chance of being statistically different than 3.9 is the 4.2 for grades in clerkship, and even that's questionable, without seeing the numbers. For people interested in a non-core specialty, that info isn't terribly useful, anyway.

 

[Fiddlergirl]

Only the grade matters, and if it's not Honors, it's not going to help you. High pass and pass will buy you the same ticket to disappointment.

This is highly dependent on your school. My school is in the middle of a campaign to deflate the grades given for clerkships, so there are now clerkships where pass is now given to over half the class and high pass is an extremely good grade.

 

[2012mdc]

This is highly dependent on your school. My school is in the middle of a campaign to deflate the grades given for clerkships, so there are now clerkships where pass is now given to over half the class and high pass is an extremely good grade.

Does your school not want to match well?

 

[lockjaw]

This is highly dependent on your school. My school is in the middle of a campaign to deflate the grades given for clerkships, so there are now clerkships where pass is now given to over half the class and high pass is an extremely good grade.

Similar here - 80-85% pass, the remaining % get honors, no high pass. And we don't seem to have problems matching at great programs at all. I'm sure PDs know by now that not all clerkship grades are created equal, and that schools have varying grading systems/grade inflation/etc., and therefore they don't just assume that pass=bad. Programs in our region that frequently match our students are already very familiar with how to interpret our transcripts, and for programs that are less familiar with it, the grading system breakdown for the school is explained in the MSPE. So I think grades only matter in the context of the school - a PD might look at a student from XYZ with mostly passes and a few high passes and consider that a mediocre transcript, while if they look at a student from my school with a few honors and the rest passes but with strong MSPE comments, they'll know the student did relatively well for our system.

 

[2012mdc]

Similar here - 80-85% pass, the remaining % get honors, no high pass. And we don't seem to have problems matching at great programs at all. I'm sure PDs know by now that not all clerkship grades are created equal, and that schools have varying grading systems/grade inflation/etc., and therefore they don't just assume that pass=bad. Programs in our region that frequently match our students are already very familiar with how to interpret our transcripts, and for programs that are less familiar with it, the grading system breakdown for the school is explained in the MSPE. So I think grades only matter in the context of the school - a PD might look at a student from XYZ with mostly passes and a few high passes and consider that a mediocre transcript, while if they look at a student from my school with a few honors and the rest passes but with strong MSPE comments, they'll know the student did relatively well for our system.

Ok I didn't know the grade system was broken down in the MSPE. Makes sense. I wonder what my school says since we don't have pre-set cut-offs.

 

[lockjaw]

Ok I didn't know the grade system was broken down in the MSPE. Makes sense. I wonder what my school says since we don't have pre-set cut-offs.

Can't say for sure that every school's is in the MSPE, but ours definitely is: it spells out exactly how our grading system works for both pre-clinical courses and each clerkship, and specifically what the honors level is set at/how many students are permitted to honor each rotation. It also puts our performance in perspective compared to our classmates if that comparison would be favorable - i.e. ABC was one of only 5 of students in the class to receive honors in 4 clerkships, etc. - because in some schools 4 honors might be no big deal, while here it's very rare and should be emphasized.

I imagine it's a bit like evaluating pre-med grades - there's a difference between an A in an engineering program at a rigorous school vs. an A at a school with rampant grade inflation vs. an A at a lower-tier state school. I think (hope) PDs know that not all clerkship grades are created equal.

 

[Fiddlergirl]

My school puts histograms next to every grade in the Dean's letter, so it's pretty clear that most of the class ends up with a pass. We don't have a lot of trouble with people not matching into their desired specialties.

 

[newbie04]

What's the source of the chart?

I forgot to cite it...here's the link: Results of the 2009 NRMP Applicant Survey.

EDIT: here's the link to the 2010 Survey --> Results of the 2009 NRMP Applicant Survey.

 

[45408]

My school puts histograms next to every grade in the Dean's letter, so it's pretty clear that most of the class ends up with a pass. We don't have a lot of trouble with people not matching into their desired specialties.

Same.


For the OP, grades are more important than comments, in your comparison.

 

[vicinihil]

For some reason I don't think much heart is put into reading MSPE which usually is not a huge factor in a lot of schools selecting candidates for interviews in the past. If you look at Charting Outcomes, 50% of all programs extended more than 50% of their interview invites before receiving MSPE with 25% of all programs extending 75-99% of all interview invites prior to receiving MSPE.

However, that being said, this year is a special year in that MSPE will actually be released earlier in the application cycle. Who knows what impact if any this will have. Most PD don't have the time to mull over >100 MSPE grade distributions and histograms. To them an Honors is a Honors and a Pass is a pass regardless if 99% of the class failed. That's my speculation.

 

[2012mdc]

For some reason I don't think much heart is put into reading MSPE which usually is not a huge factor in a lot of schools selecting candidates for interviews in the past. If you look at Charting Outcomes, 50% of all programs extended more than 50% of their interview invites before receiving MSPE with 25% of all programs extending 75-99% of all interview invites prior to receiving MSPE.

However, that being said, this year is a special year in that MSPE will actually be released earlier in the application cycle. Who knows what impact if any this will have. Most PD don't have the time to mull over >100 MSPE grade distributions and histograms. To them an Honors is a Honors and a Pass is a pass regardless if 99% of the class failed. That's my speculation.

When is it being released this year?

 

[MrPolarZero]

Well, the grade is definitely the one that many of them would base their assessment on. But positive comments would also carry weight with some the doctors, it all depends on who they are.

 

[PokerDoc]

this makes me so proud of my school that thinks honors should be reserved for 1% of the class

 

[lord_jeebus]

The MSPE is important, but you need to understand how it is used.

How a typical residency program reads the MSPE:

1. Skip to the last paragraph.
2. Read the first sentence, identify the adjective immediately preceding "student."
3. Skip to the section at the back with the graphs where the meaning of that adjective is explained.

The significance of this adjective is that it explains where you stand in your class. This is true even at schools with no class ranking or even no grades. Generally speaking,

"Outstanding" = top 10%
"Excellent" = next 15%
"Very good" = next 25%
"Good" = bottom 1/2
"Adequate" = barely passed

Some schools may choose other adjectives like "superior."

At one of my residency interviews, interviewers were given little cards that were used for making the rank list. They were very simple:

Name
School
Step I
Step II
MSPE Adjective
Interviewer 1 score
Interviewer 2 score

 

[Sol Rosenberg]

I'd previously talked to my advisor who is one of the few on the committee that selects residents for a [very competitive] program at my school about that. She said that the Dean's letter is pretty worthless aside from class rank (maybe they just look at the adjective like was posted above,) but they WILL read the comments and look for negative ones like, "...has problems relating to his patients," etc. People with those types of comments, she said, they won't touch with a 10 foot pole.

I actually went (for the free food) to a Psych grand rounds presentation at my school that talked about how they should handle Dean's letters. Bottom line was: Most are worthless (they cherry pick the good comments,) and that maybe they should pay attention to negative comments on the Dean's letter. But then they did a whole prospective study on how resident performance didn't correlate with if there were negative comments on the Dean's letter, etc, so even that wasn't a very useful metric, at the end of the day....

 

[PokerDoc]

I'd previously talked to my advisor who is one of the few on the committee that selects residents for a [very competitive] program at my school about that. She said that the Dean's letter is pretty worthless aside from class rank (maybe they just look at the adjective like was posted above,) but they WILL read the comments and look for negative ones like, "...has problems relating to his patients," etc. People with those types of comments, she said, they won't touch with a 10 foot pole.

I actually went (for the free food) to a Psych grand rounds presentation at my school that talked about how they should handle Dean's letters. Bottom line was: Most are worthless (they cherry pick the good comments,) and that maybe they should pay attention to negative comments on the Dean's letter. But then they did a whole prospective study on how resident performance didn't correlate with if there were negative comments on the Dean's letter, etc, so even that wasn't a very useful metric, at the end of the day....

I agree with this. and that is why I think the interview is such a huge factor. I think that's what ultimately is used as a barometer of how well they think you'll fit in. Its just getting that interview invitation that becomes an issue.

I think id like to even rephrase my question.. does the comments on the evals hold weight for getting an interview INVITE only.. not for acceptance. Because really, why would they take someone elses word for it when theyve already met you and can decide for themselves how your personality fits in.

 

[Docuronium]

Does your school not want to match well?

At my school 60%+ get a pass, and probably more. A grade of High Pass or Honors is difficult to get, particularly in the pre-clinical years, where better than passing grades are limited to at most, the upper 20%.

I don't understand how it's possible at other schools "High passes and Honors are the norm." It seems preposterous that PDs wouldn't see right through that. My school does quite well in the match, and I've never heard anyone say that our grading system was to blame for their issues.

And on another note, that Step 1 is rated at 4.1, and the MSPE is rated at 3.9, tells me that the MSPE is essentially almost as valuable as your performance on Step 1. I don't know how this could be interpreted any other way, as on a theoretical 50 point scale, there is only a 4% difference.

 

[Docuronium]

I think another thing we've determined from all of this is that a particularly bad MSPE is one of the fastest ways to sink your application, as apparently almost all are good.

So it would be wise to make sure this area is tucked away nicely.

 

[2012mdc]

At my school 60%+ get a pass, and probably more. A grade of High Pass or Honors is difficult to get, particularly in the pre-clinical years, where better than passing grades are limited to at most, the upper 20%.

I don't understand how it's possible at other schools "High passes and Honors are the norm." It seems preposterous that PDs wouldn't see right through that. My school does quite well in the match, and I've never heard anyone say that our grading system was to blame for their issues.

And on another note, that Step 1 is rated at 4.1, and the MSPE is rated at 3.9, tells me that the MSPE is essentially almost as valuable as your performance on Step 1. I don't know how this could be interpreted any other way, as on a theoretical 50 point scale, there is only a 4% difference.

Yeah that post was short-sighted, didn't consider the possibility of school's including their grade policy/distribution with the letter

 

[Docuronium]

Yeah that post was short-sighted, didn't consider the possibility of school's including their grade policy/distribution with the letter

This may be an SDN first, but... I forgive you.