Best way of grading clinical years?

[Qwerty122]

With step 1 being pass/fail, emphasis now shifts to step 2 and the clinical years as the main quantitative metric for a student's candidacy for residency. I've heard that many students complain about clinical grading (subjectivity, generic comments, unclear expectations from preceptors, everyone gets a 3/5, etc.). So what are some things you school did during clinical years that you think are commendable and should continue to be implemented/improved upon? Having a "well-oiled" system for clinical years will only continue to grow in importance in the coming years. I'm fortunate enough to have a student leadership role in my school at directing the clinical curriculum, so I'm hoping to draw from the collective knowledge and experience of sdn to move forward. Thanks all!

---

Members don't see this ad.

 

[Lost in Translation]

I think the notion that clerkship evals are all inflated is a meme. I think attendings no matter where you go are very honest in their feedback of students after the rotation has finished. Despite never failing a clerkship, I've had some of my attending comments (both good and bad) be brought up during interviews for residency. So to answer your question, making sure that attendings are honestly evaluating their med students is probably one of the best ways to maintain legitimacy.

At my med school, the COM actually stopped sending students to rotate with this one FM doc in the community because he was known to just give straight 5s. I'm not saying attendings should start having quotas because that doesn't help anyone either, but making it clear on the grading scheme exactly what a med student needs to do to earn that Honors eval should be implemented.

For example, my personal assessment of med students (as a resident) goes:

1. Pass - shows up, does what they're told, does a few notes, +/- read up on the patient and surgery prior to going to the case, +/- helps out with floor work after rounds and during "down time", sees consults with the resident +/- participates in the consult, carries 2-3 patients on average
2. High Pass - the above plus: always reads up on patient and surgery prior to going to the case, offers to throw in a few orders after rounds and helps out with floor work (pulling drains, changing vacs, etc.) after rounds and during "down time", sees consults with the resident and shows initiative in wanting to lead the consult, helps make the list in the morning, carries 4-5 patients on average (this grade tells me they can take care of the major moves for their patient(s))
3. Honors - the above plus: is able to see consults alone and present to the resident, follows-up on labs and orders from the AM, can present a patient to another service for a consult, carries 4-5 patients and knows something about the rest of the patients on the list even if they're not personally responsible for/assigned to that patient (this grade tells me they can take care of everything for their patient(s); the true "functions at the intern level")

I think as residents we don't do a good enough job involving the med student in clinical responsibilities. Certainly there needs to be an aspect of self-motivation on part of the student, but I hate when they ask "Is there anything else I can do" because the answer is always yes, but then it becomes a question of "is it something that the med student can do?" When I have students that rotate with me, I teach them how to put in orders and will try to give them the opportunity to see consults on their own or call consults on our patients. I try to give them tasks to do at the beginning of the rotation, and then slowly take a step back as the weeks go by to see if they can continue to do those tasks without being asked. That way I'm not hand-holding them but they aren't also just plopped there with no direction. As a med student I hated feeling useless and just a piece of decoration, but I also didn't know what my residents expected of me either because their expectations were always vague. So I want my med students to know exactly the bare minimum requirement to pass my rotation, and then build on that foundation to see if they can excel to a HP or H level.

 

[SurfingDoctor]

With step 1 being pass/fail, emphasis now shifts to step 2 and the clinical years as the main quantitative metric for a student's candidacy for residency. I've heard that many students complain about clinical grading (subjectivity, generic comments, unclear expectations from preceptors, everyone gets a 3/5, etc.). So what are some things you school did during clinical years that you think are commendable and should continue to be implemented/improved upon? Having a "well-oiled" system for clinical years will only continue to grow in importance in the coming years. I'm fortunate enough to have a student leadership role in my school at directing the clinical curriculum, so I'm hoping to draw from the collective knowledge and experience of sdn to move forward. Thanks all!

I think things like this are what helps. Having 1) extracurricular activities and 2) having those activities be something you can talk passionately about (on the application and in person).

Having reviewed applications for the past decade or so, most applicants fit in a very tight bell curve. There is little that differentiates them from a clinical or knowledge based standpoint. The step was one thing that kinda helped, but it only helped on the extremes (280... wow! or 182... yikes!). The thing that actually differentiates candidates are the things they do outside of showing up to the hospital. Did they get a higher degree? Did they significantly contribute to a project? Are they involved in leadership? (etc.). These are they things that make for standout candidates versus the run of the mill. When it comes down to rank list, the run of the mill candidates are forgettable and just get stuck in the middle. The standout candidates with interesting things to say or talk about are the ones to a memorable and stay at the top. Of course, there's always the candidates who are memorable for the wrong reasons... they get put at the bottom, or more typically get DNR'ed.

 

[Qwerty122]

@Lost in Translation I agree with your points, especially regarding the clarity of communicated expectations from the preceptor to the student. The ambiguousness will not only cause increase stress in the students, but also will increase "gunner-like" behaviors. And I'm sure no one actually wants to sabotage others, nor be sabotaged. So that leads me to this next point.

Do you think it's appropriate to shift from giving out H/HP/Ps based on percentages to a system similar to the one you mentioned above? As in, students who "perform at an intern level" will 100% of the time get Honors, regardless of how many other students get honors. The current percentage system seems like it is set up for intra-class competition. Let's say objectively speaking 70% of students from school A actually perform at an "intern level". But only 20-25% of that 70% will get honors, because of percentage limits. Let's also consider, school B, where only 5% of students actually perform objectively at an "intern level". In this case, students who are actually not performing at their highest level will get honors. Most importantly, residencies will more or less view students from school A and B with "honor" grades the same. I think if we want to move away from a percentage based tier system, foundational to that is clear communication and expectation, just like what you had mentioned. What do you think of this proposal? This will be a huge shift in approaching clinical evals, but I think now the benefits outweigh the negatives (given step 1 being p/f).

 

[Qwerty122]

@SurfingDoctor Wow, I didn't come in expecting a compliment, but thanks! That gives me motivation to continue finding things outside of simply watching lectures that I'm passionate about. I find this topic of clinical curriculum revamp not only personally engaging, but also urgent and unavoidable. By the time current M1s get to M3s, it's too late to complain about things and expect immediate change. The only way to navigate through this is to anticipate problems early on.

 

[SurfingDoctor]

@SurfingDoctor Wow, I didn't come in expecting a compliment, but thanks! That gives me motivation to continue finding things outside of simply watching lectures that I'm passionate about. I find this topic of clinical curriculum revamp not only personally engaging, but also urgent and unavoidable. By the time current M1s get to M3s, it's too late to complain about things and expect immediate change. The only way to navigate through this is to anticipate problems early on and voluntarily take responsibility for it.

The one thing I will say about education is that you typically have to some evidence of being serious about it. I mean, being on committees help, but usually it means pursuing (at some point) a higher degree in education. I'm not saying its a given, but it usually demonstrates commit.

Unfortunately, a lot of applicants say that they want to do "education" and "curriculum development", but it's usually small and unimpactful or what they really mean is that they want to give a lecture to students on a subject of their interest. However, if they've gotten (or plan to get) a M.Ed. or something along those lines, and have a specific focus in mind other than "to teach students", then I'll take them more seriously and move them up the ranking.

But overall... the most common career goal I hear from applicants is "I want to be involved in education" so that, by itself, does not move one out of the middle of the pack.

 

[Redpancreas]

With step 1 being pass/fail, emphasis now shifts to step 2 and the clinical years as the main quantitative metric for a student's candidacy for residency. I've heard that many students complain about clinical grading (subjectivity, generic comments, unclear expectations from preceptors, everyone gets a 3/5, etc.). So what are some things you school did during clinical years that you think are commendable and should continue to be implemented/improved upon? Having a "well-oiled" system for clinical years will only continue to grow in importance in the coming years. I'm fortunate enough to have a student leadership role in my school at directing the clinical curriculum, so I'm hoping to draw from the collective knowledge and experience of sdn to move forward. Thanks all!

Speaking as an ex- curriculum committee person, I will say it's noble of you to take the time to ask. As you probably know, you're essentially powerless but ultimately your role is to represent your student population which you're trying to do. To be honest, I can think of no "best practice guidelines" that someone isn't going to reply to being like "this would have hurt me", etc. Here's what I think worked.

1.) Use of histograms in the dean's letter. I feel this is universally practiced but regardless I think it's worth mentioning. Regardless of how a school decides to separate out clinical grades, there should be a histogram distribution of clinical grades for the class. This discourages grade inflation.

2.) I think both shelf scores and the subjective clinical evals (likert scores) ought to be weighed.The million dollar question is what the secret recipe is and I don't there's one policy a majority will get behind. I think the best thing to do is mitigate the subjectivity behind the clinical evaluations. I don't think the likert system itself is inherently flawed but when we try to apply it to clerkships in 2021, it falls flat. I will rant about this later, but it doesn't directly answer your question so all I will say is create a standardized, but realistic rubric. Right now we have a standardized fantasy rubric for med student competencies and people are forced to fudge things which defeats the purpose of the standardization and forces people to use subjective measures to judge. Ex.) Student A uses the scientific method and reasoning to come to clinical conclusions...what does that even mean? How about Ex.) Student A presented patients in a SOAP format?

3.) Regarding clinical evaluators, only certain people should be allowed to rate medical students. I can't tell you the number of times someone's gotten lucky and run into a "i'll give you 5s, don't worry" deal from a resident and then some other poor soul runs into an attending with good intentions who ranks them realistically and messes them up. It's an uncomfortable reality but many students are skating by and others are getting punitive actions put on them inappropriately. The schools really has no way of getting to them bottom of these things so their solution is to just put only good things on the Dean's letter and now guess what...it's their future residencies problem. In an ideal system, there should be cohorts where the clerkship director has an intimate understanding of each student's ability/performance. This is another reason to keep class sizes low.

 

[Qwerty122]

@BacktotheBasics Thanks for the input. I do think all the points you listed are valid and should be included in conversations about curriculum improvements moving forward.

I touched on this idea earlier in the thread, but I was wondering what you, especially as an ex-curriculum committee individual, think about this idea below. So only a percentage of students will get honors, and high-pass. I think this system is set up for intra-class competition, which is not that healthy. Typically, honors are reserved for students performing at an "intern level" and have good board scores. But what if a lot of students are performing at this high-level? Moreover, even if you're not performing at an "intern level", you just have to be better than others to get honors. It also doesn't matter if everyone works hard to be a great M3. The fact remains that only a small percentage will get honors. It's a self-correcting system that maintains its bell curve and restricts negative skewing. And isn't negative skewing both the goal of the student and faculty? Residencies also don't have to take time to figure out school-specific clinical grade distributions. They see an "honor" grade and can immediately assume the student functioned at an "intern level". To achieve this change, clear and detailed guidelines of what constitutes a honors/high-pass/pass first needs to be established. Grading of a student (talking more about the subjective clerkship evals, not the NBMEs) should ideally be only based on this guideline, and not comparing students to each other.

 

[Redpancreas]

@BacktotheBasics Thanks for the input. I do think all the points you listed are valid and should be included in conversations about curriculum improvements moving forward.

I touched on this idea earlier in the thread, but I was wondering what you, especially as an ex-curriculum committee individual, think about this idea below. So only a percentage of students will get honors, and high-pass. I think this system is set up for intra-class competition, which is not that healthy. Typically, honors are reserved for students performing at an "intern level" and have good board scores. But what if a lot of students are performing at this high-level? Moreover, even if you're not performing at an "intern level", you just have to be better than others to get honors. It also doesn't matter if everyone works hard to be a great M3. The fact remains that only a small percentage will get honors. It's a self-correcting system that maintains its bell curve and restricts negative skewing. And isn't negative skewing both the goal of the student and faculty? Residencies also don't have to take time to figure out school-specific clinical grade distributions. They see an "honor" grade and can immediately assume the student functioned at an "intern level". To achieve this change, clear and detailed guidelines of what constitutes a honors/high-pass/pass first needs to be established. Grading of a student (talking more about the subjective clerkship evals, not the NBMEs) should ideally be only based on this guideline, and not comparing students to each other.

Glad to have the discussion. I disagree with the majority of what's quoted. Unfortunately, we need to a way to assess/reward outstanding students and just deeming all students as high performing is inadequate in my opinion because if we do that with clinical grades, what's really left to assess students? The purpose of the high pass is to reward high effort that didn't quite meet honors criteria. Competition is inevitable, one can either accept/deny that. I do agree with the sentiment behind quoting "intern level" like wth does that even mean? It's so often used that it's lost it's meaning. I also disagree with the bolded point about residencies not taking time to figure out school specific grading distributions. I think it's very relevant and people who review these applications will pick up on a place that hands out honors like candy vs. another place. These are the sort of intricacies programs try to seek out in the deans letter hence why the histograms should be presented. It's a valuable check to grade inflation.

I do think the rotation benchmarks need to be rewritten in plain English to represent expectations of the 2021 medical student as per the example above.

 

[Qwerty122]

Glad to have the discussion. I disagree with the gist of what's quoted. Unfortunately, we need to a way to assess/reward outstanding students and just deeming all students as high performing is inadequate in my opinion. The purpose of the high pass is to reward high effort that didn't meet all the marks. Competition is inevitable. I do agree with the sentiment behind quoting "intern level" like wth does that even mean? It's so often used that it's lost it's meaning. I do disagree with the bolded point about residencies not taking time to figure out school specific grading distributions. I think it's very relevant and people who review these applications will pick up on a place that hands out honors like candy vs. another place. These are the sort of intricacies programs try to seek out in the deans letter hence why the histograms should be presented.

I do think the guidelines need to be rewritten in plain English to represent expectations of the 2021 medical student.

I'm going to push back just a bit here. Let me know what you think (and whoever else is reading for that matter). Currently, I don't think there is a good way to distinguish student's capacities in a clinical setting BETWEEN schools. Conversely, a lot of effort has been done to stratify students within a specific school. While I do agree very much that we need to reward students who perform well, but in relation to what? In relation to their peers or in relation to the highest capacity given the role as an M3. I think taking the latter approach might be the better option.

So let's say residency programs know that for school X, grading is based on their objective capabilities as an M3, and less on how well they do compared to other students in the class. Thus, PDs know that honors will mean "an excellent candidate" objectively. This allows for the possibility that a majority of the class to be "excellent interns" AND be appropriately labeled as such come residency application. Conversely, if percentages remain ingrained in the system, there will be a greater amount of false negatives (i.e. students who actually will be excellent interns objectively, but got a HP or P instead due to their class' abundance in quality students).

I would imagine this would make the lives of PDs filtering the applicants easier no? A simple "honors" = "objectively excellent clinical skills", rather than having to understand the deeper meaning behind these letter grades. A histogram might be beneficial, but that's still assuming we're sticking with the old system.

Now, with all that said, the key here is to clearly and in great detail elucidate to students the guidelines that allow students to get H, HP or P. Make sure students know very well what is expected of them through their M3 year if they want a honors.

 

[Thesimplelifeofamyloid]

I think the notion that clerkship evals are all inflated is a meme. I think attendings no matter where you go are very honest in their feedback of students after the rotation has finished. Despite never failing a clerkship, I've had some of my attending comments (both good and bad) be brought up during interviews for residency. So to answer your question, making sure that attendings are honestly evaluating their med students is probably one of the best ways to maintain legitimacy.

At my med school, the COM actually stopped sending students to rotate with this one FM doc in the community because he was known to just give straight 5s. I'm not saying attendings should start having quotas because that doesn't help anyone either, but making it clear on the grading scheme exactly what a med student needs to do to earn that Honors eval should be implemented.

For example, my personal assessment of med students (as a resident) goes:

1. Pass - shows up, does what they're told, does a few notes, +/- read up on the patient and surgery prior to going to the case, +/- helps out with floor work after rounds and during "down time", sees consults with the resident +/- participates in the consult, carries 2-3 patients on average
2. High Pass - the above plus: always reads up on patient and surgery prior to going to the case, offers to throw in a few orders after rounds and helps out with floor work (pulling drains, changing vacs, etc.) after rounds and during "down time", sees consults with the resident and shows initiative in wanting to lead the consult, helps make the list in the morning, carries 4-5 patients on average (this grade tells me they can take care of the major moves for their patient(s))
3. Honors - the above plus: is able to see consults alone and present to the resident, follows-up on labs and orders from the AM, can present a patient to another service for a consult, carries 4-5 patients and knows something about the rest of the patients on the list even if they're not personally responsible for/assigned to that patient (this grade tells me they can take care of everything for their patient(s); the true "functions at the intern level")

I think as residents we don't do a good enough job involving the med student in clinical responsibilities. Certainly there needs to be an aspect of self-motivation on part of the student, but I hate when they ask "Is there anything else I can do" because the answer is always yes, but then it becomes a question of "is it something that the med student can do?" When I have students that rotate with me, I teach them how to put in orders and will try to give them the opportunity to see consults on their own or call consults on our patients. I try to give them tasks to do at the beginning of the rotation, and then slowly take a step back as the weeks go by to see if they can continue to do those tasks without being asked. That way I'm not hand-holding them but they aren't also just plopped there with no direction. As a med student I hated feeling useless and just a piece of decoration, but I also didn't know what my residents expected of me either because their expectations were always vague. So I want my med students to know exactly the bare minimum requirement to pass my rotation, and then build on that foundation to see if they can excel to a HP or H level.

This system only works is everyone agrees to abide by it. Ultimately it comes down to is that some residents will just want to do there job and leave, and don't want to fuss with having a student slow them down.

 

[Redpancreas]

I'm going to push back just a bit here. Let me know what you think (and whoever else is reading for that matter). Currently, I don't think there is a good way to distinguish student's capacities in a clinical setting BETWEEN schools. Conversely, a lot of effort has been done to stratify students within a specific school. While I do agree very much that we need to reward students who perform well, but in relation to what? In relation to their peers or in relation to the highest capacity given the role as an M3. I think taking the latter approach might be the better option.

So let's say residency programs know that for school X, grading is based on their objective capabilities as an M3, and less on how well they do compared to other students in the class. Thus, PDs know that honors will mean "an excellent candidate" objectively. This allows for the possibility that a majority of the class to be "excellent interns" AND be appropriately labeled as such come residency application. Conversely, if percentages remain ingrained in the system, there will be a greater amount of false negatives (i.e. students who actually will be excellent interns objectively, but got a HP or P instead due to their class' abundance in quality students).

I would imagine this would make the lives of PDs filtering the applicants easier no? A simple "honors" = "objectively excellent clinical skills", rather than having to understand the deeper meaning behind these letter grades. A histogram might be beneficial, but that's still assuming we're sticking with the old system.

Now, with all that said, the key here is to clearly and in great detail elucidate to students the guidelines that allow students to get H, HP or P. Make sure students know very well what is expected of them through their M3 year if they want a honors.

Push back is welcome! I agree there is not much to distinguish capacities between schools, however, that does not mean we should not be transparent about distributions at schools. I do think people look at these histograms to see what honors means and think the more data the better. If there was a way to reward students to an established standard (i.e OSCE P/F) and not place gradations that would be great but then how would people select for residency over others? If we designate a vast majority of people as "excellent candidates" then everyone's going to apply to competitive fields thinking they've earnt it. Basically it will come down to school rank/board scores which are both less relevant than clinical skills. I do agree that the system to measure that's insanely broken right now. Honestly, we can take this further but we basically want the same thing which is a more precise measure of clinical ability.

 

[Qwerty122]

Push back is welcome! I agree there is not much to distinguish capacities between schools, however, that does not mean we should not be transparent about distributions at schools. I do think people look at these histograms to see what honors means and think the more data the better. If there was a way to reward students to an established standard (i.e OSCE P/F) and not place gradations that would be great but then how would people select for residency over others? If we designate a vast majority of people as "excellent candidates" then everyone's going to apply to competitive fields thinking they've earnt it. Basically it will come down to school rank/board scores which are both less relevant than clinical skills. I do agree that the system to measure that's insanely broken right now. Honestly, we can take this further but we basically want the same thing which is a more precise measure of clinical ability.

I suppose we can still include histograms, as it won't really have major consequences and it will help assess the general performance of the student relative to his/her class. But more importantly, I am not too confident in assuming that we will have an overabundance of "excellent candidates" which will make the competitive fields saturated. In fact, there is a chance that the amount of honors given per year will drop, if a large portion of the student body showed that objectively, they are not performing at the level "of an intern". I don't think this proposal will have consequences to the extent of step 1's p/f announcement, which completely nulls the importance of that exam for student stratification purposes.

Let's say the detailed rubric for honors is something like:
- Can be trusted to see consults alone
- Will follow up on labs and orders
- Can present clearly and articulately
- Will talk to the overnight nurse for updates
- Etc. (can be drawn from finderfee5's clinical guide, which is a fantastic thread btw)

Let's say that 25% is able to check off these boxes. Then in that case, wouldn't it be unwise to only reward 15% of these students with honors? Moreover, if only 4% of students in the class can fulfill this rubric, then it would be unfair to reward honors to the 11% (15-4) that really didn't earn it. This is to prevent evaluations to be based on unpredictable gestalt of the evaluator, which I feel most on this site would want to avoid.

There will remain stratification, enough to have it have meaning I would like to think, but now there exists a stronger, more objective basis for that stratification. At least, that's what I'm thinking right now.

 

[Walter_Cronkite]

I think things like this are what helps. Having 1) extracurricular activities and 2) having those activities be something you can talk passionately about (on the application and in person).

Having reviewed applications for the past decade or so, most applicants fit in a very tight bell curve. There is little that differentiates them from a clinical or knowledge based standpoint. The step was one thing that kinda helped, but it only helped on the extremes (280... wow! or 182... yikes!). The thing that actually differentiates candidates are the things they do outside of showing up to the hospital. Did they get a higher degree? Did they significantly contribute to a project? Are they involved in leadership? (etc.). These are they things that make for standout candidates versus the run of the mill. When it comes down to rank list, the run of the mill candidates are forgettable and just get stuck in the middle. The standout candidates with interesting things to say or talk about are the ones to a memorable and stay at the top. Of course, there's always the candidates who are memorable for the wrong reasons... they get put at the bottom, or more typically get DNR'ed.

Are EC’s really that important? I feel like there’s still a pretty big difference between a clinically strong student who understands medicine and one that mostly relies on rote memorization and scripts, with the prior being more successful in achieving high clerkship evals, exceptional LORs and step 2 CK scores. Or is this just a naive MS1’s way of thinking?

 

[Qwerty122]

Are EC’s really that important? I feel like there’s still a pretty big difference between a clinically strong student who understands medicine and one that mostly relies on rote memorization and scripts, with the prior being more successful in achieving high clerkship evals, exceptional LORs and step 2 CK scores. Or is this just a naive MS1’s way of thinking?

You're right, evals, ck, and lors will and should remain king. Its just what happens when you receive 10 apps with similar evals, cks, and lors. You would look deeper to the other qualities of the applicant made apparent in their ECs like their leadership, or desire to serve the undeserved, etc.

 

[Walter_Cronkite]

You're right, evals, ck, and lors will and should remain king. Its just what happens when you receive 10 apps with similar evals, cks, and lors. You would look deeper to the other qualities of the applicant made apparent in their ECs like their leadership, or desire to serve the undeserved, etc.

So it becomes a question of whether it's more worthwhile to spend time actually learning medicine vs. volunteering, contributing to student orgs, running in student government etc.

Problem is it seems like you can never know enough medicine, so for the sake of being a good doctor and doing right by your patients, wouldn't the best thing to do be to prioritize learning medicine?

 

[Qwerty122]

So it becomes a question of whether it's more worthwhile to spend time actually learning medicine vs. volunteering, contributing to student orgs, running in student government etc.

Nah, when you compare those two, 100% of the time there will always be one winner, which is to spend time learning medicine. But I think from the perspective of a pd, especially for competitive specialties, you have a sea of 250/260s and 6/6 honors, so letters become more important and ecs become more important in distinguishing applicants. I'm not saying I like this system of pushing students to be more competitive by having more ecs on top of studying, but that seems to be the reality and the game we have to play (even more so for competitive specialties or top 10 residencies).

 

[Walter_Cronkite]

Nah, when you compare those two, 100% of the time there will always be one winner, which is to spend time learning medicine. But I think from the perspective of a pd, especially for competitive specialties, you have a sea of 250/260s and 6/6 honors, so letters become more important and ecs become more important in distinguishing applicants. I'm not saying I like this system of pushing students to be more competitive by having more ecs on top of studying, but that seems to be the reality and the game we have to play (even more so for competitive specialties or top 10 residencies).

Ok great! Just wanted to make sure I wasn't screwing my chances by not doing many EC's lol!

I know on SDN a 260+ 6/6 honors applicant is a dime in a dozen but I feel like in real life it's still pretty rare. And even if they were commonplace, couldn't you still distinguish applicants by publications, interview vibes or school rank?

 

[Qwerty122]

Ok great! Just wanted to make sure I wasn't screwing my chances by not doing many EC's lol!

I know on SDN a 260+ 6/6 honors applicant is a dime in a dozen but I feel like in real life it's still pretty rare. And even if they were commonplace, couldn't you still distinguish applicants by publications, interview vibes or school rank?

Yep, those are all valid factors to consider as well. I think after grades, the importance of the other stuff from ascending to descending is interview vibes, pubs, LORs, ECs, school rank. So many people do fine and match by having good interviews. But there are instances where so many applicants have good interviews, so the pd has to look at the next important thing down the list, which can be pubs, lors, etc. But honestly, grades and evals still are very important, you can publish 15 papers or organize a mission trip but if you have a 220, you'll still be looked at less favorably (at least i think so) than a 260 with 1-2 pubs.

 

[SurfingDoctor]

Are EC’s really that important? I feel like there’s still a pretty big difference between a clinically strong student who understands medicine and one that mostly relies on rote memorization and scripts, with the prior being more successful in achieving high clerkship evals, exceptional LORs and step 2 CK scores. Or is this just a naive MS1’s way of thinking?

"Clinically strong" is about as subjective as one can get. You must also realize that a vast majority of students fall in a bell curve that is very tight. I mean, that's kinda by design. The biggest hurdle from a knowledge and aptitude standpoint is entry into medical school. Once you've made it that far, the variance across knowledge base is pretty small. Sure, there are some superstars... and some relative duds, but a vast majority of students are just kinda all the same. This becomes increasingly obvious when you read LORs and all of them, and yes I mean all of them, say "This is student is exceptional. I would rank them in the top (insert 1, 5 or 10%) of all students I have ever encountered". I mean, it's mostly fluff. I check LORs for the length and the grammar and that's about it. The content is rather meaningless (though a personal anecdote is always nice), but most professors have boilerplate LORs that are fill in the blank.

Step 1 scores I always thought were helpful, but they are going away, so there's that. I've never found Step 2 to be very helpful, unless the score went down in comparison to Step 1.

So what you are left, on paper, is a bunch of applications that from a "clinical" standpoint that look more or less the same. So you have to differentiate yourself from the pack somehow and generally speaking, a "clinical" metric isn't it.

 

[Walter_Cronkite]

"Clinically strong" is about as subjective as one can get. You must also realize that a vast majority of students fall in a bell curve that is very tight. I mean, that's kinda by design. The biggest hurdle from a knowledge and aptitude standpoint is entry into medical school. Once you've made it that far, the variance across knowledge base is pretty small. Sure, there are some superstars... and some relative duds, but a vast majority of students are just kinda all the same. This becomes increasingly obvious when you read LORs and all of them, and yes I mean all of them, say "This is student is exceptional. I would rank them in the top (insert 1, 5 or 10%) of all students I have ever encountered". I mean, it's mostly fluff. I check LORs for the length and the grammar and that's about it. The content is rather meaningless (though a personal anecdote is always nice), but most professors have boilerplate LORs that are fill in the blank.

Step 1 scores I always thought were helpful, but they are going away, so there's that. I've never found Step 2 to be very helpful, unless the score went down in comparison to Step 1.

So what you are left, on paper, is a bunch of applications that from a "clinical" standpoint that look more or less the same. So you have to differentiate yourself from the pack somehow and generally speaking, a "clinical" metric isn't it.

I see where you're coming from. Even if they're doing away with objective grading criteria, wouldn't being clinically strong influence your mentor's willingness to vouch for you (such as calling other PD's and raving on your behalf?). Or is the whole "life runs on phone calls" saying also a myth?

Even if clinical acumen is becoming a less important factor, wouldn't research and publications still be more important than EC's considering they provide a way for you to distinguish yourself while also forming connections with faculty (something that is harder to get through EC's).

 

[SurfingDoctor]

I see where you're coming from. Even if they're doing away with objective grading criteria, wouldn't being clinically strong influence your mentor's willingness to vouch for you (such as calling other PD's and raving on your behalf?). Or is the whole "life runs on phone calls" saying also a myth?

And even if clinical acumen is becoming a less important factor, wouldn't research and publications still be more important than EC's considering they provide a way for you to distinguish yourself while also forming connections with faculty (something that is harder to get through EC's).

I mean, sure if you know someone with connections and they know you well, are willing to pick up a phone and call someone for you, that helps. That's literally how getting a job works. Does it move you to the top of the pile come ranking time. In my experience, no.

The behind the scenes so to speak for rank lists (granted I'll be talking more about fellowship and less about residency, cause I only do residency interviews and candidate evaluations but don't sit in the rank list meetings, but do all of the above for fellowship) is that a group of core people sit in the room and run through the list. There is a bunch of randomness in my experience of where exactly people get placed. Usually there's a list generated on composite scores. Then people discuss if those composite scores actually match what the group believes is the right order. People move up, down, nowhere. This goes on an on. Occasionally, it will come up, that "PD called me and I know them and they really like this person". My usually response to that however is "Alright". I mean, I don't know that PD so their opinion doesn't hold a lot of water for me, especially if I didn't have a favorable opinion.

When I'm talking about ECs, I'm including research and publications in ECs. Those are extracurricular. Now, there are medical schools (including in the one I'm at currently) that require mandatory research blocks of several months. While this probably wouldn't be considered an EC because it's mandatory, most other programs don't know that, so it looks like EC research. But irrespective of that, the point being, that if you do something that isn't just studying or showing up to the wards, that looks better. If you can then have physical documentation of that extra work (pilot grant, abstract, or better yet, a paper), then that ups it even further. Then when you go to interview, you talk about all the amazing things you did (outside of studying and showing up to work) and tell them how you invision you are going to use those extra things you did to build on the successes you will have in the future. Actually, as this is a bit of an aside, but many candidates actually hurt themselves in the interview by not doing the latter and assuming that their application will speak for itself and but just showing up to the interview, they're in. But in reality, that's not how it works at all.

 

[Putkernerinthehall]

Question from an outsider looking in ---- with all of the discussion above about clinical grades (H / HP / P), how would residency directors view students coming out of a medical school that is P/F for both pre-clinical and clerkships ? Are those students able to adequately distinguish themselves ?

 

[Redpancreas]

Question from an outsider looking in ---- with all of the discussion above about clinical grades (H / HP / P), how would residency directors view students coming out of a medical school that is P/F for both pre-clinical and clerkships ? Are those students able to adequately distinguish themselves ?

The only schools that can afford to do that are the ones with such a high reputation already that it wouldn’t matter. If I’m wrong and such a school exists I would be curious to know which school.

 

[Lost in Translation]

This system only works is everyone agrees to abide by it. Ultimately it comes down to is that some residents will just want to do there job and leave, and don't want to fuss with having a student slow them down.

Which IMO is total **** on part of the resident. You (the resident) were once in the shoes of a med student not too long ago. Surely there were times you (again, the resident) felt left out or like just another decorative piece. You didn't want to be a burden so you didn't talk to the resident. You had no idea what you were doing but saw that the resident looked busy so you didn't talk to the resident. How did those times make you feel?

For me, I felt like the resident could have been more proactive in including me as part of the team and caring about my education. And that's how I currently practice: I actively include the students that rotate with me. If I have some free time I try to do a quick 5-10 minute powerpoint lecture on a high-yield topic relevant to their rotation. I give them responsibility like pulling drains, following up labs, working on discharge summaries, writing notes, putting in orders. And they have all told me, "Lost in Translation, I've learned more in 2 weeks working with you than I have in 2 months of 3rd year." I hate that the onus is on the student to be involved; they have zero real responsibility so that don't have the agency to try and insert themselves into the care team. So we residents need to pull them in. I tell my students that the reason why I give them stuff to do is because I trust that they can do it. If I didn't think they could handle that responsibility, I wouldn't give them anything to do.

To address the "slowing down" issue: if you the resident feels like having a student slows you down then that means you, the resident, aren't fast enough (clinic is a different story). For example, on busy services 25-30 patients long I can get all the daily notes done by myself in less than 2 hours, more often like 90 minutes. Yes, having a student do a couple of them may add some time, but I'm okay with that because if it does end up taking that much longer I can bang out those notes in 2 minutes. I would rather give the med student the opportunity to contribute to the team than just have them sit there and read/study. It makes them feel included, gives them a taste of what they'll do in a couple years, and (if I have a stellar student) cuts down on my workload, not that I really care about that last part (unless it comes to wound vac changes; I don't really like doing wound vac changes).

The only schools that can afford to do that are the ones with such a high reputation already that it wouldn’t matter. If I’m wrong and such a school exists I would be curious to know which school.

I know for a fact that Duke is pure P/F all 4 years. I think UAB is that way as well.

 

[Qwerty122]

@Lost in Translation I hope you're a resident in my institution lmao.

What you said is very true, and that goes back to the point again of ambiguous expectations of the relationship and responsibilities of a medical student and the resident/attendings in that clerkship. What ends up happening is "guess-work" on the student's part trying to figure out what to do for each preceptor, which I think is just a big unnecessary hurdle for both the student and preceptor.

I think that's also why sometimes students might want to err on the side of caution and not overburden the preceptors by remaining passive, rather than taking an active role and involving themselves in the team. There is that element of unknown that presents itself when students try to impose themselves and help out. There is a fear of judgement, doing the wrong thing, overstepping beyond their bounds, and worst, to cause any errors that may reduce the quality of care for patients.

 

[Redpancreas]

Which IMO is total **** on part of the resident. You (the resident) were once in the shoes of a med student not too long ago. Surely there were times you (again, the resident) felt left out or like just another decorative piece. You didn't want to be a burden so you didn't talk to the resident. You had no idea what you were doing but saw that the resident looked busy so you didn't talk to the resident. How did those times make you feel?

For me, I felt like the resident could have been more proactive in including me as part of the team and caring about my education. And that's how I currently practice: I actively include the students that rotate with me. If I have some free time I try to do a quick 5-10 minute powerpoint lecture on a high-yield topic relevant to their rotation. I give them responsibility like pulling drains, following up labs, working on discharge summaries, writing notes, putting in orders. And they have all told me, "Lost in Translation, I've learned more in 2 weeks working with you than I have in 2 months of 3rd year." I hate that the onus is on the student to be involved; they have zero real responsibility so that don't have the agency to try and insert themselves into the care team. So we residents need to pull them in. I tell my students that the reason why I give them stuff to do is because I trust that they can do it. If I didn't think they could handle that responsibility, I wouldn't give them anything to do.

To address the "slowing down" issue: if you the resident feels like having a student slows you down then that means you, the resident, aren't fast enough (clinic is a different story). For example, on busy services 25-30 patients long I can get all the daily notes done by myself in less than 2 hours, more often like 90 minutes. Yes, having a student do a couple of them may add some time, but I'm okay with that because if it does end up taking that much longer I can bang out those notes in 2 minutes. I would rather give the med student the opportunity to contribute to the team than just have them sit there and read/study. It makes them feel included, gives them a taste of what they'll do in a couple years, and (if I have a stellar student) cuts down on my workload, not that I really care about that last part (unless it comes to wound vac changes; I don't really like doing wound vac changes).



I know for a fact that Duke is pure P/F all 4 years. I think UAB is that way as well.

Those are top schools with UAB being regionally well known.

 

[Lost in Translation]

Those are top schools with UAB being regionally well known.

Oh, I thought you were just asking for examples of schools that followed a true 4-year P/F grading system.

 

[Redpancreas]

I think as residents we don't do a good enough job involving the med student in clinical responsibilities.

Each resident does their own thing. Some are great, some are not. The same goes for attendings many of whom fail to promote an educational environment. Until the system is adjusted to actually create incentives for incorporating medical students, nothing will change. There will always be some "overachievers" who get spread too thin.

 

[Putkernerinthehall]

The only schools that can afford to do that are the ones with such a high reputation already that it wouldn’t matter. If I’m wrong and such a school exists I would be curious to know which school.

Virginia Tech Carilion. P/F all four years. Curious how P/F for clerkships is viewed and your thoughts.

 

[Redpancreas]

Virginia Tech Carilion. P/F all four years. Curious how P/F for clerkships is viewed and your thoughts.

Very interesting. I looked at the match lists for IM (area I'm familiar with) and they are consistent with low/midtier MD schools. I have no idea how they get away with that but I guess they proved me wrong.

 

[Dantes]

Clinical grading is a complete and utter mess. If you even look at our school's evaluations, the descriptors from the lowest to the highest rating on evals are completely subjective, like "great fund of knowledge" or "very organized." Either find a way to make it more objective by giving goals and benchmarks to strive for like one of the earlier posts was mentioning, or make it P/F and place more emphasis on sub-Is.

 

[Putkernerinthehall]

Very interesting. I looked at the match lists for IM (area I'm familiar with) and they are consistent with low/midtier MD schools. I have no idea how they get away with that but I guess they proved me wrong.

Virginia Tech is a small school but as noted they manage to place folks in pretty strong residencies even with the P/F clerkships. Maybe even "punching above their weight" ? An interesting model that seems to be working. I'd be interested in others' thoughts on this move away from graded clerkships.

 

[Redpancreas]

Virginia Tech is a small school but as noted they manage to place folks in pretty strong residencies even with the P/F clerkships. Maybe even "punching above their weight" ? An interesting model that seems to be working. I'd be interested in others' thoughts on this move away from graded clerkships.

I wouldn't say punching above their weight. I've looked a lots of match lists across MD schools. VT seems to come right in with the majority of low/midtier MD schools. The key in IM is to assess the majority of applicants. There are always a handful of top IM matches at each MD school regardless of tier (UNC, UTSW, Duke, Icahn, Baylor - 5 here). The majority of of those IM matches seem to be low (11) to midtier (9) academic which is fairly typical of low/mid tier MD schools.

I agree, the model's working though better than I would have dreamed one would. I wonder what's going to happen when Step 1 is gone. Definitely a school to keep an eye on.

 

[Putkernerinthehall]

I wouldn't say punching above their weight. I've looked a lots of match lists across MD schools. VT seems to come right in with the majority of low/midtier MD schools. The key in IM is to assess the majority of applicants. There are always a handful of top IM matches at each MD school regardless of tier (UNC, UTSW, Duke, Icahn, Baylor - 5 here). The majority of of those IM matches seem to be low (11) to midtier (9) academic which is fairly typical of low/mid tier MD schools.

I agree, the model's working though better than I would have dreamed one would. I wonder what's going to happen when Step 1 is gone. Definitely a school to keep an eye on.

Logically, I don't think Step 1 going P/F will matter that much as many schools are already P/F grading (like Virginia Tech) for pre-clinical and all schools will be P/F for Step 1, creating a pretty level playing field. Step 2 CK, however, would become pretty critical for students at places like Virginia Tech that have P/F clerkships. Am I reading this right ?

 

[7331poas]

Grading on clerkship is mostly BS anyway. Half the day I just stand around waiting to be dismissed. How can you really grade that? Most services have no need for a student outside of IM and surg.

 

[Gilakend]

I think it should be P/F personally.

 

[Putkernerinthehall]

I think it should be P/F personally.

Why ?

 

[Gilakend]

Why ?

Like others said, it's largely pointless. There is no way of comparing two people from different schools based on clinical grades. Also with most schools having cut off percentages for honors, it incentivizes students to only be engaged enough clinically in order to get decent evals, then trying to squeeze in studying. It also makes for a pretty poor clinical year lifestyle.

I don't necessarily have a better system, but P/F is a good start imo. I wouldn't be opposed to some form of LOR/eval system where you choose/ask people to write narratives about you that are either looked at or compiled in some way. If I was a residency director I'd much rather have legitimate narrative assessments than a random grade I don't know how to interpret.

I think you should still have to pass the shelf, but I don't think doing great on a shelf should put you ahead of someone who integrated better into the team or outperformed you clinically. I think it would allow people to be way better prepared for residency, let them learn more, and be less stressed.

 

[TelemarketingEnigma]

I wouldn't be opposed to some form of LOR/eval system where you choose/ask people to write narratives about you that are either looked at or compiled in some way. If I was a residency director I'd much rather have legitimate narrative assessments than a random grade I don't know how to interpret.

wait is this not a widespread thing? We get a numerical grade (no honors) + a clerkship narrative that summarizes all the evals from our different preceptors. I'm told those get put in our MSPE, at least in part. I'm sure most narratives are relatively similar but there's definitely individual comments included in some parts.

 

[Lawpy]

With step 1 being pass/fail, emphasis now shifts to step 2 and the clinical years as the main quantitative metric for a student's candidacy for residency. I've heard that many students complain about clinical grading (subjectivity, generic comments, unclear expectations from preceptors, everyone gets a 3/5, etc.). So what are some things you school did during clinical years that you think are commendable and should continue to be implemented/improved upon? Having a "well-oiled" system for clinical years will only continue to grow in importance in the coming years. I'm fortunate enough to have a student leadership role in my school at directing the clinical curriculum, so I'm hoping to draw from the collective knowledge and experience of sdn to move forward. Thanks all!

There is no easy way to improve without fundamentally changing the way how clinical years are done. Some sites allow students to learn a lot and are graded fairly. Other sites screw students over by pairing them with brutal 3 bombers or pamper students with glorified shadowing and cush rotations with fun and generous 5 givers. There is too much variability within a school that's severely worsened when comparing with all schools. I don't think P/F helps because it removes another metric on something that actually is important to PDs.

 

[Gilakend]

wait is this not a widespread thing? We get a numerical grade (no honors) + a clerkship narrative that summarizes all the evals from our different preceptors. I'm told those get put in our MSPE, at least in part. I'm sure most narratives are relatively similar but there's definitely individual comments included in some parts.

It's similar here, but currently, I feel like what matters is the grade, not the comments, and since people know this most comments (unless it's a rotation you're really trying to impress on) are usually pretty generic and don't hold a lot of weight. i.e. I imagine most people would rather get an Honors with good/typical comments than a Pass with great comments.