Why should 3rd year clerkships be pass/fail?

[Sconey]

Hey, SDN - my med school is currently debating making 3rd year pass/fail. At our institution, 3rd year honors/high pass/pass/fail is almost exclusively determined by shelf scores, though theoretically they're only 20-30 percent of our grades. The issue the school has been wrestling with is how to make 3rd year grades more clinically-focused and less standardized test-focused. Most students tend to do about the same on the other 80% of their clerkship grades - the clinical portion - which means right now it all comes down the shelf tests.

My question to you is: how has this worked at other schools that have pass/fail 3rd year? Some students (mainly those pursuing especially competitive residencies) are aghast at this idea because they feel they'll lose competitiveness when applying for residencies. I can understand that. Some former residency directors at our institution are honest enough to say they're not sure how they would approach students who went through our program under a pass/fail system.

I'm not looking for arguments about why pass/fail 3rd year would be a bad idea. I get those. I want to know why it's a good idea, and what other changes my school would need to do to make it work.

---

Members don't see this ad.

 

[maxxor]

They would need to really beef up their evaluations and training system to find a way to generate lots of in-depth feedback on you that can go into your dean's letter.

Eg. Have med students rotate with the same people for longer, so they can write better evaluations. Don't just throw them into random clinics for a morning or afternoon with different people.

 

[DermViser]

Depends, where do you rank in your class right now? God forbid, people be on clerkships to learn and contribute to the team, rather than gunning to ace the NBME shelf exam. Yale and Stanford are graded Pass/Fail in their clerkships and they seem to do just fine.

 

[ChEMD]

Depends, where do you rank in your class right now? God forbid, people be on clerkships to learn and contribute to the team, rather than gunning to ace the NBME shelf exam. Yale and Stanford are graded Pass/Fail in their clerkships and they seem to do just fine.

But their names are Yale and Stanford. I wonder how students from lesser known schools would do with a completely pass/fail curriculum.

 

[DermViser]

But their names are Yale and Stanford. I wonder how students from lesser known schools would do with a completely pass/fail curriculum.

I agree, I think a lot of the reason why schools can do this is that they can rely on school name. Hence why I always laugh when people say your medical school name doesn't matter. No, it does. I'm just wondering how a student can be punished for the grading policy of his school, and I just can't see that as he/she is not in control of policy.

 

[mdeast]

Actually both Stanford and Yale have Honors during third year clerkships. Not sure exactly how it works at Yale...but Stanford used the RIME (Reporter, Interpreter, Manager, and Educator) framework. Grades (Honors/Pass/Marginal Pass/Fail) are assigned in three areas (1) Patient Care (2) Professionalism and (3) Shelf Exam..for all required core clerkships (Medicine, Surgery, Pediatrics, OB-Gyn, Psychiatry, Neurology, Outpatient Medicine, Intensive Care Unit, and Family Medicine). There are criteria by which a certain % of evaluators have to report functioning at a certain level on a rotation, taking into account time of evaluation (i.e. beginning or end of rotation). Getting Honors is not easy. I'd say most clerkships give out 3/3 Honors (in all evaluation areas) to only about 10% of students (or 8-10 students in a class of a 86). Electives and Sub-I's are still P/F. The good (?) thing is that residency programs have no idea how to interpret our grading system so it is still fundamentally P/F.

 

[Sconey]

Maxxor, definitely agree about needing to spend more time with people to get more worthwhile evaluations.

Mdeast, in Stanford's system, if a student got an honors in patient care, a pass in professionalism and a marginal pass on the shelf, would they end up with a pass for that clerkship? So the results in each of those are sort of averaged? Do most students feel like they get enough appropriate evaluations from people to produce an accurate picture of their performance? Do many students do poorly in professionalism and patient care? Maybe the issue we have is that, unless you show up naked or something, you're not likely to fail at "professionalism" or "patient care." They just don't discriminate enough. It's much easier for students to do poorly on a shelf and then that hijacks their grade.

 

[mvenus929]

I *think* VT has P/F during third year, though they compete for 'letters of distinction' or something. At least, that's what I was told when I was rotating with them on Surgery. I'm not sure why they made the decision, though. The real question would be how would the histograms in the Dean's Letter be affected. If it's just straight P vs F, that's loads different than assigning a numerical grade and then reporting it as a pass.

 

[Psai]

Honestly I think that getting into a residency shouldn't be a competition. I understand that we can't have a million psychologists and no neurosurgeons but I wish it was more of "do what you like" rather than "do what your step 1 score allows you to do".

 

[sobored]

Depends, where do you rank in your class right now? God forbid, people be on clerkships to learn and contribute to the team, rather than gunning to ace the NBME shelf exam. Yale and Stanford are graded Pass/Fail in their clerkships and they seem to do just fine.

I agree with your sentiment, but let's be honest, the system is kind of designed in a way that favors those who are gunning to ace the NBME shelf. Sad, but true nonetheless.

 

[JJMrK]

TBH, I don't think there are any good reasons for a P/F system. PDs expect grades, and when a program has a "# of clinical Hs" or "H in X specialty" cut-off for interview, there's no possible way to fulfill that criteria in a P/F system.

On the other hand, if you're content with all Ps, that's sill a viable option in an H/P/F system.

 

[PL198]

TBH, I don't think there are any good reasons for a P/F system. PDs expect grades, and when a program has a "# of clinical Hs" or "H in X specialty" cut-off for interview, there's no possible way to fulfill that criteria in a P/F system.

On the other hand, if you're content with all Ps, that's sill a viable option in an H/P/F system.

I always thought the whole P/F was initially created to prevent serious gunning. I'm not going to be naive and say that students don't still do detrimental things to other students, however I always assumed it would reduce serious offenses. And then basically the internal ranking was to still have the exact same ways to measure the students and attempt to fool the students into not gunning.

 

[NickNaylor]

TBH, I don't think there are any good reasons for a P/F system. PDs expect grades, and when a program has a "# of clinical Hs" or "H in X specialty" cut-off for interview, there's no possible way to fulfill that criteria in a P/F system.

On the other hand, if you're content with all Ps, that's sill a viable option in an H/P/F system.

I agree with everything you're saying, but that begs the question: are the PDs worrying about something that may not have any actual relation to clinical ability or performance?

In other words, is the person getting a 250+ on step 1, honors on every rotation, AOA, and even a history of fellating the president of the AAD going to be a much better dermatologist than the person who gets a 230, HP, and a respectable but otherwise not extraordinary CV? I argue no. Yet because of the inherently limited supply in fields that are desirable WRT lifestyle considerations, discrimination has to be made based on some kind of criteria. Those happen to be the things available to PDs. Whether that discrimination actually matters when it comes to clinical performance... I'm skeptical.


Sent from my iPhone using Tapatalk

 

[Wordead]

I agree with everything you're saying, but that begs the question: are the PDs worrying about something that may not have any actual relation to clinical ability or performance?

In other words, is the person getting a 250+ on step 1, honors on every rotation, AOA, and even a history of fellating the president of the AAD going to be a much better dermatologist than the person who gets a 230, HP, and a respectable but otherwise not extraordinary CV? I argue no. Yet because of the inherently limited supply in fields that are desirable WRT lifestyle considerations, discrimination has to be made based on some kind of criteria. Those happen to be the things available to PDs. Whether that discrimination actually matters when it comes to clinical performance... I'm skeptical.


Sent from my iPhone using Tapatalk

I'd say "clinical performance" is so vague and useless that I'd guess those evals are an even worse predictor of whether someone will be a "good" doctor. If you do get honors in every rotation, though, I'd call that damn impressive and a good indicator that you'll be good in any healthcare setting. But poor evals don't necessarily mean anything in my opinion.

 

[KnuxNole]

Keep in mind for quite a few attendings, their comments in eval equal one sentence, with some people 2-3 words. It's very rare for people to write paragraphs of feedback.

Which makes it hard to know how you really did.

 

[PL198]

I agree with everything you're saying, but that begs the question: are the PDs worrying about something that may not have any actual relation to clinical ability or performance?

In other words, is the person getting a 250+ on step 1, honors on every rotation, AOA, and even a history of fellating the president of the AAD going to be a much better dermatologist than the person who gets a 230, HP, and a respectable but otherwise not extraordinary CV? I argue no. Yet because of the inherently limited supply in fields that are desirable WRT lifestyle considerations, discrimination has to be made based on some kind of criteria. Those happen to be the things available to PDs. Whether that discrimination actually matters when it comes to clinical performance... I'm skeptical.


Sent from my iPhone using Tapatalk

What could possibly be more valid? It's not like there is some arbitrary policy used for medicine. Grades + standardized scores + resume is what is used by everything. I'm not sure I understand what you are saying. Maybe more weight to clerkships? I feel like that is way too subjective and the whole "fear of failing someone" prevents clerkship evaluations from being that valid.

 

[NickNaylor]

What could possibly be more valid? It's not like there is some arbitrary policy used for medicine. Grades + standardized scores + resume is what is used by everything. I'm not sure I understand what you are saying. Maybe more weight to clerkships? I feel like that is way too subjective and the whole "fear of failing someone" prevents clerkship evaluations from being that valid.

I'd start with the recommendations DermViser made earlier in this thread: straight P/F for all courses, P/F for the step exams, and an emphasis on qualitative evaluations. But that's just my own opinion, and I understand that I'm likely in the minority here.


Sent from my iPhone using Tapatalk

 

[ChEMD]

What could possibly be more valid? It's not like there is some arbitrary policy used for medicine. Grades + standardized scores + resume is what is used by everything. I'm not sure I understand what you are saying. Maybe more weight to clerkships? I feel like that is way too subjective and the whole "fear of failing someone" prevents clerkship evaluations from being that valid.

I think posing the question: does higher scores and a better CV mean that you'll be a better (insert specialty here)? Obviously specialties with limited spots need ways to decide between applicants, and what better way than a record of how you've performed during medical school. But it's still an interesting question none the less.

 

[Sconey]

Isn't it true that standardized exams (MCAT, Steps, shelf) have never been proven to correlate with any meaningful outcome of medical care, including patient satisfaction scores and patient health (however that is measured)? I was under the impression that the only thing they've even been shown to correlate with is... how those students do on future standardized tests.

 

[PL198]

Isn't it true that standardized exams (MCAT, Steps, shelf) have never been proven to correlate with any meaningful outcome of medical care, including patient satisfaction scores and patient health (however that is measured)? I was under the impression that the only thing they've even been shown to correlate with is... how those students do on future standardized tests.

I mean there have been studies that show patient satisfaction and health don't even correlate if I remember right. Step 1 pretty much tests a student's intelligence and their work ethic, I don't really know what else could be better.

 

[Anastomoses]

I'd start with the recommendations DermViser made earlier in this thread: straight P/F for all courses, P/F for the step exams, and an emphasis on qualitative evaluations. But that's just my own opinion, and I understand that I'm likely in the minority here.

P/F step exams? wtf?

I mean there have been studies that show patient satisfaction and health don't even correlate if I remember right. Step 1 pretty much tests a student's intelligence and their work ethic, I don't really know what else could be better.

I don't know that it tests everything required to be an excellent doctor...in fact, I'm pretty sure it doesn't. But I don't trust a lot of subjective evaluations either.

 

[notbobtrustme]

Depends, where do you rank in your class right now? God forbid, people be on clerkships to learn and contribute to the team, rather than gunning to ace the NBME shelf exam. Yale and Stanford are graded Pass/Fail in their clerkships and they seem to do just fine.

yea, pretty sure you could submit a giant turd instead of ERAS and still match most places if you came from Yale or Stanford.

 

[Sconey]

Yeah, the Step exams were never meant to be used to judge students for residencies. They were essentially pass/fail. But in the same way that social security cards became de facto national IDs for lack of other options, Step exams scores eventually started to matter.

 

[DermViser]

yea, pretty sure you could submit a giant turd instead of ERAS and still match most places if you came from Yale or Stanford.

Depends very much on the specialty, but for the most part, probably. Not surprising, as the student that Stanford or Yale recruits is very different than the rest of the med school population.

Realize places like Yale have the "Yale System": https://medicine.yale.edu/education/admissions/education/yalesystem.aspx, to where they have "optional" exams, no grade, no class rank. So this is a case in which the school really does give the student great leeway in his/her education allowing him/her to excel on the things that matter when the match comes around.

 

[NickNaylor]

P/F step exams? wtf?



I don't know that it tests everything required to be an excellent doctor...in fact, I'm pretty sure it doesn't. But I don't trust a lot of subjective evaluations either.

See the post above. The Steps were never designed - and are still not designed - to be used in the way they are currently used. They were always designed to be P/F exams. Per the NBME, the actual score was only provided "as a convenience." Somewhere along the way, though, someone decided that step 1 was really important, and like good little sheep we went along with it.

Considering that the Steps are used strictly for licensing, does it make sense that higher scores are better from the purpose of the test creator? Can I be more licensed if I get a higher score? Of course not. However, PDs have coopted the exam as a means for comparison even though that is exactly what the test was designed not to be. The SAT, MCAT, etc. were designed from the get-go to be exams to compare test takers to each other. The Steps were not.

I'm not saying this makes the comparisons completely invalid. But I think it's an important point to keep in mind when we think about how valuable these criteria actually are and why on earth someone decided that they were important.

 

[Anastomoses]

See the post above. The Steps were never designed - and are still not designed - to be used in the way they are currently used. They were always designed to be P/F exams. Per the NBME, the actual score was only provided "as a convenience." Somewhere along the way, though, someone decided that step 1 was really important, and like good little sheep we went along with it.

Considering that the Steps are used strictly for licensing, does it make sense that higher scores are better from the purpose of the test creator? Can I be more licensed if I get a higher score? Of course not. However, PDs have coopted the exam as a means for comparison even though that is exactly what the test was designed not to be. The SAT, MCAT, etc. were designed from the get-go to be exams to compare test takers to each other. The Steps were not.

I'm not saying this makes the comparisons completely invalid. But I think it's an important point to keep in mind when we think about how valuable these criteria actually are and why on earth someone decided they were important.

Okay. We can make one for licensing and another for specialization aptitude?

 

[DermViser]

Yeah, the Step exams were never meant to be used to judge students for residencies. They were essentially pass/fail. But in the same way that social security cards became de facto national IDs for lack of other options, Step exams scores eventually started to matter.

That's what dentistry has done with their NBDE exams, although now that has made things a bit more difficult with respect to their fellowship match:
emphasizing more non-standardized metrics: GPA, subjective evaluations, etc.

But yes, the steps were only to be used for licensure purposes only. It's morphed into the ridiculous system we have now. With all these new medical schools popping up, both MD and DO, you can bet Step scores will become even more important as a first pass screening filter, not less, esp. with the USMLE Step 1 average creeping up every year.

 

[PL198]

See the post above. The Steps were never designed - and are still not designed - to be used in the way they are currently used. They were always designed to be P/F exams. Per the NBME, the actual score was only provided "as a convenience." Somewhere along the way, though, someone decided that step 1 was really important, and like good little sheep we went along with it.

Considering that the Steps are used strictly for licensing, does it make sense that higher scores are better from the purpose of the test creator? Can I be more licensed if I get a higher score? Of course not. However, PDs have coopted the exam as a means for comparison even though that is exactly what the test was designed not to be. The SAT, MCAT, etc. were designed from the get-go to be exams to compare test takers to each other. The Steps were not.

I'm not saying this makes the comparisons completely invalid. But I think it's an important point to keep in mind when we think about how valuable these criteria actually are and why on earth someone decided they were important.

That doesn't mean it's ineffective at those additional purposes. Like I said before, I think it measures intelligence + work ethic. Completely agree that those aren't the only things that make/break a doc, however I'm not sure anything else would do better for an overall eval. If you make clerkships more important, there will be way more gunning and it'll just be more about memorizing things out of textbook so you look good when pimped. There would also be more sucking up, and the whole thing would get even more subjective. IMO the best way to take subjectivity out of it is to make it worth less and less.

 

[NickNaylor]

Okay. We can make one for licensing and another for specialization aptitude?

But... why? Why do we need to stratify an already "elite" cohort into further subdivisions? And, by the way - does it even matter? Do certain fields of medicine require smarter people? Are there some people who are just simply too inept to be a certain kind of physician?

You're making suggestions based on a premise that you haven't even demonstrated to be valid or important.

 

[NickNaylor]

That doesn't mean it's ineffective at those additional purposes. Like I said before, I think it measures intelligence + work ethic. Completely agree that those aren't the only things that make/break a doc, however I'm not sure anything else would do better for an overall eval. If you make clerkships more important, there will be way more gunning and it'll just be more about memorizing things out of textbook so you look good when pimped. There would also be more sucking up, and the whole thing would get even more subjective. IMO the best way to take subjectivity out of it is to make it worth less and less.

I pose the same question to you: what's the point in this stratification? Are you proposing that some medical students are too inept and incapable to becoming certain kinds of physicians? If not, then what's the point of the scheme you propose?

 

[dyspareunia]

I agree, I think a lot of the reason why schools can do this is that they can rely on school name. Hence why I always laugh when people say your medical school name doesn't matter. No, it does. I'm just wondering how a student can be punished for the grading policy of his school, and I just can't see that as he/she is not in control of policy.

It may not always be a punishment. For example, a PD gets one applicant from random-unknown school every other year. This school happens to have P/F clerkships, but the PD doesn't know it because the applicant is from a no-name school and the PD rarely sees applicants from that school. It's not unreasonable to say that the PD may assume the no-name school is H/HP/P/F, but the current applicant just happened to only Pass everything. So, while the PD isn't "punishing" the student, the student's school name/grading-system is certainly doing them a disservice.

I pose the same question to you: what's the point in this stratification? Are you proposing that some medical students are too inept and incapable to becoming certain kinds of physicians? If not, then what's the point of the scheme you propose?

Yes, a student who gets less than a 227 (i.e. half of all med students) is incapable of becoming a neurosurgeon, dermatologist, etc. The point of stratification is to give students as equal of a chance as possible. It doesn't matter how you or I feel about the system. What matters is how we play within it.

 

[Anastomoses]

But... why? Why do we need to stratify an already "elite" cohort into further subdivisions? And, by the way - does it even matter? Do certain fields of medicine require smarter people? Are there some people who are just simply too inept to be a certain kind of physician?

You're making suggestions based on a premise that you haven't even demonstrated to be valid or important.

Fair enough, good questions. Obviously there aren't enough slots for everyone to do anything they like. I think those showing the most "excellence" would naturally select first. And again, I allow for the fact that Step 1 or such an exam may not be the perfect indicator of overall excellence.

What do you suggest for divvying out residencies? Mere charm at interviews?

 

[PL198]

I pose the same question to you: what's the point in this stratification? Are you proposing that some medical students are too inept and incapable to becoming certain kinds of physicians? If not, then what's the point of the scheme you propose?

Of course I am. Being a neurosurgeon requires a higher caliber individual than family med on average. Is that a surprise to you? Both on an intellectual level and a career commitment level. If someone can get a 260 on steps, it would signify to me they are capable of putting in the time to excel at their pursuits. If someone gets a 200, they probably aren't trying as hard, and thus are going to have a much harder time trying to be in the OR for 10 hours straight. Making me less likely to want them. <- My thoughts if I was a PD, which I'm not.

Do you really think a neurosurg and a family med doc are the same caliber of student on average? Keep in mind I said on average. Yes I'm sure there is a 270 out there that chooses FM because they love it, and great for them! However this becomes irrelevant on the grand scale.

 

[@JackShephard]

Depends, where do you rank in your class right now? God forbid, people be on clerkships to learn and contribute to the team, rather than gunning to ace the NBME shelf exam. Yale and Stanford are graded Pass/Fail in their clerkships and they seem to do just fine.

Yeah, I think if you have a great group of students it can work.

 

[DermViser]

Actually both Stanford and Yale have Honors during third year clerkships. Not sure exactly how it works at Yale...but Stanford used the RIME (Reporter, Interpreter, Manager, and Educator) framework. Grades (Honors/Pass/Marginal Pass/Fail) are assigned in three areas (1) Patient Care (2) Professionalism and (3) Shelf Exam..for all required core clerkships (Medicine, Surgery, Pediatrics, OB-Gyn, Psychiatry, Neurology, Outpatient Medicine, Intensive Care Unit, and Family Medicine). There are criteria by which a certain % of evaluators have to report functioning at a certain level on a rotation, taking into account time of evaluation (i.e. beginning or end of rotation). Getting Honors is not easy. I'd say most clerkships give out 3/3 Honors (in all evaluation areas) to only about 10% of students (or 8-10 students in a class of a 86). Electives and Sub-I's are still P/F. The good (?) thing is that residency programs have no idea how to interpret our grading system so it is still fundamentally P/F.

The RIME framework is used across many med schools for describing the difference in behaviors to assign grades. This isn't exclusive to just Stanford. That being said I was going based off of this: http://med.stanford.edu/md/curriculum/assessment-grading.html

Edit: The only difference now seems to be a very recent change in which there is "Pass" and "Pass with Distinction"
http://med.stanford.edu/md/curriculum/CBEI/criteria.html

There is no final grade for a clerkship of Honors/Pass/Marginal Pass/Fail at Stanford.

 

[@JackShephard]

Of course I am. Being a neurosurgeon requires a higher caliber individual than family med on average. Is that a surprise to you? Both on an intellectual level and a career commitment level.

I wouldn't say higher caliber individual, that's not accurate. I would say they probably need to be smarter and willing to work all the time.

I guess if that's your definition of a high caliber person then that applies. I know plenty of people that are incredibly intelligent and work obsessed that I would call low caliber people. Likewise, I know some very average level of intelligence individuals that work very little (30-40 hrs a week) who are incredibly high caliber individuals. There are lots of these people outside of medicine who out earn the average neurosurgeon too.

 

[PL198]

I wouldn't say higher caliber individual, that's not accurate. I would say they probably need to be smarter and willing to work all the time.

I guess if that's your definition of a high caliber person then that applies. I know plenty of people that are incredibly intelligent and work obsessed that I would call low caliber people. Likewise, I know some very average level of intelligence individuals that work very little (30-40 hrs a week) who are incredibly high caliber individuals. There are lots of these people outside of medicine who out earn the average neurosurgeon too.

Yes, completely agree. I think caliber of student is more applicable.

 

[DermViser]

Maxxor, definitely agree about needing to spend more time with people to get more worthwhile evaluations.

Mdeast, in Stanford's system, if a student got an honors in patient care, a pass in professionalism and a marginal pass on the shelf, would they end up with a pass for that clerkship? So the results in each of those are sort of averaged? Do most students feel like they get enough appropriate evaluations from people to produce an accurate picture of their performance? Do many students do poorly in professionalism and patient care? Maybe the issue we have is that, unless you show up naked or something, you're not likely to fail at "professionalism" or "patient care." They just don't discriminate enough. It's much easier for students to do poorly on a shelf and then that hijacks their grade.

Apparently, you don't know how "Professionalism" is graded on clerkships. If you do something I don't like, I can label you as being "unprofessional" and destroy you.

 

[DermViser]

I agree with your sentiment, but let's be honest, the system is kind of designed in a way that favors those who are gunning to ace the NBME shelf. Sad, but true nonetheless.

That's bc medical schools have made it so. There are schools in which the NBME exam has a LOT of sway much > than the subjective evaluation and there are other schools in which NBME exam makes a very little percentage of the final grade calculation compared to subjective evals. Both have +s and -s.

 

[mdeast]

Maxxor, definitely agree about needing to spend more time with people to get more worthwhile evaluations.

Mdeast, in Stanford's system, if a student got an honors in patient care, a pass in professionalism and a marginal pass on the shelf, would they end up with a pass for that clerkship? So the results in each of those are sort of averaged? Do most students feel like they get enough appropriate evaluations from people to produce an accurate picture of their performance? Do many students do poorly in professionalism and patient care? Maybe the issue we have is that, unless you show up naked or something, you're not likely to fail at "professionalism" or "patient care." They just don't discriminate enough. It's much easier for students to do poorly on a shelf and then that hijacks their grade.

Few people would get a marginal pass or fail in any clerkship as there are ample options for remediation (i.e. retaking a shelf exam) that negate the original grade. There is no overall grade for each rotation (i.e. scores aren't averaged or something). Clerkship grades are reported as your individual grade in each of the three areas on our Dean's Letter with no overall grade.

In most cases, they make the bar for Honors fairly high. So, it's relatively easy to Pass as long as your dependable and consistent and make an effort...but relatively difficult to get Honors in any of the 3 areas. Fewest people get Honors on the shelf exams (i.e. I think the "highest" clerkship was about 30% and the lowest around 5-10% getting Honors on the shelf). Everyone in my class has different strengths and goals for rotations. I personally concentrated on Patient Care (i.e. RIME framework)...as I felt being able to effectively manage patients(i.e. develop differentials, appropriate diagnostic testing, and logical interventions) is really the goal of medical school. Professionalism was always so subjective and I personally found Shelf Exams complete and utterly useless... other than making sure everyone is exposed to some sort of core knowledge base while they're on their rotation and "proficient" enough to Pass a standardized test.

 

[DermViser]

That doesn't mean it's ineffective at those additional purposes. Like I said before, I think it measures intelligence + work ethic.

And you would be completely wrong.

I know people who destroyed the USMLE Step 1 but when it came to demonstrating clinical knowledge on the wards floundered, bc all of a sudden that patient in front of you is no longer a multiple choice question with 1 correct answer, and no textbook or review text can save you. They floundered on being able to come up with a differential diagnosis and being able to rule things in and out on that differential diagnosis.

 

[PL198]

And you would be completely wrong.

I know people who destroyed the USMLE Step 1 but when it came to demonstrating clinical knowledge on the wards floundered, bc all of a sudden that patient in front of you is no longer a multiple choice question with 1 correct answer, and no textbook or review text can save you. They floundered on being able to come up with a differential diagnosis and being able to rule things in and out on that differential diagnosis.

One(or a few) anecdote(s) doesn't really disprove what I said. Do you feel if you have student A with a 200 and student B with a 240 that they are going to be equal with respect to being able to diagnose?

 

[operaman]

I wish I could remember the reference, but there was a paper out a few years back about p/f in the preclinical curriculum and how student performance on exams was not significantly different after they switched from grades to p/f. My sense is that making clinicals p/f would have the same result. The top students will still be the top students and will work hard; those who strive to do the bare minimum required will continue to do just that.

My personal experience with third year is that the grading system has far less impact on my learning experience than the random selection of residents and attendings I'm with. Get a good group and you learn a ton; get a group comprised of those people who, as med students, did the bare minimum required and you're basically shadowing except that you're ignored more. That variability in educational experience would be my personal target.

So, along with a p/f grading system, I would also add this: there is a saying in business which goes something like, "whatever gets measured, gets done." The big change needed would be something that tied a significant part of residents' overall performance evals to their teaching and mentoring of students. Even better would be tie some part of faculty salary to their time spent directly teaching and mentoring students. Granted, neither of those things will ever happen, but it would be worth mentioning. Measuring it is pretty easy: e-surveys sent out weekly with only 1 question: "On average, how much time per day did Dr. Jones spend directly teaching and/or mentoring you? A) none B) 0-15 minutes C) 15-30 minutes D) 30-60 minutes E) >60 minutes"

Also don't forget the unintended consequence: removing more grades only makes Step scores more important.

 

[DermViser]

One(or a few) anecdote(s) doesn't really disprove what I said. Do you feel if you have student A with a 200 and student B with a 240 that they are going to be equal with respect to being able to diagnose?

Who said they were equal? You were the one who made the claim that it indicates "intelligence + work ethic". You actually think that the ability to answer multiple choice questions on basic science factoids is somehow indicative of how one is able to diagnose? Really? Even Program Directors don't believe that garbage!

 

[DermViser]

Keep in mind for quite a few attendings, their comments in eval equal one sentence, with some people 2-3 words. It's very rare for people to write paragraphs of feedback.

Which makes it hard to know how you really did.

If you look at some MSPEs from some institutions that isn't the case.

 

[@JackShephard]

.

 

[PL198]

Who said they were equal? You were the one who made the claim that it indicates "intelligence + work ethic". You actually think that the ability to answer multiple choice questions on basic science factoids is somehow indicative of how one is able to diagnose? Really? Even Program Directors don't believe that garbage!

I'm not sure what else is more likely to indicate success. Kissing up to your attending surely isn't IMO.

 

[DermViser]

Isn't it true that standardized exams (MCAT, Steps, shelf) have never been proven to correlate with any meaningful outcome of medical care, including patient satisfaction scores and patient health (however that is measured)? I was under the impression that the only thing they've even been shown to correlate with is... how those students do on future standardized tests.

In other words, is the person getting a 250+ on step 1, honors on every rotation, AOA, and even a history of fellating the president of the AAD going to be a much better dermatologist than the person who gets a 230, HP, and a respectable but otherwise not extraordinary CV? I argue no. Yet because of the inherently limited supply in fields that are desirable WRT lifestyle considerations, discrimination has to be made based on some kind of criteria. Those happen to be the things available to PDs. Whether that discrimination actually matters when it comes to clinical performance... I'm skeptical.

Bingo. No PD actually believes that Step scores are a metric of residency clinical performance. If that was the case, then Carribean grads would be vaulted to the top. PDs know that Carribean medical schools game the system by teaching very closely to the boards for 2 straight years. This is a well-known fact.

See here: http://www.im.org/Meetings/Past/2012/2012APDIMSpringConference/Presentations/Documents/PA Meeting/Wksp 203_Adams.pdf

 

[DermViser]

I'm not sure what else is more likely to indicate success. Kissing up to your attending surely isn't IMO.

Who said that anything else is more likely to indicate success? They are metrics like anything else. Medical school admissions is the same way.

Why you think that somehow a numeric metric like USMLE Step 1 has this clairvoyant ability to predict clinical residency performance, when the NBME writers and PDs don't make this assertion or believe this is beyond me. It's very child-like thinking, TBH.

 

[PL198]

Who said that anything else is more likely to indicate success? They are metrics like anything else. Medical school admissions is the same way.

Why you think that somehow a numeric metric like USMLE Step 1 has this clairvoyant ability to predict clinical residency performance, when the NBME writers and PDs don't make this assertion or believe this is beyond me. It's very child-like thinking, TBH.

I'm not saying it's going to predict it, just that it's better than anything else and thus the likely choice. I'm sure like you said there are the bookworms who 260 it all the way and then derp hard when they see a real patient. However, rather hopefully I'd assume this isn't the majority.