MD & DO USMLE will eventually become P/F?

[Lawpy]

The rise of Step 1's importance has occurred in conjunction with two other phenomena: (1) the spread of P/F unranked grading systems, and (2) the marked increase in the average number of residency applications. The former has become common because there is a strong desire among institutions to minimize competition between classmates and promote a collaborative learning environment. So in essence we traded students fighting each other with students fighting a common enemy (the USMLE). The latter is multifactorial in origin, but the push by student affairs deans to draw everyone toward the sublime middle hasn't helped.

Standardized preclinical exams would be awful. They would subvert one of the core principles of academia: the faculty is in charge of the curriculum. If every test were a mini-USMLE then I imagine most schools would move toward a "teach-to-the-test" model, which would optimize scores but forgo anything resembling an authentic education. No thanks, we have enough problems with just one monolith.

What are your thoughts on standardizing clinical year grades and making them more objective? Iirc some schools have clinical grades fully determined by shelf exams, but if there's a way to standardize clinical evaluations and make them more objective, that would be better.

---

Members don't see this ad.

 

[Med Ed]

What are your thoughts on standardizing clinical year grades and making them more objective? Iirc some schools have clinical grades fully determined by shelf exams, but if there's a way to standardize clinical evaluations and make them more objective, that would be better.

I do not know of any school using shelf exams exclusively for assigning clinical grades. That approach would undermine student participation in patient care.

If there were an easy way to standardize clinical year grades it would have been implemented long, long ago. As it is we construct elaborate rubrics that are too onerous, and then strip them down to the point where they are ineffective, and then expand them until they are too onerous, and then strip them down to the point where they are ineffective.

 

[darklabel]

The rise of Step 1's importance has occurred in conjunction with two other phenomena: (1) the spread of P/F unranked grading systems, and (2) the marked increase in the average number of residency applications. The former has become common because there is a strong desire among institutions to minimize competition between classmates and promote a collaborative learning environment. So in essence we traded students fighting each other with students fighting a common enemy (the USMLE). The latter is multifactorial in origin, but the push by student affairs deans to draw everyone toward the sublime middle hasn't helped.

Standardized preclinical exams would be awful. They would subvert one of the core principles of academia: the faculty is in charge of the curriculum. If every test were a mini-USMLE then I imagine most schools would move toward a "teach-to-the-test" model, which would optimize scores but forgo anything resembling an authentic education. No thanks, we have enough problems with just one monolith.

Good point and I hadn't considered it. I still don't like the idea of one test deciding everything but standardized preclinicals may not be the way to go. Only thing I somewhat disagree with is the false notion students have that we are no longer competing against each other. Nearly every school still ranks their students for MSPE letters so while collaboration is a great thing, we still are measured against each other for things like AOA and where we fall in rank.

 

[Med Ed]

Nearly every school still ranks their students for MSPE letters so while collaboration is a great thing, we still are measured against each other for things like AOA and where we fall in rank.

I did not mean to imply that students are no longer competing with each other. They clearly do. But the days of having every exam norm-referenced, with a limited percentage of students getting H and an inevitable percentage getting F, are fading into memory.

As for rank, it depends on how you view the term. Old school was:

47. Nancy Ray
48. Daniel Brown
49. Lawrence Crow

Comparative information, as is now common in MSPE's, is more like:

Surgery clerkship
Student: Nancy Ray
Grade: HP
% of cohort that received H: 15%
% of cohort that received HP: 60%
% of cohort that received P: 20%
% of cohort that received M: 4%
% of cohort that received F: 1%

Perhaps it's just me, but when folks talk about rank I perceive that to be precise, ordered, and summative. Comparative information involves stratification, to be sure, but it is inherently more vague.

 

[AnatomyGrey12]

I did not mean to imply that students are no longer competing with each other. They clearly do. But the days of having every exam norm-referenced, with a limited percentage of students getting H and an inevitable percentage getting F, are fading into memory.

As for rank, it depends on how you view the term. Old school was:

47. Nancy Ray
48. Daniel Brown
49. Lawrence Crow

Comparative information, as is now common in MSPE's, is more like:

Surgery clerkship
Student: Nancy Ray
Grade: HP
% of cohort that received H: 15%
% of cohort that received HP: 60%
% of cohort that received P: 20%
% of cohort that received M: 4%
% of cohort that received F: 1%

Perhaps it's just me, but when folks talk about rank I perceive that to be precise, ordered, and summative. Comparative information involves stratification, to be sure, but it is inherently more vague.

My school, and I'm sure others, only report the ranking quartile on the MSPE. Although I can request my actual rank x/115 if I want it. I think comparative info is a good compromise, you get the stratification but people also aren't fighting each other for every rung on the ladder.

 

[IMreshopeful]

I did not mean to imply that students are no longer competing with each other. They clearly do. But the days of having every exam norm-referenced, with a limited percentage of students getting H and an inevitable percentage getting F, are fading into memory.

As for rank, it depends on how you view the term. Old school was:

47. Nancy Ray
48. Daniel Brown
49. Lawrence Crow

Comparative information, as is now common in MSPE's, is more like:

Surgery clerkship
Student: Nancy Ray
Grade: HP
% of cohort that received H: 15%
% of cohort that received HP: 60%
% of cohort that received P: 20%
% of cohort that received M: 4%
% of cohort that received F: 1%

Perhaps it's just me, but when folks talk about rank I perceive that to be precise, ordered, and summative. Comparative information involves stratification, to be sure, but it is inherently more vague.

Yeah my medical school didn’t even do that. It reported a qualitative word that referred to our quintile (“excellent, outstanding”, etc), AOA status, and a qualitative statement about our career goals, demeanor, and experiences which brought us to this point etc. then it lists clinical grades by clerkship (H, HP, P, etc) with clinical evaluation comments, followed by a summary statement. It’s quite nice and always made to support the student. I’ve seen MSPEs from other schools and sometimes they’re downright cold and humiliating which does nothing to help the applicant.

 

[zero0]

Too lazy/busy to read stuff. Is this happening or no?

The main issue I can see is a sudden and precipitous drop in the medical knowledge of incoming 3rd years, but to be fair most of the content of Step 1 is useless trivia with virtually no clinical application.

But as the dinosaurs would say, I suffered so they should too.

 

[AnatomyGrey12]

Too lazy/busy to read stuff. Is this happening or no?

I think all the official motion says is that it will be more fully discussed next meeting.

I’m still pretty skeptical it will actually happen in reality. It would require almost a complete overhaul of the system.

 

[Newyawk]

I for one am 100% in favor of a p/f step score. I feel that the singular focus on step one has been a huge negative for medical education. We can’t fault students who are simply smart people doing what they must to succeed, but we can alter the system.

Yes it’s one of the few objective measures but it’s terrible! Look at what it actually measures! That doesn’t really tell residency programs much important information. APD is correct that specialties would probably develop their own exams and use those to help stratify. Personally I think this is a great idea as I feel I could design test questions to elucidate things I care about in a resident that would be much more useful than a scaled 3 digit score based off how well someone knew the Krebs cycle 2 years ago.

I’m personally optimistic that this could happen. Despite PDs liking the universal measure, enough faculty are frustrated by their students blowing off coursework to do UFAP that something has to change. Students are just too smart and efficient and will triage their time to what matters most. It really shouldn’t be some stupid test that’s sole purpose is to test if you’re ready for 3rd year (not residency).

Everyone student ive rotated with/know personally who demonstrates the ability to reason clinically did well on step 1. I am not sure how long ago you took step 1, but my exam was very clinically based.
I for one think the solution here is to make more standardized exams and allow students the opportunity to demonstrate their clinical acumen objectively. Release the shelf exam scores. Force everyone to submit step 2 and give it more weight.

As others have stated, this is an elitist move.

 

[mehc012]

Pretty sure if Step exams became pass/fail, third year would become super annoying and subjective. People would be gunning for honors like crazy and the honors threshold would soar with attendings/residents being insanely picky and biased in grading.

Gunning for AOA would increase. School name would matter even more. It would make med school life miserable and residency application process a nightmare

This is why standardized tests are important to counter the impact of subjective grades and variations

Thing is, the attendings and residents dgaf about whether you need honors or not. They dgaf about how their fine-tuned/precise their evals are or how that affects the residency application process as a whole. If we got rid of Step 1 tomorrow, the residents and attendings would care only so far as it gave them something to talk about on downtime "wow, you don't even have to take Step 1? I remember that used to be the worst part of med school..."

They're already not grading with the bigger picture in mind, so changing that picture won't change them.

 

[mehc012]

While I agree it will be difficult, I don't think being complacent with the current system is the best way to improve things. There are many ways to standardize scoring, including taking into account how frequently one gives out honors.

The only thing that would make this subjective BS worse is grading social interactions on a curve. Now, I've never gone to a school with a curve, and I thank the stars for that ALL the time. School shouldn't be a cutthroat competition, and that goes doubly so for clinicals.

 

[operaman]

Everyone student ive rotated with/know personally who demonstrates the ability to reason clinically did well on step 1. I am not sure how long ago you took step 1, but my exam was very clinically based.
I for one think the solution here is to make more standardized exams and allow students the opportunity to demonstrate their clinical acumen objectively. Release the shelf exam scores. Force everyone to submit step 2 and give it more weight.

As others have stated, this is an elitist move.

I really believe that residency programs would find their own objective measures for stratifying applicants. Some might be multiple choice tests while others might be structured phone interviews or something else entirely. There are plenty of ways to objectively assess applicants across schools.

The detrimental impact of step 1 on overall medical education is very significant. It’s just too difficult to coerce or force highly intelligent adult learners to devote serious time and thought to things that will be useful to them as a clinician when their career is far more impacted by one test. The preclinical years have become a two year boards review course in many ways, and all that for a test that isn’t really that great.

I really think we as a field can do better.

 

[Newyawk]

I really believe that residency programs would find their own objective measures for stratifying applicants. Some might be multiple choice tests while others might be structured phone interviews or something else entirely. There are plenty of ways to objectively assess applicants across schools.

The detrimental impact of step 1 on overall medical education is very significant. It’s just too difficult to coerce or force highly intelligent adult learners to devote serious time and thought to things that will be useful to them as a clinician when their career is far more impacted by one test. The preclinical years have become a two year boards review course in many ways, and all that for a test that isn’t really that great.

I really think we as a field can do better.

You didn't address any of my points.... I said 1. the test IS clinical and personally has prepared me very well for the knowledge required on wards. 2. because it is clinical, students who strive to succeed on it are learning the material.
also, your previous reference joking about the Krebs cycle is such an outdated point - the exam is like less than 5% biochemistry and even those questions that are biochemistry are asked in disease form.

 

[operaman]

You didn't address any of my points.... I said 1. the test IS clinical and personally has prepared me very well for the knowledge required on wards. 2. because it is clinical, students who strive to succeed on it are learning the material.
also, your previous reference joking about the Krebs cycle is such an outdated point - the exam is like less than 5% biochemistry and even those questions that are biochemistry are asked in disease form.

I know, I took it not that terribly long ago!

I agree that much of that knowledge was also helpful on the wards. It’s hard to tease out though just how much of that is people who are smarter overall. I agree that the people in my class who scored over 260 were also strong from a clinical knowledge standpoint, but these were also AOA folks who really crushed it in class as well.

What I see as a resident are many students who seemed to blow off everything in favor of cramming UFAPS and who really struggle with the conceptual thinking required in clinical medicine. They bring the same thinking to the clinic and heavily weight their time toward shelf study rather than trying to develop actual useful skills. Here too the smarter students have an unfair advantage in that they can study less and do well and spend additional time developing skills that let them really shine on their sub-i and whatnot.

I guess I see enough of the average students who say they just focused on boards and blew off the rest and they seem to struggle more. Maybe they would still struggle if step had been p/f! I just know that high stakes testing has made it incredibly hard for faculty to motivate students to learn and study seriously anything that isn’t high yield for step one.

I guess the question is whether people will learn better by studying for a standardized test or by studying for classes and other activities. I don’t really know the answer to that question, just the frustration of trying to teach people who are so singularly focused on the important test. The smart kids always do fine. I wonder about the average kid who only studies for boards — would he or she be better off devoting more time to classes and activities that develop clinical thinking but are maybe low yield for step, or are they better off cramming UFAP which does contain some high quality teaching?

Obviously I don’t know the answer, but making step p/f doesn’t eliminate the need to study for it and learn the material. It does remove the need to make it the entire singular focus of ones preclinical education.