Step 1 now will be P/F

[deleted171991]

It seems we are turning into a banana republic not only in Washington.

There are multiple forces at work here, trying to destroy the special standing and prestige of American medical doctors. Let's not forget that about half of the AAMC board is made-up of non-physicians, some of them nurses! The USMLE and our board exams were already a joke when compared to most developed countries, let's continue the erosion.

On one hand we have nurses and hospital administrators trying to dumb down medical education and close the chasm between how doctors and nurse practitioners are viewed, trying to reduce the former to the latter, and both to trained monkeys. Also, there has been a long-standing movement, initially started by feminist nurses and later happily embraced by business people, to castrate and diminish the American physician, to "re-educate" us, after a communist model (read up on Chinese brainwashing systems). Any resistance, any disobedience is labeled as "disruptive physician", and results in serious consequences to one's career.

This starts from medical school and especially residency. On one hand, residents are taught to behave like nurses, to adopt a shift-worker mentality and renounce the hippocratic and professional one. On the other hand, they are treated as assembly lane blue-collar workers, replaceable cogs in the healthcare factory. I got more respect as a medical student in my native country than I get as a double board-certified American attending. This results in both trainees and attendings not standing up for their own or their colleagues' rights, quietly accepting verbal and professional abuses that would have resulted in firing the culprit just 20-30 years ago.

This, plus the coddling of the recent generations, especially Z, has resulted in so many snowflakes one would think we are in Antarctica. They are also the reason why I skipped this forum for the last 6 weeks (and will continue doing so). The amount of whining is just exhausting. It's just a truly mediocre generation (with few exceptions) who needs diapers even in their tweens. If you read the AMA motivation behind dropping the score system for Step 1, you will notice this phrase: "Our student, resident and physician members voted to endorse a pass/fail policy, in part, because we know our current residency selection system is causing significant distress for our students." The babies were crying!

And let me tell you, the babies truly suck as doctors, when compared to previous generations of trainees. This will only lead to even more dilution of the value of a medical diploma, which is the whole point of this (plus a nice side effect of populism among the *****s who don't realize that the easier the path the less valuable). The current generations of residents SUCK as it is, some are downright scary; now they truly won't know much more than their nurse practitioner "colleagues". It all starts in medical school, with the same pass/fail system having been adopted even by supposedly prestigious medical schools, such as Yale. It's the healthcare version of amateur Olympics; participation is all that matters, like in (post-)Millennial kindergarten contests. Everybody gets the same diploma.

YOU CAN'T IMPROVE WHAT YOU DON'T MEASURE!

Until next time...

---

Members don't see this ad.

 

[nimbus]

It seems we are turning into a banana republic not only in Washington.

There are multiple forces at work here, trying to destroy the special standing and prestige of American medical doctors. Let's not forget that about half of the AAMC board is made-up of non-physicians, some of them nurses! The USMLE and our board exams were already a joke when compared to most developed countries, let's continue the erosion.

On one hand we have nurses and hospital administrators trying to dumb down medical education and close the chasm between how doctors and nurse practitioners are viewed, trying to reduce the former to the latter, and both to trained monkeys. Also, there has been a long-standing movement, initially started by feminist nurses and later happily embraced by business people, to castrate and diminish the American physician, to "re-educate" us, after a communist model (read up on Chinese brainwashing systems). Any resistance, any disobedience is labeled as "disruptive physician", and results in serious consequences to one's career.

This starts from medical school and especially residency. On one hand, residents are taught to behave like nurses, to adopt a shift-worker mentality and renounce the hippocratic and professional one. On the other hand, they are treated as assembly lane blue-collar workers, replaceable cogs in the healthcare factory. I got more respect as a medical student in my native country than I get as a double board-certified American attending. This results in both trainees and attendings not standing up for their own or their colleagues' rights, quietly accepting verbal and professional abuses that would have resulted in firing the culprit just 20-30 years ago.

This, plus the coddling of the recent generations, especially Z, has resulted in so many snowflakes one would think we are in Antarctica. They are also the reason why I skipped this forum for the last 6 weeks (and will continue doing so). The amount of whining is just exhausting. It's just a truly mediocre generation (with few exceptions) who needs diapers even in their tweens. If you read the AMA motivation behind dropping the score system for Step 1, you will notice this phrase: "Our student, resident and physician members voted to endorse a pass/fail policy, in part, because we know our current residency selection system is causing significant distress for our students." The babies were crying!

And let me tell you, the babies truly suck as doctors, when compared to previous generations of trainees. This will only lead to even more dilution of the value of a medical diploma, which is the whole point of this (plus a nice side effect of populism among the *****s who don't realize that the easier the path the less valuable). The current generations of residents SUCK as it is, some are downright scary; now they truly won't know much more than their nurse practitioner "colleagues". It all starts in medical school, with the same pass/fail system having been adopted even by supposedly prestigious medical schools, such as Yale. It's the healthcare version of amateur Olympics; participation is all that matters, like in (post-)Millennial kindergarten contests. Everybody gets the same diploma.

YOU CAN'T IMPROVE WHAT YOU DON'T MEASURE!

Until next time...

Do you see any irony in writing “The amount of whining is just exhausting” within that post?

 

[deleted162650]

Do you see any irony in writing “The amount of whining is just exhausting” within that post?

Reminds me of:

"I'm against picketing, but I don't know how to show it."
-Mitch Hedburg

 

[nimbus]

 

[BLADEMDA]

This is for FFP:

School district scraps traditional ‘A-F’ grading system for kinder, gentler model | The College Fix

New grading scales obscure a child’s true performance…

www.thecollegefix.com

When we no longer value competition and hard work by rewarding those who excel the entire society suffers. The "system" has worked reasonably well over the past 100 years because not every kid/student deserves an "A" or gets an award for simply "trying" to succeed. A P/F systems rewards mediocrity and that is simply not good enough IMHO.

 

[DM27]

I am not sure what the ultimate end goal is of the non-AAMC individuals who have been pushing this change. Is their viewpoint that Step 1 is overall being misused by residency PDs and that Step 2 CK needs more weight (this I can agree with), or is their end goal that all Step exams become pass/fail? If the viewpoint is "fairness" and that the test is somehow "discriminatory" then the end goal of being Pass/Fail I would argue is less fair since applicants will be judged almost entirely from their school and who they know.

 

[Mman]

Some serious psychobabble here.......

"Our student, resident and physician members voted to endorse a pass/fail policy, in part, because we know our current residency selection system is causing significant distress for our students. "

The poor students. Having to actually stress out and study hard to become a doctor. The trauma, the horror. Next thing you are going to tell me is that they would rather not have to work more than 40 hours a week as a resident because they have outside interests and families to attend to.

 

[GravelRider]

No, I don't agree, because the problem isn't necessarily the source material- it's the method in which it was taught. Perhaps when most people were cramming there wasn't a clinical correlate that made the information stick. Besides, most students who take Step I can't say with any real certainty which specialty they're going into or what kind of future research tracks they're interested in, thus the education has to be broad. The Krebs cycle may not be that important to anesthesiologists- but you may get a different story if you talk to people who went into endocrinology or genetics.

Not to mention, I worry that your argument may be prone to a reductio ab absurdum where piece by piece we do away with all the "irrelevant" knowledge until we become Nurses+ instead of physician scientists.

So you are saying that people don’t have a deep enough understanding of things, but we should keep things the way they are? I’m not sure I follow.

I’m not saying we shouldn’t learn the material, I’m saying the way it is taught and tested is antiquated. We are doing medical school like it is still 100 years ago.

This is for FFP:

School district scraps traditional ‘A-F’ grading system for kinder, gentler model | The College Fix

New grading scales obscure a child’s true performance…

www.thecollegefix.com

When we no longer value competition and hard work by rewarding those who excel the entire society suffers. The "system" has worked reasonably well over the past 100 years because not every kid/student deserves an "A" or gets an award for simply "trying" to succeed. A P/F systems rewards mediocrity and that is simply not good enough IMHO.

And that’s the problem...we are using a 100 year old system. The amount of information required to be a good “physician scientist” is multiple orders of magnitude greater than it was just 20 years ago. Yet for some reason we think wasting brain power on memorizing factoids is both efficient and productive.

 

[vector2]

So you are saying that people don’t have a deep enough understanding of things, but we should keep things the way they are? I’m not sure I follow.

I’m not saying we shouldn’t learn the material, I’m saying the way it is taught and tested is antiquated. We are doing medical school like it is still 100 years ago.

Yea, you're not following cause apparently you parsed the exact opposite of the words I wrote. I literally just said the material was important but the method of teaching was potentially the problem. Go back and read it again.

 

[VA Hopeful Dr]

The poor students. Having to actually stress out and study hard to become a doctor. The trauma, the horror. Next thing you are going to tell me is that they would rather not have to work more than 40 hours a week as a resident because they have outside interests and families to attend to.

You have to bear in mind that Step 1 prep is a whole different world than we most of us here were in school.

I graduated in 2010. Back then almost no one started studying for Step 1 before January of 2nd year. We had maybe 2-3 people who started before that but it was rare.

Go look in the MD Student forum. These kids are starting the day they matriculate and are doing hundreds of practice questions and flash cards every day for 2 years. That's on top of regular studying for exams.

I don't know if making Step 1 pass/fail is a good thing, but something had to be done.

 

[ucladoc2b]

The poor students. Having to actually stress out and study hard to become a doctor. The trauma, the horror. Next thing you are going to tell me is that they would rather not have to work more than 40 hours a week as a resident because they have outside interests and families to attend to.

Attaching life altering consequences to a single exam with a large SEM is excessive. Hope everything goes your way on test day so you don’t fall below some arbitrary exam cutoff. Never mind the average score had climbed about 20 points over the years.

The only reason it has such excessive importance is because of the match hysteria caused by the click and apply function of ERAS. With programs sometimes getting 1200 applications each and students applying to dozens of programs, it’s outsize importance was one of convenience. The backwards justification in using it for this reason was manufactured. Curricula and schedules revolved around this test. Heck, I had to pay 15K in tuition for 8 weeks that consisted of staying home and studying. The impact on education was pretty bad.

This is a good change. Hopefully the same happens for Step 2 (which has even worse score inflation).

 

[deleted171991]

You have to bear in mind that Step 1 prep is a whole different world than we most of us here were in school.

I graduated in 2010. Back then almost no one started studying for Step 1 before January of 2nd year. We had maybe 2-3 people who started before that but it was rare.

Go look in the MD Student forum. These kids are starting the day they matriculate and are doing hundreds of practice questions and flash cards every day for 2 years. That's on top of regular studying for exams.

I don't know if making Step 1 pass/fail is a good thing, but something had to be done.

I understand. As an (ex-)educator, what I have been noticing is the decrease in knowledge. I don't know whether it's because they have to waste all that time on social "sciences", on feel-good studies, but there are some scary gaps in basic science foundations. American medical students study for tests, not for their future jobs. The level of non-clinical scientific knowledge is decreasing. We are training monkeys, not scientists, and it will show in 10-20 years (it's already visible to those of us who know some medicine).

When I look at the specialty board-certification exams in Europe or ANZ, ours seem like nursing schools in comparison. We may have more hands-on training in residency, but they run circles around us in knowledge (one of the reasons being that they don't use MCQs, but traditional written exams). And, as an intensivist who sees a lot of iatrogenic crap, I can't emphasize enough how important all that knowledge and UNDERSTANDING is to FIRST DO NO HARM. It's way more important to have a doctor who knows what no to do, then what to do. For the latter, all one needs is somebody who can look up stuff in a medical book (i.e. a nurse practitioner). And that's exactly where the healthcare industry and nurse midlevel movements are trying to push us.

Also, for residency entrance purposes, one needs an exam that would stratify the candidates. Most of the recommendation letters are worthlessly syrupy. In my n=1 experience, most of my weak residents had low USMLE scores and most of my stars had great ones. I wouldn't expect the opposite to be true; still, a low score on Step 1 or Step 2 should be a red flag, regardless how much one likes the candidate as a human being. It is for me. Those huge gaps, especially the pre-clinical ones, can seldom be fixed later. Eliminating the scores is also unfair to the true stars; it's like communism, or America lately (a poor understanding of what Democracy should mean, which is equality of opportunity, not of outcomes). Not only should we have more scored exams, we should make the percentiles public at least for USMLE and specialty boards. Patients have the right to know whom they are dealing with. And those of you who embrace the current waves of populism in American university education should remember that these people will be your doctors in your old age. Be afraid, be very afraid!

The superstars will never plead for eliminating the scores. It's usually the mediocrities. The USMLE Step 1 and Step 2 CK are impressively good and thorough exams, given their time constraints. Castrating them will only lead to worse doctors, like affirmative action, for example. Another case where populism hurts education.

One more brick knocked out to hasten the fall of the American empire (and it's greatness) and to promote socialism/populism. One more reason for smart kids to avoid medical school. Nothing new for those of us who know some history. Enough time wasted, bye!

 

[Mman]

You have to bear in mind that Step 1 prep is a whole different world than we most of us here were in school.

I graduated in 2010. Back then almost no one started studying for Step 1 before January of 2nd year. We had maybe 2-3 people who started before that but it was rare.

Go look in the MD Student forum. These kids are starting the day they matriculate and are doing hundreds of practice questions and flash cards every day for 2 years. That's on top of regular studying for exams.

I don't know if making Step 1 pass/fail is a good thing, but something had to be done.

No offense, but students studying longer and longer for an exam does not mean we should stop grading the exam IMHO. It's just going to change the importance we put on something else they will have to spend an equal amount of time on.

 

[VA Hopeful Dr]

I understand. As an (ex-)educator, what I have been noticing is the decrease in knowledge. I don't know whether it's because they have to waste all that time on social "sciences", on feel-good studies, but there are some scary gaps in basic science foundations. American medical students study for tests, not for their future jobs. The level of non-clinical scientific knowledge is decreasing. We are training monkeys, not scientists, and it will show in 10-20 years (it's already visible to those of us who know some medicine).

When I look at the specialty board-certification exams in Europe or ANZ, ours seem like nursing schools in comparison. We may have more hands-on training in residency, but they run circles around us in knowledge (one of the reasons being that they don't use MCQs, but traditional written exams). And, as an intensivist who sees a lot of iatrogenic crap, I can't emphasize enough how important all that knowledge and UNDERSTANDING is to FIRST DO NO HARM. It's way more important to have a doctor who knows what no to do, then what to do. For the latter, all one needs is somebody who can look up stuff in a medical book (i.e. a nurse practitioner). And that's exactly where the healthcare industry and nurse midlevel movements are trying to push us.

Also, for residency entrance purposes, one needs an exam that would stratify the candidates. Most of the recommendation letters are worthlessly syrupy. In my n=1 experience, most of my weak residents had low USMLE scores and most of my stars had great ones. I wouldn't expect the opposite to be true; still, a low score on Step 1 or Step 2 should be a red flag, regardless how much one likes the candidate as a human being. It is for me. Those huge gaps, especially the pre-clinical ones, can seldom be fixed later. Eliminating the scores is also unfair to the true stars; it's like communism, or America lately (a poor understanding of what Democracy should mean, which is equality of opportunity, not of outcomes). Not only should we have more scored exams, we should make the percentiles public at least for USMLE and specialty boards. Patients have the right to know whom they are dealing with. And those of you who embrace the current waves of populism in American university education should remember that these people will be your doctors in your old age. Be afraid, be very afraid!

The superstars will never plead for eliminating the scores. It's usually the mediocrities. The USMLE Step 1 and Step 2 CK are impressively good and thorough exams, given their time constraints. Castrating them will only lead to worse doctors, like affirmative action, for example. Another case where populism hurts education.

Your first paragraph nails the problem. But I don't know how to train doctors not to study for a test but then put massive emphasis on a single test. A written exam pilot program might be worth looking at, I'm not familiar with how other countries train physicians so this was news to me. Seems interesting.

I don't see the major iatrogenic harm like you intensivists do but I definitely see bad medicine from other doctors since patients bounce around a lot these days and see multiple specialists.

I don't think the problem is the training most of the time, I think its what happens out in practice. People don't keep up with new research, they fall victim to patient satisfaction scores (I'm guilty of this one), or they just get plain lazy. I don't know how to prevent #1 and 3.

 

[VA Hopeful Dr]

No offense, but students studying longer and longer for an exam does not mean we should stop grading the exam IMHO. It's just going to change the importance we put on something else they will have to spend an equal amount of time on.

None taken, its why I ended that post the way I did.

 

[Mman]

Attaching life altering consequences to a single exam with a large SEM is excessive. Hope everything goes your way on test day so you don’t fall below some arbitrary exam cutoff. Never mind the average score had climbed about 20 points over the years.

The only reason it has such excessive importance is because of the match hysteria caused by the click and apply function of ERAS. With programs sometimes getting 1200 applications each and students applying to dozens of programs, it’s outsize importance was one of convenience. The backwards justification in using it for this reason was manufactured. Curricula and schedules revolved around this test. Heck, I had to pay 15K in tuition for 8 weeks that consisted of staying home and studying. The impact on education was pretty bad.

This is a good change. Hopefully the same happens for Step 2 (which has even worse score inflation).

When I was applying for residency programs in 2004, they were stratifying applicants based on Step 1 scores because they were predictive in terms of how strong you'd be as a resident so my classmates and I spent months studying for it.

It is what it is and it worked fine for a long time. Now you are either going to get a system that overloads value on Step 2 or simply awards residency slots based on the prestige of the medical school you attend which puts even more insane emphasis on the MCAT.

 

[deleted171991]

Your first paragraph nails the problem. But I don't know how to train doctors not to study for a test but then put massive emphasis on a single test. A written exam pilot program might be worth looking at, I'm not familiar with how other countries train physicians so this was news to me. Seems interesting.

I don't see the major iatrogenic harm like you intensivists do but I definitely see bad medicine from other doctors since patients bounce around a lot these days and see multiple specialists.

I don't think the problem is the training most of the time, I think its what happens out in practice. People don't keep up with new research, they fall victim to patient satisfaction scores (I'm guilty of this one), or they just get plain lazy. I don't know how to prevent #1 and 3.

Here's an example (ANZ primary exam for anesthesia certification): 50% of their exam score is short-answer questions (SAQs). And here are some examples of SAQs and how they are graded: http://www.anzca.edu.au/documents/aug-oct-primary-examination-report.pdf. This kind of system emphasizes not only knowledge, but also understanding, like real world practice. Training for their exam means training for the real world. One doesn't really know something until one can explain it to others.

The current American system encourages mechanistic learning, knee-jerk behavior, if this then that, which belongs in nursing schools, not physician training. And it's very hard/impossible to teach the WHY and WHY NOT to trainees who lack basic science foundations; once that ship sails, the individual becomes a nurse MD not a physician. I foresee a great second career for myself in retirement, consulting for rich patients whose doctors have f*cked them up by following some stupid guideline-based knee-jerk algorithm/protocol (e.g. Surviving Sepsis - should be called Surviving Bad Doctors).

 

[BLADEMDA]

You guys need to live in the real world. The educators are part of the problem these days. Now, instead of having Step 1 and Step 2 exams to stratify prospective residents the program directors will only have step 2. Does this help the average med student? I would argue no it doesn’t.

For the next 2 decades Step 2 will now be the only exam programs use to differentiate applicants.

 

[ucladoc2b]

When I was applying for residency programs in 2004, they were stratifying applicants based on Step 1 scores because they were predictive in terms of how strong you'd be as a resident so my classmates and I spent months studying for it.

It is what it is and it worked fine for a long time. Now you are either going to get a system that overloads value on Step 2 or simply awards residency slots based on the prestige of the medical school you attend which puts even more insane emphasis on the MCAT.

Research has never born this premise out. There is a very modest correlation with future board passage in some fields. There is minimal correlation with ITE. But really that’s it.

Maybe now we could award residency spots based on skills other than small differences in test taking.

 

[VA Hopeful Dr]

Here's an example (ANZ primary exam for anesthesia certification): 50% of their exam score is short-answer questions (SAQs). And here are some examples of SAQs and how they are graded: http://www.anzca.edu.au/documents/aug-oct-primary-examination-report.pdf. This kind of system emphasizes not only knowledge, but also understanding, like real world practice. Training for their exam means training for the real world. One doesn't really know something until one can explain it to others.

The current American system encourages mechanistic learning, knee-jerk behavior, if this then that, which belongs in nursing schools, not physician training. And it's very hard/impossible to teach the WHY and WHY NOT to trainees who lack basic science foundations. I foresee a great second career for myself in retirement, consulting for rich patients whose doctors have f*cked them up by following some stupid guideline-based algorithm/protocol (e.g. Surviving Sepsis - should be called Surviving Bad Doctors).

Since I'm not an anesthesiologist I can't judge the relevance of those questions, but I like the idea behind it.

Do y'all still have oral boards? That seems like a similar concept.

 

[deleted171991]

Since I'm not an anesthesiologist I can't judge the relevance of those questions, but I like the idea behind it.

Do y'all still have oral boards? That seems like a similar concept.

I am not from ANZ, just remotely interested, but yes, they do have oral boards, too. So do many other developed countries in many specialties, AFAIK.

SAQs are similar to the way old-school American physicians assess knowledge, AKA pimping. It's one thing to be good at MCQs and totally different to be able to explain something in detail. The problem with SAQs is that they are time-consuming, hence they emphasize depth of knowledge over breadth. They are better for shelf exams or specialty board exams; we should definitely use them more for the latter, like other countries do.

This is why the USMLE is such a great exam for the generalist as it is, by emphasizing breadth of knowledge. And this is why it's so great for stratifying students, by poking around in their knowledge base about 600 times (between the two written exams), not to mention doing it in an adaptive fashion; it's main fault is mostly just its low passing scores.

Regardless of the exam-type, not grading an exam beyond pass/fail, on top of a low passing score, is a recipe for mediocrity, not excellence. And it's continuing the trend towards mediocrity in American physician education, which is saddening. I am watching the fall of American greatness every day; this is just another brick for me (I remember how shocked I was when I discovered that YSM exams were pass/fail - what can you expect from a university that calls PAs physician associates, or doesn't defend a teacher when verbally assaulted by his students). But current medical students should be much smarter than to accept or encourage populist measures like this one.

 

[CopaceticOne]

Do y'all still have oral boards? That seems like a similar concept.

Yep, and now an OSCE as well. (Whoops, was answering for US boards)

 

[zero0]

I cant help feeling like all of the current issues with the exam could have been fixed by allowing students to retake it for a different score. Making it p/f was definitely not the right solution though.

 

[VA Hopeful Dr]

Yep, and now an OSCE as well. (Whoops, was answering for US boards)

I was asking about US anesthesia boards as my specialty (FM) doesn't have them

 

[VA Hopeful Dr]

I am not from ANZ, just remotely interested, but yes, they do have oral boards, too. So do many other developed countries in many specialties, AFAIK.

SAQs are similar to the way old-school American physicians assess knowledge, AKA pimping. It's one thing to be good at MCQs and totally different to be able to explain something in detail. The problem with SAQs is that they are time-consuming, hence they emphasize depth of knowledge over breadth. They are better for shelf exams or specialty board exams; we should definitely use them more for the latter, like other countries do.

This is why the USMLE is such a great exam for the generalist as it is, by emphasizing breadth of knowledge. And this is why it's so great for stratifying students, by poking around in their knowledge base about 600 times; it's main fault is mostly just its low passing scores.

Regardless of the exam-type, not grading an exam beyond pass/fail, on top of a low passing score, is a recipe for mediocrity, not excellence. And it's continuing the trend towards mediocrity in American physician education, which is saddening. I am watching the fall of American greatness every day; this is just another brick for me. But current medical students should be much smarter than to accept or encourage populist measures like this.

IF SDN is even remotely representative, most medical students aren't happy about it.

 

[Lecithin5]

It is a stupid test requiring you to memorize inane facts that you forget the day after you take it because you never use that information ever again. How many countless hours of energy and productivity have been wasted on that dumb exam? The nursing schools are running laps around us in making education more efficient and we’re sequestering ourselves in a library for 6 straight weeks memorizing the Krebs Cycle. It’s old world thinking. It’s time to move on.

I agree that it is in large a stupid test, but everything is a “stupid test.” MCAT has organic chemistry, SATs have random vocab words, etc. But the idea is that at some level you have to create some separation. Already there are medical schools that don’t have any grades other than pass/fail for the first two years of medical school. All this is going to do is have people play “the game” even more (ie you’ll see folks doing more research in order to create that separation, which is great, but for most folks, research is even more “stupid”). Isn’t this one of the arguments for physicians as opposed to nurse anesthetists? We went through a more rigorous process? Took more tests, boards, became masters at each ladder rung? Perhaps there are other ways to do it, but test taking is historically a true and tried way of doing this. Who knows

 

[Mman]

Research has never born this premise out. There is a very modest correlation with future board passage in some fields. There is minimal correlation with ITE. But really that’s it.

Maybe now we could award residency spots based on skills other than small differences in test taking.

The difference in test taking is generally large, not small, at the level programs use for cutoffs. Getting a 250 or 260 on Step 1 is significantly different than getting a 205 or 215.

And how do you suggest awarding those spots? Because if it isn't on test scores it will almost solely be based on which med school someone goes to. I say that having sat through admissions committees for med school and residency. At some point, every application looks the same. Every residency applicant has some flattering letters of rec, they all have extracurriculars, etc. If you don't use the test score, you go back to filtering out people based on their med school. You cannot have a 2 week interview of every potential candidate. Doesn't really work that way. You need to draw a line in the sand objectively somewhere to thin the pool to a more manageable number.

Also, there have been studies showing USMLE scores leading to better resident outcomes (evaluations from faculty, board passage, etc). I do now know the literature for every field but let's not pretend it isn't there.

 

[vector2]

Research has never born this premise out. There is a very modest correlation with future board passage in some fields. There is minimal correlation with ITE. But really that’s it.

Maybe now we could award residency spots based on skills other than small differences in test taking.

Notice how all the fails are clustered below ~215 while all the passes are generally above 215?

http://www.asaabstracts.com/strands/asaabstracts/abstract.htm?year=2017&index=12&absnum=4029

The correlation might be weaker for other fields because people just start working their asses off to pass as the stakes rise, but I can tell you anecdotally that I lose zero sleep about my high step 1 scoring residents inasfar as doing well on the ITE or passing the basic and advanced.

 

[vector2]

Future template for HMS’ dean’s letters

 

[ucladoc2b]

The difference in test taking is generally large, not small, at the level programs use for cutoffs. Getting a 250 or 260 on Step 1 is significantly different than getting a 205 or 215.

And how do you suggest awarding those spots? Because if it isn't on test scores it will almost solely be based on which med school someone goes to. I say that having sat through admissions committees for med school and residency. At some point, every application looks the same. Every residency applicant has some flattering letters of rec, they all have extracurriculars, etc. If you don't use the test score, you go back to filtering out people based on their med school. You cannot have a 2 week interview of every potential candidate. Doesn't really work that way. You need to draw a line in the sand objectively somewhere to thin the pool to a more manageable number.

Also, there have been studies showing USMLE scores leading to better resident outcomes (evaluations from faculty, board passage, etc). I do now know the literature for every field but let's not pretend it isn't there.

The literature I’ve seen is very modest regarding evals and USMLEs.

Yes there is a difference in test outcomes between a 205 and 260. But not so much 220 to 240. The problem is using arbitrary cutoffs.

The medical school you go to already has an impact on residency placement.

Everyone here acts that there this is the only way to select residents.

 

[BLADEMDA]

The literature I’ve seen is very modest regarding evals and USMLEs.

Yes there is a difference in test outcomes between a 205 and 260. But not so much 220 to 240. The problem is using arbitrary cutoffs.

The medical school you go to already has an impact on residency placement.

Everyone here acts that there this is the only way to select residents.

Doesn’t matter what you think or believe as program directors will turn to Step 2 CK to stratify med students. Fair? This is the real world and fair has little to do with it. The test is objective so “fair” is not the term to describe the exam. The test allows programs to compare med students from across the country and the world.

Imho, Step 2 CK should have been the go to exam all along as clinical knowledge should be valued higher than basic science for most specialties.

I do hope med schools budget in a few weeks of study time for Step 2 CK. Now that Step 1 is P/F reduce that study time to just 3 weeks and budget in time for Step 2 CK.

 

[Mman]

The literature I’ve seen is very modest regarding evals and USMLEs.

Yes there is a difference in test outcomes between a 205 and 260. But not so much 220 to 240. The problem is using arbitrary cutoffs.

The medical school you go to already has an impact on residency placement.

Everyone here acts that there this is the only way to select residents.

Having sat through interviews and admissions processes, I can tell you a lot about how to select residents. And I can tell you that having an objective test score to weed through the masses is a helpful part of it. If it isn't Step 1, it will be something else with the same effect.

Interviewing for residency isn't supposed to be some fair and even process where everybody gets an equal shot. It's more like the NFL draft. Sure things like statistics and 40 yard dash times are not necessarily going to tell you who is going to be great at the next level, but they are predictive to some extent and help teams make better decisions than having no data at all.

 

[zero0]

 

[GravelRider]

I agree that it is in large a stupid test, but everything is a “stupid test.” MCAT has organic chemistry, SATs have random vocab words, etc. But the idea is that at some level you have to create some separation. Already there are medical schools that don’t have any grades other than pass/fail for the first two years of medical school. All this is going to do is have people play “the game” even more (ie you’ll see folks doing more research in order to create that separation, which is great, but for most folks, research is even more “stupid”). Isn’t this one of the arguments for physicians as opposed to nurse anesthetists? We went through a more rigorous process? Took more tests, boards, became masters at each ladder rung? Perhaps there are other ways to do it, but test taking is historically a true and tried way of doing this. Who knows

Of course we need to find ways to stratify people, but Step 1 is a terrible way to do it. It does not adequately measure the skills to be a good resident and therefore a good physician.

The argument for physician vs midlevel has nothing to do with multiple choice tests. That’s driven by the market. Like it or not, the market doesn’t value deep knowledge of human cellular physiology and biochemistry. The market values productivity and efficiency. Let’s be honest with ourselves here, you do not need a deep understanding of biochemistry to prescribe metformin for diabetes.

The medical profession is going to have to do a deep reckoning with itself over the next decade and move into the 21st century. We keep trying to legislate or regulate our prestige into existence, but the fact of the matter is the market doesn’t care.

 

[nimbus]

Of course we need to find ways to stratify people, but Step 1 is a terrible way to do it. It does not adequately measure the skills to be a good resident and therefore a good physician.

It is a good way to measure how well somebody can ignore distractions, focus, and work hard.

 

[ambiturner]

Let’s be honest with ourselves here, you do not need a deep understanding of biochemistry to prescribe metformin for diabetes.

I would argue metformin is a bad example of a drug that requires no deep knowledge of biochemistry since its major side effect is lactic acidosis and its management in that setting depends on understanding the biochemistry of glucose metabolism. I agree with you though that prescribing it does not require that knowledge, however, and to FFP’s earlier point, if someone screwed up a patient with metformin it would likely go unnoticed and wouldn’t cause any repercussions.

I think this is a sad decision for medicine. Looking at the high quality meritocratic societies that everyone always tout as having superior medical systems, most of them require a national exam that determines the best, and only the best can become doctors. Without an exam, as others have said all you have is nonstandardized qualitative data. More sinisterly, qualitative resume boosters are easy to buy, and any decision that rewards qualitative resume boosters favors the rich (you all knew a pre-med whose parents paid for them to spend a summer in Africa saving the children, right?)

 

[Mman]

Of course we need to find ways to stratify people, but Step 1 is a terrible way to do it. It does not adequately measure the skills to be a good resident and therefore a good physician.

The thing is, we don't have a better way right now.

I mean I can describe better ways in theory. If every student did a 2 month away rotation at every program they apply to, that would be wonderful. Programs could have a nice and thorough evaluation of them in a standard clinical situation at their location. But that just can't happen in the real world. The real world is cold and hard and we have limited time and resources to make those sort of evaluations. I mean maybe we should rely on their letters of recommendation, but you only submit what 3 of them? Literally almost every med student with a pulse can find 3 attendings to right them a really nice LOR. I've seen plenty of terrible residents that had glowing letters of rec as a student.

It's easy to wax poetic about how things should be done. It's far harder to come up with actual hard suggestions that are able to be implemented and are better than what we currently do.

 

[woopedazz]

Australian schools have been P/F in the most part for a long while. Works well in our system. Doubt it will work well in the US system because of the core differences in specialty training, notably the entry point timings.

The entire point of the Australian process is you do your internship year +/- year(s) of junior MO positions before you get a sniff for training program selection. During that time you are a doctor. You get references. You do research. You complete courses. Get paid well. Etc. Then as PGY2(+) you are trying to get into a training program and the selection process is based on 3 factors in sequential order:

1. CV (20%) - Includes a P/F point threshold that needs to be reached before CV gets reviewed by a human.
2. Referees (40%)
3. Interview (40%)

With no Step 1, the CV acts as the "objectively scored" barrier to application. It relies almost entirely on work done as a doctor post-university.

Colleges previously published "point allocations" for CV sections. Some still do, some have shied away from it to try to prevent tick-boxing and encourage a more natural progression to a state of training readiness. An example is the following scoring system for one of the surgical colleges:

The CV will be scored out of a maximum 21 points.
6.2.1. Surgical Experience (maximum 6 points)
6.2.2. Skills Courses (maximum 1 point)
6.2.3. Qualifications (maximum 5 points)
6.2.4. Presentations (maximum 3 points)
6.2.5. Publications (maximum 5 points)
6.2.6. Rural Origin Applicant (maximum 1 point)

Basically the same as the US I assume.

The key difference is we are expected to languish in the pre-training wilderness for internship +x amount of years depending on the competitiveness of the specialty your applying for. Everything you achieve during medschool is basically expunged and they just want to see how you did during internship +/- extra years.

For example: Anaesthesia is internship + 1 extra year minimum before you can post a CV. Typically people enter ~PGY4, with better candidates getting in at the start of PGY3 and the drongos getting PGY6+ before getting in a ****ty program and likely failing out.

The US system doesn't function in that way. So they need a quantitative cut-off to be established during medical school for match if I'm correct? So P/F will just shift the score to Step 2 yes? Maybe good if you want to do psychiatry or obstetrics or something with less basic sciences involved, but will it actually change anything at all for anyone applying for a competitive match specialty?

 

[dhb]

I would argue metformin is a bad example of a drug that requires no deep knowledge of biochemistry since its major side effect is lactic acidosis and its management in that setting depends on understanding the biochemistry of glucose metabolism.

Well it does require reading the litterature:
"Salpeter et al. (14,15) reviewed published reports of controlled trials involving metformin that lasted 1 month or more and were reported through November 2002. They found no cases of lactic acidosis in 36,000 patient-years of exposure to metformin and concluded that there was no evidence to support a role for metformin in the development of lactic acidosis."
"A recent review by Stades et al. (17) provides additional evidence that most cases of metformin-associated lactic acidosis, particularly fatal ones, are related to underlying conditions rather than to metformin."

Phantom of Lactic Acidosis due to Metformin in Patients With Diabetes

Metformin is the only biguanide that is available in the U.S. Another biguanide, phenformin, had been used since the 1950s, but was declared an “imminent hazard

care.diabetesjournals.org

 

[tennessee2244]

Step 1 should have always been pass fail. No one in here as an anesthesiologist uses a ****in lick of the pathology that comprises the majority of the exam. Which one of yall can tell me the translation of burkitts lymphoma? Or what hairy cell leukemia does?

Chill. Step 1 is trash. No matter how much FFP beats their chest.

Step 2 is indicative of actual thinking. And should remain scored.

 

[ambiturner]

Ha! Yeah I was aware of that but step 1 still says it’s a thing. Perhaps this is instead another argument to get rid of the tests since they perpetuate old myths?

 

[Mman]

Step 1 should have always been pass fail. No one in here as an anesthesiologist uses a ****in lick of the pathology that comprises the majority of the exam. Which one of yall can tell me the translation of burkitts lymphoma? Or what hairy cell leukemia does?

Chill. Step 1 is trash. No matter how much FFP beats their chest.

Step 2 is indicative of actual thinking. And should remain scored.

the problem is historically step 2 was taken after people already had their residency interviews lined up and nobody ever studied for it because it was so easy and nobody cared about the scores.

If you make Step 1 P/F, almost by definition you have to make Step 2 scores replace it in importance which then means you have to take Step 2 and get results before you can apply for residency. But you can't take Step 2 until you've done enough of your 3rd year rotations and most people generally don't have the time to study for months specifically for Step 2...

So while it sounds nice to just replace Step 1 importance with Step 2 importance, the logistics of that get a little messy when you look at actual timelines.

 

[VA Hopeful Dr]

the problem is historically step 2 was taken after people already had their residency interviews lined up and nobody ever studied for it because it was so easy and nobody cared about the scores.

If you make Step 1 P/F, almost by definition you have to make Step 2 scores replace it in importance which then means you have to take Step 2 and get results before you can apply for residency. But you can't take Step 2 until you've done enough of your 3rd year rotations and most people generally don't have the time to study for months specifically for Step 2...

So while it sounds nice to just replace Step 1 importance with Step 2 importance, the logistics of that get a little messy when you look at actual timelines.

My guess is you might see a further shortening of pre-clinical years (some schools were already doing that) and starting rotations earlier to address the exact problem you're describing.

 

[mostwanted]

the problem is historically step 2 was taken after people already had their residency interviews lined up and nobody ever studied for it because it was so easy and nobody cared about the scores.

If you make Step 1 P/F, almost by definition you have to make Step 2 scores replace it in importance which then means you have to take Step 2 and get results before you can apply for residency. But you can't take Step 2 until you've done enough of your 3rd year rotations and most people generally don't have the time to study for months specifically for Step 2...

So while it sounds nice to just replace Step 1 importance with Step 2 importance, the logistics of that get a little messy when you look at actual timelines.

Logistics will be easy to workout, most schools will or already have moved towards a1.5 yr preclinical phase

 

[AnonymousPD]

I am all for making residency selection a more efficient, equitable, and merit-based process. But to those who feel changing Step 1 to pass/fail is a good thing, you should be careful what you wish for. Without a well thought out, comprehensive, and coherent alternative presented, eliminating another objective measure will make things worse for students. Over the last few days, every single PD and APD I have spoken to or heard from feels this way. Every single one.

I am on record in the other thread of saying the USMLE scores are limited in value. However, the exam while flawed, presents the closest thing to a level playing field for every student in the country. No matter your school or background, the test is the same and everyone knows what’s on it. It is one of the few ways a student from a new school without an established reputation can objectively compete head to head with a student from a blue-blood institution with hundreds of years of tradition. Maybe you are a late bloomer that found focus in medical school instead of undergrad. One of the ways you have to show this has now been taken away from you. And rather than de-emphasizing the importance of testing, all they have done is leave you with a winner take all USMLE Step 2. I’m sure that isn’t going to be stressful in the least.

The point was made that the exam covers material that may not have much to do with your chosen specialty with an example referencing biochemistry. I noticed how on this anesthesia forum the example given wasn’t pharmacology. I don’t imagine over on the surgical forums anyone is making the argument that Step 1 is useless because of the anatomy questions. Can someone go check the psychiatry forum to see if anyone said that time spent reviewing the behavioral sciences section is time wasted? Yes, it is flawed, but without a better alternative to assess pre-clinical knowledge, Step 1 is far better than nothing.

The point of pre-clinical education is to give students a broad exposure to subjects so they can discover what they love the most in the vast spectrum of medical science. Without a magical “sorting stethoscope” to help medical students to choose their specialty at the white coat ceremony during orientation, you must accept that you are going to be asked to learn a lot of material that will be irrelevant to your chosen field of work. But that’s how the world works. How’s all that history you took in high school helping you right now? We don’t know what we love until we are exposed to it. And then some of us become musicians, some become history teachers, some become doctors that perform surgery, and some become doctors who do need to know how the Krebs cycle works.

The root cause of selection stress isn’t being evaluated. It’s the uncertainty due to competition for scarce resources. No amount of GME expansion will prevent there from always being desirable specialties and programs with more applicants than positions. Which means there will always be stress during the match process. I can’t think of anywhere that teaches people how to deal with inevitable stress by instituting measures to avoid it. The first code you ran was stressful until you ran a bunch of them. Pilots go through all sorts of simulated disasters. Traumatic airways in super morbidly obese patients are life and death stressful situations, so is the NBME suggesting we should make residents avoid them? Or fail to critique what they did as long as the patient lives (pass/fail)? ECMO and surgical airways for everyone!

Much has been made about the effect on recruiting. I will simply say that the dumbing down of things (complete pass/fail curriculums, 2/3 of a class getting honors, schools that won’t rank their students) over the past decade has poorly served the students. Elimination of objective measures is forcing us to allocate limited positions based on things that are subjective instead of basing them on merit. Do we really want to live in a world where doing an away rotation is the most important thing? Then good luck to those with fewer resources. If we keep homogenizing the application process, thats where we’re going. Among the things I tell college students picking medical schools is that they should value places that will objectively identify hard work and sacrifice. Go somewhere where you’re measured on day one. Pick a place that will show residency programs all the things you were willing to do to excel that your classmates were not.

Is there a better way than what we’ve been doing? I’m sure there is. But we need to find more and better ways to determine merit, not less. Medical students are incredible. You’re a compassionate group with a commitment for service. You’ve been the best of the best academically since kindergarten. You put other undergrads to shame when it comes to sacrificing for the future. What you need us to do as educators is keep pushing you to be your best and setting the bar higher. You need us to support you, help you learn from your mistakes, and give you the resources to help you cope with the realities of taking care of the sick and dying. Putting you in bubble wrap is an insult to you and your potential.

 

[Dantrolene FC]

I am all for making residency selection a more efficient, equitable, and merit-based process. But to those who feel changing Step 1 to pass/fail is a good thing, you should be careful what you wish for. Without a well thought out, comprehensive, and coherent alternative presented, eliminating another objective measure will make things worse for students. Over the last few days, every single PD and APD I have spoken to or heard from feels this way. Every single one.

I am on record in the other thread of saying the USMLE scores are limited in value. However, the exam while flawed, presents the closest thing to a level playing field for every student in the country. No matter your school or background, the test is the same and everyone knows what’s on it. It is one of the few ways a student from a new school without an established reputation can objectively compete head to head with a student from a blue-blood institution with hundreds of years of tradition. Maybe you are a late bloomer that found focus in medical school instead of undergrad. One of the ways you have to show this has now been taken away from you. And rather than de-emphasizing the importance of testing, all they have done is leave you with a winner take all USMLE Step 2. I’m sure that isn’t going to be stressful in the least.

The point was made that the exam covers material that may not have much to do with your chosen specialty with an example referencing biochemistry. I noticed how on this anesthesia forum the example given wasn’t pharmacology. I don’t imagine over on the surgical forums anyone is making the argument that Step 1 is useless because of the anatomy questions. Can someone go check the psychiatry forum to see if anyone said that time spent reviewing the behavioral sciences section is time wasted? Yes, it is flawed, but without a better alternative to assess pre-clinical knowledge, Step 1 is far better than nothing.

The point of pre-clinical education is to give students a broad exposure to subjects so they can discover what they love the most in the vast spectrum of medical science. Do you have a magical “sorting stethoscope” to help medical students to choose their specialty at the white coat ceremony during orientation? Unless you do, you must accept that you are going to be asked to learn a lot of material that will irrelevant to your chosen field of work. But that’s how the world works. How’s all that history you took in high school helping you right now? We don’t know what we love until we are exposed to it. And then some of us become musicians, some become history teachers, some become doctors that perform surgery, and some become doctors who do need to know how the Krebs cycle works.

The root cause of selection stress isn’t being evaluated. It’s the uncertainty due to competition for scarce resources. No amount of GME expansion will prevent there from always being desirable specialties and programs with more applicants than positions. Which means there will always be stress during the match process. I can’t think of anywhere else where teaching people to deal with unavoidable stress is to avoid it even more. The first code you ran was stressful until you ran a bunch of them. Pilots go through all sorts of simulated disasters. Traumatic airways in super morbidly obese patients are life and death stressful situations, so is the NBME suggesting I make my residents avoid them? Or fail to critique what they did as long as the patient lives (pass/fail)? ECHMO and surgical airways for everyone!

Much has been made about the effect on recruiting. I will simply say that the dumbing down of things (complete pass/fail curriculums, 2/3 of a class getting honors, schools that won’t rank their students) over the past decade has poorly served the students. Elimination of objective measures is forcing us to allocate limited positions based on things that are subjective instead of basing them on merit. Do we really want to live in a world where doing an away rotation is the most important thing? Then good luck to those with fewer resources. If we keep homogenizing the application process, that where we’re going. Among the things I tell college students picking medical schools is that they should value places that will objectively identify hard work and sacrifice. Go somewhere where you’re measured on day one. Pick a place that will show residency programs all the things you were willing to do to excel that your classmates were not.

Is there a better way than what we’ve been doing? I’m sure there is. But we need to find more and better ways to determine merit, not less. Medical students are incredible. You’re a compassionate group with a commitment for service. You’ve been the best of the best academically since kindergarten. You put other undergrads to shame when it comes to sacrificing for the future. What you need us to do as educators is keep pushing you to be your best and setting the bar higher. You need us to support you, help you learn from your mistakes, and give you the resources to help you cope with the realities of taking care of the sick and dying. Putting you in bubble wrap is an insult to you and your potential.

Another very well written post, and as an MS-4, I agree with everything you said.

I struggled through 1st year of med school, but I had an awakening experience around the summer between 1st and 2nd year. Long story short, I worked my ass off in 2nd year to make up for my lazy 1st year. I studied all day for 6 weeks for step 1 with only 2 days off, because I knew this would be the only way to show residency programs my potential since my school is p/f and doesn’t rank our students.

Anyway, I ended up with a decent step 1 score and got some interviews at some really top notch programs, which I most likely wouldn’t have gotten if step 1 was p/f.

I can’t imagine the impact this will have for future DO students, IMGs, and students from lesser known MD schools since they won’t have their med school name recognition on their applications.

 

[DNAJB6]

Medical school tier & MCAT as well as undergrad competitiveness and SAT will therefore be more important.

Why would anyone care about your under grad school or SAT for residency?

 

[JiPo]

Why would anyone care about your under grad school or SAT for

it is not that residency programs care about your undergrad or SAT. Having done well on SAT will get you to better undergrad, and better undergrad will get you to better medical school. With Step 1 now P/F, going to a good medical school became more important.

 

[JiPo]

Step 1 allowed for “mobility” from lower to higher tier as you progress through training. This is absolutely a terrible decision, especially for DOs and IMGs, and even lower tier USMDs

 

[pgg]

Some serious psychobabble here.......

"Our student, resident and physician members voted to endorse a pass/fail policy, in part, because we know our current residency selection system is causing significant distress for our students. The AMA is currently supporting new ideas in the transition from medical school to residency through our funding of the Reimagining Residency Initiative and our Accelerating Change in Medical Education Consortium. We look forward to working with our colleagues across the continuum of medical education in developing improvements in physician education transitions.”
– Susan Skochelak, MD MPH, Chief Academic Officer for the AMA

If the AMA supports something, it's probably a bad idea. The last thing any of us need is our "colleagues" over there offering their $.02 in the matter.