Pass/Fail Boards

[BorntobeDO?]

---

Members don't see this ad.

What is the difference between a candidate that makes a 230 and one that makes a 231? 232? 233?

What is the difference between two candidates that both passed step?

The problems are (1) how are residencies supposed to objectively compare candidates and (2) does the measure of comparison have internal validity in predicting resident outcomes?

The second problem is the part that I don't think anyone has actually proven. I think your first problem is the argument for the current scoring system, however there is a better way which was already suggested.

I don't think the Steps/Levels should be used for the ranking as they are broad and not applicable in many circumstances. I also agree that a standardized specialty specific test would be much more applicable and fair for everyone involved. It makes a lot more sense to make someone wanting anesthesia to take a test on anesthesia then giving them a test that spends 99% of the time focusing on other topics. That test might actually correlate well to specialty specific board pass rates and resident outcomes.

 

[AlbinoHawk DO]

The second problem is the part that I don't think anyone has actually proven. I think your first problem is the argument for the current scoring system, however there is a better way which was already suggested.

I don't think the Steps/Levels should be used for the ranking as they are broad and not applicable in many circumstances. I also agree that a standardized specialty specific test would be much more applicable and fair for everyone involved. It makes a lot more sense to make someone wanting anesthesia to take a test on anesthesia then giving them a test that spends 99% of the time focusing on other topics. That test might actually correlate well to specialty specific board pass rates and resident outcomes.

No kidding. I always got top score in the psychiatry/behavioral section but the OMM brought down my score by over 100 points. I could see a scenario where if I had applied psych a PD would only see a number and then someone that did average on psych but 90th percentile OMM would be selected. Of course all of medicine is important, but we need to stop pretending that a radiologist remembers ob/gyn or a pediatrician remembers his surgery

 

[TheRealCookieMonster]

It seems like the COMPLEX tests should be pass/fail mostly because of the merger.
The USMLE seems good as it is. It tests students on the knowledge they had to learn in medical school with a focus on critical thinking. It seems to do a good job of testing intelligence, commitment, instincts, and study diligence in an objective manner.
With a higher step score you should have more options in specialty choice and with a lower score you should have fewer. That doesn't necessarily mean that the goobers in derm or plastics are better docs. But it indicates that they worked harder and/or were smarter to start with. Which when combined with good clinicals and LORs seems to round out nicely.
I feel like if you really wanted to make things less blatantly biased you could just blind PDs to each applicants medical school. Although, I'm not sure how realistic this is.

 

[sumtimesuwonder]

The second problem is the part that I don't think anyone has actually proven. I think your first problem is the argument for the current scoring system, however there is a better way which was already suggested.

I don't think the Steps/Levels should be used for the ranking as they are broad and not applicable in many circumstances. I also agree that a standardized specialty specific test would be much more applicable and fair for everyone involved. It makes a lot more sense to make someone wanting anesthesia to take a test on anesthesia then giving them a test that spends 99% of the time focusing on other topics. That test might actually correlate well to specialty specific board pass rates and resident outcomes.

The issue with making the tests specialty specific is that many people don't pick their specialty until fourth year and the match happens in March. That doesn't give you very long to learn your set specialty when you have other rotations and school obligations that you have to stay on top of. Also, some people apply to multiple specialties and making it specialty based would make it much more difficult to be flexible. Third, what happens when someone has to SOAP and they don't get their preferred specialty that they studied for all year? Now not only are they not happy with what they got into, but they are also behind in their clinical knowledge coming in to residency. Also, do they now have to take that specialty based test after they SOAP into it? I think it would be too complicated to do it that way.

 

[shadowlightfox]

Lets just say that this decision materializes, and USMLE is pass/fail, and lets also say that this impacts DO students like the way this topic painted it would. Wouldn't that also mean that the matriculation rate at DO schools, and even application fees accrued decrease significantly? At that point, the only people applying to DO schools would be those who resigned with the idea of only pursuing primary care.

 

[dohopeful13]

Lets just say that this decision materializes, and USMLE is pass/fail, and lets also say that this impacts DO students like the way this topic painted it would. Wouldn't that also mean that the matriculation rate at DO schools, and even application fees accrued decrease significantly? At that point, the only people applying to DO schools would be those who resigned with the idea of only pursuing primary care.

That would be the logical conclusion and apps would probably drop, but people still go into no name law schools or pharmacy schools every year with a 50% or less chance of getting a decent job after. The spots will still get filled. School expansion may stop though.

 

[Trollest]

Good job!

Every medical student in the US, and even the carrebeans, know on day 1 that they have to take step 1. It's not a secret.

It seems like the COMPLEX tests should be pass/fail mostly because of the merger.

With a higher step score you should have more options in specialty choice and with a lower score you should have fewer. That doesn't necessarily mean that the goobers in derm or plastics are better docs. But it indicates that they worked harder and/or were smarter to start with. Which when combined with good clinicals and LORs seems to round out nicely.

I feel like if you really wanted to make things less blatantly biased you could just blind PDs to each applicants medical school. Although, I'm not sure how realistic this is.

 

[atomheart]

The issue with making the tests specialty specific is that many people don't pick their specialty until fourth year and the match happens in March. That doesn't give you very long to learn your set specialty when you have other rotations and school obligations that you have to stay on top of. Also, some people apply to multiple specialties and making it specialty based would make it much more difficult to be flexible. Third, what happens when someone has to SOAP and they don't get their preferred specialty that they studied for all year? Now not only are they not happy with what they got into, but they are also behind in their clinical knowledge coming in to residency. Also, do they now have to take that specialty based test after they SOAP into it? I think it would be too complicated to do it that way.

Many students already run into the time crunch of finding a specialty during fourth year, and I feel like the current system does nothing to solve it. There are a full 4 months from the time of fourth year begins until interview season begins...ample time for taking these exams. But even if this weren't the case there are already specialty specific exams that could be used in the match...they are called shelves. At the end of third year every student has already taken at least 6 specialty specific examinations that they can then use if they needed to SOAP. Again, there is ample opportunity to decide for or against these specialties during third year. Finally, programs already make plenty of concessions during soap (rads and gas taking unmatched surgical superstars for instance). The system frequently finds ways to bend over backwards to allow programs to fill.

If an applicant wants to apply to multiple specialties they are more than welcome to take additional exams during elective periods. I feel like the system described is fair and flexible while also giving specialty programs what they need to screen out applicants.

 

[AlbinoHawk DO]

Many students already run into the time crunch of finding a specialty during fourth year, and I feel like the current system does nothing to solve it. There are a full 4 months from the time of fourth year begins until interview season begins...ample time for taking these exams. But even if this weren't the case there are already specialty specific exams that could be used in the match...they are called shelves. At the end of third year every student has already taken at least 6 specialty specific examinations that they can then use if they needed to SOAP. Again, there is ample opportunity to decide for or against these specialties during third year. Finally, programs already make plenty of concessions during soap (rads and gas taking unmatched surgical superstars for instance). The system frequently finds ways to bend over backwards to allow programs to fill.

If an applicant wants to apply to multiple specialties they are more than welcome to take additional exams during elective periods. I feel like the system described is fair and flexible while also giving specialty programs what they need to screen out applicants.

Or we could also delay the entire process. I mean, why not start applying in November, submit rank list in March and then 2 weeks later know your match? Actually, it would probably be better for everyone if interviews were January to March as flights would be cheaper than competing with Holiday travelers

 

[libertyyne]

A higher Step 1 does not make you a better doctor.
However a higher step 1 does mean a less likelihood of Board certification failure which is something PDs do care about.

If we are making stressful career defining exams p/f we might as well make the SAT and MCAT p/f as well.

I will say this that a majority of people in my school that are struggling with step are also the ones who didnt do so hot in preclinicals, would barely pass exams or just do the bare minimum for class. so they arent necessarily learning more. On the contrary the people who were driven to do well also did well in class, learned the material better and are in a better position for step.

Also it is a little weird when ever people who do poorly on standardized exams claim that they are going to be able to outshine their peers who did better than them on step on the wards. On average the people who are doing well on the wards are also the same people who did well in preclinicals, there are definitely outliers that are asses or have zero socials skills, but are the exceptions rather than the norm.

 

[Goro]

As I mentioned in one of the other threads about this subject, I suspect that if NBME thinks anything like NBOME, then they don't care about your career aspirations and whether you match into Ortho or Derm.

They merely want to know if you are competent to practice Medicine.

 

[atomheart]

Or we could also delay the entire process. I mean, why not start applying in November, submit rank list in March and then 2 weeks later know your match? Actually, it would probably be better for everyone if interviews were January to March as flights would be cheaper than competing with Holiday travelers

I'm all for a solution of all programs in a specialty interviewing all applicants across a week long conference trip then ranking and matching within a month.

It would actually be neat to watch all the drama of that week unfold and meet "everyone" in my specialty.

 

[DrWhozits]

Call me closed-minded but I have no idea how anyone could think this was a good idea. The whole argument is student well being?

I may be alone but spending months and months of time doing clubs, volunteering and research projects to keep up with the Joneses sounds way worse for my well-being than the 6 weeks that I studied for an exam on content that I had been learning for 2 years.

The competitive applicants are already doing a lot of those extra things AND have great board scores. So without Step the insane amount of fluff activity would drastically increase to get anything decent.

Just my opinion though.

 

[AlbinoHawk DO]

If we are making stressful career defining exams p/f we might as well make the SAT and MCAT p/f as well.

I think the argument goes beyond stress. Many would say that it is simply useless to have a scored USMLE. Are you really more capable to do the job because you have a higher step score? Are you a better psychiatrist because you could get high scores in the OB/Gyn or surgery sections? On top of that, we know we have doctors doing all kinds of specialties now that were never able to pass the USMLE during their time and it's not as if there is something showing that they provide inferior care.

SAT and MCAT have different purposes. Once you enter med school, you pretty much cleared that hurdle of intellectual capacity. There is a thing as being too dumb to be in college or be in medical school, but is there such a thing as being too dumb to be a dermatologist for a medical student?

 

[recycledpaper]

Horrible idea. Most med schools have P/F already for their pre-clinical curriculum, now students want P/F for standardized board exams? I am sorry but if you are not cut out for the work then something has to be done to highlight that. There needs to be a line drawn regarding how far changes are willing to be made to allow those not cut out for certain jobs and specialties to continue along that path.

If you are not cut out for med school because you cannot pass boards, you should not be allowed to practice medicine. This is a stressful occupation with severe consequences for ineptitude. You can always pursue another career with less liability and intellectual demand than a physician. Or start a career in a different field.

I agree that wellness is a big issue, and is being talked about a lot in current media. This is not the answer to that problem, which is a lot more complex than some make it out to be.

 

[libertyyne]

I think the argument goes beyond stress. Many would say that it is simply useless to have a scored USMLE. Are you really more capable to do the job because you have a higher step score? Are you a better psychiatrist because you could get high scores in the OB/Gyn or surgery sections? On top of that, we know we have doctors doing all kinds of specialties now that were never able to pass the USMLE during their time and it's not as if there is something showing that they provide inferior care.

SAT and MCAT have different purposes. Once you enter med school, you pretty much cleared that hurdle of intellectual capacity. There is a thing as being too dumb to be in college or be in medical school, but is there such a thing as being too dumb to be a dermatologist for a medical student?

Yes but you left out the part where higher step scores correlate with higher board passing rates .

You would think that the being dumb would hold people back from actually completing the first two years of medical school to even sit for step, so the MCAT and sat still don't really have a purpose besides stratification into success in completing college and successful completion of medical school. One could argue that step serves the same purpose for successful passing of specialty boards.

 

[recycledpaper]

I think the argument goes beyond stress. Many would say that it is simply useless to have a scored USMLE. Are you really more capable to do the job because you have a higher step score? Are you a better psychiatrist because you could get high scores in the OB/Gyn or surgery sections? On top of that, we know we have doctors doing all kinds of specialties now that were never able to pass the USMLE during their time and it's not as if there is something showing that they provide inferior care.

SAT and MCAT have different purposes. Once you enter med school, you pretty much cleared that hurdle of intellectual capacity. There is a thing as being too dumb to be in college or be in medical school, but is there such a thing as being too dumb to be a dermatologist for a medical student?

How good you are is multifactorial. There are technical skills, ability to build clinical knowledge, memorization, recall, etc. Everything comes into play. That being said, an individual who scored a 260 is clearly better than someone who scored 200. Let's not kid ourselves and say there is no obvious difference between these two medical students. Whether they are better in their specialty relies on other factors as mentioned above, but the individual with the 260 is starting off much stronger. I have seen this first-hand, and have also experienced this first hand. If you cannot recall knowledge or critically think effectively, how are you supposed to be a better problem solver? Seriously, I want to know.

USMLE is also useful as a screening exam as there are no other objective ways to do so. You can't compare pre-clinical grades because most are P/F. So you expect programs to filter though hundreds if not thousands of applications without a subjective filter? Easier said than done.

Finally, there is some correlation with how well you do on USMLE and how well you will do on your specialty board exam. So if you stink at taking the USMLE, there is a above average chance that you will stink at your specialty board exam.

 

[recycledpaper]

Let's not forget that it is actually pretty easy to just pass COMLEX or Step with minimal studying. Is that really the type of doctor med schools should be producing? C'mon people.

 

[BorntobeDO?]

Let's not forget that it is actually pretty easy to just pass COMLEX or Step with minimal studying. Is that really the type of doctor med schools should be producing? C'mon people.

I am not so sure you have taken these exams. If it was so easy, no one would fail it who studies for months for it and yet this happened to people in my class with no prior red flags. I absolutely do not believe it is easy to pass with just minimal studying.

 

[recycledpaper]

I am not so sure you have taken these exams. If it was so easy, no one would fail it who studies for months for it and yet this happened to people in my class with no prior red flags. I absolutely do not believe it is easy to pass with just minimal studying.

Yeah, I have. I took USMLE and COMLEX, most recently Level 3 with average amount of studying, which was so much more than you need to pass. They are difficult exams if you are trying to maximize your score, but if you were just vying to pass, I do not think it is a horribly difficult task. To those people who fail after studying for months, they likely need to be 1. studying longer and/or 2. correct an underlying issue, whether it be test-related anxiety or learning difficulties with their method of learning. Some people go through med school without ever learning how to maximize their studying methods. But a small percentage don't pass these exams because they have reached their physical and mental ceiling. There is nothing wrong with that, just like how there's nothing wrong with undergrads who fail organic chem and end up changing their major to pursue a different career.

If you are unable to complete the tasks required to graduate med school or match into residency multiple times over, the narrative should not be to change the goal post to make room for your shortcomings.

 

[Espressso]

How good you are is multifactorial. There are technical skills, ability to build clinical knowledge, memorization, recall, etc. Everything comes into play. That being said, an individual who scored a 260 is clearly better than someone who scored 200. Let's not kid ourselves and say there is no obvious difference between these two medical students. Whether they are better in their specialty relies on other factors as mentioned above, but the individual with the 260 is starting off much stronger. I have seen this first-hand, and have also experienced this first hand. If you cannot recall knowledge or critically think effectively, how are you supposed to be a better problem solver? Seriously, I want to know.

USMLE is also useful as a screening exam as there are no other objective ways to do so. You can't compare pre-clinical grades because most are P/F. So you expect programs to filter though hundreds if not thousands of applications without a subjective filter? Easier said than done.

Finally, there is some correlation with how well you do on USMLE and how well you will do on your specialty board exam. So if you stink at taking the USMLE, there is a above average chance that you will stink at your specialty board exam.

Changing step 1 to p/f would just shift the priority from a basic science step 1 exam to a clinically-focused step 2 exam. So PDs would then use step 2 grades as the main factor. Which, from my understanding, is a better predictor for clinical success than step 1. Just look at the PD surveys. Step 2 is currently fourth on the list, so if you make it to p/f step 1, then step 2 just jumps to the top and the PDs still use the other metrics (LORs, clinical grades, etc.) as other means of evaluation.

If as you say, USMLE is a useful screening exam (I assume you mean step 1), then it would make sense to make it p/f as that would mean it's screening people in or out.

Imo, it's not that radical of an idea. @Goro posted a link to a thread in the pre-med forum where @meded explained the argument behind it. And it was explained much better than any medical student can. But the history of usmle and step 1 in particular is fascinating.

 

[Ho0v-man]

Literally nothing would change if step 1 was pass/fail. The same people would just crush step 2. The only difference this would make is you’d have no clue what field you were competitive for until after step 2.

 

[atomheart]

USMLE is also useful as a screening exam as there are no other objective ways to do so. You can't compare pre-clinical grades because most are P/F. So you expect programs to filter though hundreds if not thousands of applications without a subjective filter? Easier said than done.

Finally, there is some correlation with how well you do on USMLE and how well you will do on your specialty board exam. So if you stink at taking the USMLE, there is a above average chance that you will stink at your specialty board exam.

🙄

With respect to our generation of students that have to undergo screening vis-a-vis board scores, your argument suggests that the previous generation which did not have these requirements created worse physicians than today. Of course that is also the same generation that now constantly complains about today's medical school curriculum being too concerned with "high yield" and medical students that are now expert test takers with no real skills.

Then there are the waves of premed and medical students on SDN eager to boast about how their MCAT will make them a better physician than someone with a lower MCAT because there's no subjectivity or luck in medical school admissions. Clearly their decision making and problem solving skills were proven superior by a multiple choice test.

🙄🙄🙄

 

[recycledpaper]

MCAT is more far-removed from clinical medicine so obviously it has less bearing on how you will fare as a clinician in terms of judgment, decision-making, etc. That being said, I think the MCAT tests certain aspects of an individual that, if you demonstrate excellence in, sets you up for success later on. It is not perfect, but I think it is the best we have right now. At least I cannot think of a better alternative.

Your argument about older physicians is ironic because things were so different in the past. Med students were operating at the responsibility-level of a resident, etc. Nowadays, you're lucky to hold a retractor for a few hours. Are you arguing for a return to those times, when "real skills" trump test-taking performance? When you encounter physicians who went through a different training paradigm, is it not painfully obvious whether some would have been done better than others on tests?

What is SDN's obsession with disconnecting test performance with "real skills" like they are mutually exclusive? I have yet to meet a med student who did poorly on boards/pre-clinical years that blew me away on a rotation. They are slower, less comprehensive in their presentations and plans, and tend to ask poor questions due to poor baseline knowledge of the underlying processes. If you're unable to answer a test question on ARDS management, do you really think they're going to be better at managing it in real life?

 

[Chibucks15]

MCAT is more far-removed from clinical medicine so obviously it has less bearing on how you will fare as a clinician in terms of judgment, decision-making, etc. That being said, I think the MCAT tests certain aspects of an individual that, if you demonstrate excellence in, sets you up for success later on. It is not perfect, but I think it is the best we have right now. At least I cannot think of a better alternative.

Your argument about older physicians is ironic because things were so different in the past. Med students were operating at the responsibility-level of a resident, etc. Nowadays, you're lucky to hold a retractor for a few hours. Are you arguing for a return to those times, when "real skills" trump test-taking performance? When you encounter physicians who went through a different training paradigm, is it not painfully obvious whether some would have been done better than others on tests?

What is SDN's obsession with disconnecting test performance with "real skills" like they are mutually exclusive? I have yet to meet a med student who did poorly on boards/pre-clinical years that blew me away on a rotation. They are slower, less comprehensive in their presentations and plans, and tend to ask poor questions due to poor baseline knowledge of the underlying processes. If you're unable to answer a test question on ARDS management, do you really think they're going to be better at managing it in real life?

It’s obviously not mutually exclusive but I’ve also met people who destroy boards and are either terrible people, have zero social skills, are afraid to touch patients, and treat other staff (like nurses) like garbage because they think they’re superior.

There’s exceptions to every rule I doubt anyone is actually trying to say that all people who do average on step 1 are better doctors. Some people become test robots with no (or terrible) personality and some just learn better when they have patients and real life to attribute something to. Other superstars on Step are also great people who rock clinical too while people who are average boards are the cocky douches. I mean everybody’s gotta chill about this nonsense

 

[DrWhozits]

I keep seeing people say “does your score on boards really mean you will be a better doctor?” My thought is does that matter?

Sure scoring a 270 on Step 1 doesn’t mean you will be a great Orthopod but what does?

Your preclinical grades? No.
Your research? No.
Your volunteer work? No.
Your time as the president of your ortho interest group? No.

Literally the only way someone will know if they are going to be good at being a doctor is when they get into residency and learn by doing.

 

[QueenJames]

Looks like a few people from this thread would have been better off applying to NP or PA programs. This is MEDICAL SCHOOL not clinical school.

Yeah they gonna be the same a-holes clamoring that they have the "brains of a doctor, heart of a nurse" bullssshet that a majority of them do.

Eff them.

 

[LebronManning]

Thank god the chance of step 1 becoming P/F is the same of us having universal healthcare.

Its kinda funny actually. AMA is discussing this whole thing like they have serious pull to make it happen. Similar to the donkeys discussing single payer like it will actually get through to legislation lol.

 

[atomheart]

MCAT is more far-removed from clinical medicine so obviously it has less bearing on how you will fare as a clinician in terms of judgment, decision-making, etc. That being said, I think the MCAT tests certain aspects of an individual that, if you demonstrate excellence in, sets you up for success later on. It is not perfect, but I think it is the best we have right now. At least I cannot think of a better alternative.

Your argument about older physicians is ironic because things were so different in the past. Med students were operating at the responsibility-level of a resident, etc. Nowadays, you're lucky to hold a retractor for a few hours. Are you arguing for a return to those times, when "real skills" trump test-taking performance? When you encounter physicians who went through a different training paradigm, is it not painfully obvious whether some would have been done better than others on tests?

What is SDN's obsession with disconnecting test performance with "real skills" like they are mutually exclusive? I have yet to meet a med student who did poorly on boards/pre-clinical years that blew me away on a rotation. They are slower, less comprehensive in their presentations and plans, and tend to ask poor questions due to poor baseline knowledge of the underlying processes. If you're unable to answer a test question on ARDS management, do you really think they're going to be better at managing it in real life?

Medical education is a checkpoint process. There are hundreds of tests in medical school and residency that serve as checkpoints to screen out sheer inability. Most graduating medical students have the adequate knowledge, problem solving, and decision making skills to perform the basic functions of an intern in training. The boards are screening exams; nowhere does it state that better scored performance on a screening exam equates to anything more than the same knowledge competence as a minimum passing score. But now that the process has become competitive the rules and our mindsets have changed that turn this process into a race where there is such a thing as "better". Like arguing that out of two negative PPDs one is better than the other, it's a subjective matter.

Now, does more medical knowledge equate to greater skill as a physician? Probably. Do students that perform exceptionally well on board exams work harder than those with lower scores? Probably. But boards are nothing more than a snapshot of a student passing through one of hundreds of checkpoints on their way to becoming a physician. When a program uses those scores as an arbitrary screening tool it falsely affirms to the rest of us that that checkpoint has more value than others. Perhaps more important than graduating medical school to an onlooker such as an NP.

Step 1 and other boards are useful to programs insofar as every medical student must take them and therefore serve as an easy and unique (but not necessarily valid) point of comparison. As I found out during interview season it's a mistake to believe board scores are a great equalizer of residency applicants because individual programs will still set arbitrary rules such as DO and IMG applicants must have higher scores than other students. In fact, boards have moved from a point of comparison to an effective way to justify discrimination that would have occurred anyway. A DO applicant will always be second class to an MD in my specialty. There is no validity to any of these screening processes, yet they will continue because that is the nature of the medical system that needs ways to sift through so many applications. It would not be the end of the world if boards became P/F tests, and functionally that's what their intended purpose is. Ours is not better than a system without an "objective" comparison tool; a fair system can only occur by blinding residency programs to certain information like where an applicant goes to school (how does this information correlate to being a better physician?). There is plenty of room to improve resident selection. Like other problems that occur in medical education, this will likely be solved by adding additional checkpoints so residency programs can better compare their applicants.

 

[863168]

Out of curiosity, what are the intended outcomes from pushing Step 1 into a P/F system from its current iteration? I've seen adcoms mention that it will just place more emphasis on Step 2 CK, but seriously how is that a resolution to anything? Step 1 scores being used by program directors to select students is how residency program directors are finding a solution to maintaining consistency in a healthcare profession with high responsibility, friction, and stress. By admitting that the emphasis will just be placed on Step 2 CK it basically admits that Step 1 is not the underlying issue.

The notion that auditions, LORs, and other soft factors open to consideration will be more emphasized will more likely force students attending new programs or DO programs with weak rotation sites to be judged by the capability of their medical school. The few exceptions will be individuals with strong connections who can still secure their own sub-I or interviews through tapping personal networks. The only party that I can see benefiting from this change is the schools staff responsible for the first two years of basic science as they no longer have a quantitative feedback metric to judge whether or not their system structure is actually effective in placing students into a competitive position against other students. And when students move on to Step 2 CK, it's far more easier for administrators to put the blame on "lazy students" who can't time manage both rotations and studying for boards/Step 2. It feels like schools have every incentive to shy away from a more quantitative system, because having metrics like a student able to score a 250+ and still end up in a specialty they did not want is more indicative of how the school failed the student rather than being how the student failed themselves.

Another line of argumentation that I've seen is that competition is analogous to toxicity, but if that's true then why do medical schools endorse such a system to select students based on GPA/MCAT scores? It's hypocritical if medical schools are using a quantitative system, but then removing a similar implemented system from program directors on the basis that it's flawed and detrimental to the future of physicians. Again, I have no doubt that medical schools would like to shift to an arbitrary decision to select any candidate regardless of scores, because when there is no anchor to decision making then it's virtually impossible to be held accountable especially when you reduce your standards so that students now only have to pass, they no longer have to score above a pass system. I also looked into Step 1 being a bisilateral system, but honestly there are no publications or journals on it. Nor is there evidence pointing out that how Step 1 would apply a bimodal outcome to a question varies in comparison to how the MCAT would use such a question when it comes to testing students. I would be more than willing to read a statistical paper explaining this because I wasn't able to find anything on my own search.

But I digress, one of the hardest things that our current system fails to do is to blatantly state what the outcomes are going to be when it comes to implementing changes. Correcting homeless Joe's BP type 1 and type 2 DM means that homeless Joe is going to be kicked out on the street and come back because we know he's not going to 180 his life and start bringing prescriptions to his local CVS to buy psych meds & insulin with his Monopoly money. The outcomes that have been shared on the site haven't been pro-student with statements indicating that education doesn't end with Step 1 or that any specialty should be acceptable. These aren't reasons to implement a change, these are a lot more like excuses to support obfuscation that will likely hurt students who haven't had a Yellow Brick Road paved for them from start to finish. And if this is an incorrect assumption, then I really would like to know how a P/F system would benefit the fragile M1/M2 student population who competed to get into medical school.

 

[Goro]

Another line of argumentation that I've seen is that competition is analogous to toxicity, but if that's true then why do medical schools endorse such a system to select students based on GPA/MCAT scores? It's hypocritical if medical schools are using a quantitative system, but then removing a similar implemented system from program directors on the basis that it's flawed and detrimental to the future of physicians.

Well, GPA and MCAT are good indicators of whether or not you'll make it through med school, and how easy or difficult it will be to do so.

Once you're in med school, you don't have to worry about that level anymore. You need to demonstrate competence if several different domains, only one of which has to do with mental firepower. This is, of course, anathema to the hyper-achievers who look at grades as an affirmation of their self worth.

I agree that IF one comes from a P/F school, and Steps are also P/F, that it doesn't leave much for PDs to get an idea of what to screen for. But the status quo is indeed toxic, and something has to change.

 

[shadowlightfox]

Its kinda funny actually. AMA is discussing this whole thing like they have serious pull to make it happen. Similar to the donkeys discussing single payer like it will actually get through to legislation lol.

Yeah. Didn't AMA also say that COMLEX will be viewed as equally as USMLE? haha....

 

[863168]

Well, GPA and MCAT are good indicators of whether or not you'll make it through med school, and how easy or difficult it will be to do so.

Once you're in med school, you don't have to worry about that level anymore. You need to demonstrate competence if several different domains, only one of which has to do with mental firepower. This is, of course, anathema to the hyper-achievers who look at grades as an affirmation of their self worth.

I agree that IF one comes from a P/F school, and Steps are also P/F, that it doesn't leave much for PDs to get an idea of what to screen for. But the status quo is indeed toxic, and something has to change.

Of course this cycles back to outcomes, what is the projected outcome by changing Step 1 to a P/F system if that simply shifts the burden over to Step 2 CK? There is clearly not a consensus among students, adcoms, and program directors that Step 1 is a test that is irrelevant in being a factor to determine clinical competency which means that it is a contention that has value for consideration. Therefore, if these changes are to implement a cosmetic substitution, then why is there a push to change it? It's because in all likelihood the notion of it being a cosmetic substitution is probably the best possible outcome and at worst it will result in more bifurcation between strong/weak programs and MD/DO placements. In this cycle there has been so much pride in "superstar DO" students who have been able to land competitive specialties in traditionally MD program. Why is this viewed as toxicity?

The system is competitive to a fault not due to Step 1, it's more likely because of the increased competition from new medical programs and IMG graduates from Ross/SGU who are throwing millions of dollars at hospital systems to shift the landscape for more favorable placement outcomes. At least DO representation in NY had the right idea in arguing that US medical graduates should have priority of residency placements before placement of IMGs/FMGs. However, I don't understand why you're on board for a Step 1 change when I just see it having long term negative outcomes for DO students when a traditional MD program doesn't need to give any mind to a "superstar DO" because the only metric they have to go on is MD or DO. If there is a significant mismatch problem between medical schools and program directors to the extent that it sees like medical school administrators are pushing the idea of "hiding" the data metric that program directors are currently using then that's a significant issue with leadership that ought to be dealt with on the leadership level. It feels like students who are stressed/exhausted from Step 1 prep are literally being co-opted into a false solution by parties that have absolutely no interest in what their actual outcomes are going to be from these changes.

 

[atomheart]

The notion that auditions, LORs, and other soft factors open to consideration will be more emphasized will more likely force students attending new programs or DO programs with weak rotation sites to be judged by the capability of their medical school. The few exceptions will be individuals with strong connections who can still secure their own sub-I or interviews through tapping personal networks. The only party that I can see benefiting from this change is the schools staff responsible for the first two years of basic science as they no longer have a quantitative feedback metric to judge whether or not their system structure is actually effective in placing students into a competitive position against other students. And when students move on to Step 2 CK, it's far more easier for administrators to put the blame on "lazy students" who can't time manage both rotations and studying for boards/Step 2. It feels like schools have every incentive to shy away from a more quantitative system, because having metrics like a student able to score a 250+ and still end up in a specialty they did not want is more indicative of how the school failed the student rather than being how the student failed themselves.

Actually my school is more concerned with students passing the exams, and they take little or no stake in what kind of results the scores bear. Typically every year that follows suffers the programmatic changes that result from the previous year's board failures. What my school is looking for is a 100% pass rate, and preferably a 100% match rate but not necessarily a high quality match. The outcomes that students believe are important are not that of the schools; medical schools are not in the business of making promises about matching students into their preferred residencies.

I think you underestimate how important those "soft factors" are in getting ranked at a program because they reared their head at all of my interviews this year. I assure you that DO students do not have access to all the soft factors that get MD students their choice of residency. There is plenty of nuance in this system beyond step score.

 

[Goro]

Of course this cycles back to outcomes, what is the projected outcome by changing Step 1 to a P/F system if that simply shifts the burden over to Step 2 CK? There is clearly not a consensus among students, adcoms, and program directors that Step 1 is a test that is irrelevant in being a factor to determine clinical competency which means that it is a contention that has value for consideration. Therefore, if these changes are to implement a cosmetic substitution, then why is there a push to change it? It's because in all likelihood the notion of it being a cosmetic substitution is probably the best possible outcome and at worst it will result in more bifurcation between strong/weak programs and MD/DO placements. In this cycle there has been so much pride in "superstar DO" students who have been able to land competitive specialties in traditionally MD program. Why is this viewed as toxicity?

The system is competitive to a fault not due to Step 1, it's more likely because of the increased competition from new medical programs and IMG graduates from Ross/SGU who are throwing millions of dollars at hospital systems to shift the landscape for more favorable placement outcomes. At least DO representation in NY had the right idea in arguing that US medical graduates should have priority of residency placements before placement of IMGs/FMGs. However, I don't understand why you're on board for a Step 1 change when I just see it having long term negative outcomes for DO students when a traditional MD program doesn't need to give any mind to a "superstar DO" because the only metric they have to go on is MD or DO. If there is a significant mismatch problem between medical schools and program directors to the extent that it sees like medical school administrators are pushing the idea of "hiding" the data metric that program directors are currently using then that's a significant issue with leadership that ought to be dealt with on the leadership level. It feels like students who are stressed/exhausted from Step 1 prep are literally being co-opted into a false solution by parties that have absolutely no interest in what their actual outcomes are going to be from these changes.

I'm not sure that going P/F IS the best solution or not for Step I.

But I reiterate that that it may be that NBME does not care about your career goals, just that you won't kill patients.


this reminds me of a story of a friend of mine who wanted to become a filmmaker. He gathered a group of his friends to make a film that he and I had written the screenplay too, and one day we were filming a scene where a car had to drive under bridge owned by the Long Island Railroad.

The bridge was private property and sure enough a cop car drove underneath us just as we were filming.

The cop pulled over came up to where we were and told us we had to leave.

And the first thing that of my friend's mouth was "but what about my education?"

The cop said "I don't care about your education, get off of private property"

In this case, the nbme might be the cop. I want my students to do well, but I don't want them getting burnt out and becoming suicidal, anxious, and or depressed either

can we have some crowdsourcing and come up with alternatives to pass / fail steps?

Would percentiles work instead?

 

[863168]

@Goro I have to run out. I'll edit this post with a response tomorrow. Thank you for the discourse.

 

[Feelgood MD]

Thank god the chance of step 1 becoming P/F is the same of us having universal healthcare.

Its kinda funny actually. AMA is discussing this whole thing like they have serious pull to make it happen. Similar to the donkeys discussing single payer like it will actually get through to legislation lol.

Now this is an elephant in the room we did not need and no one should address.

 

[Deecee2DO]

Honestly not in favor of P/F step. Step is my chance to set myself apart from other applicants even if i had a crap MCAT and was accepted at DO school because I didnt feel like studying pointless physics and orgo. Now I actually enjoy studying medical school info (unlike MCAT) so I dont think its unreasonable to study my butt off to know information that will be on Step which is for the most part pretty clinically relevant (some minutiae not so much). I dont want to go through med school to know I only have a shot at matching FM which would honestly suck so bad. I think the solution is not to make the exam P/F but to either stratify the step scores in quartiles like class rank or maybe schools can actually teach to boards for gosh sakes. I cant stand when professors in med school dont teach to boards it doesnt make sense to me. If they taught directly to Step board studying wouldnt be as bad as it can be when youre balancing classwork and board prep. Many students are barely passing class in M2 to crush boards-It shouldnt be like that. teach to boards!

 

[shadowlightfox]

I remember there being a podcast talking about a similar problem about this almost three-quarters a year back. One of the school administrators who was interviewed said that the more ideal solution would be to indeed look at more than just step scores, but for the sake of making the lives of program director's easy, quantify each category into a numerical representation and add them up. So you can have one category representing preclinical grades, another for clinical years, another for extracurriculars, another for research, another for boards, etc. and depending on how merited each category is, the better the quality in each facet, the higher the number, and then you add the numbers up and that's how the program directors would look at you and compare you with other people. And just like how USMLE/COMLEX breaks down the overall score into the subcategories and you get to see how you did on each of them, so, too, can this method.

This is just an idea that one of the physicians brought up as a potential alternative, not something actually in the process of being done mind you.

I think this in the long run is not a bad idea assuming it's figured out how it can be done properly. We can be done with the days of putting all of our eggs in the boards basket, stop dictating the course of the rest of our lives with literally just one or two exams, and actually focus on being well-rounded as a gauge of quality and dictate that into whether or not an applicant is truly competitive. This way someone who truly likes a specialty but has not done very well on the boards can still have his or her desired specialty attainable, and the student just simply has to show he or she is more hard-working.

But right now, having board scores graded is all we got.

 

[Deecee2DO]

Also, how was everything stratefied back in the 70s and 80s? Ive heard some of the older docs say board scores werent used with the same emphasis they are now. If you wanted to be a plastic surgeon back then did you need to score a 250?

 

[shadowlightfox]

Also, how was everything stratefied back in the 70s and 80s? Ive heard some of the older docs say board scores werent used with the same emphasis they are now. If you wanted to be a plastic surgeon back then did you need to score a 250?

The way med school worked back then was quite different. I don't remember the exact details, but after you graduated, everyone did an intern year, and then you decided your specialty after that. I could be wrong in the overall details but it went something like this.

And nothing was really very competitive back then.

 

[DNC127]

Our DO school has our own hospital with all ward based rotations, where we all round and get pimped by residents and such and can see the performance of our peers first hand and how the residents respond to students.

All the people in our class who did best on USMLE/COMLEX are largely the better students on the wards, and will likely be better interns than those who did not. (who knows if this translates further once you get deeper in your specialty). I have seen residents visibly favor these students by giving more opportunities/procedures, and asking for their opinion in a genuine way when they would never do so to the lower performing students. I can totally see why USMLE scores are valuable because the students who didn't do well on boards have less knowledge which means they will need more instruction as an intern (read baby sitting by upper levels). If PDs/residents can select someone who will likely make their life easier why would they not? Obviously there are exceptions like the dude's who gun ortho from the start despite average scores and then go on ortho auditions and kill it because they know more than all the other bros who focused on all of medicine and thus will have more ortho knowledge, but these guys are few and far between and many of them killed boards anyways.

Scored Boards = good for students = good for residents = good for PDs

Sorry not sorry.