c/o 2023 Step 1 Score Reporting Discrepancy for Residency?

[Bones&Bourbon]

Hi all,

Was curious if anyone had any input regarding this discrepancy. At my home institution (and many others in the US), members of the class of 2023 (current MS2's) will be taking Step 1 in the Spring of 2021 while it is still formally scored. However, there are many institutions whose members of the class of 2023 will be taking it in the Spring of 2022 following their MS3 year. With this in mind, when our class applies to residency, there will be some students applying with a three digit score, and others with P/F. I haven't seen anything discussing this from the USMLE thus far and was curious if anyone had heard anything about how this will be handled.

Thanks in advance!

---

Members don't see this ad.

 

[M&L]

yeah, we talked about it a few weeks ago extensively, if i am not mistaken. The latest thing that we found was that they made an official statement that for those who received 3 digit score, it will NOT be retroactively converted to P/F at the time of application. So, in other words, those 23 grads who take it after M2 WILL be applying with a 3 digit score.

 

[Bones&Bourbon]

yeah, we talked about it a few weeks ago extensively, if i am not mistaken. The latest thing that we found was that they made an official statement that for those who received 3 digit score, it will NOT be retroactively converted to P/F at the time of application. So, in other words, those 23 grads who take it after M2 WILL be applying with a 3 digit score.

Interesting - if that's the case then the class of 2023 will have some applicants reporting 3-digit scores and others reporting P/F. That's surely not going to cause ~any~ issues at all when it comes time to apply for residency.

Well at least we know who in the class of 2023 will have a 3-digit score vs. P/F...better than nothing!

 

[M&L]

Interesting - if that's the case then the class of 2023 will have some applicants reporting 3-digit scores and others reporting P/F. That's surely not going to cause ~any~ issues at all when it comes time to apply for residency.

Well at least we know who in the class of 2023 will have a 3-digit score vs. P/F...better than nothing!

Yeah , it is definitely a concern . But with all honesty, no one knows anything for sure yet . Way too early to tell what is going to happen , you know ?

 

[Bones&Bourbon]

Yeah , it is definitely a concern . But with all honesty, no one knows anything for sure yet . Way too early to tell what is going to happen , you know ?

Very true. If we've learned anything from 2020 thus far, it is that anything is possible.

 

[M&L]

Very true. If we've learned anything from 2020 thus far, it is that anything is possible.

Definitely. I decided that I won’t worry about it anymore , and just study for a scored exam , even if they tell us it won’t be scored, you know ? You have to learn this stuff anyway .

 

[Bones&Bourbon]

Definitely. I decided that I won’t worry about it anymore , and just study for a scored exam , even if they tell us it won’t be scored, you know ? You have to learn this stuff anyway .

A 100% fool proof strategy if there ever was one. Absolutely the safest bet, especially considering the uncertainty surrounding the whole situation. While I'd love for the USMLE to stick to their guns on this one, I wouldn't at all be surprised if more changes are coming.

 

[drducky.]

Life will go on, just like how some people applied to med school with a mcat of only 3 sections and others with the new 4 section mcat.

I think some people apply to residency without a step 2 score and some do. It won’t be much different for step 1 for c/o 2023

Though i think a pass from a top school might outweigh a 250 from a low md/do school.

 

[Bones&Bourbon]

Life will go on, just like how some people applied to med school with a mcat of only 3 sections and others with the new 4 section mcat.

I think some people apply to residency without a step 2 score and some do. It won’t be much different for step 1 for c/o 2023

Though i think a pass from a top school might outweigh a 250 from a low md/do school.

I see what you're getting at with the reference to the MCAT, but I think this situation is a bit different. Even though they were using two different exams/scoring scales, there were still relative percentiles assigned to both exams (e.g. you knew which percentile a 30 and 518 were within their respective scales, so even though there were two different exams, you still knew how well a student performed relative to their peers who took the same exam). Whereas now, if you have a student who reports a 250 and another who reports a "P", you can't objectively compare the two.

The Step 2 discrepancy you mentioned complicates the matter even further in my opinion. Let's say someone who reports a "P" on Step 1 hasn't completed Step 2 by the time they apply. How do you objectively compare that student to another who reports a 250 on Step 1 and has taken Step 2 when they apply. The program directors now have to compare one student reporting a "P" on Step 1 who has yet to take Step 2 with a student who's reporting scores for both Step 1 and Step 2. CV aside, these scores have historically been the "first filter" so to speak when it comes to sorting out applicants. The fact that these metrics may not be equally applied to all students when applying to residency could become problematic.

*Granted these are hypotheticals, just trying to stir the pot & generate some discussion about what will be a hot button topic down the road.*

 

[efle]

They're certainly going to leave scored tests visible (no retroactively converting to Pass). Not only because they said so, but because that's less likely to be successfully challenged.

In scenario 1, an applicant with a 250+ gets converted to Pass and fails to match a competitive specialty. They can VERY convincingly argue their odds were hurt by losing their score based on many years of public Charting Outcomes data.

In scenario 2, an applicant who answers just as many questions correct receives a Pass without ever knowing that they were in the 250+ range. They then also fail to match a competitive specialty. They cant make much of an argument, since they didnt even know what they missed out on displaying in the first place.

And that, in my opinion, is why the NBME is going to screw over thousands of people by making them compete with a Pass against high scores for competitive spots. Very unethical, but also much safer for them.

 

[Bones&Bourbon]

They're certainly going to leave scored tests visible (no retroactively converting to Pass). Not only because they said so, but because that's less likely to be successfully challenged.

In scenario 1, an applicant with a 250+ gets converted to Pass and fails to match a competitive specialty. They can VERY convincingly argue their odds were hurt by losing their score based on many years of public Charting Outcomes data.

In scenario 2, an applicant who answers just as many questions correct receives a Pass without ever knowing that they were in the 250+ range. They then also fail to match a competitive specialty. They cant make much of an argument, since they didnt even know what they missed out on displaying in the first place.

And that, in my opinion, is why the NBME is going to screw over thousands of people by making them compete with a Pass against high scores for competitive spots. Very unethical, but also much safer for them.

Agreed - it's going to be a mess. I just don't understand why it is so unreasonable to either: 1) Implement the P/F scoring system earlier, such that ALL members of the class of 2023 will take it with P/F scoring OR 2) State that ALL members of the class of 2023 will receive a 3-digit score, even if they take the exam after P/F scoring is implemented (e.g. class-wide, regardless of testing date). By applying changes to the class of 2023 as a whole, you ensure that ALL students in the class will have their Step 1 scores evaluated using the same objective metrics. Now program directors are going to have to compare apples to oranges when looking at Step 1 scores.

I feel like this is seemingly so simple and that this problem could've (and should've) been seen from miles away, yet was never addressed by the USMLE.

 

[slowthai]

The Step 2 discrepancy you mentioned complicates the matter even further in my opinion. Let's say someone who reports a "P" on Step 1 hasn't completed Step 2 by the time they apply. How do you objectively compare that student to another who reports a 250 on Step 1 and has taken Step 2 when they apply. The program directors now have to compare one student reporting a "P" on Step 1 who has yet to take Step 2 with a student who's reporting scores for both Step 1 and Step 2. CV aside, these scores have historically been the "first filter" so to speak when it comes to sorting out applicants. The fact that these metrics may not be equally applied to all students when applying to residency could become problematic.

*Granted these are hypotheticals, just trying to stir the pot & generate some discussion about what will be a hot button topic down the road.*

Everyone will have no choice but to take step 2 early enough to have a score. Programs will and already have started requiring it.

 

[Bones&Bourbon]

Everyone will have no choice but to take step 2 early enough to have a score. Programs will and already have started requiring it.

Which is a step in the right direction..but doesn't necessarily address the issue with Step 1 score reporting. If two students score the same on Step 2, but one reports a 250 on Step 1 and the other a "P", the student with the 3-digit score will invariably be looked upon more favorably than the student reporting a P. For all we know, the student reporting a P may have in fact performed better on Step 1 than the other student, but they have no way to show that and are therefore at a disadvantage purely because of when they took the exam.

My point being this: remove the ambiguity. Considering the weight that Step 1 carries (for better or for worse..that's an entirely different discussion), there needs to be an even playing field considering it's significance as one of the primary metrics used to filter applicants. Everyone in the class of 2023 should either have a 3-digit score or P/F. Having a mixture puts some students - namely those who are taking it P/F - at a striking disadvantage, because they have one less metric by which they can beef up their application.

Full disclosure: barring any extreme changes from the USMLE, I will be taking it while it is formally scored. I'm not trying to argue for change on my own behalf, as I'm not even in the group that will be most affected. I just don't like the idea of certain metrics (that are supposed to be objective and applied evenly) being used in different capacities simply based on an exam date.

 

[slowthai]

Which is a step in the right direction..but doesn't necessarily address the issue with Step 1 score reporting. If two students score the same on Step 2, but one reports a 250 on Step 1 and the other a "P", the student with the 3-digit score will invariably be looked upon more favorably than the student reporting a P. For all we know, the student reporting a P may have in fact performed better on Step 1 than the other student, but they have no way to show that and are therefore at a disadvantage purely because of when they took the exam.

Yeah, I was responding specifically to the step 2 aspect of your post. We've discussed this issue at length on here; my stance is that because there is always someone being hurt by this (current M2s taking it in M3, PhD students, etc), they should push it out like 10 years, leaving no room for unfair comparisons. This way, no one gets screwed.

 

[Bones&Bourbon]

Yeah, I was responding specifically to the step 2 aspect of your post. We've discussed this issue at length on here; my stance is that because there is always someone being hurt by this (current M2s taking it in M3, PhD students, etc), they should push it out like 10 years, leaving no room for unfair comparisons. This way, no one gets screwed.

Hit the nail on the head. I think the change was made too abruptly and didn't allow enough time for the system to adjust accordingly. It's going to be quite the interesting experience when application season rolls around for the current M2's.

 

[NotAProgDirector]

Agreed - it's going to be a mess. I just don't understand why it is so unreasonable to either: 1) Implement the P/F scoring system earlier, such that ALL members of the class of 2023 will take it with P/F scoring OR 2) State that ALL members of the class of 2023 will receive a 3-digit score, even if they take the exam after P/F scoring is implemented (e.g. class-wide, regardless of testing date). By applying changes to the class of 2023 as a whole, you ensure that ALL students in the class will have their Step 1 scores evaluated using the same objective metrics. Now program directors are going to have to compare apples to oranges when looking at Step 1 scores.

I feel like this is seemingly so simple and that this problem could've (and should've) been seen from miles away, yet was never addressed by the USMLE.

No matter what the USMLE does, it creates a discontinuity in the space time continuum. Even if they follow your recommendation, some schools take S1 at the end of MS2, some mid M2, and some at the end of MS3. Some people are in MD/PhD programs and take it years before they apply. So there's no time they can switch it where it doesn't affect someone.

And if they were to just choose an application cycle and just start reporting P/F for everyone, those applicants with a score would put it in their application anyway. And as mentioned someone would sue them for hiding the score.

I'm not a fan of this for multiple reasons, but there's no perfect way to make the change.

 

[slowthai]

Hit the nail on the head. I think the change was made too abruptly and didn't allow enough time for the system to adjust accordingly. It's going to be quite the interesting experience when application season rolls around for the current M2's.

Yeah. It's like a profit motive kills all common sense

 

[ace_inhibitor111]

Step 2 still exists. If applicants who have P/F Step 1 want to prove their test taking skills, all they have to do is take Step 2. Programs can still compare percentiles between those who have a scored Step 1 and those who have a P/F Step 1 and scored Step 2. I don’t see how this “screws over” thousands of applicants. More like inconveniences them at most.

 

[Mass Effect]

No matter what the USMLE does, it creates a discontinuity in the space time continuum. Even if they follow your recommendation, some schools take S1 at the end of MS2, some mid M2, and some at the end of MS3. Some people are in MD/PhD programs and take it years before they apply. So there's no time they can switch it where it doesn't affect someone.

And if they were to just choose an application cycle and just start reporting P/F for everyone, those applicants with a score would put it in their application anyway. And as mentioned someone would sue them for hiding the score.

I'm not a fan of this for multiple reasons, but there's no perfect way to make the change.

Why not make it by entering class AND by graduating class? So everyone entering in, say, 2023 will be on pass/fail system, regardless of when they take it and by consequence, everyone applying for residency in 2027 will have a P/F score. For research folks and MD/PhD folks, if they know at the outset that all scores will be changed to P/F for the 2027 match cycle, there wouldn't be liability. They knew going in (even if they enter before 2023) that if they want to join the 2027 match, their Step score will be P/F even if they got a numeric score. This way everyone knows going into med school what the deal is.

 

[Mass Effect]

Step 2 still exists. If applicants who have P/F Step 1 want to prove their test taking skills, all they have to do is take Step 2. Programs can still compare percentiles between those who have a scored Step 1 and those who have a P/F Step 1 and scored Step 2. I don’t see how this “screws over” thousands of applicants. More like inconveniences them at most.

As things stand now, it actually does screw over these applicants. When you have peers applying to residency and some have a score and some don't, it's a problem, especially when they acknowledge the reason for the change is the weight PDs put on Step 1.

 

[Bones&Bourbon]

As things stand now, it actually does screw over these applicants. When you have peers applying to residency and some have a score and some don't, it's a problem, especially when they acknowledge the reason for the change is the weight PDs put on Step 1.

@Mass Effect - My point exactly. Even once the change to P/F is implemented, the stress of standardized exams and the weight placed on them by PD's doesn't change (which was the entire point of the change to begin with). The USMLE has simply shifted the goal posts and now Step 2 will become the "new Step 1." Same stress on students, same overemphasis on residency applications.

Speaking on the point made by @ace_inhibitor111, even if everyone in the class of 2023 is required to report Step 2 scores with their application, this doesn't resolve the issue that some students will have a 3-digit score for Step 1 and others won't. To my previous point, even if two students report the same Step 2 score, in the event one student has a 250 on Step 1 while the other has a P, the student with a 3-digit score will be looked upon more favorably. PD's aren't likely to simply disregard Step 1 scores entirely and use Step 2 as the "new almighty metric" simply because some students are reporting 3-digit scores and others aren't. Step 1 scores will still matter - even if all applicants are required to report Step 2 scores - and that has an overwhelming impact on those taking it once it is pass/fail, and its not a positive impact.

 

[NotAProgDirector]

Why not make it by entering class AND by graduating class? So everyone entering in, say, 2023 will be on pass/fail system, regardless of when they take it and by consequence, everyone applying for residency in 2027 will have a P/F score. For research folks and MD/PhD folks, if they know at the outset that all scores will be changed to P/F for the 2027 match cycle, there wouldn't be liability. They knew going in (even if they enter before 2023) that if they want to join the 2027 match, their Step score will be P/F even if they got a numeric score. This way everyone knows going into med school what the deal is.

I'm not certain what you mean by both entering class and graduating class. What I think you mean is saying that in the 2027 match, no matter how you get there, all scores are reported P/F. The most common path would be starting in 2023. But starting in 2022 and taking a research year, or failing something an repeating a year would also result in a 2027 match, although those people would have a score, it just wouldn't be reported. The big carib schools have January starts who might graduate in 3.5 years, or 4.5 years. MD/PhD students have very unknown timelines.

But the problem is that once someone actually has a score, people will find a way to "report" it. They will put it in their PS. They'll bring a copy to their interviews (the report with the hologram on it so you know it's not a fake). They will convince a letter writer to include it, or perhaps it will even be in the MSPE. So, yet again, it seems "unfair" to someone.

And that's not even addressing the new "unfairness" built into the P/F system. if you first fail, then pass, programs will have no idea if you passed by a hair, or really improved. So those people will probably be negatively impacted by the change. Of course the people who just barely pass the first time will win in this new world order.

Changing the system just shuffles the deck of winners and losers. Perhaps a decreased focus on S1 will improve the first two years of medical school. And certainly the NBME makes a ridic amount of money selling practice exams -- that should be stopped immediately. But I remain concerned that the insanity will just switch to something else. Time will tell.