Step 2 Percentiles and Scoring Stratification Since Step 1 is going P/F

[Dryad8212]

Can anyone inform to me why the average step 2 ck score is so high relative to step 1?

How will PDs be able to straify STEP 2 ck scores since they are so close to each other and there is not much variance. For example, anesthesia average step 2 -245 Vs. Derm being 248. That is only a three point difference. Rads has a step 2 ck score of 248 while Derm is 255. Pediatrics is 241 while again anesthesia is 245. Anesthesia has always has a much higher step score than Peds and is way more competitive.

Can anyone comment on this and contribute?

These averages are taken from the 2020 NRMP. Thanks

---

Members don't see this ad.

 

[The Med-Peds Dude]

I don't understand why they kept Step 2 CK scores visible. It's nice that they qualitatively report more emphasis on MSPE and evals, but if you can just substitute one USMLE MCQ test for another, they didn't really have to adapt. The obvious question is really "does the ranklist change if we take away USMLEs"

Maybe because Step 2 is actually clinically relevant? In any event, it would be a mistake to rely on Step 2 for interview decisions since results are frequently not available during the interview selection process. As for ranklist, I've already pointed out that there is relatively little interest in either USMLE step post interview. I don't see Step 1 P/F will elevate Step 2, since neither are presently a major concern in rank list. I still hold that by the time the rank meeting is held, the only use for Step scores would be as tie-breakers for similar applicants.

 

[FindersFee5]

Can anyone inform to me why the average step 2 ck score is so high relative to step 1?

How will PDs be able to straify STEP 2 ck scores since they are so close to each other and there is not much variance. For example, anesthesia average step 2 -245 Vs. Derm being 248. That is only a three point difference. Rads has a step 2 ck score of 248 while Derm is 255. Pediatrics is 241 while again anesthesia is 245. Anesthesia has always has a much higher step score than Peds and is way more competitive.

Can anyone comment on this and contribute?

These averages are taken from the 2020 NRMP. Thanks

There's going to be more stratification when people start to study more. In the past, the amount of studying between different specialties was a little bit more homogenous. With increased importance, however, you'll see applicants to competitive specialties begin studying harder for step 2 than applicants to less competitive specialties.

 

[efle]

Nobody cares about Step 2 right now, that's why the derm and surgical specialty folks perform more similarly to the plebeian masses. As soon as Step 1 Pass/Fail kicks in, a new arms race for CK will take off.

 

[Redpancreas]

Nobody cares about Step 2 right now, that's why the derm and surgical specialty folks perform more similarly to the plebeian masses. As soon as Step 1 Pass/Fail kicks in, a new arms race for CK will take off.

It will be an immediate switch. It's human nature for us to use what we know...until what we know is not available.

 

[Redpancreas]

Can anyone inform to me why the average step 2 ck score is so high relative to step 1?

How will PDs be able to straify STEP 2 ck scores since they are so close to each other and there is not much variance. For example, anesthesia average step 2 -245 Vs. Derm being 248. That is only a three point difference. Rads has a step 2 ck score of 248 while Derm is 255. Pediatrics is 241 while again anesthesia is 245. Anesthesia has always has a much higher step score than Peds and is way more competitive.

Can anyone comment on this and contribute?

These averages are taken from the 2020 NRMP. Thanks

It's simply the way the exam is scored. CK has already started to become harder starting around 2018ish they have been gradually increasing the difficulty of the exam and I imagine NBME's already working on what to do when Step 1 becomes P/F.

Edit: Basing this off of my experience with USMLE UWorld Step 2 CK from 2018-2020 and anecdotal reports subject to bias.

 

[Dryad8212]

There's going to be more stratification when people start to study more. In the past, the amount of studying between different specialties was a little bit more homogenous. With increased importance, however, you'll see applicants to competitive specialties begin studying harder for step 2 than applicants to less competitive specialties.

I mean like how much more ? Because all of the scores are auper higher already with high test averages for each.

 

[efle]

I mean like how much more ? Because all of the scores are auper higher already with high test averages for each.

Most people only do a couple weeks of review for Step 2, instead of a couple months. And it's usually a leisurely brush up on certain topics, instead of comprehensive cram. I think over the next couple years you'll see people start taking longer dedicated periods, trying to do more (like an entire pass through all of UWorld Step 2), and probably using Anki more. It's going to be a night and day difference when it switches from an optional component of ERAS to being the single most impactful part of your ERAS.

 

[The Med-Peds Dude]

Where to start...? I'll use Derm as an example.

Can anyone inform to me why the average step 2 ck score is so high relative to step 1?

How will PDs be able to straify STEP 2 ck scores since they are so close to each other and there is not much variance. For example, anesthesia average step 2 -245 Vs. Derm being 248. That is only a three point difference. Rads has a step 2 ck score of 248 while Derm is 255. Pediatrics is 241 while again anesthesia is 245. Anesthesia has always has a much higher step score than Peds and is way more competitive.

Can anyone comment on this and contribute?

These averages are taken from the 2020 NRMP. Thanks

Because it doesn't require rote memorization. When the scores are this tight, they select a cut-off for selecting interviewees, which in Derm seems to be around 230-ish for applicants that go on to match. The 2nd-5th most important factors in selecting for interviews are 2) rotation in department, 3) grade in required clerkship, 4) MSPE, and 5) commitment to specialty

Ranking is entirely different: 1) feedback from residents, 2) interaction with faculty, 3) interaction with housestaff, and 4) interpersonal skills

It's simply the way the exam is scored. CK has already started to become harder starting around 2018ish they have been gradually increasing the difficulty of the exam and I imagine NBME's already working on what to do when Step 1 becomes P/F.

Seems to me that if Step 2 has become more difficult, you'd be seeing a drop in scores. In 2020, the mean step 2 score for matched MD seniors was 256; in 2016, it was 257.

It's going to be a night and day difference when it switches from an optional component of ERAS to being the single most impactful part of your ERAS.

I don't think anyone knows.

 

[efle]

I don't think anyone knows.

True, but history at least rhymes. Look at how insane Step 1 Mania got. I can't imagine Step 2 is going to stay a chill, optional component you review for 10-15 days and then send it.

 

[The Med-Peds Dude]

True, but history at least rhymes. Look at how insane Step 1 Mania got. I can't imagine Step 2 is going to stay a chill, optional component you review for 10-15 days and then send it.

I've been rhyming for a long time; Step 1 has never been unimportant. It has been important for a reason - it predicts the likelihood of passing specialty board exams the first time (ACGME is moving to decrease emphasis of this as a quality indicator). Step 2 does not. Also, the nature of clinical education precludes staying home from lecture to study uworld.

 

[FindersFee5]

I've been rhyming for a long time; Step 1 has never been unimportant. It has been important for a reason - it predicts the likelihood of passing specialty board exams the first time (ACGME is moving to decrease emphasis of this as a quality indicator). Step 2 does not. Also, the nature of clinical education precludes staying home from lecture to study uworld.

I don't think correlation to passing boards is the main reason it's important, especially not in the way the competitive specialties utilize scores. Sure, you're less worried about somebody with a 230 passing their boards than somebody who scored a 205, but the relationship falls off at the higher ends. Someone with a 245 and someone with a 265 are likely not going to be different in their ability to pass boards. Yet most derm PDs would strongly favor interviewing the person with the 260+ over the 240.

It's important because PDs are inundated with applications, and step 1 is an easy numeric way to sort this pile and eliminate some applicants right off the bat.

 

[efle]

I've been rhyming for a long time; Step 1 has never been unimportant. It has been important for a reason - it predicts the likelihood of passing specialty board exams the first time (ACGME is moving to decrease emphasis of this as a quality indicator). Step 2 does not. Also, the nature of clinical education precludes staying home from lecture to study uworld.

It's only predictive at the low range of scores, no significant gap in board passage for 230 vs 250 which is where most of the mania is focused. And in the late 90s when Step had already been around for years, the average matched applicant to the most competitive fields like ophtho were within a few points of the national average. This phenomenon of 85th percentile to be an average applicant only began in the 2000s-2010s as applications per capita skyrocketed and PDs required a sorting/filtering tool.

 

[The Med-Peds Dude]

I don't think correlation to passing boards is the main reason it's important, especially not in the way the competitive specialties utilize scores. Sure, you're less worried about somebody with a 230 passing their boards than somebody who scored a 205, but the relationship falls off at the higher ends. Someone with a 245 and someone with a 265 are likely not going to be different in their ability to pass boards. Yet most derm PDs would strongly favor interviewing the person with the 260+ over the 240.

It's important because PDs are inundated with applications, and step 1 is an easy numeric way to sort this pile and eliminate some applicants right off the bat.

Well, yes, it is a quick way to stratify applicants for further review, but I can tell you that if they set the lower bounds for further review at 240, 260 falls in the same group of applications to review as everyone else, for the exact reason you stated. Don't overlook all the other historically important criteria for interviewing that I listed above. Also, consider that a lot of Step 2 scores are not even available with ERAS opens for programs.

Yes, I am very familiar with being inundated with applications. I get over 100/position.

 

[efle]

Yes, I am very familiar with being inundated with applications. I get over 100/position.

How are you planning to sort/filter after Step 1 goes p/f?

 

[The Med-Peds Dude]

It's only predictive at the low range of scores, no significant gap in board passage for 230 vs 250 which is where most of the mania is focused. And in the late 90s when Step had already been around for years, the average matched applicant to the most competitive fields like ophtho were within a few points of the national average. This phenomenon of 85th percentile to be an average applicant only began in the 2000s-2010s as applications per capita skyrocketed and PDs required a sorting/filtering tool.

Exactly, which is why most matched applicants in Derm are above 230, and if that is where you set the cutoff for reviewing the rest of the application, 250 isn't going to get you any further. 85th percentile? The mean step 2 score is about 240, 75th percentile north of 256. The typical Derm match is not that high.

I've been doing this since 2000....

 

[The Med-Peds Dude]

How are you planning to sort/filter after Step 1 goes p/f?

This past season we treated all Step 1 scores as P/F. Then we added up points for AOA, publications, narratives from clinical evaluations, regional location (from this region), letters, intelligibility and originality of personal statement, reasons for prolonging school (pregnancy and personal illness are good reasons) and others. Class rank is nearly useless, as so many schools don't report it anymore. In the end we had a mix of good and very good applicants, ranked them by score (including an interview score), and then debated about them enough that our final rank list looked much different from the beginning. Hard work.

 

[efle]

This past season we treated all Step 1 scores as P/F. Then we added up points for AOA, publications, narratives from clinical evaluations, regional location (from this region), letters, intelligibility and originality of personal statement, reasons for prolonging school (pregnancy and personal illness are good reasons) and others. Class rank is nearly useless, as so many schools don't report it anymore. In the end we had a mix of good and very good applicants, ranked them by score (including an interview score), and then debated about them enough that our final rank list looked much different from the beginning. Hard work.

Wow did you have any kind of first pass filter using USMLE? Or you guys scored up every single applicant? Also very surprising you factor in an original personal statement as a reason to pick someone!

 

[FindersFee5]

Wow did you have any kind of first pass filter using USMLE? Or you guys scored up every single applicant? Also very surprising you factor in an original personal statement as a reason to pick someone!

I think med-peds is also going to be different as far as this goes. I've talked to a few PD in my specialty, and they have all admitted to stratifying (if only to sort) by USMLE score, such that a 240 IS different than a 260. I also think the data supports this - I'm sure there are confounders in that the types of students who score well on step 1 also being the types of students who try hard in other areas, but I doubt that confounding is enough to solely explain how large the difference in match % is at 240 vs 250 vs 260 in the competitive specialties.

 

[The Med-Peds Dude]

Wow did you have any kind of first pass filter using USMLE? Or you guys scored up every single applicant? Also very surprising you factor in an original personal statement as a reason to pick someone!

Honestly, if you just skim the applications, there are a large number of Apps to mark inactive without a lot of analysis. Graduated from medical school 10+ years ago. Probably no. From an international school with a sketchy history - no. Personal statement that mention an entirely different specialty - no. As for Personal Statement, as long as it is mostly grammatically correct, it's fine. Boring but intelligible get a pass. Occasionally there are excellent essays and they get extra points. In my specialty, 100% of programs cite personal statement as a factor for interview selections, though the importance score is much less than, say, "Evidence of professionalism and ethics" https://mk0nrmp3oyqui6wqfm.kinstacdn.com/wp-content/uploads/2020/08/2020-PD-Survey.pdf

 

[The Med-Peds Dude]

I think med-peds is also going to be different as far as this goes. I've talked to a few PD in my specialty, and they have all admitted to stratifying (if only to sort) by USMLE score, such that a 240 IS different than a 260. I also think the data supports this - I'm sure there are confounders in that the types of students who score well on step 1 also being the types of students who try hard in other areas, but I doubt that confounding is enough to solely explain how large the difference in match % is at 240 vs 250 vs 260 in the competitive specialties.

I've looked at this data a lot; Derm makes considerable use of Step 1 scores for interview decision, but when it comes to ranking, it falls considerably lower in importance than subjective criteria. The number one thing program directors in every specialty don't want, is someone obviously annoying, rude, or who otherwise will be disruptive. Those characteristics only get worse with time.

 

[FindersFee5]

I've looked at this data a lot; Derm makes considerable use of Step 1 scores for interview decision, but when it comes to ranking, it falls considerably lower in importance than subjective criteria. The number one thing program directors in every specialty don't want, is someone obviously annoying, rude, or who otherwise will be disruptive. Those characteristics only get worse with time.

I'm not denying that it's less important for ranking, but you can't get ranked if you don't get an interview.

 

[The Med-Peds Dude]

If you have access through your institution, check this: The Feasibility of Blinding Residency Programs to USMLE Step 1 Scores During GME Application, Interview, and Match Processes

 

[The Med-Peds Dude]

I'm not denying that it's less important for ranking, but you can't get ranked if you don't get an interview.

No one can reasonably argue this. I think we were talking about life after Step 1 becomes pass/fail.

 

[efle]

If you have access through your institution, check this: The Feasibility of Blinding Residency Programs to USMLE Step 1 Scores During GME Application, Interview, and Match Processes

I don't understand why they kept Step 2 CK scores visible. It's nice that they qualitatively report more emphasis on MSPE and evals, but if you can just substitute one USMLE MCQ test for another, they didn't really have to adapt. The obvious question is really "does the ranklist change if we take away USMLEs"