- Joined
- Jun 17, 2014
- Messages
- 63,099
- Reaction score
- 154,759
Inertia.
… well i guess that answers my other question on why Step 2 CS continues to exist
Inertia.
What's your take on Step 2 becoming Pass/Fail if it does take over this same role? Though I recall you were also a skeptic of Step 1 being Pass/Fail in 5 years, let alone 2 years.
All the other metrics were there when this decision to remove Step 1 was made. We can't pretend it isn't monumentally important to current apps. A 250+ going head to head with a Pass for interview invites seems indefensibly unfair. Given access to the data, it would not be hard to show that high scored apps received significantly more invites than Pass with the other quantitative ERAS areas held similar, which I bet would happen.
They made sure there was a big fat Percentile box right next to the scaled score, so that they could be directly compared at a glance. A year of medical admissions where some people had a 36+ and others had a "500 or better, Passed" is the true equivalent. I don't believe Step 2 CK can fill this void in 23 months.I bet an applicant with a 250 of Step 1 will be judged the same as someone with a Step 1 of Pass and a Step 2 of 250-260, rendering this apparent advantage moot. Residencies will just extrapolate. It's sort of like the old MCAT vs new MCAT. There was a time when applicants could literally send in scores for both tests.
They made sure there was a big fat Percentile box right next to the scaled score, so that they could be directly compared at a glance. A year of medical admissions where some people had a 36+ and others had a "500 or better, Passed" is the true equivalent. I don't believe Step 2 CK can fill this void in 23 months.
They made sure there was a big fat Percentile box right next to the scaled score, so that they could be directly compared at a glance. A year of medical admissions where some people had a 36+ and others had a "500 or better, Passed" is the true equivalent. I don't believe Step 2 CK can fill this void in 23 months.
There is a percentile for the Step 2 as well that they could easily look up. My analogy was equating the old MCAT to Step 1 and the new MCAT to Step 2. Also, not having Step 2 is considered a disadvantage in several specialties (IM, EM, Derm).
Haha I know. I've got extracurriculars and research going, I'm just not interested in doing a ton of them. I'd prefer to keep it at a choice few, but this makes me think I'm gonna need to do a lot more of those things now.You are still going to have a full application including good evals on clinicals, research , and other ecs. even if step 1 was not pass fail . Even in the current climate you dont just waltz into a residency position with just a step score a picture of yourself wearing a surgical mask.
If, in the next 23 months, all schools change their curriculums to ensure an early Step 2 CK and PDs treat it as the new Step 1, then sure.There is a percentile for the Step 2 as well that they could easily look up. My analogy was equating the old MCAT to Step 1 and the new MCAT to Step 2.
What's your take on Step 2 becoming Pass/Fail if it does take over this same role? Though I recall you were also a skeptic of Step 1 being Pass/Fail in 5 years, let alone 2 years.
efle said:All the other metrics were there when this decision to remove Step 1 was made. We can't pretend it isn't monumentally important to current apps. A 250+ going head to head with a Pass for interview invites seems indefensibly unfair.
I completely agree that after several years, the problem is moot. Give it 3 cycles and only MSTP will still show a Step 1 Score.I was surprised at this outcome, but it reflects the degree of pressure put on the NBME by the education people all over the country. It also reflects the extraordinary level of dysfunction that everyone with a longitudinal perspective agreed was present. The retention of Step 2 CK as a scored exam was essentially a compromise to keep the program directors and GME people from going berserk. I think the exam side is settled for now, and perhaps everyone will turn their attention to ERAS and the problem of residency over-application. Fix that and the whole system may start to function again in a (somewhat) rational manner.
What you're basically saying is that the NBME has no power to change how Step 1 is scored, and I don't think that's true.
The graduating class of 2024 will probably be 80-85% people with P/F Step 1 scores. It won't make any sense to retain Step 1 as the primary screening tool at that point, so I think your argument will become moot.
The bolded is irrelevant. The point is that the pissing and moaning was the same.
Yet people are still getting into med school, CARS and Psycho/Social categories and a three digit score notwithstanding.
This too shall pass. In the mean time, I can only handle so many people projecting.
wait what's the current median for Step 2 CK? iirc I thought it was ~240?
Clerkship grades usually take into account shelf scores. Shelf scores would be little value if you "kill it" but you get bad evals and end up with a pass in the clerkship anyways. If shelf scores would be of value then Residencies would consider them separately than clerkship grades by now. All in all, its not a good idea
Awesome, so now we do not know if we are competitive for a specialty until August or September our fourth year.
The foresight here is blinding
"The second, and in my mind, more major problem with the dental board moving to pass/fail reporting relates to residency program admissions. The typical OMS program director and directors in a few other dental specialty programs receive far more applications than they have positions. Interviewing all candidates is not feasible, so programs must use other criteria to determine whom to interview. Grade point averages are of little use owing to the different rigor and grading schemes among dental schools. Class rank is often used, but creates problems because some schools do not rank their students. Letters of recommendation vary in value; most are highly complementary of their students and therefore do not help differentiate them. Many programs require candidate essays, although there is always a question of authorship or the writer’s sincerity in such writings. This leaves national board scores as the only means of comparing candidates from across the country with one another using a nationally uniform testing instrument."
"I would have preferred that the JCNDE had followed the NBME’s lead in continuing to report numerical results to candidates, while they worked to strengthen the psychometric validity of the national dental boards"
-taken from an editorial (written by former residency PD) published on Journal of Oral and Maxillofacial Surgery back in 2012
Interesting to see how this develops over the next decade
Well in response to the shadowy out-of-touch figures in those governing bodies, all I have to say before eating the grotesque **** sandwich about to unfurl before me is "Okay Boomer."I think the governing bodies responsible for this change are aware of the consequences of the change as well as history itself. There must have been a hidden agenda.
Step 1.5Calling it right now. The ultimate plan by the nbme is to create such chaos to the point that med schools and residency directors alike are begging the nbme to create an exam to stratify applicants. More $$$.
The best part? It’s just gonna be recycled questions from step 1&2 hahaha!
I don't really get the logic of people calling out midlevel education is grossly inferior to med school (it is) while
Right. But residencies have to have some guage of medical knowledge. Former board/test performance predicts future test performance. If you have a certain percentage of board failures, you can lose your accredidation. So if someone is barely scraping by, its nice to know that. CK will give that info, but the problem is when the results come back currently, way too late to select students for 4th year AI rotations. If you are a program concerned about test scores, then you'll have zero data to figure out who to rotate.
Personally, I don't care. I allow just about anyone to rotate as long as they don't have failures. So it doesn't affect how I operate. I'll still look at Step 2 and factor that in to the candidate score come interview season when I consider that along with the rest of their application. Still not the most important part of the app, ultimately SLOEs dictate how I rank people for the most part, but its good to know/anticipate if you are getting a resident who is going to need more testing help from the beginning.
have to say though, I don't like the immediate implications here. Keeping Step 2 CK as a triple digit score is just going to lead to all residency programs requiring a Step 2 CK to apply and treating it exactly the same way.
Also going to be curious to see how people handle the relatively imminent potential implementation (January 2022). That's close enough that anyone who wanted a competitive surgical specialty, and did poorly on Step 1, could consider a couple of research years to have a brand new shot..
you bring up a good point. however, will those with low Step 1 scores who do a couple research years and then have a “new shot” be able to show their Step 1 score as P/F or will their “low score” still show? seems like the latter would happen and then there wouldn’t be a new shot.
in other words, in Jan 2022, will all the previously earned triple digit scores change to reported P/F scores?
you bring up a good point. however, will those with low Step 1 scores who do a couple research years and then have a “new shot” be able to show their Step 1 score as P/F or will their “low score” still show? seems like the latter would happen and then there wouldn’t be a new shot.
in other words, in Jan 2022, will all the previously earned triple digit scores change to reported P/F scores?
I dont think PDs have any reason to trust a number you slip into your essays or interviews. If all your official ERAS material says is Pass, they're to have to treat it as such@efle People could still report their scores through various means on ERAS even if it were reported as a Pass (if retroactively done). Wouldn't this have a big effect on the psychology and disposition of the PD towards someone who reported getting say a 265? Even if it wasn't used as a screen I think that anyone who takes the test before Jan 1, 2022 (or whatever date they choose) and gets a high score will decidedly have an advantage over others independent of whether the score is reported numerically or as a Pass. Do you disagree?
What if it’s reported on your MSPE? They won’t trust your school either?I dont think PDs have any reason to trust a number you slip into your essays or interviews. If all your official ERAS material says is Pass, they're to have to treat it as such
I like chaos, and welcome it fully.
Ultimately Step 2 wil also end up P/F. School, essays, letters, and interview performance will decide who gets the spot.
I was surprised at this outcome, but it reflects the degree of pressure put on the NBME by the education people all over the country. It also reflects the extraordinary level of dysfunction that everyone with a longitudinal perspective agreed was present. The retention of Step 2 CK as a scored exam was essentially a compromise to keep the program directors and GME people from going berserk. I think the exam side is settled for now, and perhaps everyone will turn their attention to ERAS and the problem of residency over-application. Fix that and the whole system may start to function again in a (somewhat) rational manner.
An interesting thought, but since their role is to avoid horrifically bad match outcomes more than generate a few excellent ones, I think they'd be more likely not to state people's scores if they had the option not to. Benefit of the doubt for their low performers is more valuableWhat if it’s reported on your MSPE? They won’t trust your school either?
Schools like washu dont have to care about this change really anyways. Their reputation will carry people. It's the average programs that have to weight the risk of only reporting scores for high performers.@efle i don’t think this would be true at schools that historically had high step scores. I doubt a school like WashU, which has historically had very high Step scores, would disadvantage 95% of their applicants and protect the 5% that underperform. I can see why a low performing school would do that.
In addition, you could just get a letter writer to corroborate your score. Given how easily verifiable a Step 1 score is I think the risk of lying is a lot greater than any benefit you would gain. In addition, if you claimed to have a super high Step score but then had a super low Step 2 it would probably raise a lot of flags and then people might look into it. I think with the risk of delays as well as all the different ways you could potentially convey your Step 1 score and the historic importance of it, I don’t think it matters what the NBME does with respect to retroactive scoring. Program directors want to know the score and the high scoring applicants could find some way to convey that info whether it’s directly, through letters of rec or the MSPE.
Despite what you think about P/F Step 1, all this does is put a band-aid on a much larger issue. Which is the ever increasing medical student:residency spot ratio.
I took his comment to mean that we should limit the amount of apps per person as has been discussed ad nauseum.You don't want this to change unless you want physician comp to fall off a cliff. Keep the supply artificially low is a key tactic to the high salaries US MDs receive.
I like chaos, and welcome it fully.
Ultimately Step 2 wil also end up P/F. School, essays, letters, and interview performance will decide who gets the spot.
Yes.
"The NBME allowed students at School X to show exact numerical performance on this metric, rated the #1 most commonly cited factor in residency interview invitations despite having no demonstrated validity for this purpose. Students testing a few months later at School Y were denied the ability to show their performance beyond a Pass. Students at both School X and School Y were both able to show their exact numeric performance on a separate metric which, as of 2018, was the #4 most commonly cited factor in residency interview invitations. Residency program directors also had the usual access to students' grades, letters of recommendation, and MSPE's, which contain comparative performance data and narrative feedback. Without a significant decline in the overall match rate it is therefore extremely difficult to show any harm caused as a result of this change."
Ftfy.
You are still going to have a full application including good evals on clinicals, research , and other ecs. even if step 1 was not pass fail . Even in the current climate you dont just waltz into a residency position with just a step score a picture of yourself wearing a surgical mask.
The reality is that for smaller fields this was not an equalizer, rather just a convenient screen for programs. After the screen was completed the real impact of going to a better medical school came through like being published in high impact journals, having letters from field leaders etc. Upward mobility in medical school was always difficult in those tiny competitive fields. This is why there was a significant amount of inbreeding at the top. This realistically won't impact that inbreeding as much, but will probably make it difficult for img and dos to access medium competitiveness specialties . Mds with home programs are going to continue to match at home.Regrettably, it is very tough to compare people across schools... especially in residencies with a small number of spots. What if everyone has good letters and solid grades. Then you're going by school reputation and "x-factor." Oddly enough... the latter is the reason colleges are citing for keeping asians out of top colleges. Without step 1, there's no equalizer. Everything is subjective.
While step 1 may not have been developed to sort students, they have to realize that it's the easiest way to facilitate upward mobility. Otherwise, it's harder for a student from a smaller school to make a splash.
The reality is that for smaller fields this was not an equalizer, rather just a convenient screen for programs. After the screen was completed the real impact of going to a better medical school came through like being published in high impact journals, having letters from field leaders etc. Upward mobility in medical school was always difficult in those tiny competitive fields. This is why there was a significant amount of inbreeding at the top. This realistically won't impact that inbreeding as much, but will probably make it difficult for img and dos to access medium competitiveness specialties . Mds with home programs are going to continue to match at home.
I took his comment to mean that we should limit the amount of apps per person as has been discussed ad nauseum.
Sent from my SM-G955U using SDN mobile
What do you mean by leaving opportunities on the table? Opportunities now or job opportunities later? For opportunities now, if you don’t think the top ten was the best fit, then no, you aren’t leaving anything on the table. For example, who cares if a program has all the research in the world if doing research is the bane of your existence, it’s an opportunity you wouldn’t be taking advantage of anyway. For job opportunities later, you need to be looking at your long term goals. If your career goals include academic medicine at a brand name institution then yeah, that top ten program will probably open doors for you. If your career goals are anything else (private practice, academic med elsewhere, etc) then going to the place with the best fit is not going to hurt you. And honestly many of those jobs are going to be based on your fit with the practice. If your current number 1 is in a location that you want to someday practice in with people you liked a lot, that is potentially of far greater value than an institution name on your CV because you will be networking and building connections for your future during residency.
Perhaps a better example would be Mizzou. A solidly mid-tier school with an unexpectedly high step 1 average.Schools like washu dont have to care about this change really anyways. Their reputation will carry people. It's the average programs that have to weight the risk of only reporting scores for high performers.
I think itd be weird and awkward to have my letter writer discuss my numerical step 1 score but I'm sure plenty of 250+ people would try it. I again, if I was a PD, would be extremely skeptical of anything not coming directly from the NBME via ERAS.
They can call it the “Standardized Test Extrapolating Performance On a Numerical Exam”Step 1.5
I only partly agree. Youre right that it doesn't significantly affect inbreeding. However, in some specialties, there won't always be an internal candidate.
As someone who has watched the selection process happen... taking step 1 out of a very subjective process is only going to make things harder. Primarily for mid and lower tier schools.
The kid from Harvard or Hopkins actually benefits since he can ride the school name all the way. In particular, this primarily benefits the underperformer at the top flight of schools. On the other hand, this disadvantages the overperformers at mid tier schools and makes them bank on low yield things like trying to get published. If your mid-tier school doesn't have an academic person in your chosen field, you are out of luck (the rich get richer, the poor get poorer). On top of that... A lot of publishing is a crap shoot based on who the reviewer is for your paper, priorities (like did the journal recently publish a similar paper), and just dumb luck.
It's the same story as applying to med school and college. We should not make the MCAT pass fail. We shouldn't make the SAT pass fail. Though they are imperfect, these are the only equalizers we have.
I’m not sure why people keep glossing over this piece of info. This test has a better reputation of being more clinically relevant anyways & as long as it’s still graded, I’m not sure why this would change the process that much.But there's still step 2, right? Overperformers at mid-tier schools still have a chance in 'proving' themselves by doing well on step 2...
Correct me if I am wrong, but Step 1 is more on the individual, while i could foresee Step 2 CK being more dependent on the diversity and quality of clinical education you can get, which will vary significantly based on where you go to school and how well that school is connected.I’m not sure why people keep glossing over this piece of info. This test has a better reputation of being more clinically relevant anyways & as long as it’s still graded, I’m not sure why this would change the process that much.
I’m not sure why people keep glossing over this piece of info. This test has a better reputation of being more clinically relevant anyways & as long as it’s still graded, I’m not sure why this would change the process that much.
I’m not sure why people keep glossing over this piece of info. This test has a better reputation of being more clinically relevant anyways & as long as it’s still graded, I’m not sure why this would change the process that much.