.

[Lawpy]

Advertisement - Members don't see this ad

We do have a few people on our side, lemme plug Carmody one more time. I think he's also popping off on Twitter about the abridged step exam.

Step 2 CS, Part Three: Winners and Losers Edition

Last winter, I started a series on USMLE Step 2 CS. In Part One, I reviewed the history of the exam, from its humble origins in the first NBME exam to its more recent rebirth from the Clinical Skil…

thesheriffofsodium.com

Oof. Really sucks for those who failed CS before the cancellations.

 

[Dr. Foresight]

Jeeze, the hyperacheivers need to chill.

What Part of "we're getting rid of the tryout items that aren't part of the exam score " is so traumatic?

Just take the exam and do as best as you can. That's all that's expected of you. PDs aren't going to care. They just want a screening tool.

Because taking the same question after 4 hours instead of 6 makes it inherantly easier. Changing the way the exam is taken removes a major factor of its standardization.

 

[7331poas]

We do have a few people on our side, lemme plug Carmody one more time. I think he's also popping off on Twitter about the abridged step exam.

Step 2 CS, Part Three: Winners and Losers Edition

Last winter, I started a series on USMLE Step 2 CS. In Part One, I reviewed the history of the exam, from its humble origins in the first NBME exam to its more recent rebirth from the Clinical Skil…

thesheriffofsodium.com

Carmody is the worst of them all

 

[Med Ed]

Hope they roll this back. I know I was thinking a little slower by the last couple blocks on test day, it's mentally exhausting. Seems like a huge advantage to get two hours shaved off.

Not to mention that the head-scratcher questions that you have to read 5 times to understand what they're asking are more likely to be experimental than scored. Getting rid of all those stumbling blocks and taking only 200 high-quality Q's under the same time limit per block is yet another advantage.

The NBME already calibrates scores based on exam difficulty. If the 5 hour version is noticeably easier than the 7 hour version it wouldn't surprise me if their statistics can simply adjust for it.

And who knows? Perhaps they've been putting all the unscored pretest questions in the last two books all along...

Why can't they use 280 validated items and provide a much more accurate score under the original format? Does anyone know why they need it to be shorter?

They could do that, but to keep the stock of new questions replenished they would need to add 80+ unscored pretest items to you 280 validated ones you want to take. Congrats, you've gone from a 7 hour exam to a 9 hour exam that will likely span two days.

 

[Med Ed]

"Let's mess with our validity to save a couple hours of proctoring costs."
"Agreed"

???

It may be "a couple hours of proctoring costs" from the student perspective, but these event exams are basically commandeering medical schools staff for hundreds or thousands of test administrations. Perhaps I'm at the wrong institution, but I don't think we can afford to have half our support staff get borrowed 8 hours a day for weeks on end.

 

[Ho0v-man]

I was just thinking about this regarding COMLEX PE. It seems absurd at first for them to insist on keeping PE around when CS is cancelled...but not if PE could be the new way for IMGs to get ECFMG certified (previously required CS).

Maybe they're insisting on their nonsense PE exam because they could pick up so much $$$ from the IMG/FMG crowd this year?

I’ve heard rumors of this, but honestly don’t think they’re smart enough to consider this. The big reasons they’ve held out is 1) $$$ and 2) the current NBOME president invented the exam. That stupid test is his legacy and cancelling it costs millions of dollars while simultaneously validating the fact that pinnacle of his career is nonsense.

On an unrelated note, will non us IMGs be able to apply this cycle given the CS cancellation?

 

[Med Ed]

This is actually consistent with my theory. Maybe I didn’t word it the best way, but what you are saying is basically the same thing I was saying. “Experimental questions” are scored for future examinees, not current ones.

Anyway here’s a possible example of how it could result in score creep. According to your link, a discriminatory index of 0 means a question cannot discriminate between high and low scorers, and it ranges from -1 to +1. Let’s say out of 80 unscored questions, 20 have discriminatory indices less than 0.2, which is considered “low.” These questions get tossed out, but because they have some discriminatory value, ranging from 0-0.2 (negative values are rare due to the nature of the exam, which is heavily based on factual knowledge), when you take away these questions it results in a shift in scores for future examinees. Take for example someone who scored a 230 and got 75% of the unscored questions correct or 60/80. Let’s say a future examinee takes an exam with the new questions but with some of the unscored questions taken out due to poor discrimination, he might get between 45-47/60 correct, or 75-78%. It tends toward slight score increases because questions with positive discriminatory value are taken away.

I do not follow you. If you start with 80 unscored questions, and 20 get removed due to indices less than 0.20, then you simply have 60 validated questions remaining to carry forward into future exams.

 

[efle]

It may be "a couple hours of proctoring costs" from the student perspective, but these event exams are basically commandeering medical schools staff for hundreds or thousands of test administrations. Perhaps I'm at the wrong institution, but I don't think we can afford to have half our support staff get borrowed 8 hours a day for weeks on end.

I didn't realize that was what NBME envisioned, I thought this was going to be like AP exam administration where they try to have everyone in just one or two big event tests.

 

[Med Ed]

I didn't realize that was what NBME envisioned, I thought this was going to be like AP exam administration where they try to have everyone in just one or two big event tests.

No, it's far larger and more complicated than that.

 

[efle]

No, it's far larger and more complicated than that.

Wow so these 6 med schools are really signing up to be mega-sized Prometric centers for months at a time? I am starting to see why they wouldn't want to keep it running extra hours for the sake of experimental questions...thanks!

 

[Lucca]

what if next month they announce all scores will retroactively be changed to p/f after all this drama

 

[Cath Up]

what if next month they announce all scores will retroactively be changed to p/f after all this drama

Reported.


(I'm in dedicated and don't want that)

 

[AnatomyGrey12]

Everything I've heard from this organization over the last 6 months, in addition to my interactions with my own admins, has convinced me that everyone in Med Ed is brain dead. Who else could have possibly green lighted this decision.

 

[Mass Effect]

View attachment 309009

Off topic, but what does "approximately six" medical schools mean? Is it 5? Is it 6? Is it 7? It's not like we're talking about hundreds. When you're in the single digits, you should have an exact figure, not an approximate amount. Who's writing this stuff?

 

[cybermed2424]

It’s even more comical knowing which schools were “cho$en” to administer

 

[Mass Effect]

It’s even more comical knowing which schools were “cho$en” to administer

Which schools? I must have missed that.

 

[Frogger27]

It's all about reducing risk, in my view.

I also recall you saying something along the lines of "all of our rights ended where the health of your lungs begins".... but I am sure you are totally cool with 10's of thousands of people protesting in the streets who all care about the safety of your lungs. The difference of 80 questions on a "standardized" test to reduce the risk of SARS exposure in a very controlled environment is one of the flimsiest excuses I have ever heard on this forum

And telling the "hyperachievers" to chill is ridiculous. This test determines peoples future careers, plain and simple. People have a right to be upset if there is a perceived notion that it is not standardized or that people are being treated unfairly under the system, just as everyone has the right to go out and protest during a global pandemic.

I got a 238 on Step 1.... the thought of wasting time on test questions that didn't count when I could have focused my attention on real questions is nauseating. If I knew people did not have to take these same questions during my test period, I would be irate. It is not irrational at all to think that if I did not waste so much time on pointless questions or get test fatigue from 2 additional hours, I could have scored 2 points higher and gotten above the 240 screen, which could be the difference in matching in a hypercompetitive specialty.

 

[cybermed2424]

Which schools? I must have missed that.

Originally it was Boston, Harvard, Brown, UMass, Tufts, and UConn. To be fair, it seems they have expanded the school list quite a bit. Guess I shouldn't be so cynical.

 

[Mass Effect]

Originally it was Boston, Harvard, Brown, UMass, Tufts, and UConn. To be fair, it seems they have expanded the school list quite a bit. Guess I shouldn't be so cynical.

Huh? That's crazy. So at these schools, students take a shortened exam at their school and the rest of the country goes to Prometric where they take the regular exam?

 

[AnatomyGrey12]

Huh? That's crazy. So at these schools, students take a shortened exam at their school and the rest of the country goes to Prometric where they take the regular exam?

Yeah that was the plan lol. It's amazing what terrible ideas these organizations continually cook up. You've got the NBOME refusing to suspend Level 2 PE the same as Step 2 CS because reasons (Read: Money and Gimpel's pride), and you have the NBME out here potentially eliminating the concept of a standardized test, admittedly the reasons for this are much less clear right now.

 

[Mass Effect]

I also recall you saying something along the lines of "all of our rights ended where the health of your lungs begins".... but I am sure you are totally cool with 10's of thousands of people protesting in the streets who all care about the safety of your lungs

Actually, what I believe he said was on a thread about whether or not people should be forced to wear masks. Others said it was their right not to wear a mask and Goro said their right (to not where a mask in public) ends where his lungs begin. You can hold that view (as I do) and STILL advocate on behalf of the people who have a right to protest in the streets (as I do). The protestors should wear a mask. Done.

Now let's get back on topic.

 

[Mass Effect]

Yeah that was the plan lol. It's amazing what terrible ideas these organizations continually cook up. You've got the NBOME refusing to suspend Level 2 PE the same as Step 2 CS because reasons (Read: Money and Gimpel's pride), and you have the NBME out here potentially eliminating the concept of a standardized test, admittedly the reasons for this are much less clear right now.

So what is the new school list? Did they expand it or did they scrap the idea?

 

[Mad Jack]

This isn't going to amount to an easier exam, as the spread of question difficulties should still be the same. What it will result in is more widely variable scores since there will be less questions to average out your strengths and weaknesses

 

[Matthew9Thirtyfive]

Yeah that was the plan lol. It's amazing what terrible ideas these organizations continually cook up. You've got the NBOME refusing to suspend Level 2 PE the same as Step 2 CS because reasons (Read: Money and Gimpel's pride), and you have the NBME out here potentially eliminating the concept of a standardized test, admittedly the reasons for this are much less clear right now.

Oh that’s crazy. I thought the schools were going to be acting as testing centers for students from any school to come take the test, not just offer a shorter test to their own students.

 

[Dantes]

Oh that’s crazy. I thought the schools were going to be acting as testing centers for students from any school to come take the test, not just offer a shorter test to their own students.

Southwest Regional Testing Center | CORONAVIRUS RESOURCE CENTER

The University of Arizona College of Medicine – Phoenix will be a regional testing center where administrations of Step 1 and Step 2 Clinical Knowledge (CK) are anticipated to begin later this month. At this time, this center will only serve students from a Southwest consortium of schools:

covid.usmle.org

The plan was to have one school serve as the testing site, allowing selected nearby schools to take it there as well (above is just one example of a regional testing center).

 

[Matthew9Thirtyfive]

Southwest Regional Testing Center | CORONAVIRUS RESOURCE CENTER

The University of Arizona College of Medicine – Phoenix will be a regional testing center where administrations of Step 1 and Step 2 Clinical Knowledge (CK) are anticipated to begin later this month. At this time, this center will only serve students from a Southwest consortium of schools:

covid.usmle.org

The plan was to have one school serve as the testing site, allowing selected nearby schools to take it there as well (above is just one example of a regional testing center).

Oh okay that’s sort of what I thought. I read the other post to mean that the plan was for those 6 schools to offer their students a shorter exam while everyone else had to take the regular one at a prometric.

 

[AnatomyGrey12]

Oh that’s crazy. I thought the schools were going to be acting as testing centers for students from any school to come take the test, not just offer a shorter test to their own students.

Oh okay that’s sort of what I thought. I read the other post to mean that the plan was for those 6 schools to offer their students a shorter exam while everyone else had to take the regular one at a prometric.

Yeah it's not just the students at the school site, but even with the selected schools overall it amounts to a very small portion of the testing pool. Either way it was a pretty dumb idea IMO.

 

[Med Ed]

Originally it was Boston, Harvard, Brown, UMass, Tufts, and UConn. To be fair, it seems they have expanded the school list quite a bit. Guess I shouldn't be so cynical.

Of the schools you listed only Brown was set up as the regional testing site for Phase 1.

It wouldn't make much sense to set up six regional Prometric testing centers within medical schools and put them all in the same region.

 

[Med Ed]

This isn't going to amount to an easier exam, as the spread of question difficulties should still be the same. What it will result in is more widely variable scores since there will be less questions to average out your strengths and weaknesses

The number of scored items would be the same.

 

[Med Ed]

Huh? That's crazy. So at these schools, students take a shortened exam at their school and the rest of the country goes to Prometric where they take the regular exam?

No, but this is all very confusing (in part because things change every other day).

When Prometric centers shut down this obviously created an enormous backlog. Even if they reopened fully tomorrow and only administer essential exams it would take months to resolve. The most straightforward way to approach this is to increase testing capacity using the existing system, which translates into more Prometric sites (Phase 1).

The NBME broke the country into six geographic regions and identified a centrally located medical school in each one that can serve as an ad hoc Prometric center. They have been surveying the schools in each region to figure out the relative need for additional testing, as the nation has not been uniformly impacted by Prometric closures. With mutual agreement a loose coalition of schools forms within each region, and they give students at those schools the option of taking the full-length USMLE at the regional school-based pop-up Prometric site. The main benefit is that reservations at these sites are iron clad.

They have used the phrase "approximately six" because they were still assessing the magnitude and distribution of demand for these sites. Perhaps they only need five, perhaps they'll need eight.

At this point I'm skeptical of the need to progress to Phase 2, which would involve old school exam administrations, possibly using pencil and paper, and possibly being shorter. But they have been very clear that everything is on the table to get caught up.

 

[Neopolymath]

It's all about reducing risk, in my view.

This isn't a good take, Goro. Have you been in a Prometric in the past few years? Someone could be sitting "next" to you with a shoe full of dog turds and you wouldn't be able to tell. Everyone has masks on. The cubicles are large, the walls are generous, and you interact with like one old lady manning the desk. It's arguably more safe than pumping gas and going to the grocery store and if schools are supposedly going to replicate this set up to administer the exam then making it 2 hours less doesn't change the problem points: check in and bathroom breaks. Are the schools somehow changing those two things? Because if not, it's window dressing IMO. Furthermore, if the schools are not going to replicate Prometric setting then I would argue that it's less safe and not feasible if we are going to keep going with the Prometric decision making pathway that we decided to marry. We say Prometric isn't safe but somehow we are going to put a bunch of people in a room but for 2 less hours and that's suddenly kosher after all this obsession over doing things safely (that I agree with).

You could probably argue that they shouldn't even be cancelling half of the Prometric exams and I wouldn't totally object to the idea TBH.

They can't support the decision based on testing logic. They also can't seem to support it based on risk reduction IMO. Looks like just another stupid decision in a sea of them.

 

[El-Rami]

That's nuts that 80 of the questions on Step 1 are unscored.

I think if I were an informed PD, I would want to ask applicants in what setting they took the exam, but I don't think tons of PDs will be well enough informed on this. Then again I'm not really sure what I would do with that information once I asked.

Why would a PD care? They just want to see a score so they can compare numbers among applicants. The shortened USMLEs are unlikely to make a big impact on whether or not you get into a program. They will be judged the same as the full-length USMLEs.

The AAMC is also doing the same thing to the MCAT for pre-meds, by the way.

Like others have mentioned, this will be an advantage.

 

[peachrings]

They should just report your percentile in your test taking cohort stratified by testing modality, whether at Prometric or in a school. Especially this testing cycle with the pandemic, no reason to stick with curves from non-representative years.

 

[efle]

They should just report your percentile in your test taking cohort stratified by testing modality, whether at Prometric or in a school. Especially this testing cycle with the pandemic, no reason to stick with curves from non-representative years.

Yikes I definitely wouldn't want my curve to suddenly be against a group of people who had 4-6 months of dedicated lol

 

[longhaul3]

Why would a PD care? They just want to see a score so they can compare numbers among applicants. The shortened USMLEs are unlikely to make a big impact on whether or not you get into a program. They will be judged the same as the full-length USMLEs.

The AAMC is also doing the same thing to the MCAT for pre-meds, by the way.

Like others have mentioned, this will be an advantage.

If you're saying it's an advantage, then you're implying that students who take the modified version are likely to score higher, unless you just mean it's an advantage in the moment because it's not as painful of a testing experience. I don't think it's that cut and dry, but especially if it is, the numbers will no longer be directly comparable without some sort of correction. For PDs who make decisions based on scores, that would be relevant information.

If there are two applicants who each scored 250 in a different setting, and it was known that one setting provided an advantage, as a PD I would want to know that.

They should just report your percentile in your test taking cohort stratified by testing modality, whether at Prometric or in a school. Especially this testing cycle with the pandemic, no reason to stick with curves from non-representative years.

As discussed earlier, Step 1 is not a norm-referenced test like the MCAT and SAT, in which examinees are scored against each other by design. This strikes me as a good idea in this situation but, based on my admittedly limited understanding, I don't think it would be statistically/psychometrically valid.

 

[El-Rami]

I can't see how it would be relevant, insofar as test center is concerned. Standardized tests are designed so it's irrelevant if one is shorter in length than the other; they will both result in the same score within std error.

They are only taking out experimental questions. Effects of test-taker fatigue from 6 vs 8 hrs will be irrelevant.

It's a decision made solely for the safety of test takers.

 

[longhaul3]

I can't see how it would be relevant, insofar as test center is concerned. Standardized tests are designed so it's irrelevant if one is shorter in length than the other; they will both result in the same score within std error.

They are only taking out experimental questions. Effects of test-taker fatigue from 6 vs 8 hrs will be irrelevant.

It's a decision made solely for the safety of test takers.

Maybe I misinterpreted your post then. I agree with what you're saying here. What did you mean by "this will be an advantage?"

 

[efle]

I can't see how it would be relevant, insofar as test center is concerned. Standardized tests are designed so it's irrelevant if one is shorter in length than the other; they will both result in the same score within std error.

They are only taking out experimental questions. Effects of test-taker fatigue from 6 vs 8 hrs will be irrelevant.

It's a decision made solely for the safety of test takers.

Bruh, have you taken Step? The difference in mental drain between an abridged practice (like UWSAs) and full-length game day is huge. In fact the common advice is to force yourself to take additional blocks afterwards to build proper endurance.

Plus I suspect a block of experimental + validated feels very different than a block of purely validated. The crappy test items that are confusing, vague, or so obscure they get tossed for failing to discriminate? All those stumbling blocks are gone now, you get to just cruise through validated Qs.

 

[TBD2022]

*pre-health

I can't see how it would be relevant, insofar as test center is concerned. Standardized tests are designed so it's irrelevant if one is shorter in length than the other; they will both result in the same score within std error.

They are only taking out experimental questions. Effects of test-taker fatigue from 6 vs 8 hrs will be irrelevant.

It's a decision made solely for the safety of test takers.

 

[El-Rami]

Bruh, have you taken Step? The difference in mental drain between an abridged practice (like UWSAs) and full-length game day is huge. In fact the common advice is to force yourself to take additional blocks afterwards to build proper endurance.

Plus I suspect a block of experimental + validated feels very different than a block of purely validated. The crappy test items that are confusing, vague, or so obscure they get tossed for failing to discriminate? All those stumbling blocks are gone now, you get to just cruise through validated Qs.

In fact I have. I've stated before that people taking the shorter test will be at an advantage. Read my post more closely.

What I am saying is that a PD won't give a damn whether or not someone had the shorter version due to COVID-19 or the full-length one. Goro has already weighed in on this, but I guess if you don't want to believe it, then keep on doing what you're doing. PDs only care about your score. They will not discriminate against someone who took the shorter version during the pandemic. I do not think there will be enough of a difference for people to care.

If you are worried about it, I would suggest asking your adviser and/or dean.

 

[efle]

Effects of test-taker fatigue from 6 vs 8 hrs will be irrelevant.

I've stated before that people taking the shorter test will be at an advantage.

I'm reading these as closely as I can and they still look contradictory. Help me out

Also, how'd you already take physician licensure boards as a 2022 podiatry applicant?

The majority of my science pre-reqs are definitely less than 10 years old, but I have a few pre-reqs like English and basic biology that are right at 10 years. I am not planning to apply this upcoming cycle but for the entering class of 2022. I think I started general chemistry and microbiology back in 2012. The majority of my pre-reqs are from that time onward to about 2015 (I was knocking out some courses for my biology degree that weren't exactly pre-med courses).

 

[El-Rami]

I've been in medical school before.

As for the apparent contradiction in the two posts, it really isn't. It's irrelevant in the sense that PDs aren't going to put much thought into the difference between those who took the full 8 hour exam or the shorter "COVID-19 version." You have to put yourself into the mind of a PD. Will you put much thought into some arbitrary "score correction," or just take the score at face value?

The level of fatigue you feel at the end of an eight-hour test is enormous. There's no questioning that. However, many people likely won't notice much of a difference, especially people who can power through the eight-hour test.

I would agree that it could be an overall advantage to a lot of test takers, which may seem unfair. Keep in mind that we are living through a major pandemic right now, so measures had to be taken by the NBME for the safety of all test takers. If you can take advantage of the fact that the USMLE exams are shorter right now, then be glad you got that opportunity. In the end, it isn't a big deal. People are overreacting.

 

[Mad Jack]

The number of scored items would be the same.

Ah, that I missed. Well that's fine then

 

[SuaveCardigans]

Yikes I definitely wouldn't want my curve to suddenly be against a group of people who had 4-6 months of dedicated lol

Im not sure I agree with that, burnout is very real and I wouldnt be surprised if the people with dedicated periods 2+ months long end up scoring lower.

 

[Chelsea FC]

Yikes I definitely wouldn't want my curve to suddenly be against a group of people who had 4-6 months of dedicated lol

For alot of students in that 4-6 months loved ones got sick/died during a pandemic in which they had a complete change to their normal routine. Tons of friends and family members being displaced/losing their jobs/loss of childcare etc. Lets not act like those 4-6 months are perfect studying conditions for alot of people.

 

[Med Ed]

Im not sure I agree with that, burnout is very real and I wouldnt be surprised if the people with dedicated periods 2+ months long end up scoring lower.

There are data to support this.

 

[Splenda88]

I don't see any wrong with reducing the # of questions, hence making the exams shorter...

What they should really do is combining the step 1/2 (250 questions) so med student take one test after MS2. In order to do that, med school should only be 3 yrs instead of 4.

 

[Chelsea FC]

I don't see any wrong with reducing the # of questions, hence making the exams shorter...

What they should really do is combining the step 1/2 (250 questions) so med student take one test after MS2. In order to do that, med school should only be 3 yrs instead of 4.

Dont think anyone cares about reducing the number of questions. The problem is that it suppose to be a standardized test but some students will be taking a shorter exam than others.

 

[Splenda88]

Dont think anyone cares about reducing the number of questions. The problem is that it suppose to be a standardized test but some students will be taking a shorter exam than others.

Did not read the whole thing... You mean some testing centers will administer shorter exams. If so, that would be nuts.

 

[naijacardriodoc]

they retracted the statement about this....its on twitter