9
907914
Advertisement - Members don't see this ad
Why is more emphasis put on Step 1 than Step 2?
-
CycleTrack and SDN are teaming up to make medical school admissions more transparent and accessible! Read the announcement to learn how we’re supporting the future of CycleTrack’s free application-tracking tools .
Questions about the USMLE
Started by mariposas905
Get help with your application
Use all the free resources available to you from SDN: articles, guides, expert advising, forums discussions, and school research.
I did think in reading the “make USMLE pass fail” tweets...how about you med school professors go first, make MCAT pass fail and also undergrad grades like most med school grades are. Pick yourself a good successful med school class with no objective indicators, then after that we’ll agree to pick a residency class on same basis. They won’t.
OTOH if med school classes are selected such that all of them can succeed in any field, maybe it would be better to let the students sort into specialties based on their interests and aptitudes rather than based on their step scores. You don’t actually need to be the brightest person in med school to make a good derm or ortho more than you need to be to be a good internist, really.
OTOH if med school classes are selected such that all of them can succeed in any field, maybe it would be better to let the students sort into specialties based on their interests and aptitudes rather than based on their step scores. You don’t actually need to be the brightest person in med school to make a good derm or ortho more than you need to be to be a good internist, really.
Yeah I think the other hand has it correct. You need to heavily screen out premeds because a lot of people who want to be doctors, absolutely should not be admitted to med school. Exhibit A: the tens of thousands of people every year who score below a 25 (now a 500) on the MCAT.
But do we really need to screen med students so that only the top ~15% have strong scores for surgical subspecialties or cushy lifestyle specialties? That doesn't really make any sense to me. I've yet to see anything in an operating room that makes me think "wow, good thing Dr. Surgeon was so good at recognizing lysosomal vs glycogen storage diseases twenty years ago"
Advertisement - Members don't see this ad
Right but then what do you replace it with? Programs have to rank applicants somehow, and you can’t rotate at 20 different programs.
Momentum, standardization, timing. It's the first major standardized test of medical school. Not everyone has Step 2 scores when applying for residency, and Step 2 happens way too late to help students decide what specialty to target. As such, there's less emphasis on Step 2 for students and therefore fewer resources devoted to studying for Step 2.
I imagine if this really does happen in the next couple years, it'll just turn into a system like law or MBAs. Nothing will really replace it. The new way to land a good spot will be:
1) Be from a very well known school with a reputation for producing good interns/residents
2) Be from a less known medical school, but that is a feeder into your targets so you're somewhat of a known quantity
3) Do a rotation at your targets and impress them
If they don't know you personally, and don't know if they can expect good caliber grads from your institution, you'll be pretty S.O.L.
And obviously you can only rotate at a few places, so if you're banking on #3, you better kill your audition
Yeah, which IMO is a crap system and is part of the reason the law market sucks.
Edit: and fwiw, getting into a top law school is easier than getting into a top medical school.
I’m broadly very against high stakes standardized exams, but I also understand why many if not most would be against it. I think saying med school will become like law school is a bit hyperbolic because it’s not like a significant number of US MDs are ending up jobless but it’s probably right where you go to school, who you know, networking would be more important for determining where you end up for residency.
Perhaps there’s a happy medium to be found that has not yet been considered but honestly preclinical education doesn’t seem particularly enriching if it either gets in the way of Step or is just subservient to it. I’m happy to let the medical education deans sort it out.
Just like with the mcat I think it’s silly to select for things which beyond a certain point no longer serve as useful metrics.
Only happy medium I can think of would be a broad-strokes categorization like High Pass (top 1/3rd), Pass (most of the remaining 2/3rds), and Fail. That would be a nice step down from the insane competitiveness of striving for the top ~15%, without making it worthless to work harder than the bare minimum in preclinical years. It would suck to narrowly miss the cutoff, but then again someone in the high 230s wasn't going to be very competitive for stuff like Derm and Ortho prior to the score change anyways.
Absolutely not. I don't think that would ever happen.
I hope you’re right! There’s nothing I can do at this point so I’ll stop dwelling on it, but I definitely agree with others that a P/F Step 1 will heavily diminish my residency options
The InCUS super squad (NBME, AMA, AAMC, other stakeholders) are supposed to make their final recommendations by this autumn. Someone about to start med school in a few months is currently what, 3.5 years away from their residency interview cycle? I think you or the year after you are the prime candidates for being hit with the change, if a change really is coming
9
907914
Whenever a transition occurs, that is going to suck for all of the M2s that spend M1 with Zanki 24/7.
Yeah and I was thinking about the fact that some people are going to have to lose their scores. They definitely can't allow a mixed cohort cycle where some people have numerical scores and others have only Pass/Fail, because PDs would hugely favor the numerical scores and all the P/F people will be at a big disadvantage.
So if this really does happen, there's gonna be a ton of people who gave up months of their life to a full time UFAPS cramming dedicated period, just to find out a year later it wasn't necessary at all because it only shows up as Pass.
Advertisement - Members don't see this ad
People who review residency apps in competitive fields that I've talked to seem to already have a hard time with schools going true P/F for their curriculum with no ranking. That's at least part of the reason why they have to rely heavily on Step 1 scores. If you get rid of that and make it true P/F, then that only punts the problem farther down the road. They'll still have to distinguish students somehow so they'll do it by school, by clerkship grades, etc. At least you have some degree of control over your Step 1 score. You have less control over clerkship grades, which have a huge subjective component on top of shelf exams. And you have little control over the school you go once you're a medical student. So it would just make the MCAT even more important because that will then end up determining which school you get into.
All in all, there's really no viable alternative as long as some fields remain highly competitive and the number of medical students continues to rise. However, I do think that Step 1 could be completely revamped to make it less about rote memorization of "buzzwords" and reward clinical reasoning ability more. Although there are also challenges with that.
I think the main issue is that there's no other good screening tool. We get almost no exposure to any surgery subspecialties during medical school so it's not like they can administer a test that's specific for all the particular subspecialties to figure out who's the best qualified to go into that residency. You could go off of sub-I letters, but those also are pretty difficult for distinguishing among applicants and are mostly subjective so it can be difficult for medical students.
I think one can also make a similar argument for the MCAT. Does knowing the differences between social interactionist and conflict perspectives in sociology really make you a better medical student? Probably not. But do you still have to do well on the MCAT to get into med school? Yes.
Yeah I really feel for the test writers. They're supposed to make sure part of their exam is knowledge checks. But, the appropriate level of knowledge is so stupidly easy to achieve with modern resources that fair knowledge check questions are directly in conflict with a nice valid distribution. If they have to toss any questions that too many people get right, how can they ever ask someone to recognize an india-ink stain of Cryptococcus? It's just too easy, when 80% of students know which histone proteins are inside vs outside the core, 99% will be able to recognize that image. So instead they have to drop the image and go with a more ambiguous vignette, or ask followup questions about details of the mechanism of action for Crypto treatments. Is that really what we want the USMLE to be like? Shouldn't it be allowed to just straightforwardly confirm that an American physician can recognize Crypto?
So what’s the alternative? I’m all for making step 1 p/f as long as residency selection doesn’t turn med school into law school. But how do you do that? Get rid of p/f in class grades? Find a way to standardize clerkship grading?
Well, how did it work 30 years ago? There wasn't always such extreme emphasis on step 1, by either students or residency directors. PDs still managed to decide between applications somehow. Maybe we just need to cap the number of residencies you can apply to, and have everyone go back to doing application reviews like they used to. My understanding is that high step1 cutoffs only became common because programs were getting dozens or even hundreds of applications per seat and needed a screening tool.
You mean based off the name of the school they attended...
Interesting to think that the PDs in 1980s did something right before things somehow went awry. A cap on the number of applications would be good.
I didn't realize that used to be how it worked. Guess the predictions about turning into law school aren't just speculation
Or it was worse because they went by what school you attend.
Capping would work I think only if there is a guarantee of matching. It's not really fair to make applicant A who has below-average stats only apply to 10 programs or whatever, since the more programs he applies to, the better his chance of matching. Not sure how you do that though, unless you did it like the military, where you list your top two choices of specialty and then your ranked list of locations, and someone takes you.
Hard cap on apps at all stages of application would be a good idea.
I'm just a bit confused regarding the stakeholders' desire to make standardized exams pass/fail or less numbers heavy etc. The MCAT was revamped with the AAMC explicitly saying that 500 = pass (i.e. good enough to do well in med school) but what instead happened is schools heavily utilizing that exam to select for top students and ultimately having MCAT matriculant medians in the top percentile. Now with the discussions on making Step exams pass/fail, the selection criteria gets more emphasized on clinical years or even more strongly on school name, which consequently worsens the MCAT score creep.
Now say then, the AAMC decides to make the MCAT pass/fail. That would result in a much stronger emphasis on undergrad name for applying to med school and a worsening SAT score creep. Making the SAT pass/fail would mean placing higher emphasis on high school name/quality/prestige and significantly damages educational prospects for students from low SES backgrounds.
Rather than trying to change how Steps are scored, why not instead focus on standardizing clinical years and make clinical grading a lot more objective and less susceptible to biases? Or maybe from that rant you posted, why not make Step exams like the MCAT, in that they focus a lot more strongly on critical reasoning rather than blindly regurgitating facts from flashcards?
The suggestion on placing a cap on number of residency applications is also an effective strategy. But keep standardized exams scored and just revamp the quality where necessary.
I'm not sure how a guarantee for matching would work though. A cap forces applicants to apply smartly which could probably reduce chances of going unmatched. But it seems that the risk of going unmatched is still unavoidable in any system tbh but idk.
How would you standardize clinical grading? What does that even mean? This is all pipe dreams. The reality is that things are going to keep getting more and more competitive every year forever until they stop building med schools. At that point, we have to decide if we want half of our medical education to be about flashcards or not. From a purely educational perspective, who matches what and where is secondary to having a sensible training system that lets people keep their curiosity, ambitions, passions, humanity alive while in the pipeline.
Maybe it’s because I’m not Plastic Surgery or Die Trying but ImO one of the main draws of going to a more prestigious school in the current meta is to have more room to do what you want without everything having to be about step for the first two years. It’s clear I’m in the minority given even at top schools many are obsessed with Step.
The risk of not matching increases when you cap an applicant's applications. If someone has a bad day and gets a 218 on step 1 and has an otherwise average app, he basically has to just hope that he only applies to programs where there aren't already too many other applicants of his caliber or higher applying. And so now step 1 has become even more important--unless it's P/F, in which case now programs are going to be ranking their applicants based on what? Subjective clerkship grades and school name. So how is the problem solved?
Advertisement - Members don't see this ad
I don't know there are different proposals being tossed around with some suggesting shelf heavy, others making them pass/fail or having better scoring sheets. The aim would probably be to make the grading a lot more objective and comparable.
I think a common statement thrown around here by faculty is med school is a process for going into residency so the score creep is unavoidable. But making it pass/fail would worsen everything.
There are ways to standardized performance like that. They're used in the military. Often there is also one small part that is subjective, which here would probably be the "would I have a beer with him" test. But in this case, basically it would amount to the same thing since everyone would make sure to hit the wickets, and you'd still have that bit that is subjective, which is what would distinguish you from the next person (good or bad), plus the shelf. So it'd end up about the same anyway.
Yeah, I think making step p/f would on the one hand be a good thing, because you could use it the way it should probably be used, which is as a measure to make sure student physicians can meet a certain benchmark. But on the other hand, the match would become a **** show.
I'm highly skeptical that there are any extremely aberrant clinical grading schemes out there. I don't think there's much room for improvement for standardization in this arena given the level of heterogeneity between what clerkships actually look like not just from one institution to the next but from one clerkship site to the next depending where you're at.
I think it'll be fine. It sucks everything keeps getting more competitive, but honestly I think the subordination of preclinical education to step 1 is more problematic than people getting P instead of H because they got a bad roll of the dice on who reviewed them on one of their clerkships. What do I think would happen if Step went P/F? I think the system would adapt, as it has always done. If we want to be proactive about things, a hard cap on number of apps would at least lessen the app burden so that PDs have to make fewer arbitrary decisions between marginally different candidates than they are currently doing (and hence driving up the Step requirements for their program as that is the current marginal differentiator) but the reality of any competitive selection process is that there are a high number of essentially arbitrary decisions and people cant go into this or any profession living or dying on that. Just my own attitude. I think people who are good students are currently succeeding and will continue to do so forever and always and the risk of a significant number of US MDs going jobless is near zero for the forseeable future.
But the PDs would have fewer apps to review so they counter each other. Although i guess it also depends on the overall application and what programs applicants are applying to with the cap.
Although i'm not sure if a stronger emphasis on Step 1 would be a bad thing unless it got revamped to make it something like the MCAT that tests critical thinking strongly rather than just memorizing things from flashcards.
In what way would the system adapt to a pass/fail Step 1? Because doing so looks worse in many ways so i'm struggling to see how it's a positive other than its not placing too much emphasis on a single test day.
I actually agree with you, which is why the system is probably going to just stay the same. There are issues, but so far it's working overall. The unmatched rate is really low, and in every specialty you are guaranteed a solid income and a job (radonc notwithstanding).
But what are they judging them on? If everything is P/F except subjective clinical grades, then it's just based on what school you went to. How is that better?
😕
I mean keep the current system as is besides i guess improving the quality of Step exams and just place a cap on the number of applications. I think the score creep could slow down but i'm not sure.
Ah, thought we were still talking about step being p/f. Yeah, in the current system, capping apps is a start. But it still favors high scoring applicants and hurts low scoring applicants, and so students will blow off preclinicals for step 1 even more. The higher your score, the safer you'll be with your limited number of apps. The lower your score, the more of a guessing game it will be and a higher risk of not matching.
I don't know how it would adapt. But I also know that the people who are the best positioned to imagine how to tweak things if this is made a new reality -- residency PDs and med school deans -- probably know better than med students on online forums, to be completely honest.
I'm not sure whether low scoring students would be hurt though? Especially when applying to noncompetitive programs and specialties?
Programs are still going to be going after the best applicants they can get. When your score is low, you are at higher risk because you have to try to apply smartly and hope you're applying places where you're not sufficiently below the mean of the applicants to the point where you don't match.
Not sure how anyone could look at law and think that's a better situation than what we currently have.
the situations aren't even comparable though. Going to the lowest ranked law school means you will probably not find a job practicing law after your earn that JD. The graduate of the lowest ranked med schools are still getting into residency and practicing medicine. A change in the way step is evaluated is not going to mean that everyone outside of the T20 is going to go unemployed. I doubt it even means they will stop matching competitively. But going to a med school with connections to institutions/locations you want to match into, whether or not they have reputable home departments in specialties you are interested in, whether its a brand-name school will probably matter more than it does now. Would it be so unreasonable for that to be the case? From the perspective of someone who wants to keep the "get into any med school, UFAP for 2 yrs and skip all med school, match ortho" paradigm going probably yes that seems rly bad. From the perspective of schools who want to produce well rounded physicians for their region, state, the country and not just future LA/NYC/Chicago/Boston/Houston sub-specialists probably not so much.
Advertisement - Members don't see this ad
Yeah, they aren't really all that comparable. But going to a bottom-tier med school will definitely make it significantly harder to match if it's all based on school name. Law is the closest thing to that system, but I agree that medicine would probably never get that bad since it's not really about graduating and immediately practicing, and the fact that there is a huge surplus of lawyers and plenty of room for more physicians.
We're thinking about it from our own perspectives though, where we want to be able to distinguish ourselves and have all doors open to us because of it.
Put yourself in a faculty member's shoes. Ten years ago your small groups would be an active discussion where people asked about things that really need an expert to clarify. You felt you were teaching people how to think in ways they will use on the wards.
Now, it's a small minority of students that want to engage with you because most people are grinding flashcards on their phone. Nobody cares about spending 20 minutes building a good thorough differential for your example case, because the skill they really need to practice is skimming a vignette in 50 seconds and eliminating options on a multiple choice list.
I can see why students and residency directors love the scaled scores, but I can also see how stakeholders in good preclinical medical education are growing to hate it. And if I could flip a switch that reset things to the former kind of classroom, even if that comes at the expense of Joe Stateschool being able to match Plastics, I think I'd flip it for the greater good.
Or maybe revamp Step exams to make it more strongly based on critical thinking just like the MCAT? Why get rid of the scores? It's really hard to just memorize your way to a high score in a critical thinking/reasoning based standardized exam. And that in turn could make preclinical years a lot more meaningful.
Yeah, I don't disagree. I really don't think what's happening in law would totally happen here. I think it would suck a little more for people at low ranked schools because while they may have been able to make up for that with a 260, they won't be able to with just a P, which will limit their career paths and ultimately mean that the MCAT and undergrad GPA will become even more important because pedigree will matter even more. So while you'll possible get a better education in med school and still be a doc, your career might already be limited before you even start med school.
Or maybe it won't. I dunno.
I think this is a better idea.
