Lmao. You act like writing exams is easy. The step 1 is actually one of the best written exams I've ever taken. You really think neurosurgery is going to put out a product of similar quality that is constantly being updated with different questions throughout the year. Multiply this by every single specialty. It's unsustainable, especially when we are already doing shelf exams that fill a similar role.
Pass/ Fail Step 1
- Thread starter sc2016
- Start date
- Status
Lmao. You act like writing exams is easy. The step 1 is actually one of the best written exams I've ever taken. You really think neurosurgery is going to put out a product of similar quality that is constantly being updated with different questions throughout the year. Multiply this by every single specialty. It's unsustainable, especially when we are already doing shelf exams that fill a similar role.
- Joined
- Apr 25, 2016
- Messages
- 1,770
- Reaction score
- 2,967
For the latter part; I can't really explain, since I'm not apart of the discussions on changing the usmle system lmao. But I was paraphrasing what @aProgDirector mentioned in a previous thread (see here: comment #204 -- MD - Heard a rumor that Step 1 (and maybe Step 2 CK) may change from scores to P/F. Is that true?)
For the former part; I never said it's "beneficial". Also, it won't be the "only metric". Never said that either. I simply said that Step 2 would bounce up to #1 on the PD survey's list of things they look at. Which isn't all that different than it is now. And students ankiing throughout their clerkships, again.. these speculations are wild. My point is none of us know exactly what will go down if anything changes. The program directors do have a better idea.
I understand that as med students we have to cling to the current step 1 scoring system, heck, I'm at a DO program so I understand the value of a numerical score showing PDs that I'm just as solid of a student as the MD counterparts. I get it. But is it safe to say that it the mere fact that they had this convention to discuss USMLE scoring means that there is room for improvement?
I never acted like writing exams is easy, did I ever say anything like that? Lol. Nor did I say that every single specialty will have their own exam. Take it easy buddy. It's alllll good. Ya'll need to relax lmao. I think we're going to be okay
D
deleted938090
A big reason for this is pass/fail preclinicals. Pass/fail anything isn't helpful to residencies because the buckets are so large. The fewer numbers there are attached to each student, the more emphasis the remaining numbers will have.
- Joined
- Apr 25, 2016
- Messages
- 1,770
- Reaction score
- 2,967
Which brings up a good point... why do you think a lot of schools are changing to p/f clinicals?
And if they are-- do you think it makes sense that the usmle would change too?
D
deleted938090
I suppose it's due to student feedback and student mental health.
It was mentioned that "Step 1 is a poor predictor of future clinical performance; the negative impact of the test on students’ mental health and engagement with institutional curricula; and the discriminatory effect of the test on underrepresented minorities, women, and those from lower socioeconomic backgrounds."
www.medscape.com
It's perceived that students actually want pass fail STEP but that is not unanimous by any means. I encourage those who do not want Pass Fail step to talk to your medical school deans to show them that this isn't what students really want. This could be another case of admin/doctors who are no longer in medical school thinking they know what's best for the students...
www.statnews.com
In this piece, the author also brings up a few points... but I'm rather skeptical because he ended up at BWH despite his low step 1 and this may show how much your medical school matters in placing you into good residencies. Individuals like the author would certainly benefit from pass fail step because it places more emphasis on their medical school.
It was mentioned that "Step 1 is a poor predictor of future clinical performance; the negative impact of the test on students’ mental health and engagement with institutional curricula; and the discriminatory effect of the test on underrepresented minorities, women, and those from lower socioeconomic backgrounds."
Students Urge Step 1 Fixes, Physician Leaders Reply With Insult
The response seems to indicate a generational divide in the medical community.
www.medscape.com
Make the Step 1 test for medical students pass/fail
Step 1 "cutoff scores" are being used to help medical residency programs choose candidates. That's a misuse of the test, which is a poor predictor of future clinical performance and harms learning in medical school.
www.statnews.com
In this piece, the author also brings up a few points... but I'm rather skeptical because he ended up at BWH despite his low step 1 and this may show how much your medical school matters in placing you into good residencies. Individuals like the author would certainly benefit from pass fail step because it places more emphasis on their medical school.Make the Step 1 test for medical students pass/fail
Step 1 "cutoff scores" are being used to help medical residency programs choose candidates. That's a misuse of the test, which is a poor predictor of future clinical performance and harms learning in medical school.
Pretty convenient opinion coming from a guy that was AOA at Johns Hopkins. I'm sure his definition of "bombing" isn't the same as the rest of us. Change his school to Joe Blow State U, change his curriculum to P/F taking away his AOA, and let's see how he feels.
By and large, there are no winners by removing Step scores. Certain specialties have implemented additional metrics (EM) and perhaps instead of blaming students as per usual, specialties can step (pun) up and determine what other actual distinguishing characteristics they are looking for in an applicant if they are unhappy with the current available options
- Joined
- Feb 13, 2017
- Messages
- 4,303
- Reaction score
- 7,922
This is my problem with P/F step one. It serves only to benefit those at prestigious schools, who can be hurt by their step 1 score if it's poor, but would match well based on their school name. I understand the "stress" argument, but making step 1 P/F isn't a solution. This will hurt anyone at a lower tier school/DO school.
- Joined
- Sep 7, 2016
- Messages
- 2,518
- Reaction score
- 6,175
I'm just throwing this out there, but what if the perceived is actually the reality? SDN is in no way ever an accurate depiction of MOST medical students. I don't think it'll solve anything to change it honestly. But revamping the system to test in a more logical way rather than some of the vague nonsense they throw out there in the name of stratification is definitely not a bad idea.I suppose it's due to student feedback and student mental health.
It was mentioned that "Step 1 is a poor predictor of future clinical performance; the negative impact of the test on students’ mental health and engagement with institutional curricula; and the discriminatory effect of the test on underrepresented minorities, women, and those from lower socioeconomic backgrounds."
It's perceived that students actually want pass fail STEP but that is not unanimous by any means. I encourage those who do not want Pass Fail step to talk to your medical school deans to show them that this isn't what students really want. This could be another case of admin/doctors who are no longer in medical school thinking they know what's best for the students...Students Urge Step 1 Fixes, Physician Leaders Reply With Insult
The response seems to indicate a generational divide in the medical community.
In this piece, the author also brings up a few points... but I'm rather skeptical because he ended up at BWH despite his low step 1 and this may show how much your medical school matters in placing you into good residencies. Individuals like the author would certainly benefit from pass fail step because it places more emphasis on their medical school.Make the Step 1 test for medical students pass/fail
Step 1 "cutoff scores" are being used to help medical residency programs choose candidates. That's a misuse of the test, which is a poor predictor of future clinical performance and harms learning in medical school.
Sure I'm only studying for it right now, but the amount of times I get the first 4 steps right in my head, but then don't know some tiny little mechanism and end up getting the question wrong is insane. I will never need to know that tiny little mechanism unless I do research in it. The way that the trend is currently going, its only going to get more and more away from testing knowledge and more about test taking skills. I'm not saying make it all softball questions, but the way NBMEs have been worded and some of the answer choices make me more than a bit concerned about the direction of the exam in the future.
I'm at a DO school and am super average, but I would rather be tested on stuff that actually resembles knowledge rather than following a convoluted path to an answer 5 steps down the line. Full P/F isn't a good idea, but adjusting in some way before the crap really hits the fan probably is a good idea. The BS that is associated with doing well on step 1 and stress and the like is a recipe for disaster long term
A
AnatomyGrey12
No it's not. People already do that instead of paying attention during anything in pre-clinicals so why would you think it would change and is "wild speculation?" Hell, most of the 3rd years I know already do that right now as is without a P/F Step 1.
And literally every one I've talked to has said they hate the idea of changing it to P/F and want the system to stay the way it is.
Change for the sake of change is even worse than maintaining the status quo. Before any changes are made there needs to be a VERY clear path forward with a clear outline of how it improves from the current situation. This rushed meeting is not the answer. It will do far more harm than good.
It's not a coincidence that these articles are almost invariably written by individuals at schools that wouldn't be affected by a change to P/F.
And according to the 5 people I know that I am really close with who have already taken it say the NBMEs are not even close to the real thing. So basing your opinion off of how you do on the NBME exams is a bit premature. It is not simply a minutia memorization exam. In fact my buddy who took it yesterday's first comment was, "that was the most beautiful exam I've ever taken" because of how well written it is and how it made him think. I can't wait to take it myself.
Med school is hard. That stress will never go away. In fact, a change from the current model will likely create even more stress as the emphasize will simply be placed further down the line and I don't know about you but I'd rather have a year or so to adjust my residency application plan than to find out last minute that it needed to change after I had already planned sub-i's and auditions. There has to be a way to stratify applicants and none of the proposed suggestions are better than the current model we have. Changing stuff just for the sake of change is never a good thing.
As to perception that students want a P/F exam, the only people I've heard even kind of support this are the ones who are most concerned about performing poorly. They are the same ones asking if they have to report their USMLE score on their ERAS app. The majority of students are against this pretty vehemently, at least at my school.
Last edited by a moderator:
- Joined
- Sep 7, 2016
- Messages
- 2,518
- Reaction score
- 6,175
Well if the exam is better than awesome. Which brings up another point....why in the hell do we all still take NBMEs if they're nothing like the real thing
- Joined
- Oct 21, 2017
- Messages
- 209
- Reaction score
- 336
off topic, but if you're in dedicated and posting on sdn you need to get off right now! It will not end well.
A
AnatomyGrey12
Neuroticism honestly. And somehow the NBME has created exams that are nothing like the real thing but that actually give you an ok prediction of how you'll do.
- Joined
- Jul 28, 2014
- Messages
- 1,982
- Reaction score
- 5,648
I was at preceptor recently, dipping my feet in the clinical waters... Preceptor asks me if I know what causes fever.
I launch into an explanation of PRR's and interleukins. He looks at me like I'm the dumbest person he's ever seen and goes... yeah so the patient has an infection, and inflammation helps to fight it off...
- Joined
- Jun 11, 2010
- Messages
- 74,888
- Reaction score
- 120,134
Its less stressful on the students.
If Steps go P/F, wouldn't that give a (greater) benefit to students who do audition rotations?
- Joined
- Oct 21, 2017
- Messages
- 209
- Reaction score
- 336
This is DO world thinking. Only a minority of students do away rotations and most of them in specific specialties (Ortho, neurosurg, derm, ophthal).
Away rotation isn't someone's "golden ticket" into locking down a program... Lots of people rotate at programs where they have no chance of matching. My state MD school average student has no chance at matching UPenn Gen Surg even with an away. And if they are competitive to match UPenn Gen Surg (as a few do every couple years) they don't need an away there, since they look super good on paper (AOA, Honors, 250+ step, first author pubs). So aways don't really help an average MD student unless it's in a specialty where aways are expected and it's a red flag not to do one.
Aways are kind of annoying to do (learn new EMR, hospital layout, new team culture and dynamics, etc) and can be very expensive. Not every med student has the money or the time to have the privilege do one.
- Joined
- Sep 7, 2016
- Messages
- 2,518
- Reaction score
- 6,175
If you wanted to do UPenn Gen surg don’t go DO? Either that or be okay with the most likely fact you won’t attain that.
It all comes back to people being in denial about what they’ll actually be competitive for it never fails in DO land on SDN
EDIT: I misread your post. So disregard haha but my point stands so I’ll leave it
- Joined
- Oct 21, 2017
- Messages
- 209
- Reaction score
- 336
If you want UPenn Gen Surg don't think an away there will help you in any way and will most likely hurt you, since if you're competitive for that program you likely look better on paper.
Okay, then what do you do with the MD/PhDs who take it now and will be applying to residencies 5-6 years down the road? They won't have known about this, will work their ass off for this exam now, and will have scores when nobody else will 3-4 years later. There's no good time where you can just say "Oh, this is when we're going to start doing this." Unless it's like 10 years into the future where people will enter med school knowing this and it will have been one long, drawn out rollout.
mingomango
Full Member
- Joined
- Mar 8, 2019
- Messages
- 13
- Reaction score
- 6
Yes, which screws over everybody else who doesn't have a score. So there's no good away about this. Imagine if half the applicants have scores ("unofficial") and the other half don't. What do you think will happen then?
It has to be a really long process of rolling it out if they want to do it right without hurting anyone. I don't understand why many medical students even want this. It doesn't make any sense after you think about it for a while. Getting rid of pre-clin grades was a good idea because it allows you to not worry about the useless material that you get taught and only focus on the relevant stuff. It gave us more time to focus on Step 1 and beyond. But getting rid of Step scores is one step too far. There's nothing else to distinguish people after that. Unless you really want to go off of rotation grades, Step 2, and all the subjective measures.
What doesn't make sense to me is - if you're a student at a not-top school, you don't want to get rid of this because it's how you can become competitive for top residencies. If you're a student at a top school, you still want to be able to distinguish yourself from all the other students competing for coveted residency slots. Like imagine if 20 people from your school now want to do derm. That makes it a whole lot harder for you unless you have a top Step score. So I don't understand why anybody would want this.
- Joined
- Jun 11, 2010
- Messages
- 74,888
- Reaction score
- 120,134
As an outsider looking in, here's what I've learned from these threads on Steps going to P/F
1) The NBME is making the change due to the problem of student stress and burnout.
2) Steps were never meant to be a tool for lazy PDs and their residents to help select who to rank
3) Steps are a useful tool for screening purposes for PDs and their residents to help select who to rank
4) NBME intended Steps to be a screen for competency and nothing else.
5) People are REALLY fearful that without a convenient screen, PDs will choose grads of high class med schools over anyone else.
I am NOT saying I am in favor of the proposed change. I'm a firm believer in the law of unintended consequences.
I agree that the hardest hit victims will be grads of P/F schools, unless they have some sort of rankings in their MSPEs.
Having heard this firsthand from someone in NBOME, I strongly suspect that the mindset of NBME is similar...namely, NBME doesn't care what your career aspirations are. They just want to know that you are competent. And on top of that, the guy from NBOME was frankly dismissive of student's aspirations. It was one of the few times in my career I've wanted to break the nose of a fellow professional.
So what to do? Don't complain on SDN...complain to NMBE. And involve your Faculty as well. And come up with alternative solutions to the problems that Steps are making now. I've seen good suggestions in these fora; make them heard.
1) The NBME is making the change due to the problem of student stress and burnout.
2) Steps were never meant to be a tool for lazy PDs and their residents to help select who to rank
3) Steps are a useful tool for screening purposes for PDs and their residents to help select who to rank
4) NBME intended Steps to be a screen for competency and nothing else.
5) People are REALLY fearful that without a convenient screen, PDs will choose grads of high class med schools over anyone else.
I am NOT saying I am in favor of the proposed change. I'm a firm believer in the law of unintended consequences.
I agree that the hardest hit victims will be grads of P/F schools, unless they have some sort of rankings in their MSPEs.
Having heard this firsthand from someone in NBOME, I strongly suspect that the mindset of NBME is similar...namely, NBME doesn't care what your career aspirations are. They just want to know that you are competent. And on top of that, the guy from NBOME was frankly dismissive of student's aspirations. It was one of the few times in my career I've wanted to break the nose of a fellow professional.
So what to do? Don't complain on SDN...complain to NMBE. And involve your Faculty as well. And come up with alternative solutions to the problems that Steps are making now. I've seen good suggestions in these fora; make them heard.
- Joined
- Oct 22, 2013
- Messages
- 8,599
- Reaction score
- 19,864
there will be a period open for public comment once the actual recommendations are posted in June. I hope the public comments are made, well, public. It'll be interesting to hear what the rest of the med ed / GME / UME community has to say beyond the SDN echo chamber. Unpopular opinion, but that most on Allo appear to be against S1 going P/F suggests to me that it's probably a good idea. Not a med student yet, but it makes sense to me that the critical exam be the more clinically relevant exam. I don't mind grinding flashcards for hours tbh it's all the same to me but I'd rather spend more time in lab / having a life. I suspect the knowledge decay post-Step 1 grind is hella steep so even though M2s might know less medicine once the exam goes P/F than they once did I highly doubt it'll affect the overall quality of training/education.
- Joined
- Jun 17, 2014
- Messages
- 63,099
- Reaction score
- 154,758
I think before changing Step 1, the focus should be on getting rid of Step 2 CS altogether. That exam is widely hated, unduly stressful and an expensive rip-off.
A
AnatomyGrey12
Sorry @Lucca , I like you but you aren't really in a position to make a statement like this. Step 1 IS pretty clinically relevant. The days of getting a high score because you memorize the steps of the kreb cycle or that currant jelly sputum is klebsiella are long over. It is a basic medical science exam, i.e. gets at a lot of the WHY things are happening, but don't mistake that for not being clinically relevant. It's been pretty well known that for a number of years Step 1 has been trending to far more clinical relevance.
Making Step 2 the big exam for residency has a whole host of problems that I could get into if you'd like, not the least of which being that it is taken right before you apply to residency and if someone has been a neurosurgery gunner their whole time in med school and then bombs Step 2 they are stuck in a porta-potty in an F-5 tornado. It also only gives you one shot at a test, whereas right now many applicants are able to make up for a mediocre Step 1 with a solid Step 2 and still end up in their career of choice.
While Step 1 certainly does test clinically-relevant material, it also tests material that, while relevant, is unlikely to come up in most peoples' careers. Step 1 likes to test things for which the pathogenesis is very clear because it can then tie in physiological concepts, which is really the idea of the exam. So things like Double Y males and things like that which have tie-ins to genetics. If Step 1 were to test only on concepts that were representatively relevant, then 90% of the exam would have to deal with heart failure, cirrhosis, nephrosis, or diabetes. Although these concepts do get a huge proportion of attention on Step 1 as it is now, if it was to be representative, it would be much much more.
So while I do think that everything Step 1 teaches is clinically relevant in some way, not everything is practically relevant. Your career as a physician isn't going to hinge on whether you can look at a super aggressive male, order a karyotype, and then diagnose him as Double Y.
A
AnatomyGrey12
Ok. I don't see how that negates anything I said. If we were only to be tested on topics that are practically relevant we should just memorize algorithms and call it a day. You even admit that the big topics are a large portion of the test, yet knowing the stuff that isn't necessarily going to walk in your office on a daily basis is the whole reason we exist. If we don't have a grasp on how to apply physiological principles to novel situations we are nothing more than NPs. I stand by what I said, the idea that Step 1 isn't clinically relevant is laughable and a myth.
Step 1 also gets a lot more credit than it deserves with regard to residency selection. You don't actually need a 250 to match any specialty if you are a USMD student despite what SDN says. Literally none of them. There are hoards of people that match even the most competitive specialties with average to slightly above average scores, hell even a decent amount of below average scores, and they do so because they often make up for the Step 1 by performing well on Step 2, doing well during the clinical years, getting good LORs, etc.
Getting rid of Step 1 scoring:
1. Won't decrease student stress at all unless you go to an elite school. This is a complete lie for the vast majority of students.
2. Simply gives students less opportunity to make up for a poor performance in other areas
3. Only serves to kick the can further down the road.
3. Puts a lot more emphasis on subjective junk like rotation grading, school prestige, how well known your home department is in your specialty of interest for LORs.
4. Completely screws over the hoards of students who use the standardized test scores to "move up" in the medical world. It will effectively create classes of medical students that will have a much harder time getting residencies outside of their perceived level.
Last edited by a moderator:
- Joined
- Jun 22, 2016
- Messages
- 644
- Reaction score
- 1,866
I do agree with this for the most part, but I also think there is something to be said for learning the rare diseases at some point in your medical career. If you don't have something like step 1 hanging over you, are you ever really going to teach yourself about Goodpastures?
Had a path professor who diagnosed Goodpasture her second year of residency, nobody else knew what was going on, but some aspect of it triggered her memories it from step 1 studying. Even if a lot of that knowledge is consciously lost, some residue remains and can be helpful to patients with rare conditions.
- Joined
- Oct 4, 2015
- Messages
- 1,616
- Reaction score
- 3,251
there was a cool post on r/medicine about a stumped rads doc with a woman with pulmonary infilitrates and signs of renal injury or urinalysis. she refused biopsy of lung or kidney initially and there was a whole host labs with pANCA positivity standing out. ended up likely being some kind of vasculidity likely EGPA or MPA. turned out the woman had childhood asthma.
i have no clue about the prevalence of these diseases, but it was cool as hell to make that connection for the differential
Was I trying to negate what you said? As I stated clearly above, all of Step 1 is "clinically relevant." The reason physicians exist is not to recognize the one case of Double Y male they will see in their entire career. The reason physicians exist is, like you say, to apply physiological principles to novel situations. Specifically, to apply physiological principles to solve problems that present in ways that may be outside what is typical. It is still most important that you understand the physiology underlying 99% of what you see. That 99% just happens to be the super common syndromes, i.e. nephrosis, cirrhosis, heart failure, diabetes, URI, pneumonia, etc. Then if someone comes in with something you've never seen before, what's important is that you have a basic understanding of how to work it up diagnostically. And if something is confusing you, you know how to search and critically appraise the literature to better help your patients.
What I'm saying is that while Step 1 is clinically relevant, it is not proportionate to what you see in clinic. Not by a long shot.
Even with a solid Step 2 score, coming back from a low Step 1 score is an uphill battle. For the most competitive specialties, if you don't do well on Step 1, you better choose your away rotations wisely and perform very well on those rotations. Because you're less likely to get interviewed at other places. Or if your department chair is well known, get him/her to make some calls for you.
But the point is that that isn't the only way to make the diagnosis. As a physician, you will have learned throughout your training that nobody can know everything and that 1) if you don't know, you know how and where to look it up and 2) you have colleagues who do know. Medicine is a team sport, not House MD. Even if Step 1 didn't exist, I would hope that medical school curricula would still cover Goodpasture's so everybody will have learned about it at some point. I would argue that if you're learning a lot of things for the first time during Step 1 studying, then your medical school curriculum isn't sufficient.
A
AnatomyGrey12
I'm still not getting your point. Are you saying you think it should be P/F? It's not supposed to be proportionate to what you see in the clinic. Even so, those topics are covered very heavily on Step. It is supposed to test the underlying basic medical science behind clinical decisions.
Medical training is a lot of repetition. There are concepts from exams that I got 98s on that when I pulled them up to review for Step I could barely remember them. Seeing something in your curriculum isolated in its system isn't the same as seeing it as it ties in to the bigger picture. Step studying ties in a LOT of concepts together and creates that big picture. I'm of the opinion that people would enter rotations less prepared if Step was P/F, as honestly simply passing the test really isn't the most difficult thing if you've put in even average effort in your curriculum and dedicated. The pass rates are very high. Once people hit that passing point they would stop studying and simply go take it. Personally I could have passed Step without any of my dedicated because I was prepping during the semester, and I can tell you for a fact I would start rotations with crap knowledge because of how much i've learned and solidified grinding the last month trying to score as high as I can.
Depends on what you mean by "low Step score." And at least under the current system you have another chance to prove yourself and balance out the first exam. If it were P/F then you would have even less opportunity, and If it were P/F and you go to a low tier school you wouldn't even really get a chance at all.
I have yet to see a good reason for changing it to P/F.
No, I'm very against it being P/F. I don't think there's any good reason why it should be P/F. At least, nobody has ever presented to me a good reason why it should be P/F.
I was kind of in the opposite boat. I put in a lot of work during the pre-clinical years trying to really understand everything (PhD mindset) in depth and I started dedicated at a 250. So dedicated was a whole lot of review and refreshing but I found that I hadn't really forgotten about anything. Maybe obscure things like Behcet's but not really anything for which I already understood the pathogenesis. If Step was P/F, I would have exactly the same amount of knowledge I started dedicated with. I think what you're getting at is that for some/many students, Step 1 forces you to review the material all at once. But I would argue that continuous, longitudinal learning is something that you need for long-term success (and success on Step 1). If you don't motivate yourself to learn and review now unless there's a huge exam, what are you going to do when your boards are every 10 years?
Any low Step 1 score. It's a spectrum. If you're applying Derm and your Step score is 228, it's an uphill battle. Step 2 isn't going to erase that. If your Step 1 score is low, I think more important ways to balance that out is to choose your away rotations wisely because those places are going to be your best shots at matching. In other words, you come to rely more on the subjective aspects of the process, i.e. performance on aways, calls from your program, etc.
A
AnatomyGrey12
Where did I ever say one shouldn't be reviewing during pre-clinicals? I literally said I did that, and most people do. I'm still not getting the point that you're getting at and how it relates to the current discussion.
Again, still not sure what the point is with this. I didn't say anything different, except that in the current model you at least have more than one opportunity to try and balance out a lower Step 1.
Then why were there concepts that you could barely remember when you started dedicated? Presumably, you would have been reviewing/reinforcing those concepts all along. As for my point, I've been saying it. Step 1 = clinically relevant but not practically relevant. Knowing random **** that helps you make 1-2 diagnoses in your entire career is not what makes you a physician. Being able to problem solve and appraise common problems that present atypically or in conjunction with other problems is.
You were quoting a statement that wasn't made in response to you, so I understand how you wouldn't see how it's relevant.
Last edited:
- Joined
- Apr 24, 2017
- Messages
- 114
- Reaction score
- 107
Why don't we just argue this come June, when their recommendation is public for discussion.
- Joined
- Jun 17, 2014
- Messages
- 63,099
- Reaction score
- 154,758
Alternatively, you can just close your eyes until June
Do you really think that's a good reason? As others have already stated, it just punts the ball down the road and I have seen many reasons why Step 2 should not be given that level of relevance.
- Joined
- Jun 17, 2014
- Messages
- 63,099
- Reaction score
- 154,758
I can't find anything better
I think P/F Step 1 is a terrible idea but people are wasting time focusing on Step 1 and trying to ruin it with unnecessary changes, when Step 2 CS, which itself is pass/fail, is a terrible, expensive and stressful exam that literally tanks students' careers if they fail them for whatever unfortunate circumstances. Yet medical education leaders don't care because they're so pathologically obsessed with the "horrors" of Step 1 mania.
I agree that Step 1 changes are all unnecessary at this point. You're more familiar with the Step 2 CS issue than I am though so I'm not educated enough on that to comment...
- Joined
- Jun 17, 2014
- Messages
- 63,099
- Reaction score
- 154,758
i'm not really familiar with Step 2 CS actually (it's from reading dozens of Step hate threads and personal encounters, which admittedly got me more paranoid than i'd like to admit). I just don't like the idea of schools and education leaders trying to ruin medical education and making med school a nightmare. which is why i hate the idea of changing Step 1 for something that clearly looks a lot worse.
- Joined
- Oct 11, 2006
- Messages
- 10,733
- Reaction score
- 15,563
There are several threads talking about this, so this will be somewhat repetitious.
I don't find the argument that the USMLE was designed to be a test of minimum competence, and hence shouldn't be used for selection, terribly persuasive. It was in fact developed to test for minimum competence, to replace individual state exams. However the test characteristics do lend themselves to discriminating student performance. If the test really was "designed" (rather than developed) to test for minimum competence, then it should be relatively easy to get all the questions correct. You wouldn't want to ask rare or obscure facts -- instead you'd ask basic questions looking for minimum understanding. Consider the many online training modules you've probably had to complete to test your understanding of fire drills, or hand hygiene / precautions. Most modules like this have a test at the end to make sure you learned the material -- and usually you'll get 100% because it's pretty basic, and the minimum pass is 90% or so. That's a test that's been designed to test minimum competence -- it has maximal discriminatory power at that minimal knowledge level, and tells you nothing about middle or high achievers. The USMLE scoring is nothing like this.
What does S1 test? Honestly, I think it tests your ability to learn material. The actual content of the exam doesn't matter all that much. If you can learn the S1 material well and get a good score, you can probably learn the material I need you to know and get a good score.
What will the NBME do? I have no idea. But what is clear is that they feel their stakeholders are the FSMB and the AAMC, and they don't really care what anyone else thinks. They told us this, directly, at our national meeting. The FSMB really doesn't care what happens -- they only care about the P/F result in any case. The AAMC thinks this will improve the student experience and decrease stress. I doubt that -- the stress will just move to something else.
What's the likely timeline? Also unclear, but given the downstream effects I expect there would be a long lead in period to any change. The one change I could imagine might happen more quickly is quartiles -- instead of reporting scores, only reporting quartiles. This is a middle compromise -- takes away the pressure to get a 270 vs 260 vs 250. But unclear if it would really change how anyone preps for S1, and clearly hurts people who just miss a quartile cutoff (but helps people who just beat a quartile cutoff).
What's the public comment period? It's a period of time where the NBME will accept comments from the public. There's no way they would actually publish those comments publicly.
How likely are changes from the initial proposal in June? Honestly, I think it's zero. They will take public comments, then just finalize their recommendations. I doubt anything people say will be anything new that they didn't consider beforehand. And as mentioned, the only people they care about are the FSMB and the AAMC.
So what happens if they make S1 P/F? In the short term, programs will probably just demand a S2 score prior to application. This will simply put the current pressure on S1 onto S2, and as mentioned above on this thread that has all sorts of consequences -- some good, some bad. But if they make S1 P/F, it's only a matter of time until they do it to all the Steps. So, my recommendation to the IM community should the NBME propose that S1 be P/F is that we develop our own scored exam that all applicants will need to take. I expect that the major fields would follow suit -- Surgery, Peds, OB, etc. Alternatively, we could all join together and create one single exam. I would start doing this if they leave S2 as a scored exam, because it will take years to develop and it would seem that the writing is on the wall. I would announce this during the public comment period -- that's perhaps something the AAMC would listen to, since they won't want their students taking yet more exams. And most importantly, I would insist that the NBME not have anything to do with this new exam -- I don't want them profiting from this mess.
I don't find the argument that the USMLE was designed to be a test of minimum competence, and hence shouldn't be used for selection, terribly persuasive. It was in fact developed to test for minimum competence, to replace individual state exams. However the test characteristics do lend themselves to discriminating student performance. If the test really was "designed" (rather than developed) to test for minimum competence, then it should be relatively easy to get all the questions correct. You wouldn't want to ask rare or obscure facts -- instead you'd ask basic questions looking for minimum understanding. Consider the many online training modules you've probably had to complete to test your understanding of fire drills, or hand hygiene / precautions. Most modules like this have a test at the end to make sure you learned the material -- and usually you'll get 100% because it's pretty basic, and the minimum pass is 90% or so. That's a test that's been designed to test minimum competence -- it has maximal discriminatory power at that minimal knowledge level, and tells you nothing about middle or high achievers. The USMLE scoring is nothing like this.
What does S1 test? Honestly, I think it tests your ability to learn material. The actual content of the exam doesn't matter all that much. If you can learn the S1 material well and get a good score, you can probably learn the material I need you to know and get a good score.
What will the NBME do? I have no idea. But what is clear is that they feel their stakeholders are the FSMB and the AAMC, and they don't really care what anyone else thinks. They told us this, directly, at our national meeting. The FSMB really doesn't care what happens -- they only care about the P/F result in any case. The AAMC thinks this will improve the student experience and decrease stress. I doubt that -- the stress will just move to something else.
What's the likely timeline? Also unclear, but given the downstream effects I expect there would be a long lead in period to any change. The one change I could imagine might happen more quickly is quartiles -- instead of reporting scores, only reporting quartiles. This is a middle compromise -- takes away the pressure to get a 270 vs 260 vs 250. But unclear if it would really change how anyone preps for S1, and clearly hurts people who just miss a quartile cutoff (but helps people who just beat a quartile cutoff).
What's the public comment period? It's a period of time where the NBME will accept comments from the public. There's no way they would actually publish those comments publicly.
How likely are changes from the initial proposal in June? Honestly, I think it's zero. They will take public comments, then just finalize their recommendations. I doubt anything people say will be anything new that they didn't consider beforehand. And as mentioned, the only people they care about are the FSMB and the AAMC.
So what happens if they make S1 P/F? In the short term, programs will probably just demand a S2 score prior to application. This will simply put the current pressure on S1 onto S2, and as mentioned above on this thread that has all sorts of consequences -- some good, some bad. But if they make S1 P/F, it's only a matter of time until they do it to all the Steps. So, my recommendation to the IM community should the NBME propose that S1 be P/F is that we develop our own scored exam that all applicants will need to take. I expect that the major fields would follow suit -- Surgery, Peds, OB, etc. Alternatively, we could all join together and create one single exam. I would start doing this if they leave S2 as a scored exam, because it will take years to develop and it would seem that the writing is on the wall. I would announce this during the public comment period -- that's perhaps something the AAMC would listen to, since they won't want their students taking yet more exams. And most importantly, I would insist that the NBME not have anything to do with this new exam -- I don't want them profiting from this mess.
- Joined
- Jun 17, 2014
- Messages
- 63,099
- Reaction score
- 154,758
how are specialty exams different from shelf exams?
D
deleted103644
olabrador23
Full Member
- Joined
- May 5, 2019
- Messages
- 279
- Reaction score
- 189
Last edited:
- Joined
- Nov 12, 2017
- Messages
- 207
- Reaction score
- 420
I see the “step 1 was supposed to be P/F anyway” remark a lot, but that was during a time where residencies were a lot less competitive and there was a surplus of spots. Taking the major objective measure away in today’s match is going to be chaos. The only people who stand to benefit are those at elite institutions, which is why it will probably happen.
- Joined
- Oct 21, 2017
- Messages
- 209
- Reaction score
- 336
The hypothetical specialty exam would likely contain more than 110 questions (that's the current shelf length, 100 scored questions and 10 experimental). Shelfs aren't really used in residency section right now (insofar as they determine the clerkship grade). The questions may have a different "style" also.
A
AnatomyGrey12
You literally quoted me...
Perhaps because there is simply a lot of material and keeping up on class and reviewing every past topic is challenging? Nice job making passive aggressive comments at what kind of student I am 👍
And I say BS. Most of Step IS practically relevant. You can't tell me it isn't practically relevant to have a deep understanding of heart failure, diabetes, renal functioning and how it interacts with all the other systems, pharm, etc. This is all over Step 1 prep in copious amounts. You are arguing a strawman, and I'm not even sure why because on the actual topic of if Step should be P/F we agree completely.
Let's not derail the thread anymore. We agree Step shouldn't be P/F. Moving on.
- Status
