USMLE Comprehensive Review

[NotAProgDirector]

---

Members do not see this ad.

I was just made aware that the USMLE is undergoing a comprehensive review. By "comprehensive", they mean that everything is on the table. A summary of their findings to date is located on the USMLE's website here.

I'll quote the end of the article:

Major themes emerging in this process thus far include the following:

1. There is a strong sense that the licensure examination program should be more explicitly designed to support decisions at two points. The first of these is a decision about readiness to begin provision of direct patient care under supervision, at the interface between undergraduate and graduate medical education. The second decision relates to readiness for a physician to provide unsupervised patient care and to obtain a license to enter into unrestricted practice.
2. At the time they enter post graduate training, doctors must have minimum competency in basic clinical knowledge and the skills necessary to safely care for patients. A higher level of these competencies, together with other competencies acquired during graduate medical education, is necessary at the time of primary licensure. To the extent that these competencies can be measured in a valid, reliable, and practical manner, they should be incorporated in the USMLE.
3. From both a licensing and an educational perspective, the separate design and administration of an examination of the basic sciences seems to create an artificial separation of basic and clinical sciences. This was a sentiment frequently expressed by stakeholders, including faculty members from both the basic sciences and clinical sciences. The weight of opinion gathered to date favors the integration of basic science and clinical science concepts throughout all examination components rather than the current segregation of basic science content in Step 1.
4. The current Step 1 component of the USMLE is used by many medical schools to support promotion or graduation decisions. If the USMLE is redesigned in a manner that eliminates Step 1 in its current form, then NBME should be prepared to provide similar valid, reliable, and secure assessment tools to schools that still wish to use them.
5. There are conflicting opinions on the value of numeric versus pass/fail reporting for both the primary and secondary uses of USMLE. It is likely that this issue will not be resolved until the final recommendations of CEUP are made and the implications of those recommendations fully examined.

#1 and #2 are simply redefining what the USMLE has become. #4 is simply stating that no matter what happens, there will be some test available to medical schools for use at the end of the second year.

The real juicy material is #3 and #5. To my read, #3 is suggesting either that Step 1 and 2 become both a mixture of basic sci and clinical, or (combining #1, #3, and #4) that step 1 be dropped completely and that a combined basic sci / clinical knowledge exam be required to start residency training, timed where Step 2 currently sits.

#5 is suggesting that the USMLE become pass/fail.

Although I am happy to post my thoughts on this issue, I was hoping to get some input as to what people think about these changes. I expect this will be a topic of discussion for program directors and medical schools shortly.

NOTE: Although it may seem that this topic would be more appropriate for the USMLE forums, I thought that I would get a better perspective from this more general forum. If a mod feels it fits somewhere else, feel free to move it!

 

[nykka3]

Wow thanks for the link! I find this very interesting esp. #3-#5. As an MS4, it sure would take the stress off of studying for usmle in order to get that golden score and just focus on learning the material to the level of a predesignated bar which equals pass. I guess it MIGHT make it harder for residency prgm's to evaluate the tons of applications they receive if there is only a pass/fail designation. I wonder how they evaluated applicants in the "old" days when scores were not as impt..... It will also be interesting if there was a combined basic science/clinical science test taken after 3rd year b/c that is when the basic science actually meant anything to me. Applying a clinical context to the basic science facts is key.

 

[buckley]

To be honest I actually feel Step 1 and CK should have been combined. What I thought (or assumed) was that Step 1 was for US medical schools to make sure their students have at least some working knowledge to start in the clinics. And then CK and CS to make sure the students are ready to be residents.
As an IMG, I took all of them post med school.
As for pass/fail...I was under the impression that numeric scores really were not that comparable statistically speaking, but real-world speaking, of course residencies use that. I honestly don't feel it was comprehensive and the variety of questions really do matter. E coli petri dish questions? My friend was asked to identify muscles! Hayayaya...For the IMGs, we have been told again and again that it's the only way we can "prove" we're as good as the US grads. So above 90s always has been the goal. Though I love the idea of a pass/fail exam, I feel it would be on a down-note for future IMGs who want some errr competitive slots.
On another note, Im not from the US so I have to ask---anybody feel these exams are simply way too expensive? I can't imagine students shelling out 1200 USD for CS. Ugh. I'm done with it already, but I honestly felt it was a little overboard. I could bring my own lunch and snacks, maybe my own pen, if that was what made it so expensive.

But then again, as my friends--and fellow IMGs--remind ourselves, who are we to have an opinion? we want US training, we swallow everything thown our way...hahaha...but it is nice to see USMLE questioning itself. At least that progressive spirit is what makes the US such an inviting place to train in.

 

[RxnMan]

With respect to #3, my attendings complain that the students seem to forget everything when they step onto the wards for the first time. They believe that students have no experience to associate their book learning with.

Some schools have switched from lecture/self-paced learning to PBL formats, which kinda integrates things. From the article you posted, it looks like they're trying to integrate the science and clinical curricula even more.

Hijacking your thread a bit, how about this: Remove MS2 and MS1 summer and get the students out on the wards 15 months earlier. They don't have any path yet, just basic phys and anatomy. Maybe give them some Pharm. Then have them rotate through the specialist services first. Later rotations would be the FM, IM, and EM, where you'd integrate everything. Since you have more time (15 months worth), you could stretch out rotations and supplement them with organ- or specialty-based lectures, similar to how most residencies are formatted.

Feel free to say why it won't work.

 

[yaah]

I worry that the whole process is going to start to become "dumbed down" (obviously an extreme way to put it, but just go with the expression) akin to how many med schools are going to pass/fail. Does this really help anyone?

Knowledge is important and crucial and the fact the med schools are already moving away from more knowledge at the expense of "feel good" and "practical" type of courses irritates me a lot, especially in the current atmosphere of medicine where new discoveries happen daily, the volume of information grows exponentially, and having the scientific and medical knowledge to understand these developments makes one a much more effective physician. People may disagree with that of course, that another way to look at it is that medicine is becoming more algorithm and guideline based, and the job of many physicians is to know these aspects. Or alternatively, that medicine is becoming so specialized that it only is necessary to understand things within a narrow part of medicine (unless you are a generalist, in which case you fall back under the first algorithmic description). Obviously as you can tell, I disagree with those presumptions.

I know the trend these days is to see all standardized tests as somehow biased or uninformative, but without standardized tests, what do we have? Evaluations will become even MORE biased and subjective to evaluation and all the inherent politics that goes into that. How do you compare med school teaching? How do you really know someone actually knows the material?

Obviously the way standardized tests are, one is going to have to learn some trivial information that doesn't impact your future knowledge or practice one bit, but that's part of learning!

I worry that these proposals are going to increase the number of physicians who are technically "competent" because they can follow the algorithms and guidelines and refer people appropriately, but that isn't necessarily a good thing if there are tradeoffs involved. You can't decrease evaluations based on knowledge and reasoning and then expect people to develop MORE knowledge and reasoning. You have to have the knowledge to know what questions to ask and where medicine is going.

I am a physician, but I also could be a patient. I don't want my future care directed by people who only feel it prudent to know enough to "pass" the USMLEs. I also don't want my care directed by someone who only knows how to empathize with me but doesn't understand why a certain drug is causing my liver to fail.

I strongly agree with the premise above that med schools need to do a better job of preparing students for direct patient care. But I don't think the USMLEs can have anything to do with that. It has to be through direct clinical exposure and experience (which is NOT at the expense of classroom learning!) - if this means extending med school, so be it.

 

[Faebinder]

The licensing and the grading for spot needs to be separate processes. Currently, they are integrated in the infamous USMLEs. Funny, both chiefs that rotated on my service had not yet taken step 3 yet graduating this year after 5 year residencies. So my opinion of the USMLE determining your license is pretty low. It's pretty amazing to me that we still have the USMLE as necessary for the licensing. Don't the medical boards realize that there are NPs out there with 1 year post nursing school that can prescribe virtually anything?

I would like to see a federal license instead of a state license as well. You get tired of hearing stories how doctor A can't practice in state X but can in state Y. What? Is he missing a necessary organ that he is not good enough for state X but is okay for state Y? I think a federal license is a good solution.

 

[Faebinder]

With regards to step 1. I dont think it's possible to have an integrated step 1 and 2. I cant think of a single step 1 component that is not useful later in your career. Did you all walk in knowing how a muscle looks under the microscope, what toxoplasma is and what the anterior-thalamic track is? But I do agree that it is a bad test to use to tell whether someone is fit for a residency or not.

 

[NotAProgDirector]

As a program director, I cannot imaging trying to evaluate residents without USMLE scores. As mentioned by yaah, everything else is so subjective it's hard to know where people stand. In some medical schools, 90+% of everyone gets honors. In others, 5% get honors and everyone else gets pass. Either way, I often get almost no usable information about someone's performance. No med school is P/F in the clinical years yet, but I forsee that coming also. And don't get me started about LOR's, which uniformly tell me that this student is in the "top 5% of students with whom I've worked". The USMLE is imperfect, for sure. However, at least it's standardized.

Earlier clinical exposure is certainly possible, as suggested by RxnMan. In fact, at Duke both years 1 and 2 are crunched into a single year, clinical rotations are second year, and the third year is a complete year of research. I'm not convinced that this is an ideal model, but the fact remains that it's possible.

My thoughts are this:

1. If Step 1 and Step 2 are combined into a single exam, then students will all need to take it prior to interview / match decisions. I can't imagine PD's being willing to interview or rank people without these scores.
2. I think the exam needs to remain "scored" rather than pass/fail.
3. I wonder if medical students should be allowed to retake the exam if they are unhappy with their score, perhaps once (or twice).
4. If Step 1 is replaced by an NBME shelf type exam for medical school use, there will be a HUGE pressure to report those scores to residency programs. Hence, switching Step 1 to a shelf exam will be unlikely to change much, although I guess a school could refuse to release those scores (but most will, my guess)

 

[nykka3]

1. If Step 1 and Step 2 are combined into a single exam, then students will all need to take it prior to interview / match decisions. I can't imagine PD's being willing to interview or rank people without these scores.
2. I think the exam needs to remain "scored" rather than pass/fail.
3. I wonder if medical students should be allowed to retake the exam if they are unhappy with their score, perhaps once (or twice).
4. If Step 1 is replaced by an NBME shelf type exam for medical school use, there will be a HUGE pressure to report those scores to residency programs. Hence, switching Step 1 to a shelf exam will be unlikely to change much, although I guess a school could refuse to release those scores (but most will, my guess)

i like the idea of a combined step1/2. i think it should not be pass/fail, and it should be reported b4 interview season. shelfs are also ok b/c they can help prepare you for the step exam, and they can include basic science stuff if needed. i just feel that i learned more and understood more of the basic science after my 3rd year clinical exposure.

 

[Faebinder]

As a program director, I cannot imaging trying to evaluate residents without USMLE scores. As mentioned by yaah, everything else is so subjective it's hard to know where people stand. In some medical schools, 90+% of everyone gets honors. In others, 5% get honors and everyone else gets pass. Either way, I often get almost no usable information about someone's performance. No med school is P/F in the clinical years yet, but I forsee that coming also. And don't get me started about LOR's, which uniformly tell me that this student is in the "top 5% of students with whom I've worked". The USMLE is imperfect, for sure. However, at least it's standardized.

Earlier clinical exposure is certainly possible, as suggested by RxnMan. In fact, at Duke both years 1 and 2 are crunched into a single year, clinical rotations are second year, and the third year is a complete year of research. I'm not convinced that this is an ideal model, but the fact remains that it's possible.

My thoughts are this:

1. If Step 1 and Step 2 are combined into a single exam, then students will all need to take it prior to interview / match decisions. I can't imagine PD's being willing to interview or rank people without these scores.
2. I think the exam needs to remain "scored" rather than pass/fail.
3. I wonder if medical students should be allowed to retake the exam if they are unhappy with their score, perhaps once (or twice).
4. If Step 1 is replaced by an NBME shelf type exam for medical school use, there will be a HUGE pressure to report those scores to residency programs. Hence, switching Step 1 to a shelf exam will be unlikely to change much, although I guess a school could refuse to release those scores (but most will, my guess)

A combined step 1 + 2 is a bit bulky. Though it's possible that they should combine both exam scores together to get 1 score only. Being able to repeat the exam to increase scores should be an option, Why? Because if this exam does what it is supposed to do (which is measure the medical knowledge/skills of a potential resident) well then it should be able to demonstrate the variability in knowledge/skills with time because time can only increase your knowledge/skills when you study the material some more.

 

[physicsnerd42]

I have a rather selfish concern about Step I and II being combined: I'm a MD-PhD student who is in second year med school right now and will be starting his PhD next year. I'll be taking Step I in June. If the rumor mill is to be believed, the combined Step I/II will be rolling out ~2011, which may be a year before I get back to the wards. If this happens, I'll have the pleasure of having to take Step I twice: once as Step I and once as the new combined test. Trying to re-memorize all the minutiae required for Step I after 4 years in the lab will be quite a challenge. I suspect this will be a huge problem for all other 2-P-2 format MD-PhD programs out there.

Anyway, I just thought I'd raise a concern for the ~2500 of us in MD-PhD programs at the moment.

 

[Faebinder]

Well, on the other hand... paying for one exam is so much ..... "humane" to my pockets which scream bloody murder every time I swindle some poor relative of mine for some money for an exam/interview/course/BS-requirement(such as first time medical board application for the hospital).

 

[physicsnerd42]

Well, on the other hand... paying for one exam is so much ..... "humane" to my pockets which scream bloody murder every time I swindle some poor relative of mine for some money for an exam/interview/course/BS-requirement(such as first time medical board application for the hospital).

Do you really think the NBME is going to be willing to forgo revenue to combine two tests into one? I bet they'll find a way to justify charging more for the combined test than Step I and Step II CK added together cost right now. 👎

 

[Faebinder]

Do you really think the NBME is going to be willing to forgo revenue to combine two tests into one? I bet they'll find a way to justify charging more for the combined test than Step I and Step II CK added together cost right now. 👎

it costs less to grade 1 exam rather than 2, chances are it will be more expensive than either of the exams but less than their combined total.... although i do find yor lack of faith....dist..err.. on the money.