Getting rid of USMLE?

[Gabby]

---

Members don't see this ad.

A post in one of the other forums mentioned that on the med school interview trail, a school told applicants that the USMLE is being replaced within a year or two with a Gateway exam. The scores (I assume pass/fail?) won't be made available to residency directors.

Anyone know if we're really a year or two away from this and how the residency selection system is going to be restructured?

 

[chessknt]

Last year on the interview trail I heard about this too, but I think its very far off. I also heard they were just going to combine steps 1/2 into one test, which would be problematic for assessing readiness for clinical years.

 

[ChiDO]

Believe that the combined step 1/2 will affect those entering school in 2012/13 (ie. Class of 2016/17).

 

[chessknt]

Did they figure out how they were going to overhaul residency apps & fix the legal issues of letting med students with no proven scientific competence into hospitals?

 

[Ableton]

Did they figure out how they were going to overhaul residency apps & fix the legal issues of letting med students with no proven scientific competence into hospitals?

I'm pretty sure passing your preclinical coursework demonstrates enough "scientific competence" to begin 3rd year. Shelf exams can also be used to help in that assessment.

 

[ChiDO]

I'm pretty sure passing your preclinical coursework demonstrates enough "scientific competence" to begin 3rd year. Shelf exams can also be used to help in that assessment.

Also, step 1 was never meant to be used as a residency selection tool.\; but that is what it became. The NBME stated this and that is why they are combining the steps.

 

[Gabby]

Also, step 1 was never meant to be used as a residency selection tool.\; but that is what it became. The NBME stated this and that is why they are combining the steps.

What was the initial intent of the Steps?

 

[lord_jeebus]

What was the initial intent of the Steps?

To determine your qualification for medical licensure

 

[Typical Premed]

In Canada we don't have to write any board exams before clerkship. We write the MCCQE Part I (which is similar to Step II CK) at the end of M4 after the residency match has already occurred. We also write the MCCQE Part II (which is similar to Step III) in the fall of PGY-2. This system seems to work fine as we produce many competent doctors. However, I've heard it is difficult for residency directors to distinguish between applicants since all the schools are Pass/Fail for the first 2 years and there is no standardized test to compare applicants from different schools. So basically it comes down to clerkship performance, reference letters, research, etc.

 

[Gabby]

To determine your qualification for medical licensure

So if they combine them now for that purpose, when will you take it? After third or fourth year, I assume since Step II is more clinical? Of course, that means remembering all the Step I material while on rotations for two years.

 

[chessknt]

While I admit it would be super awesome to not have to worry about a score for residency selection, what exactly does that leave directors to use to evaluate? Will this make pre-clinical grades become supremely important?

 

[RegularNihilist]

Get rid of licensure instead.

 

[physics junkie]

I highly doubt the step1 would be replaced by a "pass/fail" system.

It would be unfortunate if they got rid of the step1. As much as I dislike standardized tests I know that preparing for them is absolutely essential for reorganizing your knowledge from resembling a patchwork quilt to a reliable foundation.

 

[sunset823]

I agree it's not a good idea to get rid of Step 1- every school is different, so you need a way to compare across schools, which Step 1 does, and I can't imagine what residency selection will be based on - just clinical grades? recommendations? As a person who's pretty good at standardized tests but incapable of brown-nosing (and therefore worried about what will happen in third year), I think that subjectivity would be unnerving.

 

[Slack3r]

Also, step 1 was never meant to be used as a residency selection tool.\; but that is what it became. The NBME stated this and that is why they are combining the steps.

While I admit it would be super awesome to not have to worry about a score for residency selection, what exactly does that leave directors to use to evaluate? Will this make pre-clinical grades become supremely important?

They're trying to do this for dental schools and the dental equivalent of the USMLE. There has been some speculation that residency programs will either develop their own tests or will use some combination of clinical/pre-clinical grades and the GRE.

 

[Gabby]

I highly doubt the step1 would be replaced by a "pass/fail" system

I think it's a pretty done that Step 1 will be pass/fail, at least the last I heard of it. The only question is when will it happen. I thought we were five or more years away.

 

[surag]

I think it's a pretty done that Step 1 will be pass/fail, at least the last I heard of it. The only question is when will it happen. I thought we were five or more years away.

thats ridiculous...this will put an unnecessary burden on students at "lower" ranked schools to get into good residency programs. The residency directors will have to put greater perference on school reputation and assumed rigors of the programs. Schools like Yale will need to do away with their no test policies and many pass fail academic policies will turn to grading

Currently students at state schools would get into good residency programs because they impress with their step 1s along with school work/curriculars. without the step 1(pass/fail really fails to separate students), residency directors would become more dependant on the medical school reputation itself. I know harvard carries huge respect over acme state med school but its still nto impossible for acme student to get into good residency programs at competitive places. my cousin went to a state school(not uva, ucsf, michigan, pitt, etc.) and is now an ophthalmology resident at a top 15 ophthalmology program. if the step one gets changed then this would be nearly impossible short of awesome research/extras that are very hard to come by at state programs to begin wtih.



am I wrong to think like this? its certainly possible...

 

[Gabby]

On the other hand, there are some people who aren't good test-takers, but make incredible doctors. There's something to be said for placing so much emphasis on third-year grades, subjective as they may be.

 

[meister]

Get rid of licensure instead.

This is a great idea, the free market should decide all.

 

[Dr Lyss]

Did they figure out how they were going to overhaul residency apps & fix the legal issues of letting med students with no proven scientific competence into hospitals?

not all schools require passing step 1 to go onto third year. I don't think the concern is about competency.


I think it's a good idea because I hate the fact that our performance on the preclinical subjects is so heavily emphasized when it isn't all that important for real practice. The focus should be on the clinical aspect since that will have the greatest correlation to how you would perform in that residency. I'm not saying the preclinical years aren't important, just not as important as we make them because of this exam.

 

[Hurricane95]

Sucks for those of us who destroy the Steps.

 

[Dr Lyss]

Sucks for those of us who destroy the Steps.

but great for those of us who hate standardized exams

 

[dude1344]

So... does anyone know when this will actually be implemented (if it will be)? Probably won't affect those of us who are already in school correct?

 

[VincentAdultman]

Evey year someone starts a rumor that Step One will be abolished, that CS is going to be graded numerically, or that medical liscensure will be determined in a manner similar to the news team brawl in "Anchorman."

 

[Perrotfish]

There has been some speculation that residency programs will either develop their own tests

I've heard this is going to be the solution, yeah. The Steps were only designed as a minimum standard, not to differentiate students who got a 200 vs a 220 vs. a 240. There's just no data showing that these kinds of incremental improvements in step score actually correlate with an improved quality of resident. The idea is that residency programs will now develop their own screening exams that they can prove actually correlate to your performance in their field.

But I'm still stuck with the old system.

 

[physics junkie]

On the other hand, there are some people who aren't good test-takers, but make incredible doctors. There's something to be said for placing so much emphasis on third-year grades, subjective as they may be.

What is being a "bad test taker"? Does it mean that when asked to demonstrate your knowledge you are unable to do so? Its a bizarre excuse to hide behind...

 

[ChimpanzeeMinky]

What is being a "bad test taker"? Does it mean that when asked to demonstrate your knowledge you are unable to do so? Its a bizarre excuse to hide behind...

I always wondered the same thing when people make that claim. And it's not like you're going to avoid taking tests throughout your career. Maintaining your license and board certification is all about test-taking.

 

[twostep376]

this report (http://advan.physiology.org/cgi/content/full/32/2/109) says that

"If the governing bodies of the USMLE program approve major changes in the program, the first components of a new examination program would probably become available in 2012 or 2013, for students in the LCME graduating classes of 2013 or 2014."

SO...LCME graduating class of 2013 or 2014 is the same as regular old class of 2013 or 2014....right? That would be people who started this past august or the people who start next august...yah?

 

[ChimpanzeeMinky]

this report (http://advan.physiology.org/cgi/content/full/32/2/109) says that

"If the governing bodies of the USMLE program approve major changes in the program, the first components of a new examination program would probably become available in 2012 or 2013, for students in the LCME graduating classes of 2013 or 2014."

SO...LCME graduating class of 2013 or 2014 is the same as regular old class of 2013 or 2014....right? That would be people who started this past august or the people who start next august...yah?

Written in 2008. "If" they approve major changes. Looks like they missed the boat already for 2013 and probably 2014. Fact is, we don't know what's going to happen.

 

[lord_jeebus]

There's just no data showing that these kinds of incremental improvements in step score actually correlate with an improved quality of resident. The idea is that residency programs will now develop their own screening exams that they can prove actually correlate to your performance in their field.

I know of at least one residency program that has done rather extensive analysis of their residents over the years and found that Step I and Step II were both strong predictors of certain desirable traits and performance measures in residency. I think you are right that there is no publicly available data.

 

[Gabby]

What is being a "bad test taker"? Does it mean that when asked to demonstrate your knowledge you are unable to do so? Its a bizarre excuse to hide behind...

I don't think so. Some people are great at the oral stuff, but choke on tests. And when I say choke, I don't mean that they fail. I just mean they don't do as well. For example, I have a classmate who can dance circles around me when it comes to explaining topics and remembering minute details. We quiz each other and she constantly kicks my ass with her level of knowledge. She's a very hard worker (attends all lectures and studies for several hours at home in the evenings). She even tutors some of us on things like pathology, giving us ways to remember things, mnemonics for anatomy, etc. For whatever reason, she can't break a high-70s score on any test even though many of the people she tutors score much higher. Then when we do clinical skills type stuff, she's at the very top of the class with scores in the upper 90s. I think she's one of those people who will be a star on rotations, but will do average on the Steps.

 

[Gabby]

this report (http://advan.physiology.org/cgi/content/full/32/2/109) says that

"If the governing bodies of the USMLE program approve major changes in the program, the first components of a new examination program would probably become available in 2012 or 2013, for students in the LCME graduating classes of 2013 or 2014."

SO...LCME graduating class of 2013 or 2014 is the same as regular old class of 2013 or 2014....right? That would be people who started this past august or the people who start next august...yah?

The Class of 2013 will be taking Step 1 in 2011. If this new exam won't be available until 2012, when are they planning to tell us? LOL

 

[Perrotfish]

What is being a "bad test taker"? Does it mean that when asked to demonstrate your knowledge you are unable to do so? Its a bizarre excuse to hide behind...

There is a LOT of information presented in multiple choice tests, and only a small fraction of it is in the actual question. Good test takers are people who are good at things like POE, looking for trends in the answers, hunting for answers in the test's other questions, and even understanding the psychology of the teacher who wrote it. Bad test takes are people who approach the question as though it were a stand alone short answer question: if they can't answer the question on their own they basically have to guess. I've tutored people for various kinds of tests, and how to take multiple choice tests intelligently is one of the hardest things to teach.

Eaxmple: Which of the following does X

A) 1 &2

B) 2 & 3

C) 1 & 3

D) 3 & 4

E) 1 & 6

Bad test taker: I have no idea what does x (treating it like a fill in the blank question), guess

Good test taker: Well, the question used option 1 several times and option 3 several times, so odds are those are the ones I'm looking for. The correct answer is C.

See how the good test taker gets the right answer even though he has NO IDEA about what's going on in the question?

Now I'm assuming that when the people you're talking about say they're 'bad at tests' they're in the majority that really mean that they're bad at multiple choice scantron tests. If they also mean they're bad at rounds, practicals, lab reports, and every other kind of examination then yes that's a problem.

 

[chessknt]

I know of at least one residency program that has done rather extensive analysis of their residents over the years and found that Step I and Step II were both strong predictors of certain desirable traits and performance measures in residency. I think you are right that there is no publicly available data.

Really? That would be a really interesting report to read as well as their guesses as to why that is.

 

[unsung]

I don't think so. Some people are great at the oral stuff, but choke on tests. And when I say choke, I don't mean that they fail. I just mean they don't do as well. For example, I have a classmate who can dance circles around me when it comes to explaining topics and remembering minute details. We quiz each other and she constantly kicks my ass with her level of knowledge. She's a very hard worker (attends all lectures and studies for several hours at home in the evenings). She even tutors some of us on things like pathology, giving us ways to remember things, mnemonics for anatomy, etc. For whatever reason, she can't break a high-70s score on any test even though many of the people she tutors score much higher. Then when we do clinical skills type stuff, she's at the very top of the class with scores in the upper 90s. I think she's one of those people who will be a star on rotations, but will do average on the Steps.

People like this baffle me. What it sounds like is she has memorized certain clinically useful details, but perhaps lacks an overall level of conceptual understanding. I may be 100% off. But if what you describe is true, she really needs to re-evaluate how she's studying! Putting that much effort to be so consistently not rewarded has to be frustrating and demoralizing...

As a prof once told me, if you can verbalize it, you can write it down during a test. It's the same thing. I don't see how someone could verbally "know" an answer and not know it when it's addressed in writing. Also, why is she tutoring people who are scoring better than her? If I were her and I found out they did better than me, I'd be begging them to share their thought process and trying to learn how to do better from them.

Lol... not trying to be harsh, but ahhhh it just doesn't make sense.

 

[Gabby]

People like this baffle me. What it sounds like is she has memorized certain clinically useful details, but perhaps lacks an overall level of conceptual understanding. I may be 100% off. But if what you describe is true, she really needs to re-evaluate how she's studying! Putting that much effort to be so consistently not rewarded has to be frustrating and demoralizing...

As a prof once told me, if you can verbalize it, you can write it down during a test. It's the same thing. I don't see how someone could verbally "know" an answer and not know it when it's addressed in writing. Also, why is she tutoring people who are scoring better than her? If I were her and I found out they did better than me, I'd be begging them to share their thought process and trying to learn how to do better from them.

Lol... not trying to be harsh, but ahhhh it just doesn't make sense.

That's just it -- she can explain all the concepts just fine and if people are confused, they ask her for clarification. But when it comes to tests (multiple choice tests), she gets test anxiety or something. I'm not sure why she can't pick out the right answers. We had a clinical exam the other day where it was short answer and she got a 91%. Now, if you gave her a multiple choice exam on those same questions, I bet she'd get a 75%. I know it's frustrating for her. She's also one of those high GPA, low(ish) MCAT people. I used to think the "bad test-taker" thing was an excuse, but with her, it's absolutely true.

 

[Dral]

On the other hand, there are some people who aren't good test-takers, but make incredible doctors. There's something to be said for placing so much emphasis on third-year grades, subjective as they may be.

At my school, we all pretty much got similar rotation scores, and the shelf exam was what determined our final grade for each rotation most of the time...and well, you know shelf exams. Time is even more of the essence than most class exams, so if someone's going to crash and burn on any exam other than steps, it's going to be shelf exams.

So that idea doesn't hold across all schools.

 

[Lokhtar]

Another thing that would be interesting is finding out how then the residency directors would judge international students. USMLE seems like a great equalizer, so people from different schools and backgrounds can show scientific knowledge. Whether you're from a top MD, state school, DO, Caribbean or international, you take the same test and you're on at least somewhat of a level playing field.

 

[SouthernSurgeon]

I know of at least one residency program that has done rather extensive analysis of their residents over the years and found that Step I and Step II were both strong predictors of certain desirable traits and performance measures in residency. I think you are right that there is no publicly available data.

I would think any program that has this type of analysis would publish it, as it would be very big news in the med ed field. The existing data suggests very little predictive capacity for any of the current metrics.

 

[MSSM2013]

Another thing that would be interesting is finding out how then the residency directors would judge international students. USMLE seems like a great equalizer, so people from different schools and backgrounds can show scientific knowledge. Whether you're from a top MD, state school, DO, Caribbean or international, you take the same test and you're on at least somewhat of a level playing field.

actually, it's not a good comparison for USMD vs. caribean/international because they may spend years and years study for this exam.

 

[surag]

That's just it -- she can explain all the concepts just fine and if people are confused, they ask her for clarification. But when it comes to tests (multiple choice tests), she gets test anxiety or something. I'm not sure why she can't pick out the right answers. We had a clinical exam the other day where it was short answer and she got a 91%. Now, if you gave her a multiple choice exam on those same questions, I bet she'd get a 75%. I know it's frustrating for her. She's also one of those high GPA, low(ish) MCAT people. I used to think the "bad test-taker" thing was an excuse, but with her, it's absolutely true.

I dont buy into this one bit....and I do believe that the concept of 'bad test taking' is an excuse. let me explain...

test taking requires a different mental preparation than information studying. There are several purposes of a test-(im not a med student yet but I have taken the MCATs...)- in many cases for such standardized tests(and this pertains to STEPS though Im not sure about her schoole exams)- in many cases, the purpose is severalfold-

1. to ensure that the distribution of students is such that only a small percentage can succeed in the exam off the charts, i.e. make sure the exam is hard enough that its a valid examination. basically the test is made so that you make mistakes.
2. to test material(obvious)
3. as a way to compare students on several levels not just knowledge based. I dont agree that the MCAT and SATs are true IQ examinations, but they are passage dependant and require you to APPLY what you have learned in some manner(while the effectiveness of addressing critical thinking through this method may be controversial you cannot deny that these exams are NOT straight up knowledge tests)

As a result, students who have command of INFORMATION, may not have a command of APPLICATION. Thats different from concepts. you're friend may be a master of concepts but applying them in unique situations may be a difficulty for her. reading through a passage and then finding out ONLY the relevant information and ignoring the rest is a form of reading comprehension/application. this sort of thing is usually NOT covered by ur information studying. that could be only one of the issues at hand here.

In any case, my point is that saying one is 'not a good test taker' is a trivialization of the test taking process. It implies somehow that this individual has not done as well only because of some un important aspects that the test seems to examine and the person would be a good doctor/student otherwise.


On that note, just because there is a strong correllation between the step 1 and good residents doesnt mean that the step 1 is a good indicator. there are a lot of confounding variable involved(didnt you guys take stats for crying out loud?) maybe...students who study hard do well on step 1s and residency because they are HARD WORKERS not because the step 1 perpares them or measures the same variables necessary for residency success. and thats just one (haphazard and obvious) example i came up with.

 

[womp]

They're trying to do this for dental schools and the dental equivalent of the USMLE. There has been some speculation that residency programs will either develop their own tests or will use some combination of clinical/pre-clinical grades and the GRE.

The GRE? Isn't that like a step backward? The MCAT already kicks the GRE into the dirt.

 

[coldweatherblue]

glad we get to take Step 1 next year. It's gonna be a PITA but any standardized exam is just an opportunity to objectively distinguish yourself.

edit: wrong word lol.

 

[DrJD]

-(im not a med student yet but I have taken the MCATs...)-

I agree with your post, but just wanted to let you know that before you start medical school, it is MCAT, not MCATs... You just sound like an idiot if you tell people you've taken the MCATs...

Pet peeve...

BTW -
I too hope that the USMLE stays the way it is at least for the class of 2013... Seems to me form researching the web, that the earliest class that could be impacted would be the class of 2014...

 

[womp]

If the USMLE is no longer used for residency placements, watch everyone aim for the bare minimum to pass. Matching will also become even more of a crapshoot, ever-dependent on subjective 3rd year grades as more and more schools are P/F.

I thought the whole point of modifying the USMLE was to ensure greater basic science coverage in the later "steps," since right now basic sciences is mostly clustered in Step 1 and program directors are tired of residents forgetting their basic sciences.

 

[womp]

Also if they do make the new Gateway USMLE pass/fail and not in time for matching, what will be the implications of picking different tier medical schools for residency placement? Would it be more advantageous to go to an easier medical school (with lower MCAT/GPA median) and be in the top of the class with AOA, or be in the middle of the pack of a higher tier school? Or would school reputation carry the day (as in law schools)?

Step 1 was the great equalizer.

Who wants to bet that program directors will be asking for MCAT scores if they don't have access to the board scores anymore...

 

[Lokhtar]

actually, it's not a good comparison for USMD vs. caribean/international because they may spend years and years study for this exam.

That's fair enough.

 

[Ableton]

There is a LOT of information presented in multiple choice tests, and only a small fraction of it is in the actual question. Good test takers are people who are good at things like POE, looking for trends in the answers, hunting for answers in the test's other questions, and even understanding the psychology of the teacher who wrote it. Bad test takes are people who approach the question as though it were a stand alone short answer question: if they can't answer the question on their own they basically have to guess. I've tutored people for various kinds of tests, and how to take multiple choice tests intelligently is one of the hardest things to teach.

Eaxmple: Which of the following does X

A) 1 &2

B) 2 & 3

C) 1 & 3

D) 3 & 4

E) 1 & 6

Bad test taker: I have no idea what does x (treating it like a fill in the blank question), guess

Good test taker: Well, the question used option 1 several times and option 3 several times, so odds are those are the ones I'm looking for. The correct answer is C.

See how the good test taker gets the right answer even though he has NO IDEA about what's going on in the question?

Now I'm assuming that when the people you're talking about say they're 'bad at tests' they're in the majority that really mean that they're bad at multiple choice scantron tests. If they also mean they're bad at rounds, practicals, lab reports, and every other kind of examination then yes that's a problem.

Uh yeah, I really don't think that is the difference separating good test takers from bad ones. I'm a very good test taker, and I would never consider using such flawed logic you suggested.

A good test taker can read the question and pull out the pertinent information to answer the question, while also picking up on tricks here and there. A bad test taker, for one reason or another, has a problem getting from the question to the answer. It has absolutely nothing to do with intelligently guessing.

 

[surag]

Uh yeah, I really don't think that is the difference separating good test takers from bad ones. I'm a very good test taker, and I would never consider using such flawed logic you suggested.

A good test taker can read the question and pull out the pertinent information to answer the question, while also picking up on tricks here and there. A bad test taker, for one reason or another, has a problem getting from the question to the answer. It has absolutely nothing to do with intelligently guessing.

I agree, which is basically what I said in my above post. that being said and going off what i said...i dont think its fair to say that bad test takign is a legit excuse...it still tests reading comprehension etc.

 

[womp]

If there were no tests in medical school and everything was based on a subjective score, I bet a sizable portion of students will stop pursuing medicine on that alone 🙄... medical school will be like art class, soft and fluffy.