The ABEM (written) Qualifying Exam Needs To Be Improved

[ChangeABEM]

I started a petition about this in reaction to recently taking the 2023 exam.

---

Members don't see this ad.

 

[RustedFox]

You're not wrong.

 

[GonnaBeADoc2222]

No it doesn't.

It's a standardized test. It's a reasonable examination of the base knowledge required of an ER doc.

If anything, it should have a higher fail rate.

Y'all whine about midlevels and docs from garbage residencies, but then don't want standards for practice.

 

[RustedFox]

Are you high?
Bro. 10 years ago when I took mine, it was the same garbage.
What biomarker is the best for WeirdRheumatic disease.
"A patient has symptoms. What is the second best next step?"

 

[Mount Asclepius]

1) Close the bad residencies.
2) Keep the written exam. Make decently rigorous and community clinically based. That's what most EPs do. No need for obscure diseases you will never see.
3) Get rid of the oral exam. Too out of touch with the real world.
4) Make maintenance of certification even simpler and cheaper. We're residency trained and board certified. Medicine doesn't change that much or that fast. If anything it moves towards the basics when therapies are disproven with meta analysis.
5) Bye, bye private equity. You don't do medicine.

 

[ChangeABEM]

No it doesn't.

It's a standardized test. It's a reasonable examination of the base knowledge required of an ER doc.

If anything, it should have a higher fail rate.

Y'all whine about midlevels and docs from garbage residencies, but then don't want standards for practice.

I am not saying the exam is too difficult. I’m saying it is poor quality.

 

[ChangeABEM]

Are you high?
Bro. 10 years ago when I took mine, it was the same garbage.
What biomarker is the best for WeirdRheumatic disease.
"A patient has symptoms. What is the second best next step?"

I was not even referencing the relevancy of questions like that. (Although agree that it’s debatable whether or not those type of things should be included on our exam.)

These were my issues:
“The 2023 ABEM (written) Qualifying Exam was full of ambiguous question stems and answer choices. Multiple questions were lacking basic information such as omission of the most important vital signs; it did not provide necessary information to make diagnoses or management decisions. Many questions used unusual terminology that seemed to be trying to “trick” the test taker into choosing the wrong answer; it was full of “gotcha questions” that seemed hostile and unfair. The test this year had multiple errors including typos and timeline discrepancies in question stems. The questions with “stimulus” media to view were blurry or otherwise difficult to interpret because of the poor quality.”

 

[RustedFox]

I was not even referencing the relevancy of questions like that. (Although agree that it’s debatable whether or not those type of things should be included on our exam.)

These were my issues:
“The 2023 ABEM (written) Qualifying Exam was full of ambiguous question stems and answer choices. Multiple questions were lacking basic information such as omission of the most important vital signs; it did not provide necessary information to make diagnoses or management decisions. Many questions used unusual terminology that seemed to be trying to “trick” the test taker into choosing the wrong answer; it was full of “gotcha questions” that seemed hostile and unfair. The test this year had multiple errors including typos and timeline discrepancies in question stems. The questions with “stimulus” to view were blurry or otherwise difficult to interpret because of the poor quality.”

I'm agreeing with you, amigo.

 

[JacobMcCandles]

I agree with the sentiment but “hostile” questions is a very millennial phrase.

 

[ChangeABEM]

I'm agreeing with you, amigo.

I saw thanks. I just wanted to clarify it wasn’t just the irrelevant questions.

On another emergency medicine forum there have been multiple complaints (here and here for instance). I think the quality of the exam has been going downhill for several years, and this year it finally went off the rails.

 

[RustedFox]

Oh, Christ... you went to ... Reddit?!
You poor, poor soul.

 

[RustedFox]

1) Close the bad residencies.
2) Keep the written exam. Make decently rigorous and community clinically based. That's what most EPs do. No need for obscure diseases you will never see.
3) Get rid of the oral exam. Too out of touch with the real world.
4) Make maintenance of certification even simpler and cheaper. We're residency trained and board certified. Medicine doesn't change that much or that fast. If anything it moves towards the basics when therapies are disproven with meta analysis.
5) Bye, bye private equity. You don't do medicine.

But all those administrators would lose their jobs!

 

[thegenius]

are there examples of bad questions?

 

[cyanide12345678]

What actually needs to change how there’s some case report/research requirement for renewing my board certification.

It’s ridiculous

 

[ChangeABEM]

are there examples of bad questions?

Not without violating code of conduct. However, as I linked before (here and here) there are many people with the same concerns.

 

[ScepticSkeksis]

They probably have to fail a certain percentage every year. With studying resources becoming better, the only way to do that is to make the questions weirder.

That's how most standardized tests are designed. An organization has to fail X% of test takers to justify their existence. If the test was on real emergency medicine, then the pass rate would be too high. That's why we need questions on bovine encephalopathy.

 

[Brigade4Radiant]

No it doesn't.

It's a standardized test. It's a reasonable examination of the base knowledge required of an ER doc.

If anything, it should have a higher fail rate.

Y'all whine about midlevels and docs from garbage residencies, but then don't want standards for practice.

Why should it have a higher fail rate? The oral is standardized to but I wouldn’t call it a reasonable test

Doctors do medschool and residency we do t have to justify for the NP sake. the exam doesn’t test basic EM otherwise nearly all would pass

 

[GonnaBeADoc2222]

Why should it have a higher fail rate? The oral is standardized to but I wouldn’t call it a reasonable test

Doctors do medschool and residency we do t have to justify for the NP sake. the exam doesn’t test basic EM otherwise nearly all would pass

The oral is extremely reasonable.

It has a defined format which to need to learn ahead of time and follow.

The cases are all bread and butter EM. Very easy.

 

[ScepticSkeksis]

The oral is extremely reasonable.

It has a defined format which to need to learn ahead of time and follow.

The cases are all bread and butter EM. Very easy.

I disagree. Some crazy stuff on it. I remember I had a thyroid storm case. I remember that I knew the medications to give in the corrrect order. The examiner asked me the dosing. I said "refer to the package insert" he said there was no package. I said "I'll call down to pharmacy". He said the phones were down. Seriously crazy to have to remember dosages for non-emergent, rarely-used medications.

 

[Birdstrike]

I am not saying the exam is too difficult. I’m saying it is poor quality.

The written exam should be kept but improved.

The oral exam should be eliminated. An oral exam to “simulate your clinical skills in the real world” can never be of quality great enough to supplant 3 years of training. If a program isn’t of sufficient quality to do that, improve Credentialling of programs, and shut down bad programs. Don't try to claim you at ABEM can judge a new doctor’s skills better in 4 hours, than a team of ABEM certified attendings can over 3 years

The oral exam serves no purpose but to generate revenue for ABEM.

 

[ChangeABEM]

They probably have to fail a certain percentage every year. With studying resources becoming better, the only way to do that is to make the questions weirder.

I think this is the problem. I don’t agree that there should be a guaranteed 10% of candidates that fail each year.

The ABEM written exam should have high quality, difficult, relevant, and vetted questions that test core knowledge. If a candidate meets a certain threshold of correct answers, they should pass. You should walk out of the exam knowing if you passed or failed. If they want to add 25-50 experimental questions for the following year (as part of the vetting process), those should be included in the exam but not considered in score calculations.

I also agree that the oral boards are extraneous. Clinical competence should be ensured prior to graduation from an accredited, high quality residency program.

There is a petition about that here: Remove American Board of Emergency Medicine Oral Exam Requirement

In response to rising opposition to the oral boards, ABEM funded and conducted a study: Early validity and reliability evidence for the American Board of Emergency Medicine Virtual Oral Examination
which concluded that “There was substantial validity evidence and reliability to support ongoing use of the ABEM [oral boards] to make confident and defensible certification decisions.”

Some argue that this study is biased because of conflict of interest.

Few sign the petition (just like no one will sign mine) because once they pass, no one cares anymore. But we should care. This is the specialty we devoted ourselves to and our board certification/recertification process should be widely accepted by its members.

 

[TooMuchResearch]

I disagree. Some crazy stuff on it. I remember I had a thyroid storm case. I remember that I knew the medications to give in the corrrect order. The examiner asked me the dosing. I said "refer to the package insert" he said there was no package. I said "I'll call down to pharmacy". He said the phones were down. Seriously crazy to have to remember dosages for non-emergent, rarely-used medications.

You should absolutely never try to pull doses of critical medications out of your ass.
I'd refer to up to date - what's that, the internet is broken?
Okay, I'd call pharmacy. They're dead?
I'd page endocrine or ICU or someone else. All dead?
I'd refer to an old textbook sitting on the shelf above my desk. I guess this is why we didn't throw them out. I hope nothing has changed with this disease process or treatment since 2002.

 

[TooMuchResearch]

I think this is the problem. I don’t agree that there should be a guaranteed 10% of candidates that fail each year.

The ABEM written exam should have high quality, difficult, relevant, and vetted questions that test core knowledge. If a candidate meets a certain threshold of correct answers, they should pass. You should walk out of the exam knowing if you passed or failed. If they want to add 25-50 experimental questions for the following year (as part of the vetting process), those should be included in the exam but not considered in score calculations.

I also agree that the oral boards are extraneous. Clinical competence should be ensured prior to graduation from an accredited, high quality residency program.

There is a petition about that here: Remove American Board of Emergency Medicine Oral Exam Requirement

In response to rising opposition to the oral boards, ABEM funded and conducted a study: Early validity and reliability evidence for the American Board of Emergency Medicine Virtual Oral Examination
which concluded that “There was substantial validity evidence and reliability to support ongoing use of the ABEM [oral boards] to make confident and defensible certification decisions.”

Some argue that this study is biased because of conflict of interest.

Few sign the petition (just like no one will sign mine) because once they pass, no one cares anymore. But we should care. This is the specialty we devoted ourselves to and our board certification/recertification process should be widely accepted by its members.

I guess I'm not seeing a good reason to make board certification easier as the quality of residency training falls apart.

 

[ChangeABEM]

I guess I'm not seeing a good reason to make board certification easier as the quality of residency training falls apart.

Again, not advocating for easier. Advocating for questions that actually assess core knowledge, understanding, and mastery of the relevant material.

That as opposed to missing questions because they put in EKGs that are too old and degraded to read because the line is so fuzzy, or chest xrays that are terribly over penetrated, etc.
That as opposed to missing a question because yellow fever in the answer choices is referred to as “Bronze John” (not an actual question, but an example of unusual terminology).

That as opposed to knowing the content and missing it because of a poorly written question, bad stimuli media, or confusing answer choices.
Make the questions very hard. I’m into that. But make the questions themselves easy to understand what they are asking.

If you’re saying we still need oral boards to assess clinical competency, I agree with previous posters that it is the duty of residency to teach and evaluate residents during training. No one should graduate residency until they have demonstrated that they are capable. Residency programs that are substandard and not able to properly teach or evaluate their residents should be discontinued.

I'd refer to an old textbook sitting on the shelf above my desk. I guess this is why we didn't throw them out. I hope nothing has changed with this disease process or treatment since 2002.

I like that. Oral board dungeons and dragoned yourself into victory.
“I opened to the correct section and breathe a sigh of relief as I see the dosing on the page and notice that it’s in English and the book isn't damaged at all”

 

[gruemd]

I like that. Oral board dungeons and dragoned yourself into victory.
“I opened to the correct section and breathe a sigh of relief as I see the dosing on the page and notice that it’s in English and the book isn't damaged at all”

"I put on my robe and my wizard hat..."

 

[ScepticSkeksis]

You should absolutely never try to pull doses of critical medications out of your ass.
I'd refer to up to date - what's that, the internet is broken?
Okay, I'd call pharmacy. They're dead?
I'd page endocrine or ICU or someone else. All dead?
I'd refer to an old textbook sitting on the shelf above my desk. I guess this is why we didn't throw them out. I hope nothing has changed with this disease process or treatment since 2002.

It was literally that as you described. I think the examiner was having a bad day and was out to get people. In the age of the pocket supercomputer, there isn't really any need to memorize anything except code medications.

 

[TooMuchResearch]

It was literally that as you described. I think the examiner was having a bad day and was out to get people. In the age of the pocket supercomputer, there isn't really any need to memorize anything except code medications.

1 mg epi. 1 gram calcium. 1 amp bicarb. 1 moment of silence. Literally nothing to remember here either.

 

[ChangeABEM]

From here: The passing rate for all candidates who took the 2023 Qualifying Examination is 86%. The passing rate for all candidates who completed an approved Emergency Medicine residency program and are first-time takers of the examination is 88%.

I think this is pretty similar to previous years. I still think that some that failed probably knew the content and failed because of the poor quality of the exam questions (ambiguous and deceptive, omission of important information, unusual terminology, typos and timeline discrepancies, hard to interpret “stimulus” media, etc).

 

[EctopicFetus]

I saw thanks. I just wanted to clarify it wasn’t just the irrelevant questions.

On another emergency medicine forum there have been multiple complaints (here and here for instance). I think the quality of the exam has been going downhill for several years, and this year it finally went off the rails.

Been going downhill? Honestly, how do you know. It was trash when i took it a decade ago..

 

[EctopicFetus]

Again, not advocating for easier. Advocating for questions that actually assess core knowledge, understanding, and mastery of the relevant material.

That as opposed to missing questions because they put in EKGs that are too old and degraded to read because the line is so fuzzy, or chest xrays that are terribly over penetrated, etc.
That as opposed to missing a question because yellow fever in the answer choices is referred to as “Bronze John” (not an actual question, but an example of unusual terminology).

That as opposed to knowing the content and missing it because of a poorly written question, bad stimuli media, or confusing answer choices.
Make the questions very hard. I’m into that. But make the questions themselves easy to understand what they are asking.

If you’re saying we still need oral boards to assess clinical competency, I agree with previous posters that it is the duty of residency to teach and evaluate residents during training. No one should graduate residency until they have demonstrated that they are capable. Residency programs that are substandard and not able to properly teach or evaluate their residents should be discontinued.


I like that. Oral board dungeons and dragoned yourself into victory.
“I opened to the correct section and breathe a sigh of relief as I see the dosing on the page and notice that it’s in English and the book isn't damaged at all”

If this is so mission critical to you I would advise you Figure out how to get on that group that does this. It’s hard to be empathetic. The SAT sucked, the MCAT USMLE sucked etc etc.

With the more HCA and similarly craptastic residencies coming online make it hard make it weird let people fail.

 

[ChangeABEM]

Been going downhill? Honestly, how do you know. It was trash when i took it a decade ago..

I don’t know for sure. However, the past few years (maybe since pandemic?) there have been increasing complaints about the test every year. It’s possible something else is at play (increasing online presence/interactions/participation in forums of the candidates etc).

This year had more complaints by far than others, which is what prompted the petition and OP.

 

[ChangeABEM]

If this is so mission critical to you I would advise you Figure out how to get on that group that does this. It’s hard to be empathetic. The SAT sucked, the MCAT USMLE sucked etc etc.

With the more HCA and similarly craptastic residencies coming online make it hard make it weird let people fail.

I will try to get on a committee to improve the exam when I am able.

I think people from whatever residency should fail if they don’t know the content/material, not because they guessed wrong on confusing/convoluted questions.

 

[JacobMcCandles]

I don’t know for sure. However, the past few years (maybe since pandemic?) there have been increasing complaints about the test every year. It’s possible something else is at play (increasing online presence/interactions/participation in forums of the candidates etc).

This year had more complaints by far than others, which is what prompted the petition and OP.

The general public has been complaining more since the pandemic.

 

[ChangeABEM]

The general public has been complaining more since the pandemic.

Fair enough, but I am sure you have also noticed there has been a lot of restructuring in almost every entity since the pandemic that has often made things worse.

I can tell you this year’s exam from my perspective was shockingly bad (not hard, bad). It was the worst quality standardized test I have ever encountered. It was unprofessional.
Maybe it’s always been like this, maybe it’s a new problem (possibly related to pandemic restructuring). Either way it shouldn’t be like that.

 

[JacobMcCandles]

Fair enough, but I am sure you have also noticed there has been a lot of restructuring in almost every entity since the pandemic that has often made things worse.

I can tell you this year’s exam from my perspective was shockingly bad (not hard, bad). It was the worst quality standardized test I have ever encountered. It was unprofessional.
Maybe it’s always been like this, maybe it’s a new problem (possibly related to pandemic restructuring). Either way it shouldn’t be like that.

I think you're going to have to have quite the pile of evidence to prove that the test is unprofessional and hostile. Outdated? Yes. Unprofessional and hostile? Those are big (and difficult to prove) accusations.

 

[ChangeABEM]

I think you're going to have to have quite the pile of evidence to prove that the test is unprofessional and hostile. Outdated? Yes. Unprofessional and hostile? Those are big (and difficult to prove) accusations.

Other than getting signatures from other people that agree that the test needs to be improved, I don’t know how to go about obtaining evidence to support my assertions.

I’d be happy to provide examples if it was allowed, but I believe discussing specific questions is against code of conduct.

 

[EctopicFetus]

I will try to get on a committee to improve the exam when I am able.

I think people from whatever residency should fail if they don’t know the content/material, not because they guessed wrong on confusing/convoluted questions.

I’m not mad at you. Simply saying that it’s always been that way. On my exam I had some straightforward but bizarre questions.

Like what nerve innervates (insert whatever muscle). I’m like is this anatomy and why does this matter. I guess in my eyes I took it, I passed and moved on. If this is your thing then by all means. There is a ton of room for improvement. Seemingly they are using the ABEM money to do cool trips and not make a better test.

 

[NicksMD]

Scores are up for everyone who took the exam.

 

[ChangeABEM]

Scores are up for everyone who took the exam.

Passed, but still think the exam needs to be improved.

 

[TooMuchResearch]

I’m not mad at you. Simply saying that it’s always been that way. On my exam I had some straightforward but bizarre questions.

Like what nerve innervates (insert whatever muscle). I’m like is this anatomy and why does this matter. I guess in my eyes I took it, I passed and moved on. If this is your thing then by all means. There is a ton of room for improvement. Seemingly they are using the ABEM money to do cool trips and not make a better test.

They used outdated (i.e. no longer generally clinically used) terms and had images that were so comically bad you knew they probably weren't necessary to figure out the answer. Not my first standardized test rodeo. That's why we practice.

 

[TooMuchResearch]

Edit: whatever year I took it

 

[southerndoc]

A friend took it and failed by one point. Feel sorry for him because his raw score was >82% correct. The fail rate was unusually high for this year when compared to others. Isn't the traditional pass rate around 96%?

 

[hopefulgasman]

A friend took it and failed by one point. Feel sorry for him because his raw score was >82% correct. The fail rate was unusually high for this year when compared to others. Isn't the traditional pass rate around 96%?

This year was the lowest in at least 20 years. Highest pass rate was 95% in 2018.

I thought the exam as a whole was fine. Like any USMLE question, they’re not going to give you every piece of information presented in a classic, perfect world. This is a non-issue if you know the information cold.

 

[emergentmd]

This is just another typical barrier to show they ABEM has some standards, give ABEM a purpose, Give ABEM funds.

You are not going to change anything. Even if you get things changed, people will still complain that it stinks. Learn to game the system, study to pass the test because its actually easy to pass. How do I know? I am strong clinician but terrible at knowing how to choose the right answers. I still passed all my boards and recert without ever failing.

I mean, you walk in the room and if the question is what do you do next then I will give you 5 reasonable next steps. Dumb question but just another hurdle we need to jump over.

 

[Bill Brasky]

DOs taking the test will notice they've seen a test like this before: The COMLEX. When I was sitting for the ABEM I immediately recognized the horrible, vague question writing and many, many questions that you can absolutely argue more than one answer is right.

 

[JacobMcCandles]

DOs taking the test will notice they've seen a test like this before: The COMLEX. When I was sitting for the ABEM I immediately recognized the horrible, vague question writing and many, many questions that you can absolutely argue more than one answer is right.

It’s why you pick the answer that is MOST correct…