Physics & Radbio

[Tony&lisa]

---

Members don't see this ad.

My latest email to Dr. Wallner after he wrore back basically saying I am sorry you feel like you were treated unfairly but you most certainly were not, and we are working on making improvements in the study material for next year.

Dr. Wallner,
Again, I appreciate your response and the time you are taking to answer our concerns. However, the answers that I, and those that many of my co-residents, have received, can not be the end of the discussion. I feel like I was treated unfairly and I know for certain that many others feel the same way. There is A LOT of frustration going around among PGY 5s as we speak.
First of all, we don't know what the pass rates were for either exam. It is all speculation, because the ABR won't tell us. We need this transparency to better know if our grievances are warranted or not. The overwhelming feeling is that they are, but we have no data to work with. The longer we go without those numbers, the greater the speculation grows.
Secondly, I am not at all surprised, as you told me, that the majority of test takers passed. Of course. It seems, however, that a much larger minority failed, significantly larger than in previous years. THIS is the concern. We are a field of intelligent people. There is no reason why 20%, 30%, who knows what % should be failing this exam. The only reason why that would suddenly happen one year out of nowhere is that suddenly, out of nowhere, the exam changed.
I appreciate your efforts and those of the stakeholders, to work on improving the study material for next year. But the question I specifically raised in the last email was not answered: what is being done for the people THIS year. Next year is next year. If there are broad efforts being made to improve study material for next year, that clearly says that there is a recognition that something was off this year. My number one concern, and that of my colleagues, is what is the ABR doing for the people who took it THIS year. You said we were not treated unfairly, but the overwhelming feeling on our side right now is that we were, and until we are provided any evidence to prove us wrong, we will keep pressing forward with this sentiment and need for an answer.
At the end of everything, we are all on the same team. We are all in the same field and want the same things for our patients. I am asking the ABR that we work together on this, right now, because what we now have is an entire generation of soon-to-be graduating residents who are very upset with the ABR.
Please work with us, the residents of THIS year.
Best

 

[RO28]

My latest email to Dr. Wallner after he wrore back basically saying I am sorry you feel like you were treated unfairly but you most certainly were not, and we are working on making improvements in the study material for next year.

Dr. Wallner,
Again, I appreciate your response and the time you are taking to answer our concerns. However, the answers that I, and those that many of my co-residents, have received, can not be the end of the discussion. I feel like I was treated unfairly and I know for certain that many others feel the same way. There is A LOT of frustration going around among PGY 5s as we speak.
First of all, we don't know what the pass rates were for either exam. It is all speculation, because the ABR won't tell us. We need this transparency to better know if our grievances are warranted or not. The overwhelming feeling is that they are, but we have no data to work with. The longer we go without those numbers, the greater the speculation grows.
Secondly, I am not at all surprised, as you told me, that the majority of test takers passed. Of course. It seems, however, that a much larger minority failed, significantly larger than in previous years. THIS is the concern. We are a field of intelligent people. There is no reason why 20%, 30%, who knows what % should be failing this exam. The only reason why that would suddenly happen one year out of nowhere is that suddenly, out of nowhere, the exam changed.
I appreciate your efforts and those of the stakeholders, to work on improving the study material for next year. But the question I specifically raised in the last email was not answered: what is being done for the people THIS year. Next year is next year. If there are broad efforts being made to improve study material for next year, that clearly says that there is a recognition that something was off this year. My number one concern, and that of my colleagues, is what is the ABR doing for the people who took it THIS year. You said we were not treated unfairly, but the overwhelming feeling on our side right now is that we were, and until we are provided any evidence to prove us wrong, we will keep pressing forward with this sentiment and need for an answer.
At the end of everything, we are all on the same team. We are all in the same field and want the same things for our patients. I am asking the ABR that we work together on this, right now, because what we now have is an entire generation of soon-to-be graduating residents who are very upset with the ABR.
Please work with us, the residents of THIS year.
Best

I send another email today with a similar sentiment. Something just isn’t adding up for me. There is no reason the fail rate should be what it is. I’m also asking for my raw score.

 

[KHE88]

I think it's also worth mentioning to those reading this who are not PGY-5 residents that while this little online discussion is raging, it is really the tip of the iceberg. There is a lot of offline commotion. Publicity helps. Email ARRO, and post and share your thoughts if you took this year's exam, even if you agree with the ABR apologists.

 

[Corgi4Prez]

I am sorry to everyone on here that is dealing with this issue. Full disclosure, I took rad bio/physics last year and clinical this year and I passed all 3 (luckily). I thought all were poor tests with both weird questions about expected content and also surprising content. There is no question these tests are more disconnected from prep materials than the MCAT or the Steps. l had a great physics course and poor rad bio course. I studied by drilling many, many questions. Raphex (~8 years worth) and ASTRO rad bio study guides and ROQ physics/rad bio. It sounds like many/all of you did this too and it did not work, so my advice is probably not helpful. That really sucks and I am sorry you have to deal with it.

I think you all could benefit a lot by focusing on what can actually be changed. My humble opinion is that the primary issue is the disconnect between the guidance from ABR/study guides and the test. On this thread even a rad bio teacher is in the dark. That is honestly absurd and my feeling is that this has been a problem for a while. If we set aside the assumption that the ABR is tanking residents for cheap labor, everyone is on the same team. ADROP wants you to pass, their program looks bad if you dont. ABR has easier ways to make money than tanking residents on purpose to charge for retake exams. It is a safe assumption that the goal is to make a test that ensures quality of training and have people pass.

If you read the exchange in PRO between Amdur/Lee and Wallner/Kachnic again and ignore the weird sentence about resident quality, there is a lot there. In addition to saying the test is harder, Amdur and Lee argue that the curriculum has "narrowed" and people are teaching to the test at the (implied) expense of teaching/testing the less objective ACGME competencies that make good physicians. IMO the response by Wallner/Kachnic misses this very important point, but they say other things about their test. Additionally (importantly), they explicitly defend their method and explicitly note that they do not set specific pass targets any year. If you take that exchange in context with Wallner's prior article about a desire for more biology focus in Radiation Oncology, it is not surprising to see a slow creep of terrible/irrelevant/etc questions leading to the situation we have now.

Again, opinion here, but the ABR wont view this as an anomaly with their test. There is no pass rate target, so the test is not abnormal based on pass rate alone. At the ACR conference this year, the ABR gave a presentation about an IT problem with their breast fellowship exam and the steps they took to address it for examinees. They offered a retake that was done outside of testing centers and was quite successful. When they view their test as a failure (technical in this case), they do work to remedy the situation. They also presented some thoughts on the future of MOC (caveat it was for Radiology), but it seemed like a genuine effort to make MOC educational and not terribly invasive in our lives. I am not arguing about whether MOC is worth while or not, and of course the ABR is going to think MOC is worth while. This is outside the scope of this present issue of a low pass rate on the current exams.

Another opinion, the ABR will not take kindly to saying the test is irrelevant. You cannot tell a bunch of radiation biology question writers that you dont need to know radiation biology to be a good clinician because they wont listen. Whether that is true or not is a matter of opinion and reasonable people can disagree. Clearly at least a task force convened by our society feels rad bio is important. Its probably not going away.

So (another opinion), you might do the most good by forcing an open discussion of this test, its content, and a revision of the prep materials/study guide. Again, the teachers are in the dark and the study guide is embarrassingly superficial. The lack of prep materials compared to all our other tests is really poor. These are all objective issues that can easily be remedied and would likely solve this problem. Everyone is smart in this field. If people are told what to study and they study they will pass.

ASTRO/ABR/ADROP/ARRO are not good at having discussions about contentious issues, we can all agree there. Force it through ARRO, ADROP, and by emailing the ABR. There are plenty of venues outside of academic journals if people dont want to do it in writing. Final opinion, this would be way more productive than hate mailing Wallner/ABR, writing the NYT, and fighting about recalls or not.

Everyone that failed, I hope you pass next year and put it behind you. And again, sorry you have to deal with this.

 

[teiresius]

Very interesting to see the people taking the side of the ABR and status quo. Some people who have not even posted in 10 years are even coming out. i guess the ABR is sending out their sycophant army and trolls.

So, because I disagree with you, I am now a sycophant ABR troll? What a bunch of horsesh*t. Get over your persecution complex.

In actuality, I am simply a PGY-5 at an upper-mid-tier program who worked hard to pass the same test we all took. I had no recalls to use, I also read (about half of) a steaming pile of Hall, and had a fairly well-meaning but ineffective Radbio teacher who was largely absent from didactics. I made a lot of flashcards, and did all the questions I could from the past few years and RO questions. The ONLY resource I can think of that I used that might not be known to all was a Nature reviews cancer on cell death pathways, and a (probably ten year old) gyn-onc book that had a wonderful summary of classical rad bio. It had a green cover.

While I don't disagree that the test was crummy, random, oddly specific about stuff I hadn't thought about since undergrad, and way out of left field, I don't know what to say about that. I also felt prepared, and walked out feeling like I passed... Complain all you want, but sometimes life isnt fair, and I'm not sure yelling into the void will fix it. You can't change the world, you can only change yourself and your reactions to it.

To whit: I strongly disagree that the test is some nefarious plot by the senior members of the specialty to keep their boots on our throats and I think that some of the rhetoric on this thread makes us all look ridiculous by association. It does a disservice to legitimate criticisms. I agree with many here that more transparency is a good thing, a better test with more vetting and more clinical relevance is a good thing, and agree that major changes to the examination should be accompanied by revised study materials and guidance. I think that everyone SHOULD pass, but acknowledge that not everyone will. There is truth to much of what has been said above, but I can't agree with the arguments that everyone should get a trophy simply for showing up. I came out of hibernation to call out the momentum to write an expose about the "conspiracy" to the Gray Lady, lest my silence be construed as agreement with a vocal minority. FWIW: I feel like arguments that your time is too valuable to bother with testing, or that keeping up with cutting edge cancer biology and targeted agents and immunotherapy is somehow irrelevant crap to AN ONCOLOGIST is unlikely to engender much support - even from those of us that understand the truth and frustrations being expressed.

I am an also an unapologetic advocate of the status quo: I believe our specialty is amazing, that our physicians, mentors and colleagues are some of the most talented, intelligent and capable in all of medicine, and I am working hard to join their ranks as an equal. In all honesty, I'm sure you are all also working incredibly hard to be the best you can be, and that is why I certainly commiserate with anyone that failed this year. I really hope that after the rage subsides, that everyone will do whatever it takes to pass on the next go-round, and we will all meet up for bacchanalian festivities in Valhalla (the open bar at the space-OAR sky-lounge). in the words of Walter: "nothing is f*cked here, dude)

With that, I'm signing off to go finish contouring a CNS case in someone who really has the right to be pissed off about the unfairness of life. See you in another ten years perhaps.

- ABR Sycophant

 

[KHE88]

If you read the exchange in PRO between Amdur/Lee and Wallner/Kachnic again and ignore the weird sentence about resident quality, there is a lot there. In addition to saying the test is harder, Amdur and Lee argue that the curriculum has "narrowed" and people are teaching to the test at the (implied) expense of teaching/testing the less objective ACGME competencies that make good physicians. IMO the response by Wallner/Kachnic misses this very important point, but they say other things about their test. Additionally (importantly), they explicitly defend their method and explicitly note that they do not set specific pass targets any year. If you take that exchange in context with Wallner's prior article about a desire for more biology focus in Radiation Oncology, it is not surprising to see a slow creep of terrible/irrelevant/etc questions leading to the situation we have now.

Bingo. Although I would argue that the slow creep of terrible/irrelevant questions went exponential this year. Perhaps some of us saw it coming. I didn't and just studied the way my wise elders told me to. It may be correct to expand the focus of the exam but the way they did it suddenly and secretly and to such a level of detail way beyond what any reasonable person would consider minimal knowledge for competence is the crux of the issue. The other problem is that the ABR is talking out of both sides of its mouth and saying on one hand "we need to change the exam" and on the other "the exam didn't change." While not a technical issue as bad as the computer presenting us with the CPA exam or something, it wasn't the exam we prepared for, and worthy to consider a resolution other than having to retake it the day before the clinical exam next year.

While I don't disagree that the test was crummy, random, oddly specific about stuff I hadn't thought about since undergrad, and way out of left field, I don't know what to say about that. I also felt prepared, and walked out feeling like I passed... Complain all you want, but sometimes life isnt fair, and I'm not sure yelling into the void will fix it. You can't change the world, you can only change yourself and your reactions to it.

I am glad that you also feel the test was out of left field. However, again, here is somebody who had a background in something other than classical radiation biology that felt this helped with the exam. I would point out that there are people who also walked out feeling like they passed who did not. I don't know the margin you passed the exam by (and you don't either), and I would ask that you try to imagine yourself in the situation, right now, where you scored second quartiles and failed. Because there are a lot of people who studied just as hard, if not harder, than you and are in that situation due to a couple of unlucky guesses. Do you think you would be a little more empathetic to those who are mad at our ABR overlords for changing the exam content without any warning on one of the most high-stakes career-impacting exams we will ever take?

 

[thecarbonionangle]

PGY-5 at an upper-mid-tier program who worked hard to pass the same test we all took.

but I can't agree with the arguments that everyone should get a trophy simply for showing up. I came out of hibernation to call out the momentum to write an expose about the "conspiracy" to the Gray Lady, lest my silence be construed as agreement with a vocal minority.

I am an also an unapologetic advocate of the status quo

With that, I'm signing off to go finish contouring a CNS case in someone who really has the right to be pissed off about the unfairness of life. See you in another ten years perhaps.

- ABR Sycophant

How nice of you to grace us with your presence, ye upper mid-tier program, hard working, reasonable, self-described unapologetic advocate of the status quo to lecture people on not expecting that “everyone gets a trophy”. You took time off from drawing circles to sprinkle on us some of your wise pixie dust. I hope the readers will be able to work as hard as you one day.

 

[smq123]

That all said, if this was the second time I was taking this exam, I'd sue. And if you don't think people would agree with you, you're wrong.

People are afraid of making the field look bad. Give me a break. Look, I don't like having to have this conversation either, but it has to be had to fix the problems in this field.

Again, keep writing to whoever. For me it's going to be [email protected].

Clearly, I am not a radiation oncologist, but my husband is, so sometimes I read this forum out of curiosity.

I'm genuinely sorry for anyone who failed the exam. I think that any physician can empathize with the frustration of such a professional setback.

That being said, while I understand being angry and frustrated, I would urge you to consider your response to this carefully. For example, why sue the ABR? What would you hope to accomplish? That is several years and thousands of dollars for....what? They'll argue that it doesn't prevent you from becoming a practicing radiation oncologist (which is true, as you said yourself). And I would honestly hesitate to hire someone who is suing the ABR over $1000 and a failed exam. To me, that indicates a lack of foresight that is concerning and would make me seriously question if I want to work with you as a colleague. I also don't imagine that I would be the only one to think so - as others have pointed out, the people who are in charge of hiring you are likely to a) embrace the status quo, and b) argue that they themselves took this exam without much difficulty.

I would also encourage you to think twice about writing to the New York Times. You are far overestimating their readership if you think that their readers will care. As others have said, the New York Times readers are NOT pro-physician.

On a personal level, I would really urge you to approach this very cautiously. A few years ago, the NY Times wrote an article, after the DEA announced that they were going to institute laws prohibiting physicians from writing for more than 3 days worth of opiates for non-oncological pain. I wrote a comment (just a comment! Not even an article!) saying that I supported this. As a family medicine doctor, I agree that most, if not all, primary care offices are poorly equipped to deal with chronic pain patients. We don't have the security or the training. In addition, as we all know, patients who are dependent on opiates can get very angry if they are denied their narcotics for any reason. I mentioned in my comment that angry patients will end up taking out their rage on young medical assistants - I wrote about one incident where a patient "lost" their prescription for Percocet. When my 24 year old MA (who made $14 an hour) told him that the office could not replace lost prescriptions, he screamed at her in the waiting room, and told her that she deserved to be raped, "for being an unfeeling bit**."

The vitriol that I got in response to my comment was horrendous. People told me that a doctor who makes as much as I do should be able to "suck it up" and learn to have some compassion for these poor, suffering patients. They told me that I was a horrible doctor for caring more about my staff than about my patients. They actually supported the patient who screamed at my MA in the waiting room, saying that I should take the time to imagine how much pain he is in to be so rude and angry. The overwhelming sentiment was that doctors are entitled, privileged, and far removed from "real patient concerns." One person wrote that this is why he went to an NP instead of a physician, "because nurses actually care about their patients." The only commenter who supported me was...another physician (pain management, actually). But that was one voice in a sea of about 15.

So imagine the response to someone who is in a field where even the bottom 10% of academic radiation oncologists make ~$300K a year, a salary that most Americans cannot even dream about. You will not find the support that you are looking for; I guarantee that most of the responses will be worse than what you have read here. "You should have studied harder; how will you expect to give good quality care if you can't even pass this exam?" "Other people last year passed it just fine, why can't you?" "OF COURSE we need stringent board exams! How else are you going to weed out bad quality doctors?" etc. Most people do not understand the training that I go through; they will understand an esoteric field like radiation oncology even less. And I can assure you that they will have little sympathy.

I'm sorry that you failed and I hope that you pass the exam next year. But I think that you're wasting time trying to fight windmills.

 

[Tony&lisa]

Just to repest my question because I don't know - what email should I use to send a message to ARRO?

 

[Tony&lisa]

Another note of emphasis - from the fact that Dr. Wallner is saying that they are working closely with stakeholders to develop updated study materials and guides for next year, that is a clear admission that something was off this year. There would be no reason to try to fix something moving forward if nothing is broken. When you email him, thank him for what is being done for future years, but our primary concern is what is being done FOR US THIS YEAR.

 

[RO28]

Another note of emphasis - from the fact that Dr. Wallner is saying that they are working closely with stakeholders to develop updated study materials and guides for next year, that is a clear admission that something was off this year. There would be no reason to try to fix something moving forward if nothing is broken. When you email him, thank him for what is being done for future years, but our primary concern is what is being done FOR US THIS YEAR.

 

[Tony&lisa]

Thank u

 

[Phantom1]

I think best option is to push for a mid year retake. Its a computer exam so could be rescheduled. Also this would not necessarily admit any blame on ABR part, and would help to ammeliorate the situation.

 

[mikezab]

I want to make the point that there are many PGY-5's such as myself who failed this years exam and are following this thread but may not be posting regularly. Our main concern is what is FAIR. I do not agree that there is a conspiracy. I don't think we should be emailing the NY Times, because as people have said.. the general public would just be in favor of more stringent regulations on physician certification. But if something was wrong with this years test then we deserve to have it evaluated. If the pass rate is the same as it was in prior years (or even close) then we can/will suck it up. But lets review the facts:

1. The average physics pass rate over the last 3 years (on ABR website) is 92%
2. The average physics pass rate from the 12 years prior: was 88.9% with a std. dev. of 5.8% (Rad bio during this time was 7.2% failure, 3.9% std. dev)
3. What MAY be fact (but yet to be confirmed): RO28 posted yesterday that the physics pass rate was 70% (which he heard 'from the man himself'. We need more clarification if this was indeed from Dr. Wallner. But that would be GREATER THAN 3 standard deviations from the norm

We need to confirm the pass rates from the ABR. This isn't about PGY-5's whining. If the pass rates were in line with prior years then I and many others will shut right up. But if the failure rates were that far off this year it wasn't due to a lack of quality residents, something else was wrong. At a minimum it deserves further review.

 

[Corgi4Prez]

Just to repest my question because I don't know - what email should I use to send a message to ARRO?

[email protected]

 

[KHE88]

After getting them to offically publish the pass rates, which is really indefensible at this point and they know it, the next step is getting them to admit that it is theoretically possible they created a test with content and a cut-rate that was not consistent with prior years. You would think a pass rate 4+ standard deviations below the TWELVE YEAR average would be enough evidence for them to re-evaluate it. They are still stuck with this sophomoric cognitive bias that it is impossible for them to create an unreasonable exam.

Reasonable options:
1. Have another team of "experts" including clinicians re-evaluate this year's test and re-determine the cut score based on what a minimally competent PGY-5 "would know."
2. Let us take an old exam given within the past 5 years. Give it in December. Waive the re-take fee.

They admitted that study resources are needed. This is tantamount to admitting the exam was not fair. They gave us an exam without study resources that the class of 2020 will have. The class of 2018 and below all had an ASTRO guide that year. We did not. How is that a level playing field?

Unreasonable options:
1. Make everyone restudy for physics and/or bio and then take the clinical written the next day. Expect the clinical written pass rate to drop a lot next year in this case. Of course, that's just due to declining resident quality with the idiots that make up the class of 2019.

 

[DebtRising]

This troll account is one of the leader conspiracy theorists about the job market here , supposed to be a gal who hates her dating-less life in the middle of nowhere while feeling helpless about her small town and her know nothing colleagues at tumour board, now wants to lecture people on being reasonable. Funny stuff.

A conspiracy theorist basically makes tings up in absence of data. At this point there has been wild speculation about people and program behavior, abolishing tests since they just waste time, etc, and the pass rate still isn’t known. It should be known, no hesitation, but it is not.

While I have made many posts about the job market, my constant emphasis has been that it is based on facts (supply mis match is published, expansion is documented, fellowship expansion documented, workforce survey presented at national meeting). There is a difference. My passion comes from the fact that data is available and it is being ignored.

That the only person who liked your personal attack was the poster who was annoyed I called Wallner out for his hypocrisy in another thread and then was forced to admit his mistake when I posted the paper and quote to prove so is telling.

Hope it all works out for everyone, in this small town we only get internet 2 hours per day and time is up. Won’t clutter this thread any further.

 

[SexyPhotonBabe86]

Instead of waiting for the ABR to release the pass rate (which they still need to do), all the PGY-5s (and repeat takers) can post their status on an anonymous google doc.

 

[KHE88]

Google doc is pointless. No way to verify entries.
Rumor pass rate is 70% physics, 67% rad bio. We can sit here and say the ABR swore on a bible when they told it to us, but it doesn't matter unless it's coming from them. We all know these numbers are more likely than not accurate and that there is no way in hell that 92% of people passed rad bio this year.

DebtRising, data is available and is being ignored in this situation as well (pass rate). Rad onc has a lot of problems, including overtraining and about 2 dozen other raging debates. Board exams are a huge problem as well. I'm annoyed you think your fight is the only one that matters and only you have the moral authority to tell the rest of us where to direct our outrage. I think any med students looking at the mess that rad onc forum has become over the past 5 years and thinking "hey, i really want to be a part of this" need their heads examined.

 

[Corgi4Prez]

@KHE88

The ABR have not been responsible for study resources for these exams, and whether they are available has nothing to do with whether a test is "fair". ASTRO publishes the study guide. There has been one published every year. It has a ton of authors. Why don't you email them and discuss how it could be improved? Some might even be at your institutions.

 

[KHE88]

Corgi4Prez: The ABR is responsible for providing candidates with expected content on the exam. Their "study guide" is a joke and might as well just say "Included are questions on the following topics: Biology."

I do not know why ASTRO did not release a study guide this year for the first time in over a decade. It is suspicious. I suspect we will have better guides next year to help us deal with the exam content changes. I have already reached out to instructors at my program, who were very interested to hear about these exam changes, so I think word is going around the radbio community now.

I really wish I could post some examples of questions I saw on this year's exam so you all could have a better understanding of just how ridiculous they were. But of course I can't as that's all part of the racket. We were used as guinea pigs.

 

[RO28]

I want to make the point that there are many PGY-5's such as myself who failed this years exam and are following this thread but may not be posting regularly. Our main concern is what is FAIR. I do not agree that there is a conspiracy. I don't think we should be emailing the NY Times, because as people have said.. the general public would just be in favor of more stringent regulations on physician certification. But if something was wrong with this years test then we deserve to have it evaluated. If the pass rate is the same as it was in prior years (or even close) then we can/will suck it up. But lets review the facts:

1. The average physics pass rate over the last 3 years (on ABR website) is 92%
2. The average physics pass rate from the 12 years prior: was 88.9% with a std. dev. of 5.8% (Rad bio during this time was 7.2% failure, 3.9% std. dev)
3. What MAY be fact (but yet to be confirmed): RO28 posted yesterday that the physics pass rate was 70% (which he heard 'from the man himself'. We need more clarification if this was indeed from Dr. Wallner. But that would be GREATER THAN 3 standard deviations from the norm

We need to confirm the pass rates from the ABR. This isn't about PGY-5's whining. If the pass rates were in line with prior years then I and many others will shut right up. But if the failure rates were that far off this year it wasn't due to a lack of quality residents, something else was wrong. At a minimum it deserves further review.

Yes that 70% came in a reply to my second email to the ABR. The email was authored by Dr. Wallner.

I agree, something needs to be done about THIS YEAR. There is no reason one of the most selective specialties should have a board pass rate of 70%. This doesn’t make sense.

 

[948863]

PD here.
I have been following this thread with interest particularly because our program's residents were also affected by this year's exams, but wasn't sure the best route of action to take.

*MOD EDIT*- Due to concerns about maintaining anonymity I have asked for my original post to be moderated. As a PD, the lack of transparency from ABR about this year's exam is very concerning to me. I have contacted ADROP leadership to express these concerns and would strongly encourage other PDs to do the same.

 

[thecarbonionangle]

That the only person who liked your personal attack was the poster who was annoyed I called Wallner out for his hypocrisy in another thread and then was forced to admit his mistake when I posted the paper and quote to prove so is telling.

Guurl well thankfully I ain’t looking for validation with “likes” for my views.

 

[evilbooyaa]

Users are reminded to keep their discussions and interactions with each other professional. Do not type anything you would not say to that user in person. While disagreement with other users viewpoints is allowed, directly attacking other users of this forum will not be tolerated. This thread is quickly approaching dumpster-fire territory and is teetering on the edge, but there is progress being made here, and thus I ask that we try to stay as on-topic as possible and minimize the clutter and side discussions.

Extensive, unnecessary off-topic discussions that do not add to the original topic of this thread will be pruned and warnings will be handed out.

 

[radbioistheworst]

PD here.
I have been following this thread with interest particularly because our program's residents were also affected by this year's exams, but wasn't sure the best route of action to take.

*MOD EDIT* - Due to concerns about maintaining anonymity I have asked for my original post to be moderated. As a PD, the lack of transparency from ABR about this year's exam is very concerning to me. I have contacted ADROP leadership to express these concerns and would strongly encourage other PDs to do the same."

Thank you for your help. Forgive my ignorance on the topic but does the ADROP have any formal, organized correspondence with the ABR where these exams are addressed? I understand that there is a PD meeting at ASTRO every year - can any collective action be taken at that time? Would the ADROP have better luck uncovering the pass rates than we do? I’ve asked our PD to contact the ABR but will suggest Dr. Chen as well. Appreciate the support.


Sent from my iPhone using SDN mobile

 

[deleted18755]

I've been posting on this forum for over a decade and viewing it much longer and have nothing to add other than my deepest sympathies and a word of caution since this field is so small (and apparently maybe corrupt to a certain degree) with so much favoritism, nepotism, etc so please be very careful when you post identifying information like I wrote an email to xxx and here is the reply - it's way easier than you might think to now know exactly who you are (and you have even less power than you realize in this tiny field dominated by a few very powerful and well connected people).

For what it's worth I clearly remember walking out of the physics/radiobiology exam (and vaguely recall taking that exam and the clinical one on the same day or at least same year) not only thinking that I failed the physics exam but seriously wondering if I was given the correct exam since it wasn't that I thought the questions were difficult so much as I honestly had no idea what so many of the questions were getting at or even asking. I somehow passed and never thought about it again after all these years until this thread came up (but this is why it's so difficult to get these things changed - everybody knows these tests are too much and largely ridiculous but the second you pass it's hard not to move on with your life vs help the future test takers).

For the reasons others listed I don't see how going to the media could help anything. I wish I had better suggestions but please do something constructive because between this debacle and the job market/over training I don't see how or why any medical students would apply to this wonderful field of medicine anymore.

Sorry you guys are going through this.

 

[948863]

I've been posting on this forum for over a decade and viewing it much longer and have nothing to add other than my deepest sympathies and a word of caution since this field is so small (and apparently maybe corrupt to a certain degree) with so much favoritism, nepotism, etc so please be very careful when you post identifying information like I wrote an email to xxx and here is the reply - it's way easier than you might think to now know exactly who you are (and you have even less power than you realize in this tiny field dominated by a few very powerful and well connected people).

For what it's worth I clearly remember walking out of the physics/radiobiology exam (and vaguely recall taking that exam and the clinical one on the same day or at least same year) not only thinking that I failed the physics exam but seriously wondering if I was given the correct exam since it wasn't that I thought the questions were difficult so much as I honestly had no idea what so many of the questions were getting at or even asking. I somehow passed and never thought about it again after all these years until this thread came up (but this is why it's so difficult to get these things changed - everybody knows these tests are too much and largely ridiculous but the second you pass it's hard not to move on with your life vs help the future test takers).

For the reasons others listed I don't see how going to the media could help anything. I wish I had better suggestions but please do something constructive because between this debacle and the job market/over training I don't see how or why any medical students would apply to this wonderful field of medicine anymore.

Sorry you guys are going through this.

What, precisely, do you think I have to worry about?

I would argue that the lack of transparency from the ABR here is one of the major drivers of the problem, and being shrinking violets about serious concerns with the specialty accreditation board, and actions that are being taken to attempt to remediate the same, out of concern for some unspecified retribution, benefits absolutely no one. I have nothing to hide regarding contacting the ADROP president, and in fact I feel it is the responsibility of PDs to advocate on behalf of their residents when such concerns arise. The only purpose of creating the throwaway account was to protect the anonymity of my residents who for obvious reasons would not want identifying information about their program to be posted.

I trust that the moderators will address doxxing or other efforts to disrupt legitimate reasons for maintaining anonymity and sincerely hope that your post is not a veiled effort to do that.

 

[scarbrtj]

This is a really concerning thing *if* pass rates have gone down in the 7 out of 10 range.

In my psych rotations in med school, one of the smartest docs (a psychiatrist) I've ever known said "keep a dirty mind"... about patient histories, motivations, etc. It's sort of a riff on the famous quote by House M.D.: "Everybody lies."

So the ABR and the higher-ups say it's impossible to prune back residency slots or numbers because it would literally violate federal law (anti-trust). (On the other hand, I hear talk among other specialists how other specialties do this all the time.) But you wouldn't have to prune residency numbers if you literally produced less board-certified rad oncs. Certainly a one year "time ripple" and momentary dip in the board-certified-radiation-oncologist production rate will help the job market a little bit, yes? Well... it will moor the recently graduated, un-certified as cheap labor whilst perhaps helping out in the private practice arena simultaneously. (It seems like a geologic age ago, but "in my day" one came out of residency board certified--even the young and naive could sense the agita this caused the non-young/non-naive at the time and its fleetingness.) I don't know. My mind might just be too dirty.

 

[smq123]

What, precisely, do you think I have to worry about?

I would argue that the lack of transparency from the ABR here is one of the major drivers of the problem, and being shrinking violets about serious concerns with the specialty accreditation board, and actions that are being taken to attempt to remediate the same, out of concern for some unspecified retribution, benefits absolutely no one. I have nothing to hide regarding contacting the ADROP president, and in fact I feel it is the responsibility of PDs to advocate on behalf of their residents when such concerns arise. The only purpose of creating the throwaway account was to protect the anonymity of my residents who for obvious reasons would not want identifying information about their program to be posted.

I trust that the moderators will address doxxing or other efforts to disrupt legitimate reasons for maintaining anonymity and sincerely hope that your post is not a veiled effort to do that.

Interesting; I did not interpret his post in that way.

I would agree with him that the residents who are posting here should be careful. While it seems like you are an understanding PD who truly wants to help his/her residents, I'm sure that you can also understand that not every PD or chairman will necessarily feel that way. I can easily imagine some becoming unhappy and irritated by the idea that his or her residents would email the New York Times or the administration at the ABR to complain about this exam. Heck, I had a great relationship with my PD, and he was pretty kind and generally understanding, but I can imagine his facial expression if he found out (second hand) that I was emailing the New York Times to complain about our hospital system or the ABFM or something. And that's in family medicine, which is a huge field. Rad onc is much smaller, and it seems like everyone knows everyone else.

I don't think that it's any secret that residents do not ever bargain from a position of power, so advising them to be cautious about posting something that can easily be traced to them is prudent. SDN is committed to protecting our privacy, but I don't think that it hurts to be careful about what you post.

 

[RadOncLazers]

Interesting; I did not interpret his post in that way.

I would agree with him that the residents who are posting here should be careful. While it seems like you are an understanding PD who truly wants to help his/her residents, I'm sure that you can also understand that not every PD or chairman will necessarily feel that way. I can easily imagine some becoming unhappy and irritated by the idea that his or her residents would email the New York Times or the administration at the ABR to complain about this exam. Heck, I had a great relationship with my PD, and he was pretty kind and generally understanding, but I can imagine his facial expression if he found out (second hand) that I was emailing the New York Times to complain about our hospital system or the ABFM or something. And that's in family medicine, which is a huge field. Rad onc is much smaller, and it seems like everyone knows everyone else.

I don't think that it's any secret that residents do not ever bargain from a position of power, so advising them to be cautious about posting something that can easily be traced to them is prudent. SDN is committed to protecting our privacy, but I don't think that it hurts to be careful about what you post.

Are you suggesting to stay silent due to fear of retaliation from your program? That in itself represents the issues we have as a field. Speak up. Email. Tell your PD to help. Isn’t that their job???

 

[evilbooyaa]

OK everybody, let's take a step back.

Can we all accept that there is a big difference between asking your PD for assistance, and going to the media to antagonize all of the ABR and more importantly, bring negative attention to your residency program and hospital.

Some people are saying that it's not ideal to jump directly to the nuclear option in regards to this.

 

[smq123]

Are you suggesting to stay silent due to fear of retaliation from your program? That in itself represents the issues we have as a field. Speak up. Email. Tell your PD to help. Isn’t that their job???

No, not at all.

But maybe use a throwaway account that isn't tied to a personally identifiable email. Make sure that what you've posted in the past doesn't automatically identify where you matched or where you live, etc.

And it's one thing to ask your PD for support and help; it's another to post something on an internet forum talking about a conspiracy from the ABR and how this should be exposed in the national media.

I don't think it's cowardice to want to protect your identity and your future. As a resident, that's basically all you have - the opportunities that you will have once you leave residency. I know, the residency system sucks, but it's what we have to work within for now.

(D'oh! @evilbooyaa posted what I was trying to say, but in a much more eloquent fashion!!)

 

[mikezab]

Thank you to those that have helped get this thread back on track. We appreciate all of the help and feedback we have received. As a PGY-5 who is interested in having the ABR take a fair and honest assessment of this past year's Rad Bio/Physics exams, what would you recommend that we all consider doing? I have not and will not ask anything of the ABR in a disrespectful way-- but I think it is entirely fair to ask them to treat us with the same respect. We want them to collaborate with us and find a solution that all parties are happy with. Here are some of the ways I am interpreting others saying we can push forward in a respectful manner (please correct me where I am wrong, just trying to get everyone on the same page)

1. Email Dr. Wallner of the ABR
2. Email Dr. Valerie Jackson (Executive Director of ABR)
3. Contact ARRO: [email protected]
4. Get your program director involved: Have them contact ADROP Leadership (Dr. Allen Chen of Kansas University)

What else have I missed? In my opinion any and all residents should be doing all of the above. It would be great if by ASTRO we had confirmed pass rates (physics appears to be 70%, rad bio is still unknown) so that we could have all concerned PD's get together to discuss next steps. Studying for these exams takes countless hours, you can't tell me that our pass rates are 3+ standard deviations below the norm and it's just the residents aren't as intelligent.. if that were the case this same class of 2019 would (theoretically) have been scoring 3+ standard deviations lower on all in service exams..... which I'm pretty sure was not the case.

 

[RO28]

Thank you to those that have helped get this thread back on track. We appreciate all of the help and feedback we have received. As a PGY-5 who is interested in having the ABR take a fair and honest assessment of this past year's Rad Bio/Physics exams, what would you recommend that we all consider doing? I have not and will not ask anything of the ABR in a disrespectful way-- but I think it is entirely fair to ask them to treat us with the same respect. We want them to collaborate with us and find a solution that all parties are happy with. Here are some of the ways I am interpreting others saying we can push forward in a respectful manner (please correct me where I am wrong, just trying to get everyone on the same page)

1. Email Dr. Wallner of the ABR
2. Email Dr. Valerie Jackson (Executive Director of ABR)
3. Contact ARRO:
4. Get your program director involved: Have them contact ADROP Leadership (Dr. Allen Chen of Kansas University)

What else have I missed? In my opinion any and all residents should be doing all of the above. It would be great if by ASTRO we had confirmed pass rates (physics appears to be 70%, rad bio is still unknown) so that we could have all concerned PD's get together to discuss next steps. Studying for these exams takes countless hours, you can't tell me that our pass rates are 3+ standard deviations below the norm and it's just the residents aren't as intelligent.. if that were the case this same class of 2019 would (theoretically) have been scoring 3+ standard deviations lower on all in service exams..... which I'm pretty sure was not the case.

5. Don’t give up. We can be polite and assertive and very very persistent.

 

[radbioistheworst]

Thank you to those that have helped get this thread back on track. We appreciate all of the help and feedback we have received. As a PGY-5 who is interested in having the ABR take a fair and honest assessment of this past year's Rad Bio/Physics exams, what would you recommend that we all consider doing? I have not and will not ask anything of the ABR in a disrespectful way-- but I think it is entirely fair to ask them to treat us with the same respect. We want them to collaborate with us and find a solution that all parties are happy with. Here are some of the ways I am interpreting others saying we can push forward in a respectful manner (please correct me where I am wrong, just trying to get everyone on the same page)

1. Email Dr. Wallner of the ABR
2. Email Dr. Valerie Jackson (Executive Director of ABR)
3. Contact ARRO: [email protected]
4. Get your program director involved: Have them contact ADROP Leadership (Dr. Allen Chen of Kansas University)

What else have I missed? In my opinion any and all residents should be doing all of the above. It would be great if by ASTRO we had confirmed pass rates (physics appears to be 70%, rad bio is still unknown) so that we could have all concerned PD's get together to discuss next steps. Studying for these exams takes countless hours, you can't tell me that our pass rates are 3+ standard deviations below the norm and it's just the residents aren't as intelligent.. if that were the case this same class of 2019 would (theoretically) have been scoring 3+ standard deviations lower on all in service exams..... which I'm pretty sure was not the case.

Thanks for laying out these ideas, they all sound reasonable. I tried to allude to this before but I think we should push for these exams to be on the ADROP’s agenda when they meet at ASTRO this year (if it isn’t already). It would be nice to see solidarity among PDs, even from programs who had no failures. Does anyone know of other educational committees or meetings at ASTRO? Does anyone know if the ABR and ASTRO leadership meet with each other? Perhaps if discussed in a live, public, forum where all parties are present the message will be better received.


Sent from my iPhone using SDN mobile

 

[xrt_md_2019]

Hm, where to start?

1) Let's rise above being trolls to one another. Doesn't help anyone. Yes, while our job puts big life events into perspective, failing a major board exam after dedicating years of our lives to medicine still sucks. We should all have compassion for our colleagues.

2) No media contact whatsoever. I agree that the public will just throw us under the bus.

3) Whether you passed or failed, I think we can all agree that the material tested on the physics and radbio exams is mostly irrelevant to daily radiation oncology. If they want to cover these topics, they should be done in a way that tests practical knowledge & not minute detail.

I studied my butt off for both exams, and very unexpectedly failed radbio (which I felt more confident about).

It may have been stated in prior posts, but I wrote to the ABR & they told me that the subsections are not equally weighted. Of course they did not specify which subsections are weighted the highest.

I spoke with my radbio professor after getting the exam results because I was so shellshocked. I walked out of that exam believing I had passed.
My professor is prominent in the field & told me some inside scoop from their friend who is closely involved with the ABR exam.
This friend revealed that the fail rate (for at least radbio this year) was over 30%.
This person is a very reliable source.
Apparently residents from top programs have failed & everyone is losing their mind.

My professor suggested that my PD email the ABR on my behalf & I think that's a good place for people to start. No one cares when residents complain....when PDs do, it could be different.

I think some reasonable efforts the ABR could make include:
-revisiting the questions and lowering the failure rate on the 2018 exam
-allowing residents who failed to retake the exam in early 2019, thus sparing them the need to take 3 exams in July 2019 (my professor confirmed that a mid-year test retake was offered in the past)
-publish more detailed study guides for both test sections
-be explicit about the weighting of test subsections, organize the test by subsection so that test takers can strategize their question answering
-for radbio, publish a 2019 study guide (there was no 2018 guide)
-for physics, offer samples of more representative test questions (RAPHEX grossly underestimates the exam difficulty)

If anyone from ARRO is reading, if you could post a weekly update about discussions with the ABR that would be amazing. Preferably on Facebook -- in general this website makes me want to vomit...

Fingers crossed that some change happens that will benefit us all.
My goal is that once I pass these exams, I want to speak out about this ridiculous process from the stance of an attending.
We need to make things better for future residents.

 

[deleted18755]

What, precisely, do you think I have to worry about?

I would argue that the lack of transparency from the ABR here is one of the major drivers of the problem, and being shrinking violets about serious concerns with the specialty accreditation board, and actions that are being taken to attempt to remediate the same, out of concern for some unspecified retribution, benefits absolutely no one. I have nothing to hide regarding contacting the ADROP president, and in fact I feel it is the responsibility of PDs to advocate on behalf of their residents when such concerns arise. The only purpose of creating the throwaway account was to protect the anonymity of my residents who for obvious reasons would not want identifying information about their program to be posted.

I trust that the moderators will address doxxing or other efforts to disrupt legitimate reasons for maintaining anonymity and sincerely hope that your post is not a veiled effort to do that.

Oh very good, I don't really pay attention to who is actually posting what and didn't realize you were a program director. In fact, I think the only way anything is going to get done is if residents go through their program directors in an organized fashioned, rather than individually emailing or contacting directly (this would also "shield" the residents for what that's worth).

I don't know what anybody actually could do to anybody else but all residents in general are already in such a compromised position and our field in particular is so small (and I don't know what other fields are like or even what is going on here but it seems like at least some degree of shadiness).

I used to keep in touch with residents from my program but I haven't had any contact with anybody in awhile and I'm just a random doctor in community practice but you all have my sympathy and support (morally is probably all I have to offer).

 

[948863]

Thank you for your help. Forgive my ignorance on the topic but does the ADROP have any formal, organized correspondence with the ABR where these exams are addressed? I understand that there is a PD meeting at ASTRO every year - can any collective action be taken at that time? Would the ADROP have better luck uncovering the pass rates than we do? I’ve asked our PD to contact the ABR but will suggest Dr. Chen as well. Appreciate the support.

Sorry, I don't know the answers to any of your questions. ADROP is an affiliate of ASTRO, but I don't know the specifics of how the ABR is connected with ASTRO or any of the ASTRO affiliates (ie ADROP). My best guess would be that they are totally independent organizations. I wouldn't be surprised if this comes up at the ASTRO PD meeting, but only time will tell. Assuming this is the first time that more than a handful of residents have failed the basic science exams, there is probably no formal mechanism for addressing it, since it hasn't really been a problem in the past.

 

[Tony&lisa]

Just sent a long email to ARRO. If anyone hasn't yet, in addition to getting PDs involved, we should all be emailing ABR and ARRO. I agree that we have little sway, but the more emails they both get, the greater the issue will appear and will help in hiw seriosuly we are taken. We won't effect change necessarily, but mass emails are effective in hilighting and lending weight to an issue.

 

[metallica81788]

Thanks so much to everyone who has reached out to us via email, Twitter, FB, SDN regarding resident concerns with the ABR exam. We wanted to let you know that we are collectively working with several groups and have talked with leaders of ADROP as well to make sure the ABR hears them. The comments that have been shared are truly important to the process, so we encourage you to reach out to us to send any direct concerns or specific comments via DM on social media or email arro AT astro org.

Is there any consideration of making a statement or reporting results/findings at the ARRO resident seminar at ASTRO this year? That could be a good opportunity to address some things and give a sense as to where all of this is going. I'm sure there will be many interested individuals in that room.

 

[ramsesthenice]

Before everyone attacks me let me make it very clear, I do not agree with how this exam transpired. It means much more to those of you who failed than it does to me and other PDs and assistant PDs. But you are not the only ones with consequences. Some of our residents had issues and we ended up taking a double hit. We had some transfers and fair or not, the ACGME only cares about where people started their residency when they evaluate your first time pass rates and this can really screw a program. And of course, this is the year for a site visit. Bottom line, I am also incentivized to see this resolved.

I also don't want to discourage anyone from writing letters to ARRO, ABR, etc. They have and will continue to hear from us. That much I assure you.

But I have to point out that the reality is you all need to embrace the fact that their action plan is to provide better guidance on what will be covered on future exams. You are going to be retaking these tests the day before your clinical writtens. Period. There are really only 4 things I can think of that would motivate them to do anything more drastic:

1) Public pressure: not going to happen for the reason stated above
2) Litigation: Despite what some people have suggested this is dead in the water. The simple fact is that if the ABR decides it thinks that 70% is a more appropriate pass rate they are within their rites to do so. Even if you argued that the historic pass rate provides some kind of precedent you only have one data point and it will be very difficult to prove the problem was with the test and not the test takers. Even if by some miracle it moved forward this would take years to settle. No chance of helping any of you now.
3) Political Pressure: Not external politics but input from ARRO, PDs, etc. This is your best hope but lets face it, the powers that be have climbed the ladder and don't like being told what to do. They have made it clear what they think the best solution is and I just don't see them backing down. Realistically you should expect people to dig in their heals.
4) Its the right thing to do: They have already made it very clear this is not going to happen. They don't think that there is anything inherently wrong with the test or process that warrants re-evaluation. If they were going to do this it would have already happened.

The absolute best case scenario that you can realistically hope for would be an earlier retake (maybe January). If you are planning to make any concrete suggestions/requests this would be it.

 

[KHE88]

Before everyone attacks me let me make it very clear, I do not agree with how this exam transpired. It means much more to those of you who failed than it does to me and other PDs and assistant PDs. But you are not the only ones with consequences. Some of our residents had issues and we ended up taking a double hit. We had some transfers and fair or not, the ACGME only cares about where people started their residency when they evaluate your first time pass rates and this can really screw a program. And of course, this is the year for a site visit. Bottom line, I am also incentivized to see this resolved.

I also don't want to discourage anyone from writing letters to ARRO, ABR, etc. They have and will continue to hear from us. That much I assure you.

But I have to point out that the reality is you all need to embrace the fact that their action plan is to provide better guidance on what will be covered on future exams. You are going to be retaking these tests the day before your clinical writtens. Period. There are really only 4 things I can think of that would motivate them to do anything more drastic:

1) Public pressure: not going to happen for the reason stated above
2) Litigation: Despite what some people have suggested this is dead in the water. The simple fact is that if the ABR decides it thinks that 70% is a more appropriate pass rate they are within their rites to do so. Even if you argued that the historic pass rate provides some kind of precedent you only have one data point and it will be very difficult to prove the problem was with the test and not the test takers. Even if by some miracle it moved forward this would take years to settle. No chance of helping any of you now.
3) Political Pressure: Not external politics but input from ARRO, PDs, etc. This is your best hope but lets face it, the powers that be have climbed the ladder and don't like being told what to do. They have made it clear what they think the best solution is and I just don't see them backing down. Realistically you should expect people to dig in their heals.
4) Its the right thing to do: They have already made it very clear this is not going to happen. They don't think that there is anything inherently wrong with the test or process that warrants re-evaluation. If they were going to do this it would have already happened.

The absolute best case scenario that you can realistically hope for would be an earlier retake (maybe January). If you are planning to make any concrete suggestions/requests this would be it.

I understand what you are saying in terms of having realistic expectations. However, I don't understand why you think that we should just accept the fact that there was nothing wrong with the test or evaluation process because they say so. At some point, the scores are so low as to demand re-evaluation. If 95% of people failed, would the test be re-evaluated this year? Of course. I would argue that an overall failure rate of 40-50%, which I think is what we are looking at this year with individual failure rates of 30-35% on bio and physics and a few people failing both, when historical average failure rates are around 10% with a 4% standard deviation well exceed that threshold.

The ABR should have a policy that states that if a failure rate in a single year exceeds more than, say, 3 standard deviations of the moving average over the past 10 years, that it should trigger an automatic re-review and by outside consultants resulting in either adjustment or verification of the cutscore. This benefits everybody as it improves the reliability and objectivity of their "Angoff method." Perhaps this already happened and that's why it took them >6 weeks to release our results. We don't know. There is zero transparency. This apparent revelation about some questions being weighted more than others? I was basically called an idiot earlier in this thread for thinking it could be possible. Apparently it actually is? We don't know how these things are graded at all. I disagree with your assessment that we just need to accept this lack of transparency because they're the board and we just blindly do what they say lest we get doxxed online and our careers ruined for simply having an opinion, as other posters suggested we should be afraid of.

 

[RO28]

I understand what you are saying in terms of having realistic expectations. However, I don't understand why you think that we should just accept the fact that there was nothing wrong with the test or evaluation process because they say so. At some point, the scores are so low as to demand re-evaluation. If 95% of people failed, would the test be re-evaluated this year? Of course. I would argue that an overall failure rate of 40-50%, which I think is what we are looking at this year with individual failure rates of 30-35% on bio and physics and a few people failing both, when historical average failure rates are around 10% with a 4% standard deviation well exceed that threshold.

The ABR should have a policy that states that if a failure rate in a single year exceeds more than, say, 3 standard deviations of the moving average over the past 10 years, that it should trigger an automatic re-review and by outside consultants resulting in either adjustment or verification of the cutscore. This benefits everybody as it improves the reliability and objectivity of their "Angoff method." Perhaps this already happened and that's why it took them >6 weeks to release our results. We don't know. There is zero transparency. This apparent revelation about some questions being weighted more than others? I was basically called an idiot earlier in this thread for thinking it could be possible. Apparently it actually is? We don't know how these things are graded at all. I disagree with your assessment that we just need to accept this lack of transparency because they're the board and we just blindly do what they say lest we get doxxed online and our careers ruined for simply having an opinion, as other posters suggested we should be afraid of.

I completely agree. Throughout my career when I have taken a standardized test of any sort I assume that each question is weighted equally unless otherwise told (ex back in college where an essay as part of an exam is 50% of the total grade). To let us know after the fact that there is differential weighting of sections is not forthcoming. It would have changed my study approach and I’m sure the approach of others as well. I don’t disagree with needing board exams to ensure competency, but man let us know what it takes to achieve this goal otherwise we will end up so frustrated we feel like giving up.

 

[ramsesthenice]

I disagree with your assessment that we just need to accept this lack of transparency because they're the board and we just blindly do what they say

I think you need to re-read my post. They should continue to hear from test takers and PDs and any other vested party and hopefully that will bring about change. But whether you want to accept it or not change takes time. My opinion. Your opinion. What seems fair. None of these things hold weight. If you want to become board certified you are at the mercy of the board and you have to play the game by their rules. At present there is no higher power to tell them what they can and can't do. If they had any intention of addressing the pass rate for this years cohort they would have already done it. I would love to be wrong but the overwhelming likelihood is that everyone who failed one or both of the tests is going to be retaking them. If you want something tangible to ask for, a broad effort to get some separation between these and the clinical writtens would be a good goal.

 

[KHE88]

I think you need to re-read my post. They should continue to hear from test takers and PDs and any other vested party and hopefully that will bring about change. But whether you want to accept it or not change takes time. My opinion. Your opinion. What seems fair. None of these things hold weight. If you want to become board certified you are at the mercy of the board and you have to play the game by their rules. At present there is no higher power to tell them what they can and can't do. If they had any intention of addressing the pass rate for this years cohort they would have already done it. I would love to be wrong but the overwhelming likelihood is that everyone who failed one or both of the tests is going to be retaking them. If you want something tangible to ask for, a broad effort to get some separation between these and the clinical writtens would be a good goal.

For the most part I agreed with your post. I am not going to deny that there were a few histrionic outbursts here. I think most of us realize that change is going to come from within and not by riling up the general public to go and picket the ABR's headquarters. Although some appropriate publicity within the broader medical community may help, as some have proposed. I greatly appreciate that you are on our side (I don't know if you are a PD, but sounds like you are involved in a residency -- and there are some that are unfortunately apathetic to this situation and don't want to be bothered with it). My concern was that you sounded a little defeatist. The results were only released a few weeks ago, and there are 10 months until the next scheduled test. We should continue our appeals to reason and push for a tangible solution of a re-take, and I would also add a separate independent review of this year's test and adjustment of the passing score as a tangible solution. At the very least, get the ABR to release an official statement with the pass rate and be willing to formally engage us in discussion through ARRO.

 

[radbioistheworst]

Before everyone attacks me let me make it very clear, I do not agree with how this exam transpired. It means much more to those of you who failed than it does to me and other PDs and assistant PDs. But you are not the only ones with consequences. Some of our residents had issues and we ended up taking a double hit. We had some transfers and fair or not, the ACGME only cares about where people started their residency when they evaluate your first time pass rates and this can really screw a program. And of course, this is the year for a site visit. Bottom line, I am also incentivized to see this resolved.

I also don't want to discourage anyone from writing letters to ARRO, ABR, etc. They have and will continue to hear from us. That much I assure you.

But I have to point out that the reality is you all need to embrace the fact that their action plan is to provide better guidance on what will be covered on future exams. You are going to be retaking these tests the day before your clinical writtens. Period. There are really only 4 things I can think of that would motivate them to do anything more drastic:

1) Public pressure: not going to happen for the reason stated above
2) Litigation: Despite what some people have suggested this is dead in the water. The simple fact is that if the ABR decides it thinks that 70% is a more appropriate pass rate they are within their rites to do so. Even if you argued that the historic pass rate provides some kind of precedent you only have one data point and it will be very difficult to prove the problem was with the test and not the test takers. Even if by some miracle it moved forward this would take years to settle. No chance of helping any of you now.
3) Political Pressure: Not external politics but input from ARRO, PDs, etc. This is your best hope but lets face it, the powers that be have climbed the ladder and don't like being told what to do. They have made it clear what they think the best solution is and I just don't see them backing down. Realistically you should expect people to dig in their heals.
4) Its the right thing to do: They have already made it very clear this is not going to happen. They don't think that there is anything inherently wrong with the test or process that warrants re-evaluation. If they were going to do this it would have already happened.

The absolute best case scenario that you can realistically hope for would be an earlier retake (maybe January). If you are planning to make any concrete suggestions/requests this would be it.

Thanks for fighting on our behalf. I understand ARRO plans to ask for a retake earlier next year, but I’m not holding my breath. I think what we need to demand before anything else (after official pass results) are assurances that the next exam will test material that we actually have an opportunity/resources to study for. It doesn’t sound like the grading system will change but how could that possibly not at least be reviewed? The exam was so obviously, empirically, flawed according to the pass rates. Wallner’s alleged attempt to procure more relevant study references is an admission of this.

What concerns me the most from your post is that “if the ABR decides 70% is a more appropriate pass rate then that’s their right.” I mean sure it is (I agree litigation isn’t the way to go), but do you really think that the ABR will be ok with half of the 2019 class being board ineligible in 2020 (assuming similar content and grading method)? That’s not a rhetorical question, genuinely asking. And if that is the case/new standard, then is the ADROP so powerless that they can’t influence the ABR to make the pass rates more reasonable (and historically consistent)? Again, genuine question.


Sent from my iPhone using SDN mobile

 

[ramsesthenice]

do you really think that the ABR will be ok with half of the 2019 class being board ineligible in 2020 (assuming similar content and grading method)?

I sure hope not. They had the option of doing something about it when evaluating the results and deciding who passed and who failed and decided it was ok to at least let it go this year. I can't think of a reason why they would want to see this but they have shown that at the very least the idea does not bother them terribly.

And no, I am not a PD but I am heavily involved in our residency program and training.

 

[RadOncG]

I would add email Lisa Kachnic (President of the board of governors of the ABR and a radiation oncologist) to the list of things to do- Respectfully tell her that residents, PDs, and other concerned parties have been reaching out to the ABR with concerns but have so far not received adequate answers.

 

[Rad-Onc-PD]

For current PGY-5s who failed biology or physics ... Your PD just received an email from the ABR. Please ask him or her to share its contents. The 70% pass rate for physics and 74% pass rate for biology are now confirmed by the ABR