Radiobiology - ABR history

[Rad Onc SK]

I am hoping I can get some help from this forum, which is the main (self-serving) reason I joined.

I am trying to better understand some of the major inflection points over the past several years which have affected younger rad oncs. The 2018(?) radiobiology exam and subsequent back forth with the ABR seems to be at the top of that list. I did not live through that and did not pay much attention to it. I was not on the BoD.

(As a reminder, I am STILL not on the ASTRO BoD, despite what many here seem to think. That starts at the AM in October.)

Would/could anyone summarize the high points from the residents perspective? I suspect that this topic has a potential to get off the rails quickly, but if we could stick to objective facts and keep personal names (and shaming) out of the thread, I would personally appreciate that very much!

Thanks in advance

SK

---

Members don't see this ad.

 

[thecarbonionangle]

There are entire threads on this which might be worth reading, as well as editorials from Lord Wallner and Kachnich. Basically the scoring of these exams uses some bizarre “Angoff” method. PW and LK basically said that quality of residents had gone down and approximately half of all residents failed an exam. The ABR refused to be transparent. The resident association put out a paper totally disproving the fake news that residents just got dumber. Simul posted on ROHUB asking why is a guy with close ties to 21C, an organization accused of massive corruption in charge of our boards and was threatened with litigation and banned. The ABR was accused also of failing to accomodate lactating women, and giving zero flexibility for retakes. LK went to ASTRO and doubled down on “you guys are just dumb” narrative. Yeah she is NEVER getting my ASTRO leadership vote. I would vote for your cologuard box over her.
Essentially a whole class got totally screwed. LK would go on to be associated with multiple hellpit programs (one shut down) and expand one. “Leaders” being them. I believe Amdur and Lee were one of few people to push back while the silence was deafening from our “leaders”.

Anyways please someone correct me if im wrong. Im just thinking back and doing a quick summary of what i remember.

Perhaps @elementaryschooleconomics might be able to dive deep on it.

 

[NotMattSpraker]

Im a little biased, but this is excellent.

 

[deleted1111261]

There are entire threads on this which might be worth reading, as well as editorials from Lord Wallner and Kachnich. Basically the scoring of these exams uses some bizarre “Angoff” method. PW and LK basically said that quality of residents had gone down and approximately half of all residents failed an exam. The ABR refused to be transparent. The resident association put out a paper totally disproving the fake news that residents just got dumber. Simul posted on ROHUB asking why is a guy with close ties to 21C, an organization accused of massive corruption in charge of our boards and was threatened with litigation and banned. The ABR was accused also of failing to accomodate lactating women, and giving zero flexibility for retakes. LK went to ASTRO and doubled down on “you guys are just dumb” narrative. Yeah she is NEVER getting my ASTRO leadership vote. I would vote for your cologuard box over her.
Essentially a whole class got totally screwed. LK would go on to be associated with multiple hellpit programs (one shut down) and expand one. “Leaders” being them.

Anyways please someone correct me if im wrong. Im just thinking back and doing a quick summary of what i remember.

Perhaps @elementaryschooleconomics might be able to dive deep on it.

Excellent and accurate and even though hard for ASTRO to see it, balanced summary of what happened.

(Also, the residents were nice enough to include me on that paper!)

 

[Chartreuse Wombat]

I am hoping I can get some help from this forum, which is the main (self-serving) reason I joined.

I am trying to better understand some of the major inflection points over the past several years which have affected younger rad oncs. The 2018(?) radiobiology exam and subsequent back forth with the ABR seems to be at the top of that list. I did not live through that and did not pay much attention to it. I was not on the BoD.

(As a reminder, I am STILL not on the ASTRO BoD, despite what many here seem to think. That starts at the AM in October.)

Would/could anyone summarize the high points from the residents perspective? I suspect that this topic has a potential to get off the rails quickly, but if we could stick to objective facts and keep personal names (and shaming) out of the thread, I would personally appreciate that very much!

Thanks in advance

SK

I am board-certified and in the MOC program voluntarily (took boards before 1995).
I have volunteered with the ABR in many different contexts (item writer, Angoff participant, oral examiner).
This is my take on the episode.

The Angoff process is a system that is widely used in many different contexts. Like any process it is imperfect but there are few alternatives when trying to use a criterion-referenced examination (as opposed to a normative examination). Angoff participants are asked to imagine the person barely competent and guess whether they would answer the item correctly. Multiple Angoff participants "guesses" and then collated and questions can be rated as hard or easy and the test construction should provide a mix of hard an easy.

The fundamental problem is that the sample size is relatively small (realtively few test takers per year) and this can lead to strange results (big fan of Taleb).

I have some inside knowledge but it is second hand. In this case the "data" was such that a large % of scores on biology and physics were in the fail category. The % was a big outlier when compared to years of previous results.

In this case the decision maker (I don't know who it was) decided to "follow the data" and a large fail rate resulted.

An example of how turning "knowledge" into a "number" has downsides.

The decision maker COULD have decided to ignore the data and draw the line at fail rate similar to previous experiences but I am told that there was no obvious "cutpoint" at 10-15% fail.

I think all of this information could have been shared with the test takers.

Instead the ABR leadership decided to blame the test takers and label them as stupid in public and in the peer-reviewed literature. This decision was a massive mistake and they are still recovering.

In a result that is such an outlier is it more likely that the test takers have changed or the test has changed. I favor the latter.

Hope thi shelps. You can DM me if you want more granularity

 

[thecarbonionangle]

The best thing to come out the Wallner editorial is the outloud admission that there are terrible training programs out there, the hellpits. Why isnt the specialty closing down these places? We know they exist! Close them ALL down!

 

[deleted1111261]

I am board-certified and in the MOC program voluntarily (took boards before 1995).
I have volunteered with the ABR in many different contexts (item writer, Angoff participant, oral examiner).
This is my take on the episode.

The Angoff process is a system that is widely used in many different contexts. Like any process it is imperfect but there are few alternatives when trying to use a criterion-referenced examination (as opposed to a normative examination). Angoff participants are asked to imagine the person barely competent and guess whether they would answer the item correctly. Multiple Angoff participants "guesses" and then collated and questions can be rated as hard or easy and the test construction should provide a mix of hard an easy.

The fundamental problem is that the sample size is relatively small (realtively few test takers per year) and this can lead to strange results (big fan of Taleb).

I have some inside knowledge but it is second hand. In this case the "data" was such that a large % of scores on biology and physics were in the fail category. The % was a big outlier when compared to years of previous results.

In this case the decision maker (I don't know who it was) decided to "follow the data" and a large fail rate resulted.

An example of how turning "knowledge" into a "number" has downsides.

The decision maker COULD have decided to ignore the data and draw the line at fail rate similar to previous experiences but I am told that there was no obvious "cutpoint" at 10-15% fail.

I think all of this information could have been shared with the test takers.

Instead the ABR leadership decided to blame the test takers and label them as stupid in public and in the peer-reviewed literature. This decision was a massive mistake and they are still recovering.

In a result that is such an outlier is it more likely that the test takers have changed or the test has changed. I favor the latter.

Hope thi shelps. You can DM me if you want more granularity

I don't know that I knew this part about the possibility to draw a different cut-off.

Statistically, it is suboptimal. But, politically, this would have been way smarter.

Count on these people to always pick the greater of two evils.

 

[grenz]

I don't know that I knew this part about the possibility to draw a different cut-off.

Statistically, it is suboptimal. But, politically, this would have been way smarter.

Count on these people to always pick the greater of two evils.

Manipulating the cut line to not anger people is the greater of two evils?

 

[fiji128]

Since about 1995 the pass rate for physics and rad bio exams had been in the 85 to 95% range. Then all the sudden in 2018 (candidates would have been in the 2015 match class ie peak rad onc) the pass rate for the combined exam was somewhere around 50%. ABR "leadership" very publicly just said the test takers that year were basically not very bright and no accommodations would be made on their part and better luck next year doing rad bio/physics/clinicals exams all in one go.

 

[RickyScott]

Manipulating the cut line to not anger people is the greater of two evils?

Sure, when data is flawed. Subsequently, the "Quality" of examinees (from those same programs) has worsened, but the pass rates are way up.

 

[deleted1111261]

Manipulating the cut line to not anger people is the greater of two evils?

It’s just my opinion, failing a lot of brilliant kids and then calling them dumb is the greater of two evils. But we can agree to disagree!

 

[RickyScott]

The critical underlying issue here is over-supply. The ABR fiasco just brought awareness to the terrible leadership, decision making, greed and self-interest, which of course underlies oversupply. Cant blame it for the demise of radonc in the match or the rising anger among the majority of radoncs who feel that astro works against their interests.

 

[fiji128]

I don't think this one shameful incident in of itself would be enough to tank interest in the field. Had the same thing happened in derm/ortho/uro/plastics in 2018 they would still be as competitive as ever today. However, it may have pricked the bubble. It also made it abundantly clear the complete lack of quality/seriousness there is in terms of "leadership" at the top of the field. The same club/group of people that did this are also the same club/group that thought it was fine to 2x the number of people we are training because of a sloppy paper in JCO. In my opinion the damage is done and is likely not fixable (from a practical stand point given the huge COI) in a specialty who's foot print is shrinking. Potential applicants also see this and now stay away.

 

[communitydoc13]

The fundamental problem is that the sample size is relatively small (realtively few test takers per year) and this can lead to strange results (big fan of Taleb).

I have some inside knowledge but it is second hand. In this case the "data" was such that a large % of scores on biology and physics were in the fail category. The % was a big outlier when compared to years of previous results.

In this case the decision maker (I don't know who it was) decided to "follow the data" and a large fail rate resulted.

An example of how turning "knowledge" into a "number" has downsides.

The decision maker COULD have decided to ignore the data and draw the line at fail rate similar to previous experiences but I am told that there was no obvious "cutpoint" at 10-15% fail.

I think all of this information could have been shared with the test takers.

Should be a lesson for many people. ABR had "priors" as in "informative priors" in a Bayesian sense. I'm no statistician, but I would think years of score distributions would count as "informative" information. Meaning, you expect distributions to roughly resemble themselves year over year. Yes, the sample size is low, but 150+ kids taking a test is not the smallest sample size.

You take a process where a panel of experts determines how hard the test is and then you get a result that doesn't look anything like any of your priors? I would argue that the correct thing to do would have been to throw out your determined pass line and pick something reasonable based on your prior score distributions.

There can be exceptions of course (Covid year in grade school for example), but there was no evidence of this from that time. As mentioned above, probably one of the most competitive classes in terms of board scores out there.

There may also be a deeper thing going on here with regards to rad-bio. (Just a hypothesis here). Rad-bio as tested is no longer the foundational rad-bio from Gordon Steel's Basic Clinical Radiobiology text. Even when I took the exam (roughly 5 year before this cohort) it was becoming about oncogenes and tumor suppressor genes and pathways. If this is the stuff that is viewed as an essential foundation for radiation oncologists, maybe radiation oncologists should be giving targeted systemic therapy.

There are hellpits out there, but I wouldn't take the radbio scores at present to be a good indicator of hellpitness.

 

[RickyScott]

Pretty sure I know what happened. Test was absurdly hard, but big programs with large recalls were able to provide their test takers with buffer of 100% correct answers on 30-50% of the test. If analysis was limited to just the new questions, could easily see that this was the case.

 

[elementaryschooleconomics]

As someone who has obsessively followed this topic since the proverbial D-Day...

I think the "truth", as close as we will ever get to it, is in this thread. There's some information here that hasn't been so plainly stated yet.

If you listen to the podcast @NotMattSpraker posted, it's very dense but tells the "entire" story for context. Or, at least, close to it.

Then read this thread again.

And...that's what happened.

 

[GapCalc]

Dr. Keole, I appreciate you taking an interest. For a certain number of early- and mid-career folks on RO SDN, this episode really put a fine point on just how little the leadership in radiation oncology did not (and I would argue, does not) care about the future of trainees.

As you may be aware, SDN has been used for study prep (premeds, USMLEs, shelf exams, etc). Our very own rad onc forum had a mildly-active thread discussing the Physics and Rad Bio exams. Different threads would pop up year to year on how to study. One particular thread was started in 2013: Physics & Radbio

But on page 3 things start to heat up. You're a busy man, so I suspect you don't have time to read through much of it, but this is a time capsule about what trainees were experiencing in near-real time.

A number of posters shared their quartile scores for each section and overall exam result. It just didn't add up.

Just to save you a trip, here's the data from ABR about pass rates:

Notice anything funny?

I personally know of two residents who took this exam, failed one or both, later interviewed for jobs they wanted, and were later told they would not be candidates for employment because of a failed exam. Imagine that stress as a PGY4 with 200k+ loan debt in a poor job market. Both people eventually passed, and were employed elsewhere.

I've chatted with study group partners and other friends who did training in/around this era and are now enjoying the coerced participation longitudinal learning that is ABR's Maintenance of Certification (MOC). The contrast in questions between initial certification and MOC is laughable. I mean, it's truly remarkable. I don't want to spark the ire of ABR's legal eagles by sharing MOC content but suffice to say, the IC test is the Olympics, and MOC is remedial middle school math.

The ABR doesn't exist for radiation oncologists (or radiologists). It exists for a vague notion of 'the public good.' It cannot admit or accept flawed methodology in initial training questions or low-quality MOC content because that would call into question the very reason for it's existence. I get that. It'll never change.

I had great training in residency, and thankfully passed my boards. I'm gainfully employed. I love my job, and my patients.

But I'll never frame my ABR certificate because I don't believe it's worth the paper it's printed on.

 

[Linaculous]

...
I personally know of two residents who took this exam, failed one or both, later interviewed for jobs they wanted, and were later told they would not be candidates for employment because of a failed exam. Imagine that stress as a PGY4 with 200k+ loan debt in a poor job market. Both people eventually passed, and were employed elsewhere.
...

To underscore the bolded (my emphasis). I was in this class of test takers. I (luckily) passed.

During the job hunt, interviewers knew that a higher proportion of residents failed. I was asked if I passed or failed multiple times, and was told by these interviewers that their practices would not hire someone who failed.

This was not just the hassle of having to take the exams the following year (which in itself was a big deal), but this affected employment in a very real way. It was a huge deal.

 

[wandering star]

Just my two cents: the Angoff method sounds reasonable, but there's a lot of noise in asking a small number of subject matter experts to estimate how well a minimally competent rad onc should score. Imagine that a "generous" expert rotates off and a "harsh" expert rotates on -- that would skew the results. Or, imagine that in 2019-2020 the subject matter expert wants to avoid a repeat debacle and tries to overcompensate by being generous -- of course no resident will complain, but the data shows that probably happened too (at 99% pass rates) and it defeats the whole point of Angoff scoring.

How would you fix the system? Here's one possible solution. Let's say that instead of creating one test each year, we create 25% of the test over 4 years. That means the same bank of questions, with their Angoff criterion scores, is distributed over 4 years in a randomized fashion (probably stratified by content area). That way, we can calibrate to how generous/harsh the subject expert matters were. We're still using the Angoff criteria, but we're able to detect if a batch of subject matter experts are being difficult. I think that if in 2018, we had known that questions were randomly sampled and this year's group truly performed worse than 2017, people would be able to accept it with that data. As it stands, there is no such data, and from an outsider's perspective it's much easier to imagine that the subject matter experts were calibrated wrong, not that one cohort of test takers was suddenly (and inexplicably) stupid.

 

[winstonfoot5]

The resident association put out a paper totally disproving the fake news that residents just got dumber.

Here is the paper:
Radiation Oncology Resident Quality by National Resident Matching Program Metrics From 2007 to 2018 - PubMed

 

[winstonfoot5]

I am trying to better understand some of the major inflection points over the past several years which have affected younger rad oncs. The 2018(?) radiobiology exam and subsequent back forth with the ABR seems to be at the top of that list. I did not live through that and did not pay much attention to it. I was not on the BoD.

The radbio exam fiasco was indeed a major event.
But dont overlook other "major inflection points" including the Shah blood bath Expanding the number of trainees in radiation oncology: has the pendulum swung too far? - PubMed and the oversupply discussions which have been published/discussed on ad nauseam

 

[elementaryschooleconomics]

Just my two cents: the Angoff method sounds reasonable, but there's a lot of noise in asking a small number of subject matter experts to estimate how well a minimally competent rad onc should score. Imagine that a "generous" expert rotates off and a "harsh" expert rotates on -- that would skew the results. Or, imagine that in 2019-2020 the subject matter expert wants to avoid a repeat debacle and tries to overcompensate by being generous -- of course no resident will complain, but the data shows that probably happened too (at 99% pass rates) and it defeats the whole point of Angoff scoring.

How would you fix the system? Here's one possible solution. Let's say that instead of creating one test each year, we create 25% of the test over 4 years. That means the same bank of questions, with their Angoff criterion scores, is distributed over 4 years in a randomized fashion (probably stratified by content area). That way, we can calibrate to how generous/harsh the subject expert matters were. We're still using the Angoff criteria, but we're able to detect if a batch of subject matter experts are being difficult. I think that if in 2018, we had known that questions were randomly sampled and this year's group truly performed worse than 2017, people would be able to accept it with that data. As it stands, there is no such data, and from an outsider's perspective it's much easier to imagine that the subject matter experts were calibrated wrong, not that one cohort of test takers was suddenly (and inexplicably) stupid.

Wow, this is a cool idea.

There's just...so many alternative ways to do our board certification process.

Years ago, discussing this with a senior faculty member, I was told "everyone agrees this is bad, but there's no motivation to fix it, because it's a lot of work for no reward for the people doing the work, so you should just suck it up and get it over with and forget it ever happened".

I think about that conversation a lot, because that logic explains so many things in Radiation Oncology.

 

[thecarbonionangle]

Wow, this is a cool idea.

There's just...so many alternative ways to do our board certification process.

Years ago, discussing this with a senior faculty member, I was told "everyone agrees this is bad, but there's no motivation to fix it, because it's a lot of work for no reward for the people doing the work, so you should just suck it up and get it over with and forget it ever happened".

I think about that conversation a lot, because that logic explains so many things in Radiation Oncology.

It is sad but not surprising. It is an issue with a lot of issues in society. All it takes often is for “good people” to do nothing. If many people who supposedly agree do nothing, we have no real “leadership”. It is a blatant admission of such. This had a huge influence in my views of the field and many people I know. This malaise has only grown stronger with repeated examples of complete lack of leadership.

 

[winstonfoot5]

Just to add a little context for the "peer reviewed" article. It was really Wallner, Kachnic, and Gerdeman using PRO to put out a hit piece on a class trainees. It cites only 6 references, including wikipedia and a personal conversation. It was reviewed and accepted in 3 days. It didn't end there.

A small group of ASTRO members later went on to lead a grassroots campaign to elect write-in Chelsea Pinnix to the ASTRO board over Kachnic for the education seat.

Just talking to many PDs, this event was not viewed as a one off. Rather it reflects a pattern of behavior by leadership that came to a head around this event, and people really noticed how these individuals reacted to discussion. Some very experienced PDs with a lot of med student mentees believe this pattern is a bigger deterrent to prospective med students than the perceived job market issues.

Unfortunately, Im not sure much has changed with leadership today, and this may be why there continues to be decreased interest in this field from US medical school graduates.

View attachment 374872

You really have to re-read it to believe it. Such a preposterous paper that is so incredibly out of touch with the constituents the organization (he leads) claims to serve - truly the higher you climb, the smaller your world becomes
"Fourth, generally available National Residency Match Program data suggest that, during the past decade, regardless of a belief within the radiation oncology community, trends in the quality of residents accepted for training have been drifting slightly downward." -PW
Commentary on: Thoughts on the American Board of Radiology Examinations and the Resident Experience in Radiation Oncology - PubMed

 

[DebtRising]

This is definitely the time to address this pressing issue.

 

[deleted1111261]

This is definitely the time to address this pressing issue.

I visualize deadpan Aubrey Plaza
With severe audible eye roll at end.

 

[deleted1168505]

But dont overlook other "major inflection points" including the Shah blood bath Expanding the number of trainees in radiation oncology: has the pendulum swung too far? - PubMed and the oversupply discussions which have been published/discussed on ad nauseam

Nah. Lets just direclty ignore that topic for the ten thousandth time and distract with something else instead...
Whose cocaine was it in the white house? What... What was that? A question about UFOs...? Glad you asked...

 

[thecarbonionangle]

People who are actually good leaders who care about the field, trainees, even if issues will not affect THEIR career, are the ones we need. We don’t need more careerism, look at the pictures of my family, im really just here for another title junk.this is what we constantly get and why nothing ever gets done. Yeah i don’t care that it is “your turn”.

 

[deleted1111261]

People who are actually good leaders who care about the field, trainees, even if issues will not affect THEIR career, are the ones we need. We don’t need more careerism, look at the pictures of my family, im really just here for another title junk.this is what we constantly get and why nothing ever gets done. Yeah i don’t care that it is “your turn”.

Cmon can’t we have caring leaders and family pics?? Cramping my style

 

[elementaryschooleconomics]

Manipulating the cut line to not anger people is the greater of two evils?

"Evil", in this context, is subjective.

How would you define "evil" here?

This event catalyzed the collapse of American Radiation Oncology's reputation.

We are unlikely to ever fully recover.

We haven't even slightly recovered. Those exams were 5 years ago. Even partial recovery is years away, perhaps a decade or more.





Was it worth it?

 

[Dare2Dream]

One thing i remember is that no mid year retake was offered (at least initially). Not sure if that changed later after further requests or if residents had to wait a full year.

 

[Rad Onc SK]

I am hoping I can get some help from this forum, which is the main (self-serving) reason I joined.

I am trying to better understand some of the major inflection points over the past several years which have affected younger rad oncs. The 2018(?) radiobiology exam and subsequent back forth with the ABR seems to be at the top of that list. I did not live through that and did not pay much attention to it. I was not on the BoD.

(As a reminder, I am STILL not on the ASTRO BoD, despite what many here seem to think. That starts at the AM in October.)

Would/could anyone summarize the high points from the residents perspective? I suspect that this topic has a potential to get off the rails quickly, but if we could stick to objective facts and keep personal names (and shaming) out of the thread, I would personally appreciate that very much!

Thanks in advance

SK

Thanks! This has been helpful. I appreciate the feedback.

 

[evilbooyaa]

On the topic of the workforce, we keep blowing past the fact that SCAROP is running an anti-trust racket against it's own trainees. When I explain this to non-medical friends, they think I am making it up because it seems so obviously illegal. There was an outpouring of support last summer from a huge range of radiation oncologists of all ages when I brought this up on Twitter. Another society even reached out to discuss solutions to this problem. I promise you that action on this issue would be nearly universally appreciated, maybe except by the chairs running the racket.

When I brought it up last summer, I had an exchange with Emily Wilson from ASTRO. She at first tried to lie about the SCAROP survey purchase policy, and those shenanigans were documented on Twitter at the time. Once we started having a real conversation about buying the survey, here is what she said to me on August 24, 2022:

"As you can imagine, the salary information was given by the institutions with the understanding that it would not be made public, so it will remain with SCAROP members. But as I mentioned, I am hoping that the SCAROP leaders will agree to creating some kind of executive summary with trends and regional snapshots. More to come after the next SCAROP leadership call next month. If there is specific information you are looking for in such an exec summary, let me know."

(Ha, yes Emily, I can imagine. This is one reason why we have anti-trust laws in the US. Anyway...)

I told her no. First of all, I couldn't even see the table of contents, so it was hard for me to pick out specific parts. But, really I wanted the whole report.

In December, this article was submitted to Advances: https://www.advancesradonc.org/article/S2452-1094(23)00039-8/pdf. It is supported by SCAROP, as stated.

This paper is making important claims. I know some of the authors and think they are good scientists, but this paper is unusual in that it so copiously reports salary difference risk ratios without a single absolute number in the entire manuscript. If I reviewed for ASTRO, I would have brought this up in review. I want to be clear that I do not refute the claims in the paper. However, if I was a graduating resident, these data in a SCAROP funded study that contains only risk ratios, I'd be a little curious about what was left out of this paper.

When I saw this article, I emailed Advances to see if I could have access to the data. It was in pre-publication and no data sharing statement was published. Im sure you will be shocked to hear the answer was no, I could not access the data.

An excerpt from the ASTRO Journals Data Sharing Policy: "In the interest of transparency and in support of Findable, Accessible, Interoperable, and Reusable (FAIR) data principles, however, authors are asked to include a data availability statement with their submitted work"

Okay, thats FAIR (ha puns). Here is the data availability statement from the paper, now fully published: "Research data are not available at this time."

Here are some examples of data sharing statements ASTRO gives, you decide if this is a high quality sharing statement: Data Sharing - ASTRO Journals - American Society for Radiation Oncology (ASTRO)

Do you think that anyone should accept these events as an honest response to a serious concern about wage fixing?

Keep in mind at the time, I was an ASTRO member who had put in many hours of volunteer effort and worked for the newly elected ASTRO president. That is probably as close as I will ever get to the ASTRO secret society and it wasn't enough.

We are all very smart adults here. How many "root cause analyses" of these inflection points need to point to leadership for people to accept that we have a leadership problem?

We have a leadership problem.

We have a leadership problem.

Maybe if I say it one more time someone will pop out of my mirror and help?

I would support a re-vitalization effort of this on Twitter and perhaps some open transparent communication from ASTRO* on this pressing issue to so many radiation oncologists.


*LOL I MADE MYSELF LAUGH AT THAT ONE

 

[deleted1168505]

 

[elementaryschooleconomics]

Thanks! This has been helpful. I appreciate the feedback.

 

[Mandelin Rain]

My memory:

Wallner had voiced some reservations about the training at “small” programs just prior to the 2018 debacle, setting an arbitrary level of 6 residents as too small to exist. This gained mild traction here, but nowhere else. Note: Wallner hires/oversees a lot of rad oncs and new grads in his business and sees results of tests (obviously)

Test day 2018. We started hearing reports of a cancer biology test (opposed to rad bio test) and an incredibly hard physics test. In the 5+ years prior, we had not heard as much immediate consternation.

Results start to trickle in and it became clear it was an outlier from previous. Looked like about 50% of takers failed one, or both.

Program directors get upset. Residents get upset. ABR get indignant. Cite their angoff methodology and that their hands are tied.

Wallner et al. Publish article stating resident quality is down citing Wikipedia. No actual evidence presented, but wouldn’t you know it, the 6 resident cutoff was again reported. Seems like this was an agenda item.

Speculation here that Wallner was “trying to save specialty” by closing programs amid oversupply concerns. Nope. He starts threading posters here and lactating mothers with lawsuits for their speech. More likely, the larger programs had recalls for repeat questions which allowed them higher pass rates. ABR recently embroiled in recall problem on the diagnostic side, so this would NEVER be investigated.

ARRO definitively disproves the “declining quality” argument. Basically told to pound sand. Kachnic doubles down at ASTRO.

Nothing ever happens to the benefit of the residents, and lo… everyone is smart again the following year.

Obvious impacts on stress, time, job search, monetary, etc…. For those who failed. No consequences for ABR. Wagons circled and all that.

Plenty of “small” (and large) residency programs still out there using residents as cheaper PAs.

 

[Mandelin Rain]

Also, I venture to guess if you asked practicing radiation oncologists, including those who failed in 2018, if this issue remained in their top-5 concerns about the health of the specialty moving forward, roughly 100% would say “no”

 

[thecarbonionangle]

One cannot overstate how bad this was for the relationship between newer rad oncs and the specialty organizations. What began as extreme frustration with ABR “leadership” (PW and LK) spilled into ASTRO as LK DOUBLED DOWN at ASTRO, ASTRO “leaders” did nothing, you had leaders like Steinberg weigh in supporting the other “leaders”, lactating mothers being threatened by PW, ASTRO members being threatened and banned for expressing their views.
I know a good amount of people who completely disengaged after that and have very negative feelings toward the specialty, the “leadership”, the SCAROP chairs etc etc. the whole thing was just absolutely disgusting! We have a crisis of leadership in this field and it is killing us! Yeah so much for “diversity” and caring about our women!

Is it fixable? Maybe but it will take a lot more than just listening. It will take a lot of doing and LEADING.

 

[medgator]

One cannot overstate how bad this was for the relationship between newer rad oncs and the specialty organizations. What began as extreme frustration with ABR “leadership” (PW and LK) spilled into ASTRO as LK DOUBLED DOWN at ASTRO, ASTRO “leaders” did nothing, you had leaders like Steinberg weigh in supporting the other “leaders”, lactating mothers being threatened by PW, ASTRO members being threatened and banned for expressing their views.
I know a good amount of people who completely disengaged after that and have very negative feelings toward the specialty, the “leadership”, the SCAROP chairs etc etc. the whole thing was just absolutely disgusting! We have a crisis of leadership on this field and it is killing us!

Is it fixable? Maybe but it will take a lot more than just listening. It will take a lot of doing and LEADING.

I pretty much cancelled my ASTRO membership not long after that. I know i wasn't the only one.

They were just a disgusting and vile organization at that point, given how terrible they were historically for those of us out in community and private practice in terms of how we were viewed and how they lobbied for (against) us.

 

[NotMattSpraker]

Ha, Im now sitting here wondering if this thread was some kind of trap

Cool.

 

[sirspamalot]

I cancelled ASTRO years ago. Lots of dough, no show.

Buh bye!

Platitudes and I'm a nice guy.. but here's why you should consider ROCR? Nothing new, same sandwich as last year. Taste like..

 

[communitydoc13]

It’s a fair question, ABR faced a really tough situation.

When faced with a known anomaly of scoring, I do think doubling down on the validity of the test and calling the examinees lower quality is relatively evil (classically speaking) among the range of options.

I did some more digging on this. Angoff is widely but not uniformly used and there are significant questions about validity. (Testing is just hard period and assessing real competence is even harder.)

I'm guessing ABR was surprised by the results and scrambled for a response. I personally think they had lots of options and they chose the worst one (hindsight is 20/20). They chose to stick with their Angoff cut score and then blame the residents.

I do firmly believe that no matter what you do, full disclosure is the best policy. I do think either of the following would have been acceptable, and admittedly, no solution is perfect.

1. They could have abandoned the Angoff cut score as it resulted in an aberrant result.

They could have then graded on a curve with some sort of normalization based on collections of past performances. I think this is reasonable as long as they did not pick a curve based on individual test takers identities (worst joint center resident passes or something like that). They should have then disclosed what occurred and encouraged changes in curriculum at the residency level with specific guidance.

2. They could have stuck with their cut score, but then explained the aberrant result with positive and reasonable messaging. For example:

" We understand that an unusually large number of residents did not pass this year's subject exams. The exams were different in content this year, however, we employed our longstanding validation process for determining cut-off scores and basic competence. We believe that the disparity regarding pass rate this year is likely due to a combination of new material, the longstanding effects of test-bank style preparation and inadequate adjustments made in didactics at the residency level. We encourage residency programs to adjust their curriculum appropriately and will be providing guidance for this going forward. We will provide short interval opportunity for non-passing residents to remediate their scores prior to graduation and the typical job search deadline."

What happened later is even more baffling. They go ahead and pass essentially everyone in the following two years, completely obliterating their own narrative of inferior test takers, as well as eliminating the testing as a meaningful measure of competence (no test of competence should be passing everyone in a class of 200). Whether this implies some impact of prior controversy of Angoff panel members' scoring or other phenomenon, I don't know, but it's clear as mud that the class of 2018 was not dumber than the classes that followed.

Ha, Im now sitting here wondering if this thread was some kind of trap

Yeah, between this thread and the "Dickstein Prompt" on the twitter thread, I'm pretty sure that the establishment is trolling us and counting on us to expose ourselves as a bunch of prurient outliers with long memories and penchant for di&k jokes...

...I mean, if the shoe fits

I like to think of us as "the best people".

 

[dieABRdie]

There may also be a deeper thing going on here with regards to rad-bio. (Just a hypothesis here). Rad-bio as tested is no longer the foundational rad-bio from Gordon Steel's Basic Clinical Radiobiology text. Even when I took the exam (roughly 5 year before this cohort) it was becoming about oncogenes and tumor suppressor genes and pathways. If this is the stuff that is viewed as an essential foundation for radiation oncologists, maybe radiation oncologists should be giving targeted systemic therapy.

This is definitely one of the reasons for that catastrophic outcome. I actually have a heavy cancer molecular biology background... but unfortunately I REALLY suck at math. I usually put a decimal in the wrong place.... but I always **** up some mundane detail.

So when there was a whole bunch of genes, molecular targets and actual molecular biology lab tests..... man I was happy as a pig in ****. There were only a few classic rad bio calculations. I flew through that bitch and walked on out with a skip in my step.

My co residents.... they had faces of sheer terror when I saw them afterward. They passed. But all had to change their underwear afterward.

I think the worst part for me of the process was sending cover letter after cover letter (tailored for that place); and applying for the jobs; only to have very few options and get not one bit of ability to negotiate; or not hear back at all. It was a really long road to get there, and fairly discouraging to have so much struggle to find something that felt right. There were three times I almost just threw my arms up and took a crappy job; and at the last minute changed my mind. I've landed where I could not be happier; but it was a three year period that seriously felt like a decade.

These years were 2018 - 2020. The Covid pandemic did really change things a bit by culling the heard; once things opened, seems like it got a bit better. Its definintely better than those years, but if this for some reason doesn't work out. I would probably not have enough soul left to go back into the job market. Maybe put my toes in the water.... but I would defninitely do something else if it was cold.

 

[elementaryschooleconomics]

This is definitely one of the reasons for that catastrophic outcome. I actually have a heavy cancer molecular biology background... but unfortunately I REALLY suck at math. I usually put a decimal in the wrong place.... but I always **** up some mundane detail.

So when there was a whole bunch of genes, molecular targets and actual molecular biology lab tests..... man I was happy as a pig in ****. There were only a few classic rad bio calculations. I flew through that bitch and walked on out with a skip in my step.

My co residents.... they had faces of sheer terror when I saw them afterward. They passed. But all had to change their underwear afterward.

I think the worst part for me of the process was sending cover letter after cover letter (tailored for that place); and applying for the jobs; only to have very few options and get not one bit of ability to negotiate; or not hear back at all. It was a really long road to get there, and fairly discouraging to have so much struggle to find something that felt right. There were three times I almost just threw my arms up and took a crappy job; and at the last minute changed my mind. I've landed where I could not be happier; but it was a three year period that seriously felt like a decade.

These years were 2018 - 2020. The Covid pandemic did really change things a bit by culling the heard; once things opened, seems like it got a bit better. Its definintely better than those years, but if this for some reason doesn't work out. I would probably not have enough soul left to go back into the job market. Maybe put my toes in the water.... but I would defninitely do something else if it was cold.

This. This story and others like it is why I feel white-hot rage when establishment RadOncs benefiting from status quo respond to oversupply concerns with "the unemployment rate is low, everything is fine".

Judging the job market by the unemployment rate for Radiation Oncology is as ridiculous as saying "the terrorist attacks in 2001 only involved a couple buildings". While technically true, it misses the whole point.

While it's distasteful to say plainly, it needs to be acknowledged now much our specialty benefitted from the pandemic.

It triggered a lot of people to either retire, or leave clinical practice, or engage in lateral job moves, etc. It will artificially boost the job market for a few years.

Not only that, the pandemic stopped APM implementation.

American Radiation Oncology is like...hanging on by making wishes on a cursed monkey paw. IMRT saved us 20 years ago but catalyzed intense government scrutiny we couldn't handle, and enabled weak leaders to bloat the specialty unsustainably.

Then, a pandemic shut the whole world down, but stopped a draconian payment model and caused a dramatic job market shakeup.

Can we get off the roller coaster please?

 

[Mandelin Rain]

I honestly can’t believe we all forgot that Amdur and Lee published an article in PRO like two months before the debacle questioning the usefulness and steps needed for initial certification in PRO. Implied it was actually detrimental to education as everyone had to teach to the test.

https://www.practicalradonc.org/article/S1879-8500(18)30080-8/fulltext

Wallner responded in same ePub.

2 months later half of that resident class is failed. Totally legit, imo.

let’s wonder what is more likely.1. A set of top tier med students turned residents got dumb for a day only or 2. A guy whose primary business went bankrupt and settled massive fraud cases, working for an organization that was already embroiled in public confidence crisis on the dx rad side, had his role called into question within that same org by a couple giants in the field so he threw a hissy fit a failed a bunch of said top tier brains.

The real issue was no one within the specialty leadership at the time had the will/backbone to stop it.

 

[fiji128]

I took oral boards in person during that years academic cycle.

They used to give a little intro presentation and Wallner gave ours. He talked about conflicts and how we should tell them if we have an examiner where there might be a conflict. Then he made a joke about the job market and said something along the lines of “don’t worry, you all can just come work for me”.

I remember thinking “wow this dude is really professional, we are so blessed”.

lol, I think I was in the same room.

 

[Mandelin Rain]

 

[RadOncDoc21]

I took oral boards in person during that years academic cycle.

They used to give a little intro presentation and Wallner gave ours. He talked about conflicts and how we should tell them if we have an examiner where there might be a conflict. Then he made a joke about the job market and said something along the lines of “don’t worry, you all can just come work for me”.

I remember thinking “wow this dude is really professional, we are so blessed”.

 

[TheWallnerus]

I took oral boards in person during that years academic cycle.

They used to give a little intro presentation and Wallner gave ours. He talked about conflicts and how we should tell them if we have an examiner where there might be a conflict. Then he made a joke about the job market and said something along the lines of “don’t worry, you all can just come work for me”.

I remember thinking “wow this dude is really professional, we are so blessed”.

Paul W is a weird amalgam of Greta Thunberg, Tracy Enid Flick, James Comey, Emperor Palpatine, and a kiln of farts.

 

[thecarbonionangle]

Paul W is a weird amalgam of Greta Thunberg, Tracy Enid Flick, James Comey, Emperor Palpatine, and a kiln of farts.

“Oh so you are going to complain to ABR? I am THE ABR, bitch!”