Physics & Radbio

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
Done deal. It’s time for a lawsuit and publicity. This is absolutely appalling. Please PM if interested.

Members don't see this ad.
 
I just can't yet believe that after all this, the advocacy, the awesome ARRO letter, all the emails, Wallner/Kachnik could just get up, spout the same rhetoric they've been saying all along, which we all know is garbage and biased, and then shut down the few little attempts at comments? Its like we made it easy for them. They barely had to put up a fight at all, and just had to read the content of the letter they already wrote but in speech form.
Really? Is that it? Is he that intimidating and all-powerful that the PDs, ARRO, and all of us just lay down and put our tails between our legs? I just can't accept that one bitter man has so much control over all of our lives. What about the masses not just "asking" for a re-score but demanding it? What about a much-needed changing of the gaurd?
ARRO, PDs, what is the next step? (Short of lawsuit).
 
Members don't see this ad :)
Sounds like the field is SOL. there is zero true leadership at the very top.
 
I don't know what you all thought was going to happen at the circus that is ASTRO.
PDs are mostly junior faculty trying to climb the ladder of the popularity contest that is academics, which means playing politics and carefully navigating pathologic personalities. Challenge the ABR's big lie? To what end? I am sure many mean well and want to help but can't risk self-immolation.

Walking around this meeting and overhearing conversation, I have never felt so out of place. Our field is filled with truly odd, insecure, and jealous people. Academics attract the oddest of the odd. What is a normal person to do? I suppose go present a retrospective review "poster" to a group of people who don't care. What a waste. I know that there are a handful of us who can see through all the b.s., and as much as I'd like to believe we are really a silent majority, I sadly not really sure anymore.

The ABR wants to break it down between big and small programs. Why stop there guys? Why not break it down between sex, race, age, socioeconomic status, undergrad institution, political affiliation, marriage status, etc. I am sure if they looked hard enough they could find all sorts of shocking ways to group people who failed. But there's only one grouping they care about.

The next step? We need to fix this system that allows the ABR to administer physics/bio repeat exams the day before the clinical. That is absurd and setting people up for failure.

These exams test how well you can cram for a test and memorize a load of useless and easily forgettable material, not how competent you are. It's step 1 all over again. We all know that. The ABR will never admit it. The idea you could pull a PGY-5 aside in clinic on any random day and expect him to be able to perfectly draw the MAPK/ERK pathway is laughable.

What we can do is keep pushing them to separate the dates between the physics/bio and clinical exam. There should be a one month buffer between them, minimum. There is no reason to have them on sequential days unless the ABR wants to purposely punish and set a higher bar for failing candidates. We should ask the ABR to administer the bio/physics exam in July and the clinical in August. There is no reasonable objection they could offer to this. We need to push them to admit that sequential exams are unfairly punitive and is a problem that is easily solved by separating the exams by a month. This solves the biggest problem, and we should focus on this before moving on to the higher level problem of the exam content and scoring.

Reality? Not going to happen. They want the class of 2019 to have repeat failures on bio/physics, new failures on clinical, and a 2-3 year delay in getting to orals. With higher bars at each step, including orals, hopefully getting a handful of us past the 6 year hard limit on board eligibility. They used these exams to identify their "small number" of residents that they want forced out of the field (or at least into the locums pool driving rates even further down) and build Wallner's proscription list of programs that need to be shut down.

This scandal has killed any enthusiasm I had left after enduring the pointless tribulations of residency. It's really sad. I know people are suffering through some serious depression and anxiety over this affecting their personal and professional lives. Not even the tiniest effort to help or compromise from the ABR. Not a shred of compassion, even. Instead they patronize and humiliate the 60% of us not in the big programs. I'm so over it. ASTRO is a joke and I doubt I'll come back any time soon after I graduate.
 
  • Angry
  • Like
Reactions: 1 users
I don't know what you all thought was going to happen at the circus that is ASTRO.
PDs are mostly junior faculty trying to climb the ladder of the popularity contest that is academics, which means playing politics and carefully navigating pathologic personalities. Challenge the ABR's big lie? To what end? I am sure many mean well and want to help but can't risk self-immolation.

Walking around this meeting and overhearing conversation, I have never felt so out of place. Our field is filled with truly odd, insecure, and jealous people. Academics attract the oddest of the odd. What is a normal person to do? I suppose go present a retrospective review "poster" to a group of people who don't care. What a waste. I know that there are a handful of us who can see through all the b.s., and as much as I'd like to believe we are really a silent majority, I sadly not really sure anymore.

The ABR wants to break it down between big and small programs. Why stop there guys? Why not break it down between sex, race, age, socioeconomic status, undergrad institution, political affiliation, marriage status, etc. I am sure if they looked hard enough they could find all sorts of shocking ways to group people who failed. But there's only one grouping they care about.

The next step? We need to fix this system that allows the ABR to administer physics/bio repeat exams the day before the clinical. That is absurd and setting people up for failure.

These exams test how well you can cram for a test and memorize a load of useless and easily forgettable material, not how competent you are. It's step 1 all over again. We all know that. The ABR will never admit it. The idea you could pull a PGY-5 aside in clinic on any random day and expect him to be able to perfectly draw the MAPK/ERK pathway is laughable.

What we can do is keep pushing them to separate the dates between the physics/bio and clinical exam. There should be a one month buffer between them, minimum. There is no reason to have them on sequential days unless the ABR wants to purposely punish and set a higher bar for failing candidates. We should ask the ABR to administer the bio/physics exam in July and the clinical in August. There is no reasonable objection they could offer to this. We need to push them to admit that sequential exams are unfairly punitive and is a problem that is easily solved by separating the exams by a month. This solves the biggest problem, and we should focus on this before moving on to the higher level problem of the exam content and scoring.

Reality? Not going to happen. They want the class of 2019 to have repeat failures on bio/physics, new failures on clinical, and a 2-3 year delay in getting to orals. With higher bars at each step, including orals, hopefully getting a handful of us past the 6 year hard limit on board eligibility. They used these exams to identify their "small number" of residents that they want forced out of the field (or at least into the locums pool driving rates even further down) and build Wallner's proscription list of programs that need to be shut down.

This scandal has killed any enthusiasm I had left after enduring the pointless tribulations of residency. It's really sad. I know people are suffering through some serious depression and anxiety over this affecting their personal and professional lives. Not even the tiniest effort to help or compromise from the ABR. Not a shred of compassion, even. Instead they patronize and humiliate the 60% of us not in the big programs. I'm so over it. ASTRO is a joke and I doubt I'll come back any time soon after I graduate.

In my more naive days, I used to think that observed oddity in this field was a sign of underlying brilliance...boy have I learned my lesson. I have since found it’s nothing more than a group of petty, narcissistic, socially awkward, synchophants. It’s as if they’re still fighting the same battles they were in high school and undergrad. Truly a pathetic bunch all around.
 
  • Like
Reactions: 1 user
Couple things I heard second-hand:

Wallner wasn't there.
Kachnic defended the position as said above. There was apparently some discussion of potentially moving the re-take to sometime a little bit earlier than clinical (so it wasn't back to back days), but nothing finalized.
Kachnic apparently frustrated that she's getting blasted for this as she's just the second in command for this.
No plans for mid year re-take.
Kachnic shut down suggestions of "simply increase the pass rate" to 85-90%.
Small programs (< 6 residents) only had statistically significant results compared to large programs (> 16 residents). There were 4 columns in terms of resident split, only the first and 4th column were 'significantly' apart.
Apparently some maybe better resources for study might come out? However, apparently when they state that 'cancer biology' or 'immunology' are testable concepts, they mean ALL of cancer biology and immunology.
Angoff method may have been influenced by increased clinical physician input into the exams.
No analysis of what percentage of old exam questions (40-60% any given year) were correctely answered.

My takeaways:
What a joke.
Apparently we're all only supposed to train at MSKCC, MDACC, HROP, Stanford, Emory, or other places with 16+ residents. Or expand programs until they hit 16 residents, at which point we'll just osmosis all the rad bio and physics we need from each other.
Kachnic is likely just the fall gal given Wallner didn't even have the decency to show up.
Better have a PhD in both cancer biology AND immunology AND whatever other topics are listed in the 'study guide' if you wanna be sure you know enough material to pass.
Wish somebody had discussed the hit piece Wallner and Kachnic co-authored in PRO as a conspiracy.

If your PD went to ASTRO, I encourage you to ask them directly what the proposed solution to any of this will be, as they can potentially post more details.
 
  • Like
  • Wow
Reactions: 1 users
They really should not be allowed to hold these positions without any term limits. Kachnic and Wallner need to go from these positions. How many positions does kachnic need? She’s leading the plenary and trials sessions, she doesn’t give a single F about this educational position she’s stated as much and people say she laughs when people bring up this boards issue. People need to figure out a way to rid of both of them
 
Angoff method may have been influenced by increased clinical physician input into the exams.

Also was not there, but just want to confirm I heard this too last week from some people involved.

It is pretty lame that Wallner did not show up. He should have been there.
 
Couple things I heard second-hand:

Wallner wasn't there.
Kachnic defended the position as said above. There was apparently some discussion of potentially moving the re-take to sometime a little bit earlier than clinical (so it wasn't back to back days), but nothing finalized.
Kachnic apparently frustrated that she's getting blasted for this as she's just the second in command for this.
No plans for mid year re-take.
Kachnic shut down suggestions of "simply increase the pass rate" to 85-90%.
Small programs (< 6 residents) only had statistically significant results compared to large programs (> 16 residents). There were 4 columns in terms of resident split, only the first and 4th column were 'significantly' apart.
Apparently some maybe better resources for study might come out? However, apparently when they state that 'cancer biology' or 'immunology' are testable concepts, they mean ALL of cancer biology and immunology.
Angoff method may have been influenced by increased clinical physician input into the exams.
No analysis of what percentage of old exam questions (40-60% any given year) were correctely answered.

My takeaways:
What a joke.
Apparently we're all only supposed to train at MSKCC, MDACC, HROP, Stanford, Emory, or other places with 16+ residents. Or expand programs until they hit 16 residents, at which point we'll just osmosis all the rad bio and physics we need from each other.
Kachnic is likely just the fall gal given Wallner didn't even have the decency to show up.
Better have a PhD in both cancer biology AND immunology AND whatever other topics are listed in the 'study guide' if you wanna be sure you know enough material to pass.
Wish somebody had discussed the hit piece Wallner and Kachnic co-authored in PRO as a conspiracy.

If your PD went to ASTRO, I encourage you to ask them directly what the proposed solution to any of this will be, as they can potentially post more details.

It was pretty insulting to hear all of this live at the ARRO session.

Heard via some other sources at the meeting that the Angoff method wasn't even done correctly this year. True? Who knows but this whole thing is pretty ridiculous. Just waiting for the impossible clinical boards this year to double down on the class of 2019's "stupidity".
 
. Just waiting for the impossible clinical boards this year to double down on the class of 2019's "stupidity".

Can't even imagine how malignant the oral boards will be in a couple years....traditionally that was the one test most of us worried about.

At this rate, they might as well just keep baseball bats in the examination hotel rooms for the examiners... would be a lot quicker for the examinees.
 
Can't even imagine how malignant the oral boards will be in a couple years....traditionally that was the one test most of us worried about.

At this rate, they might as well just keep baseball bats in the examination hotel rooms for the examiners... would be a lot quicker for the examinees.

Can someone please elaborate on how the angoff score may have been done incorrectly this year? Did they change the point value for questions that have been used in the past? That should absolutely be illegal for them to do.

I still don’t understand how/why we have no power to force the ABR to demonstrate that this was a fair exam. The Onus should be on them. It shouldn’t be a “take our word for it” situation. If they changed the value of questions compared to years prior (which wouldnt surprising) or if our class performed similarly on questions used on prior exams they should be forced to re-evaluate.
 
For anyone who hasn't, listen to Kachni's ARRO presentation. Its recorded on the virtual meeting. Mind boggling.

Can someone please elaborate on how the angoff score may have been done incorrectly this year? Did they change the point value for questions that have been used in the past? That should absolutely be illegal for them to do.

I still don’t understand how/why we have no power to force the ABR to demonstrate that this was a fair exam. The Onus should be on them. It shouldn’t be a “take our word for it” situation. If they changed the value of questions compared to years prior (which wouldnt surprising) or if our class performed similarly on questions used on prior exams they should be forced to re-evaluate.
 
Members don't see this ad :)
I have a hard time believing she knew that was going to be recorded.
 
For anyone who hasn't, listen to Kachni's ARRO presentation. Its recorded on the virtual meeting. Mind boggling.

Is there a link? It doesn’t seem to be posted on the ASTRO website yet that I can see.
I have a hard time believing she knew that was going to be recorded.

Anyone have the transcript? I’m curious but didn’t go to astro this year and don’t feel like shelling out $$ for the virtual meeting. Gotta save that for when I have to retake this stupid thing.
 
When people tell you who they are, BELIEVE THEM!! Her and Wallner need to go. I feel sorry for the academic department she “leads”.
 
In the main room, LK managed to say "for those who don't have the courage to come up front, there is an overflow room". Its one piece of the LK puzzle. pretty sure she is the bully, and a poor leader. reminds me of trump's attitude, to be honest.
 
Thanks for letting us know about the ARRO presentation.

There was not a single mention of any concern on their end about problems with exam content or scoring. Everything is fine according to them - they can't explain the "blip" this year, which they don't seem to think is a very big blip because "passing standards" didn't change that much. Residents were reluctant to ask any questions. Somebody finally stood up and politely asked them to prove that the exam was reliable and valid by comparing to old exams. She said she would ask if she was allowed to share this data.

Quick notes I took:

- Lisa Kachnic has been the victim of cyberbullying. Somebody once claimed she had a private jet. She volunteers her time to give back to us, her children, because of the mentoring she received at her time at Harvard (nb: This is one of the oldest tricks in the instruction book for bullies: Pretend to be the victim in order to manipulate those you are bullying and to avoid changing your behavior. Happens all the time, especially in professional/workplace settings). It is telling that this is how she chose to start this conversation. Notably, Paul Wallner did the same thing in his emails that were posted earlier in this thread where he claimed he was the subject of "attacks."

- Cost to produce exams "outweighs" cost we pay for them.
$150-300 cost to develop an exam item question. (no reference at all as to where this number came from)

- Only charge in 2018 was the addition of more clinicians to the radiation AND CANCER BIOLOGY committee (note the repeated new emphasis on cancer biology by the ABR).

- A "bunch of very intelligent PhDs - this is all they do and learned at the ABR" psychometrically review questions. She can't answer any questions on this because she is not a PhD in psychometrics. It took Lisa Kachnic 10 years to learn what the Angoff method meant. (I realize this was somewhat in jest, but seriously? You are at the helm and you thought this was an appropriate thing to say?)

- Exam is valid because everyone sits in a room and votes with cards they hold up (showed a picture of this) whether a "minimally competent resident" would get it correct.

- For first-time takers:
Clinical pass rate was 84% in 2017 (she remarked that it was funny nobody complained about this)
Physics 88% in 2017, Bio 88% in 2017

(these are the first time I have seen these numbers revealed).

- 2018 test results are valid because the passing standard was within the range of the past 5 year range.

- ABR was confused why results were lower this year, so did some digging and figured out there are heterogeneous program sizes and basic science teaching.
They realized that ASTRO core curriculum was "out-dated"

- Biology performance by program size (of course first-time takers only - they never share the lower overall rate):
4-7 residents: 65%
8-9 residents: 79%
10-14 residents: 86%
16+ residents: 95%
Physics was similar

- ABR wants to help and this is what they want to do:
Improve study guides
Provide updated reference sources
Make ASTRO update update their core curriculum
Ask Hall and Giaccia to update their book to include more cancer biology and she admited that cancer biology is more important now. (This was especially hilarious because an update with minor revisions literally came out after this year's exam and Giaccia is on the bio committee - I guess he just didn't notice)
Encourage ACGME to fix physics core curriculum.
Investigate the possibility of giving some time inbetween the clinical and basic science exams ("a couple of days to a week"), notably they can't do it THIS YEAR, but can do it for later years. They want to do it, but PearsonVUE won't let them.

- 60% of questions are reused old questions and Angoff standards are recalculated every year.
In-service exam questions are not correlated and appear to be more difficult than ABR exam.


The above mostly speaks for itself, but I'll add some comments of my own:
- She spoke rather smugly throughout the entire presentation. She referred to us as children jokingly at first, but then I realized that it might not be completely a joke given the ongoing patronizing manner of discourse.
- The defense of the passing standard continues to be insultingly asinine.
- It seems stupidly obvious that larger programs are unfairly benefiting from recall strategies by virtue of having more residents taking the exam each year, something she actually admitted towards the end of this presentation (it was subtle, but if you listen closely, she did), given that 60% of the exam has re-used questions. When the new 40% are outrageously difficult questions, this benefit is greatly multiplied. Old questions need to be discarded completely to prevent recall strategies and level the playing field between large and small programs. Of course, I am sure the ABR with its severe lack of funds can't afford to do this.
- The most infuriating thing about this is their insistence that what was tested is what a "minimally competent radiation oncologist would know." Anybody who took this exam and saw the absurd content on it knows that this is an outright lie.
- LOL @ this being due to "more clinicians involved with the bio exam" Those questions were not anything any practicing clinician would know. Pure basic science trivia. Knowing low-level enzyme names is not clinically relevant. This is a flat out lie to people who did not take the exam and are not able to see the absurdity of the questions.
- I also want to know the breakdown between big programs and small programs in terms of answering old vs. new questions on the bio exam. Thank you to whoever asked a similar question as it is really key. Hopefully she will be "allowed" to release these statstics (which, like all other "data" presented so far could be simply made up as it's not verifiable).
- I feel more certain than ever that they are manipulating the exam results at a high level. There is no oversight. We can't see the questions, we can't verify the numbers, we can't do anything. It's all a secret. They act like this and they expect us to just take them at their word? You've got to be kidding me.
- There is no reason these exams can only be given once a year. Lots of other professional licensing exams are given throughout the year with questions pulled from an extensive question bank. Is the USMLE only given once per year? Their precious Angoff method with annual volunteer-get-together question writing session (which I'm sure they are all paying for out of their own pockets) is not the only way to generate a reliable exam. Hell, the FAA written exam is generated by pulling 60 random questions from a testbank of 1000+ questions (which is available openly for study -- imagine that -- knowing what you could be tested on!) -- this is for the people that fly us 500 mph blindly through clouds to our pointless meetings, no biggie right?
- In short, nothing but lies and excuses. I don't blame the residents there for not taking a stand. These are petty, nasty, and vindictive people, and I have zero doubt they would go out of their way to harm you for challenging them more aggressively. It's like she was almost daring you to challenge her, as some sort of way to assert their absolute power and dominance over us.

Bill Maher said in a recent episode regarding political leadership, "power begets power." I thought immediately of the ABR: "Power is a lot like owning rabbits: The more you have the easier it is to get a lot more."

So sure, the ABR needs drastic change in leadership and examination philosophy. Yet nobody has the power to do this but them, and it is fairly obvious that they will use their power to exercise even greater power over us in the future. If anybody has a solution, I'm all ears.
 
  • Like
  • Angry
Reactions: 8 users
A few thoughts:

1) Shout out to the woman who asked the first question in the session after LK spoke. Her question was succinct and exceedingly reasonable. For those who didn't attend the session (I wasn't able to attend) or listen to the 'virtual meeting' presentation, she asked LK if the ABR had compared the performance on the 'recycled' questions questions which appeared on the 2018 exams and the prior exams. In other words, did 2018 examinees answer these questions correctly at the same rate as prior years? LK did not have an answer (she "left out the slides") but assured us additional results would be presented during ADROP (the whisperings did not include this in their review).

2) LK's explicit acknowledgement of recalls playing a role in larger programs is really, really frustrating. More residents = more people to 'inform' radiobiology instructors after taking rad bio and physics = better instruction in larger programs = better pass rates in larger programs. I have trouble believing she would say this out loud knowing the session was recorded. I'm not in the the pro-lawsuit group because I hold out hope this thing will correct itself when enough PDs/chairs complain, but this is the kind of thing that very strongly suggests an unfair advantage for certain residents in larger programs.

3) The alleged disconnect between ABR and ASTRO is a farce. It's the same people who are on committees and task forces and research groups. It's like they pretend there's a firewall and they don't talk to each other professionally or otherwise. As such, they can shrug off the (very reasonable) request for better exam prep materials.

Then again, maybe these three points are really just cyber bullying...
 
If it wouldn't get me in trouble, I would have loved to ask her a few questions from the rad bio test in front of everyone...Minimally competent my butt.
 
  • Like
  • Haha
Reactions: 2 users
A few thoughts:

1) Shout out to the woman who asked the first question in the session after LK spoke. Her question was succinct and exceedingly reasonable. For those who didn't attend the session (I wasn't able to attend) or listen to the 'virtual meeting' presentation, she asked LK if the ABR had compared the performance on the 'recycled' questions questions which appeared on the 2018 exams and the prior exams. In other words, did 2018 examinees answer these questions correctly at the same rate as prior years? LK did not have an answer (she "left out the slides") but assured us additional results would be presented during ADROP (the whisperings did not include this in their review).

2) LK's explicit acknowledgement of recalls playing a role in larger programs is really, really frustrating. More residents = more people to 'inform' radiobiology instructors after taking rad bio and physics = better instruction in larger programs = better pass rates in larger programs. I have trouble believing she would say this out loud knowing the session was recorded. I'm not in the the pro-lawsuit group because I hold out hope this thing will correct itself when enough PDs/chairs complain, but this is the kind of thing that very strongly suggests an unfair advantage for certain residents in larger programs.

3) The alleged disconnect between ABR and ASTRO is a farce. It's the same people who are on committees and task forces and research groups. It's like they pretend there's a firewall and they don't talk to each other professionally or otherwise. As such, they can shrug off the (very reasonable) request for better exam prep materials.

Then again, maybe these three points are really just cyber bullying...

I recently "re-cerftified" and am good for almost another decade but the way things are going I'm not sure if I'll still be doing this a decade from now and I very well may have taken my last exam and interaction with the ABR, so I don't have much "skin the game" but I feel so absolutely awful for you guys, many if not most of whom are way smarter and harder working than I am (and the "leaders" in our field) and any event (knowing what it takes to be an excellent radiation oncologist vs what it takes to land a residency position and now actually get a job) would be excellent physicians. I can't imagine working so hard for so many years and doing everything you could to succeed and being screwed like this, with so many hurdles still to go (retaking this exam, then clinical written). Then you have to take oral boards, which I honestly conditioned many years ago due to what I firmly believe was bias and a malignant examiner and that was many years ago so who knows what it's like now and in the future and after likely fighting/scrambling for a job you don't want in a location you can't stand.

All of this is obviously terrible for the field and in that sense affects me of course but I just really feel so awful for you guys, especially since all of this was totally unnecessary and due to lack of leadership. For what it's worth coming from some random lowly community practitioner you'll most likely never meet (I went to my last ASTRO meeting many years ago) all of you residents have my deepest sympathies (not you medical students though . . . seriously if still go into this field you deserve whatever you have coming in 8-10 years).

PS: I just remembered why I replied to this post - excellent point above in #2. Even many years ago I heard first hand accounts of the examines in many programs walking out of the exam and going to directly to lunch with multiple junior residents who sat there feverishly taking notes as the people who took the exam ten minutes ago vomited every specific question and then any additional piece of information they could. I can imagine that if you have multiple residents many can recall specific questions in their entirety and among themselves they can piece together much of the exam. Do this year after year (and "trade" with other schools) for an exam where 40% of questions are recycled and I bet many people know most of the exam before they take it. While this should clearly be illegal, I never really considered how while not as blatant as what is mentioned in #2 it is very possible that after the exam the PhD who teaches the course says "so how did it go?" and the test takers say "I was surprised there were so many questions about x and not about y" and the course instructor naturally adjusts the lectures for next year accordingly. The effect would obviously multiply for larger programs.

At the very least I think you guys should demand to know correct answers to recycled questions for small vs larger programs
 
The whole idea of residents taking a rad bio exam, and then telling their instructors "hey you really should cover these additional 5-10 topics, since they were on the test this year and we didn't learn about them" is in the same spirit as a recall, right? Why is that felt to be OK?

I still want to see the data that stratifies exam results based on presence of a faculty member on the exam committee. I imagine that will also be positive. Obviously doesn't fit their narrative of 'hurr hurr small programs need to get geud', but that's what anybody who knows anything about research would ask in regards to their 'conclusions'.
 
The whole idea of residents taking a rad bio exam, and then telling their instructors "hey you really should cover these additional 5-10 topics, since they were on the test this year and we didn't learn about them" is in the same spirit as a recall, right? Why is that felt to be OK?

I still want to see the data that stratifies exam results based on presence of a faculty member on the exam committee. I imagine that will also be positive. Obviously doesn't fit their narrative of 'hurr hurr small programs need to get geud', but that's what anybody who knows anything about research would ask in regards to their 'conclusions'.

Totally agree and I think this point needs to emphasized. It's almost the same thing but in any event totally unfair.
 
Does the ABR actually talk about how residents create and save recalls for all the written board exams? Was that discussed at all at the ASTRO session?
 
  • Like
Reactions: 1 user
"Maybe this is a phenomenon of larger programs I don’t know, but the residents talk. You know… you’re not suppose to have recalls and there are no recalls but you kind of talk about what is on the exam, and a couple years back as cancer biology was coming on the exam our residents would say to our radiobiologist that teaches the course… you know… you really need to cover this. So, we get not all programs have this so we will go back with all of the stakeholders to make sure what you need to prepare will be there."

Lisa Kachnic, ASTRO 2018
 
  • Wow
Reactions: 1 user
Lisa,

Thank you for admitting direct cause for an unjust examination while being recorded.

Lawsuit.

"Maybe this is a phenomenon of larger programs I don’t know, but the residents talk. You know… you’re not suppose to have recalls and there are no recalls but you kind of talk about what is on the exam, and a couple years back as cancer biology was coming on the exam our residents would say to our radiobiologist that teaches the course… you know… you really need to cover this. So, we get not all programs have this so we will go back with all of the stakeholders to make sure what you need to prepare will be there."

Lisa Kachnic, ASTRO 2018
 
  • Like
Reactions: 1 user
"Maybe this is a phenomenon of larger programs I don’t know, but the residents talk. You know… you’re not suppose to have recalls and there are no recalls but you kind of talk about what is on the exam, and a couple years back as cancer biology was coming on the exam our residents would say to our radiobiologist that teaches the course… you know… you really need to cover this. So, we get not all programs have this so we will go back with all of the stakeholders to make sure what you need to prepare will be there."

Lisa Kachnic, ASTRO 2018

Recalls by proxy
 
Kachnic: "You know… you’re not suppose to have recalls and there are no recalls"

There are no recalls? What if there is proof there were recalls? Would that be a big deal? How old would recalls have to be to "prove" there's still a problem with the exam?
 
Kachnic: "You know… you’re not suppose to have recalls and there are no recalls"

There are no recalls? What if there is proof there were recalls? Would that be a big deal? How old would recalls have to be to "prove" there's still a problem with the exam?
What she described is the definition of a recall. If an examinee tells their professor what is on the exam to help future students that is a recall. I am not sure how anyone could argue otherwise.

She also stated this may be a phenomenon of larger programs and then admitted in the last sentence that not all programs have access to this information and are at a disadvantage.

Nobody needs to find the smoking gun, this is the smoking gun.
 
  • Like
Reactions: 1 user
Basically, you can talk about topics on the test. However, specific question recall is illegal. See below for the official rules:

Examinee Guidelines

Examinees are expressly prohibited from disclosing, publishing, reproducing, or transmitting exam content, in whole or in part, in any form or by any means, verbal or written, electronic or mechanical, for any purpose. Examples of unauthorized use are listed below. This list is not all inclusive.

  • Accumulating memorized questions
  • Sharing recalled questions
  • Giving or receiving confidential exam information at any time prior to, during or after the exam administration (please refer to guidelines listed below)
  • Study materials in any media in an examinee's possession during the time period of an exam
For individuals (before and after an examination):
  1. It is permissible to discuss topics covered on the exam, the manner of test administration, and test-taking strategy. It is fine to say, "There is a lot of ________; I had almost no _______." Examinees should realize that each exam form is a sample of an overall domain, and the samples included on subsequent forms will be different.
  2. The objective is for examinees to study and master the knowledge, skill, and understanding required for practice. It is NOT permissible to memorize specific questions and answers that might be encountered on a future exam.
  3. It is NOT permissible to give or receive explicit recollection of exact questions, answer choices, and (supposed) correct answers. Both oral and written transmissions are prohibited by the policy.
For residency training programs (exam preparation activity):
  1. Exam preparation groups, as well as topical reviews, are permitted and encouraged as means of preparation for examinations. The writing of sample questions and sample examinations, and the simulation of the exam setting, are also permitted and encouraged, as long as the questions used are not recalled questions.
  2. The objective is for examinees to study and master the knowledge, skill, and understanding required for practice. The program should NOT facilitate or condone the memorization of specific questions and answers that might be encountered on a future exam.
  3. No collections of recalled items should be tolerated in any media format.
  4. Program directors should monitor their programs for evidence of violations in examination security and take necessary steps to prevent and stop this behavior.
 
Topic: There is a a lot of lymphoma translocation questions. I had almost no questions on uncommon translocations, only the most common ones
Question: What translocation is most common in Burkitt's lymphoma?

Either would be beneficial for recall purposes. Distinction without a difference.

What Lisa Kachnic is describing is a recall method. Residents go to their professor and say "teach this...". They didn't say "teach cancer biology." Cancer biology is everything. That would be like going to a program director and saying "I just took my clinical boards teach the residents about radiation oncology". That would be a worthless conversation.
 
There are lots of recalls. Idk if they are useful. Many programs have them. Bigger programs have throves of it from clinical, physics, rad bio and oral boards. they are definitely out there.
 
Im suprised no one is questioning the method of implementing the angoff method. Holding up cards in a room of your peers to rate questions is subject to bias. Hard questions are rated easier than they really are because of fear of looking dumb.
 
Last edited by a moderator:
The whole idea of residents taking a rad bio exam, and then telling their instructors "hey you really should cover these additional 5-10 topics, since they were on the test this year and we didn't learn about them" is in the same spirit as a recall, right? Why is that felt to be OK?

I'm the first to admit that this is kind of a gray area, but is allowed, technically. (See above.) Certainly less egregious than maintaining a multi-year stockpile of exactly-recalled exam questions, however.

Of course, this would be way less of an issue if the ABR bothered to come up with an in-depth biology curriculum (NOT a half-assed outline) and get the information to the actual instructors. Other than those who serve as their chief question writers, I seriously doubt the ABR could even name more than a couple of them nationally.
 
How about they actually wrote useful questions? The ABR is a money hungry scam. Large programs have tons of recalls. Professors who work for the exam allude to questions. This recent set of tests was bogus. And there lack of concern is the reason we’re taking more action.
 
Just being devil’s advocate here. Everyone is talking about how RO is overtraining people. Isn’t this actually something being done to cut down the clot of the ROs (albeit in an extremely sinister fashion if it’s actually done for this purpose).

There has been a glut of diagnostic radiology residency spots, which was somewhat resolved by making some of those spots into IR/DR spots. But one of the big issue radiology face is smaller community programs that have no business training DRs.

Would folks rather have 20 spots be divided up between MDA and MSK or MDA and MSK have 16 spots total between them and the next 4 go to random DO school’s rad onc department staffed by 4 attendings?
 
.

Would folks rather have 20 spots be divided up between MDA and MSK or MDA and MSK have 16 spots total between them and the next 4 go to random DO school’s rad onc department staffed by 4 attendings?

Not the situation in RO actually.

More like mdacc and msk getting 24 spots now and a bunch of random smaller university and community programs opening up 4-6 spot programs as well
 
Not the situation in RO actually.

More like mdacc and msk getting 24 spots now and a bunch of random smaller university and community programs opening up 4-6 spot programs as well

I guess the big question is then if 20-30% of the spots should go, should they come from smaller community programs or come from spots at Stanford?

Personally, I think the bottom 150-200 community rad spots should be eliminated because they do not provide adequate training.
 
What community programs are there? Most rad onc positions are affiliated with a university albeit some smaller than others.

I don’t think the education varies all that much, more like access to resources.
 
What community programs are there? .
A few off the top of my head: New York Methodist, Allegheny general, Beaumont (although to be fair, that's really an outlier in terms of its reputation).

There are a lot more "small" academic programs, some of which are recently established... Mississippi rings a bell.
 
  • Like
Reactions: 1 user
Unlike diagnostic radiology, rad onc doesn't really have a problem with programs flagrantly not training residents well. In general with only a few exceptions in the past, you'll be equally competent no matter what program you go to.

This assertion by the ABR that resident performance at bigger programs is due to poorer teaching and residency complacency is a complete lie. Resident quality is excellent throughout the field with little variation. We don't have IMGs, FMGs, DOs. Most residents came from top medical schools with top USMLE scores. It just isn't so. The ABR is lying.

What IS different between big programs and small programs is the capacity for exam recall use, as Lisa Kachnic inadvertently admitted when she said what we all already knew. I tried to bring this up earlier in the thread and nobody really seemed to care that much, but I think what really came from the ASTRO "meeting" was that the recalls are the issue.

My program of 4 residents does not have recalls. We don't "report back" to our instructors. Only one person takes the test each year. I barely failed rad bio. Now that we know 60% of the exam is recycled, I feel confident that had I had access to those recalls and used them, I would have passed. Likewise, I feel confident that many who passed using recalls would have otherwise failed.

The ABR has allowed this problem to happen. By recycling questions, they engender the creation of recalls. Period. If the ABR actually cared about people cheating, then they would write new tests each year. But they don't. They hate the small programs as evident by their raving publications on the matter, so it's just peachy that big programs are doing this. What this has done is reward the dishonest (cheating) residents and programs and harm the honest people. Recall use falsely elevates exam scores and misleads the test creators by biasing their opinions on what we "would know" per the Angoff method (funny how Lisa Kachnic wouldn't comment on any shortcomings of this method namely the glaringly obvious problem of over-confidence bias). As a result the honest people are punished by creep in question difficulty.

What can we do? I think a couple of things.
1. Mainstream media has been interested in the recall story before. No reason to think they wouldn't be again, especially since the ABR has gotten busted on it before. They wouldn't care about doctors failing a hard test. They would care about large programs conspiring to shut down the little ones by implementing cheating methods that benefit programs with larger numbers of residents. Especially when a maldistribution problem has been well-documented and the small programs are the ones that are filling the rural and midwestern need for rad oncs. Now the ABR wants everyone to be training at the big programs? Harvard grads are not typically going to work in Salina, KS.
2. We need to quantify recall use. What's out there, what are people doing, what kind of strategies are these programs using? If you know something, speak up. Use a VPN and post anonymously.
3. Someone could post recalled questions publicly and anonymously on a different website like reddit or something. This is really the only thing we can do to actually fight back (and it's no surprise the ABR forbids it as it is the only power we have to take back some control). Making the recalls public does two things: 1. It forces the ABR to create a fair test by generating 100% new questions each year, and 2. It levels the playing field for everyone and prevents under-qualified residents from passing while qualified residents fail.

Keeping recalls private allows cheating. Making them public prevents it.

I have never seen an ABR recall. If you have this, the power to do something is in your hands. We need whistle-blowers, people that are willing to make the recalls public and talk to the media. This is not ok, and the ABR has no right to bully us the way they are.

If we can prove that recalls were used, there may be a legitimate argument to invalidating this year's test.
 
Last edited:
Top