Would it be worth rotating through a rad onc department when one of the weeks would be during ASTRO?

not really if all the big boys will be out of town; not many consults and no one to see you shine.
But if its one week out of say four, i wouldnt worry too much (lots of time to relax and read).