I'm same volume as you and same mindset.
I want to clarify to the OP: If your goal is a single 2 week vacation back home a year, that's a reasonable goal. You can do it, but it's hard. There are a few practices where this basically won't be possible (I worked for one where basically nobody took vacation, they wouldn't pay for locums, and having a partner cover for you was frowned upon). Multiple trips >2 weeks each per year, this probably isn't a great field for you.
If you are a hospital employee, you will typically get 25-35 days of PTO in your contract then technically you can tell admin I will be gone for 3 weeks in a row these days, I'm not asking, I'm telling you -- It's your problem to deal with it now and find someone to fill in for me and if you don't you are in breach of contract. So far, I have not been able to do this in good conscience. For two main reasons: (1) I care about my patients, and (2) I don't have full faith that the hospital will fail to staff it and blow up my vacation plans at the last minute.
The other option is to work part-time/locums. I don't know how old you are, but if you don't need a full-time income and benefits, you can do short term locums positions for a few months at a time and set your own schedule. There are also practices where you can split the assignment (a few months at a time, typically in pretty remote locations). But unless you really love rad onc, why? Just do radiology or anesthesia.