Like many have already said, the field is awesome. In comparison to many other fields the level of intellectual stimulation, pay, work life balance, and not to forget time with the patients cannot be beat. I do agree with other here that the compensation and the allure of PP of the field is going down, but much like you, I never experienced those things so it doesn't bother me too much. I have been told I could make $100-200k more, but honestly I am very comfortable with what I am making. Money should not be a major determining factor after $250k. Someone once told me that any doctor making over $400k is working HARD to make that kind of cash. I am very pleased with my non-Coast work situation.
On the flip side, it is true that it is very difficult to live in major cities or on the coast. If your family needs to be in LA, NYC, or DC it is not going to be good for you. Here is the catch, for many other fields it is the same thing. So unless you are thinking IM, OBGYN, or other PCP then it will be tough to get a good job in a major city anywhere (ENT, Urology, etc...). If your child needs to be by a pediatric specialist (my friends in a super specialized field in this situation) they had to move from their midwest home to another place. The other thing I don't like about this field is that we are really on the way bottom of the totem pole, but I just accept that for the lifestyle perks. If you can't do inpatient, night float, or need to work on the weekends, don't be surprised when your referring partners who have to rotate weekend rounds, inpatient shifts, or stand in the OR for 8 hours don't feel sorry for you as you are making just as much $$$ or more then them,while you are sitting at home enjoying life with your family.
Finally, RESEARCH SUCKS FOR RAD ONC. Listen to me carefully, you are in the ivory tower daze. You will likely NOT DO ANY MEANINGFUL RESEARCH AT ALL. You can do retrospective reviews (I do) and participate in clinical trials, but really rad oncs are good at knowing the clinical data and doing retrospective reviews, but there is a reason we get something like <5% of all NCI funding. For the rad oncs who actually do great research, they suck up all the funding (and they should b/c they are the ones who are proving they can move this field and give me job security - let's go OligoGOMEZ), and for you to try to break in this little teeny weeny field of real rad onc researchers overpopulated and saturated by MD PhDs - don't do it. Med Onc has WAY more REAL research opportunities and funding (think pharma for better or worse).
Don't confuse the JV competition between radiation oncology residents vs med onc fellows. I say we win that battle much more often then not in terms of average med student scores, publications, and trial knowledge, but the real researchers who play in the NCI, NIH, SWOG, / big boy leagues are dominated by med onc. I have absolutely ZERO intention of writing a grant or working in academia and probability wise you will not either, so the fact I am "not doing research" does not bother me at all.