I spend a good chunk of my time (see Salty's post above) at the VA affiliated with a large, urban, academic medical center with a large residency. I ended up here a bit by accident (within our system, it had the most ICU time available when I was finishing my fellowship). I didn't intend to stay as long as I have, but, well, it'll be 10 years in June. There are many opinions and comments made about working for the VA; some are true, and some are shades of truth. Many, you'll note, include something like "...at least that's how it was in residency (however many years ago)..."
Getting hired: Yeah, it's hard in that the fed has rules about what you can tell an applicant before they have an official, actual, offer letter. We have lost good applicants over the years to this. It is VERY frustrating, and emblamatic of larger hiring (and other administrative) inefficiencies in the system. These issues will make doing anything other than clinical work very difficult. The VA gets a lot of things right, but, man, it is HARD to get things done.
Working there: I can only speak for my VA, but, by and large, the clinical work and environment is high quality, collegial, and even fun. I teach residents, work with CRNAs who are dedicated and capable, and do some of my own cases. We have amazing technicians, solid equipment (new GE machines, 5 ultrasounds for 12 locations, a Glide in every room), surgeons who aren't jerks, and a good amount of espirit de corps. It's true that the VA OR will staff for a certain number of rooms at a certain hour, so if you're not going to be "down to X number of rooms by Y time," your case might not go. Keep in mind a couple of things. We are a capitated healthcare system, meaning there's a budget, and everything we do takes money out of that budget. If you have to pay overtime because the surgeon didn't show up on time and went long (this happens), or because the case was "really hard," or because they let the med student close, well, that costs money. Also, keep in mind there's no overtime (or even set # of hours) for the MDs, so if we all stay late, it's for free. Do that too much, and there's attrition (see above about hiring). Believe me, it is MUCH easier to say yes and do a case, but you have to have the people. What I really like about working there is that we do big cases on sick, deserving people, but in a small shop where people know one another, and where people's incentives and motives are MOSTLY aligned. I do research with a mentor at the U, and get invited a few times a year to speak at a thing. I spent 2 years at 50% clinical to work with the Lean process improvement group. I write a chapter in a named anesthesia text book. Which is all just to say, there are opportunities for career development in this kind of position.
The benefits: Benefits are described publicly, and salaries are a matter of public record. I can easily compare my pay with that of colleagues who are more primarily paid by the U, and there's no doubt the U salaries are mostly a bit higher. I think I work about the same number of hours that they do, but I know the hours I work are less stressful than theirs. I'm at most 2:1 (they go 3:1 at times), and it's rare that both of my rooms are super intense. Vacation is 5+ weeks per year + 3-ish weeks sick leave + meeting time (which is poorly defined and not really limited if it's legit; many of us in academics have a lot of conferences), + the new 12 weeks paid family leave. Well-subsidized health/vision/dental. The retirement has been debated quite a bit here and in other posts. I can say that the 401K is very well managed with good funds at low fees, and a strong match. The pension I think has changed over time, in that newer employees are paying a larger amount toward it (I'm not paying anywhere near what is described in other posts above). It is 1% times your high 3, times your years of service. If you stay more than 20 or beyond 62, it becomes 1.1%. So for the sake of argument, if you end your career at 400K and spend 25 years, you'd have $110K per year.
The bad/good: Sometimes people think you're a week candidate because you work in a VA. Even within our own department, the VA docs are kind of the red-headed step-children, so to speak. There is a perception that people who work here couldn't cut it elsewhere, or are lazy, or whatever. I'm happy to put my CV up for consideration, but I think most of us are here because, like the OP, we like the Vets, and believe that they deserve to be cared for by smart, capable accomplished people like ourselves. I really like that I never once have to think about whether the patient is insured. I can go talk to the patient's cardiologist in person. I like that I never feel taken advantage of for a surgeon's financial gain; the surgeon isn't making more money to book this appy at night, so if it's going, the patient needs it. Putting it all together, I feel like it's been a good fit for me. For many people, the pace might feel slow, and that they'd be willing to work harder/faster/more for more money. Those jobs are 100% out there. But for a university faculty type job with decent opportunities, decent lifestyle, relatively pleasant working conditions, fair pay, and no one in a suit telling me how high to jump? Not bad.