Upside: Good pay, very good benefits, and it's hard to beat the value of a VA position on your CV, particularly for early career psychologists.
Downside: Politics, administrative red tape, bureaucracy.
The way psychology is treated as a discipline generally, and the way psychologists are treated individually varies widely by VISN, system, and facility.
Example: one psychologist I know of got hired as a director of a prestigious clinic at one of our other facilities. She got paired with an administrative assistant that was supposed to work for her, but she had no formal authority over, and worse, this admin person was personality disordered as well - but no one could figure out how to get rid of her. To make a long story short, this psychologist friend of mine quit her prestigious job basically because this administrative assistant was literally making her crazy. She works somewhere else now in an equally prestigious position, but likely makes less money.
Another example: I know of a psychologist who has the exact same position I do at a neighboring VISN. Unlike me, she has to record her time in three different ways (an excel spreadsheet, an "event capture" program, and through the electronic charting program we use), and she has to do her own MDS careplanning. Not only that, unlike where I work, her boss is a psychiatrist and doesn't seem to respect psychology as a discipline.
So, as you can see, not all VAs are the same.