Certain government and non-profit jobs are eligible for the
PSLF program as well.
Veterinarians can take on lots of jobs across multiple government agencies. I've met vets who work with the USDA, CDC, FDA, and DHS at the federal level. I've met veterinarians that work for state agriculture departments as well as state health departments. I've also met veterinarians that work for city and county health departments. Also, there are vets who work with international programs like WHO and OIE.
In terms of the actual job titles people can hold:
Veterinary Medical Officer (VMO)
State Public Health Veterinarian
State Veterinarian
Area Veterinarian in Charge (AVIC)
Searching for those titles specifically will give you some ideas about what a veterinarian does in government work.
Also, there are many vets who work in upper level management positions across every level of government that don't necessarily have the work title of "veterinarian" (like county health commissioner, director of this or that, etc.).