Yes, PHS officers get many of the same benefits that active duty military get, including PX privileges, free health benefits, including dental (Tricare), and VA benefits (including VA home loan and the 9/11 GI bill). And yes, PHS get military pay, which includes a housing allowance and a subsistence allowance that is tax free. The housing allowance depends on where your duty station is, as well as rank, and whether or not you have dependents (there is only two rates in regards to dependents, either you have dependents or you don't, you don't get more pay per each dependent). For a better understanding of why USPHS is structured like the military and is one of the seven uniformed services of the federal government, see this nice concise essay about the militarization of PHS:
https://lhncbc.nlm.nih.gov/files/archive/pub2001060.pdf
Additional benefits for PHS pharmacists specifically is a $30,000 accession bonus (taxed) if you sign a 4-year contract (and since most people go in to PHS with the plan of doing a 20-year career, it rarely makes sense to not sign the 4-year contract and not get the bonus), an additional $1,250/month specialty pay (taxed), and if you are board certified, you get an additional $2000/year (taxed). Also after 20 years you can retire with 50% of your base salary (average of last 3 years of service) as your pension for life (this will change for anyone called to active duty after Jan 1, 2018). There are a bunch of other fringe benefits, like moving expenses are covered when you are first called to active duty if your new duty station is greater than a certain distance from your current residence, or whenever you relocate for a job (permanent change of station), 30 days of annual leave per year, space A travel on military aircraft, low-cost life insurance for you and your spouse, low-cost health insurance for your dependents, don't need to spend time planning what to wear to work everyday, opportunities to go on deployments and have a really unique and nontraditional pharmacy career, the sense of fulfillment and satisfaction you get from serving the greater good and protecting, promoting, and advancing the health and safety of the nation, etc.
This PDF provides a good run-down of the benefits and pay, although it is a couple of years old and so some of the numbers are outdated:
http://www.usphs.gov/docs/pdfs/bks/PBKS 2014 12-2-14.pdf
You can get updated numbers for base pay, housing and subsistence allowance, and more using this military pay calculator:
http://militarypay.defense.gov/Calculators/RMC-Calculator/
(use O-3 and less than two years of service to figure out the beginning salary of a pharmacist without any prior service, keep in mind that the total does not include pharmacist specialty pay or board certification pay)
You still have to put in a few years of service before your salary becomes comparable to the private sector, even with all the tax advantages. Although, keep in mind that the military pension cannot be beat by the private sector, unless you are maxing out your 401k plus build up other IRAs and are getting more than 10% returns , or you die really young so you don't get a chance to capitalize on the value of the pension... So PHS only really makes sense if you are dedicated to the long haul (and plan on living to an old age). Also, PHS only makes sense if you don't mind that you are limited to working in the agencies PHS pharmacists work in (IHS, BOP, and FDA being the top three, but we also have PHS pharmacists in CMS, NIH, CDC, SAMHSA, HRSA, USCG, ICE, DHA(DOD), and probably some more I'm forgetting). You don't really get moved around like in the military (unless you are detailed to the Coast Guard, they typically move you around every 4-5 years). For the most part, you have a choice about what jobs/locations you move to and when, but you are expected to move around and take on jobs with greater responsibilities and/or supervisory roles throughout your career, or you wont be able to move up in rank. You can sometimes work this out so you change jobs but stay in the same geographical area, although doing so severely limits your options unless you are in the DC area to begin with. If all you want to do is be a clinical pharmacist and you never want to do anything else, you can still do PHS, but don't expect to make it to captain (O-6). You also don't have to worry about being deployed unless you specifically and willingly decide to put yourself in a situation where you have a higher chance of being deployed, and most deployments are 30 days or less.
Regarding loan repayment, that is an agency specific program and not a PHS-wide program. PHS does not have any loan repayment programs for PHS pharmacy officers. The only PHS agency that still provides loan repayments to pharmacists is IHS. In order to qualify for loan repayments you have to work at specific, high need, IHS sites. You don't have to be a PHS officer to qualify for the loan repayment, civil servants qualify as well. IHS will pay up to $40,000 in loan repayment per 2 years of service commitment. More info here:
https://www.ihs.gov/loanrepayment/lrpbasics/