Yeah, from what I remember, the service dog study was pretty bad and seemed basically setup so that it was guaranteed to find a "therapeutic effect."
Here were the two MAJOR (and I mean MAJOR flaws in design--and there is no excuse because they had YEARS?/DECADES? and plenty of funds to plan and execute it):
1) there was no 'no-dog' condition. There were only two 'arms' of the study. a) let's give veterans a 'service dog' b) let's give veterans an 'emotional support dog.' There was no arm where they just did 'treatment as usual' or 'treatment with an already established EBP. And, if you read the manuscript, the 'rationale' they gave was utterly laughable. They claimed that it would be 'unethical' to not offer a dog as 'treatment' (mind you, they admitted in the introduction that it was an experimental 'treatment.' They could have easily had an actual (EBP) established treatment arm with some form of established EBP (CPT/PE) as a comparison group. They could have easily had an arm where they simply delayed giving them the dog for several weeks. What they did was give people either a service dog (trained to do hypothetically useful 'tasks') or an 'emotional support dog' (basically, a companion animal or pet). The basic findings were that veterans, over time, on average, in both groups 'improved' on their symptoms to a very mild/moderate degree over the course of the study (a few months? I can't remember) but nowhere near the degree of improvement generally observed when people do an EBP protocol or take medication. Which gets to the second godawful critique...
2) Not only did they not CONTROL factors such as, gee, are they actually doing an EBP before or during the course of the 'doggie' treatment?...they didn't even TRACK, or take a peek at CPRS to see what other treatments they were engaged in at the time of the study. That's right, as I read the manuscript (and I think they even tepidly address this 'limitation' in their discussion section)...we don't even know what--if any--medication or psychotherapy treatments the participants were engaged in while 'getting' the 'doggie' intervention. People could have been doing anything. Individual, group, CBT, EMDR, SSRI's, whatever. We just don't know and they didn't bother to look. Inexcusable.
My own personal theory is that the bigwigs were terrified at the possibility of Congress passing the PAWS act mandating service dogs for all vets with mental health disorders. Now, from what I've seen, the cost to train a service dog is in the TENS of THOUSANDS of dollars...like a price tag of 20k -40k or so PER VETERAN. Comparatively, providing veterans with an 'emotional support dog' would be the equivalent of hooking them up with a pet (considerably cheaper). So the clever folks at VA designed the study that would provide evidence of 'equivalence' between the 'efficacy' of service dogs and emotional support dogs so when PAWS is made into law, they could say, 'hyuck, hyuck, lookie here...we here at VA have 'evidence' of equivalence between the two and, here at VA, hyuck hyuck, we do 'the evidence-based medicine...yesiree.'
What truly needs to happen (if people are really interested in if service dogs 'work' or not) is that individual components (e.g., 'my puppy wakes me up from my nightmares and that 'helps') need to be independently and rigorously evaluated as to whether they help or hurt. For example, there's plenty of reason to believe that having a dog wake you up during all your nightmares would actually further FRAGMENT your sleep and even result in REM sleep deprivation and you could have MORE and more severe nightmares as a result and worse PTSD symptoms and quality of life. We simply don't know. It hasn't been tested empirically in well-designed research and with reference to the basic and clinical scientific literature. But because of the 'aaaaw...isn't that CUTE...isn't that PRECIOUS!' (the idea of your puppy waking you up and 'rescuing' you out of your nightmares) factor, it passes as a compelling argument because it is consonant with the cultural narrative of infantalizing adults in our society.
True scientists know that the strongest research designs are based on the principle of 'strong inference' whereby the study design is set up in such a way to pit two competing theories against one another and--no matter what the actual empirical results of the study are--there will be evidence that will be relatively more favorable to theory A or theory B at the end of the study. This study appeared to be designed to be 'weak' and to, if anything, only 'demonstrate' 'equivalence' (they are equivalently useless...or barely useful) between the 'interventions.'
