Recent Big Psych Replication Problems

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
Is the ~20% attrition rate VA-specific? I honestly haven't looked into it, but can imagine there are multiple factors that would contribute to attrition and/or poorer outcome (e.g., questionable/inaccurate PTSD diagnoses, incentive to not improve, experiences Sanman and WisNeuro mentioned).

I saw the 20% in a review of PE a few weeks ago (I don’t have it in front me), but I don’t think the cited research was VA-specific.

One study I just looked up for a veteran population (Tuerk et al., 2011), sample of 65 total, found that 34% of vets completed less than 6 sessions (this was considered the attrition cutoff; this means they completed only up to 3 actual exposure sessions when you include the first two introductory sessions). A second study with virtual reality PE vs imaginal exposure (Reger et al., 2016) had a 40-44% attrition rate for various reasons.

These numbers seem to more accurately reflect what folks have said in here about strong avoidance and incentive to not improve (ie losing service connectedness/pay, which happened with a client I’d worked with and he vaguely threatened violence to the psychologist who did the comp & pen eval).
 
In my experience, there is a lot of discussion among vets that PE is "dangerous." I heard that from more than a handful of Vets.

Wow, that’s too bad. If facing your fears is dangerous, I guess healing and growth are dangerous, too.
 
One bit of good news is that some of the 'big dogs' in the field have recognized that the 'protocol for syndrome' approach has many serious issues as a main approach to treatment planning and implementation. One recently published book that summarizes their position (which is mostly to re-focus efforts on ensuring actual 'competencies' in assessment/treatment and implementing transdiagnostic principles of behavior change in context) is Process-Based CBT--The Science and Core Competencies of Cognitive Behavioral Therapy.

Have you read this book? It looks great from the index, and I'm teaching a course next semester for first year grad students on intro to therapy practices and this might be great. Just wanting to get any input beyond the amazon reviews...
 
Have you read this book? It looks great from the index, and I'm teaching a course next semester for first year grad students on intro to therapy practices and this might be great. Just wanting to get any input beyond the amazon reviews...
Yeah, I've read the first several chapters and sampled various others later in the book on particular methods/interventions. I think it's good and I notice that it's doubled in price (at least on Amazon) since I bought it. I truly think this is the 'next wave' of change with respect to how 'evidence-based' psychotherapy practices are taught and supervised (as do the authors).
 
Yeah, I've read the first several chapters and sampled various others later in the book on particular methods/interventions. I think it's good and I notice that it's doubled in price (at least on Amazon) since I bought it. I truly think this is the 'next wave' of change with respect to how 'evidence-based' psychotherapy practices are taught and supervised (as do the authors).

THanks! I just ordered the Kindle version to read in my spare time before I figure out my required books for spring. I agree with this approach in general--of teaching evidence based processes rather than "packages" for disorders; this is already my approach for supervision and I sometimes get pushback from the (newer, more rigid) students. The trick, of course, will be helping these newer students figure out how to integrate these strategies with case conceptualization so they don't just run to technique without considering aspects of the person and context.
 
I think in general the attrition rate for PTSD EBPs (PE and CPT) is 20%. I've even heard 40% for PE. But drop out, per research, is more related to things like scheduling and homework completion issues than difficulties tolerating the treatment.

Concerning the research discussion: someone I knew on fellowship had already completed another research fellowship beforehand. And I knew other research psychologist-hopefuls that completed multiple post docs. That also scared me.
 
I agree with what you say.

Additionally, I'd like to highlight that there is no such thing as a flawless study in psychology or health research. We are always leveraging validity with reliability against pragmatics of studying people.

Consequentially, I often run into psychologists questioning the validity of all RCTs/evidence because of this and falling back on clinical judgment or intuition or the dodo bird effect. My point is that while all studies are flawed some are much more flawed than others. Similarly, some theories are much more flawed than others. While there is still a lot clear up, we do have very strong evidence of certain psychological phenomenon and certain psychological treatments.

So please, no more energy therapies, acupuncture, or repressed memories (many more I could add) {steps down from soap box}

Why go to school at all if one is just working on intuition?

FYI: the WHO has a statement on acupuncture that says something like, “it’s hard to reconcile the idea of affecting chi with science”.

FYI II: did you know that David Palmer, the founder of chiro said that he learned about chiro from a ghost named Jim Atkinson? Sadly this is true.
 
Why go to school at all if one is just working on intuition?

FYI: the WHO has a statement on acupuncture that says something like, “it’s hard to reconcile the idea of affecting chi with science”.

FYI II: did you know that David Palmer, the founder of chiro said that he learned about chiro from a ghost named Jim Atkinson? Sadly this is true.
For me it's more about trying to artfully combine the idealized 'roadmap' of how a person gets from, say, 'clinically depressed,' to 'recovered'
I think in general the attrition rate for PTSD EBPs (PE and CPT) is 20%. I've even heard 40% for PE. But drop out, per research, is more related to things like scheduling and homework completion issues than difficulties tolerating the treatment.

Concerning the research discussion: someone I knew on fellowship had already completed another research fellowship beforehand. And I knew other research psychologist-hopefuls that completed multiple post docs. That also scared me.
One of the issues I have is that VA National (and, by extension, all of VA administration down to the local level) appears to be pursuing 'full rollout' of the EBT protocols under the assumption that the main (or even only) barrier to EBT uptake is the clinician at point of care, i.e., that the clinician is somehow choosing not to encourage veterans to engage in a protocol or somehow (I suppose, due to laziness) sabotaging the process or at least not giving a full effort to enroll their patients in these protocols. In my view, this is a result of the organizational schemas (core beliefs) that are often enacted by VA administration that basically fall under the umbrella of a core belief that 'If something isn't working, it's the provider's fault.' Unfortunately, prior to doing therapy on the 'front lines' of a post-deployment clinic for the past four years, I kinda bought into this assumption (and that's what it is...an assumption) as well. What I have found is that the vast majority of the time the barrier is on the client-side rather than the therapist-side of the equation. I actively encourage (with a blend of psychoeducation, motivational interviewing, and even leveraging of the therapist-client relationship as appropriate) full participation of the veterans I work with in an EBP protocol (say, CPT or PE for PTSD and even the full range of other protocols available at our facility). There are multiple reasons why a veteran would choose to decline an offered course of an EBT for their mental health condition and some of them relate to other systems-level issues. For example, in my clinic, I have 'program manager' folks accepting consults for psychotherapy from veterans who live up to FOUR HOURS drive away from our facility when there is a CBOC 10 mins away from them and they even have to pass another two CBOCs on their way here. This is a ridiculous barrier to weekly therapy. But when I raise the issue, they say, 'it's the veteran's preference and we respect veteran choice.' My perspective that it is inimical to the prospect of full engagement in a weekly psychotherapy process falls on deaf ears as does the empirical observation that I've never had a single case who had to travel so far for 'psychotherapy' that lasted more than 1-3 or so sessions. There are other realities that VA admin refuse to acknowledge such as the obvious reality that some veterans do choose to malinger or at least amplify their symptom self-report for various reasons (avoiding employment, avoiding responsibilities, generous benefits for themselves/spouses/children, etc.) and this, I suspect, does serve as a disincentive to participate in any rigorous process of treatment/assessment of their underlying service-connected condition. And of course there's the garden variety avoidance that is part and parcel of PTSD (and, frankly, any psychopathological process) to any trauma-focused therapy approach. Working through this--even skillfully utilizing techniques that are appropriate to such reluctance--is often anything but a straightforward process with anything like 100% efficacy. It's my observation that the more a person in the system is removed from daily provision of direct care to patients who walk into an open access mental health clinic, the more they are likely to be facile with their response to any lack of 'uptake' on the part of veterans being that somehow the provider hasn't 'done their job' or been 'convincing enough' in their psychoeducational efforts. Of course we're imperfect as providers and of course we could always improve anything we're doing clinically. However, I think that there is a pretty big gulf between the assumptions of some of the 'higher level' MH folks within the VA system and what actually happens routinely at the point of care delivery.
 
FYI II: did you know that David Palmer, the founder of chiro said that he learned about chiro from a ghost named Jim Atkinson? Sadly this is true.
I had no idea but I am not shocked.
 
For me it's more about trying to artfully combine the idealized 'roadmap' of how a person gets from, say, 'clinically depressed,' to 'recovered'

One of the issues I have is that VA National (and, by extension, all of VA administration down to the local level) appears to be pursuing 'full rollout' of the EBT protocols under the assumption that the main (or even only) barrier to EBT uptake is the clinician at point of care, i.e., that the clinician is somehow choosing not to encourage veterans to engage in a protocol or somehow (I suppose, due to laziness) sabotaging the process or at least not giving a full effort to enroll their patients in these protocols. In my view, this is a result of the organizational schemas (core beliefs) that are often enacted by VA administration that basically fall under the umbrella of a core belief that 'If something isn't working, it's the provider's fault.' Unfortunately, prior to doing therapy on the 'front lines' of a post-deployment clinic for the past four years, I kinda bought into this assumption (and that's what it is...an assumption) as well. What I have found is that the vast majority of the time the barrier is on the client-side rather than the therapist-side of the equation. I actively encourage (with a blend of psychoeducation, motivational interviewing, and even leveraging of the therapist-client relationship as appropriate) full participation of the veterans I work with in an EBP protocol (say, CPT or PE for PTSD and even the full range of other protocols available at our facility). There are multiple reasons why a veteran would choose to decline an offered course of an EBT for their mental health condition and some of them relate to other systems-level issues. For example, in my clinic, I have 'program manager' folks accepting consults for psychotherapy from veterans who live up to FOUR HOURS drive away from our facility when there is a CBOC 10 mins away from them and they even have to pass another two CBOCs on their way here. This is a ridiculous barrier to weekly therapy. But when I raise the issue, they say, 'it's the veteran's preference and we respect veteran choice.' My perspective that it is inimical to the prospect of full engagement in a weekly psychotherapy process falls on deaf ears as does the empirical observation that I've never had a single case who had to travel so far for 'psychotherapy' that lasted more than 1-3 or so sessions. There are other realities that VA admin refuse to acknowledge such as the obvious reality that some veterans do choose to malinger or at least amplify their symptom self-report for various reasons (avoiding employment, avoiding responsibilities, generous benefits for themselves/spouses/children, etc.) and this, I suspect, does serve as a disincentive to participate in any rigorous process of treatment/assessment of their underlying service-connected condition. And of course there's the garden variety avoidance that is part and parcel of PTSD (and, frankly, any psychopathological process) to any trauma-focused therapy approach. Working through this--even skillfully utilizing techniques that are appropriate to such reluctance--is often anything but a straightforward process with anything like 100% efficacy. It's my observation that the more a person in the system is removed from daily provision of direct care to patients who walk into an open access mental health clinic, the more they are likely to be facile with their response to any lack of 'uptake' on the part of veterans being that somehow the provider hasn't 'done their job' or been 'convincing enough' in their psychoeducational efforts. Of course we're imperfect as providers and of course we could always improve anything we're doing clinically. However, I think that there is a pretty big gulf between the assumptions of some of the 'higher level' MH folks within the VA system and what actually happens routinely at the point of care delivery.

There should probably be a "VA grievances/issues" thread, as this is a recurring (valid) issue. I wish we could paste them all together somehow, because there have been many, many, many good points and testimonies over the past 3-4 years. Its stable, its national, it is a high starting salary considering most "employed" positions as a psychologist, it has research opps, you are protected from almost all malpractice issues (mostly useful if you are a program manger running a full-service SUDs service or something similar though), and it can be very functional and focused on actually practicing clinical psychological science and training. The variability of the latter point is disturbing though.

I did not experience non-clinical folks taking over key clinical admin positions, but I did of course notice the disconnect between what we said we do versus what clinicians are actually able to do given the resources/time they auth to do it. There was not as much focus as I would have wanted (during my time there) on time-limited treatment and various access and resource management issues. When I heard the VA actually had utilization management nurses and other associated staff, I was floored! Where are these people?! VA healthcare, as currently practiced, desperately needs widespread utilization review by someone. More care is not "better care" and it is actually mostly harmful (i.e., poorer outcomes). This is NOT new and is well-known across non-VA healthcare settings. When I brought this up at various meetings, you might have thought I had just ran over a bus full of Catholic Nuns or something because we happened to be working with "veterans" as opposed to "normal" people. Seriously, how infantalizing is that?!
I also started to become skeptical of the PCMHI model (both its fundamentals AND how we had to practice it given the first issue I mentioned). They mandated a group therapy program for wife beaters when I was there. They used a euphemism for this, of course. I did not like having to constantly treat "doc, I can't sleep." I did not like treating, "I feel fine but my wife thinks I'm "different" after deployment." I did not like having to be "at a work station" from 8-4:30 no matter what.
 
Last edited:
There should probably be a "VA grievances/issues" thread, as this is a recurring (valid) issue. I wish we could paste them all together somehow, because there have been many, many, many good points and testimonies over the past 3-4 years. Its stable, its national, it is a high starting salary considering most "employed" positions as a psychologist, it has research opps, you are protected from almost all malpractice issues (mostly useful if you are a program manger running a full-service SUDs service or something similar though), and it can be very functional and focused on actually practicing clinical psychological science and training. The variability of the latter point is disturbing though.

I did not experience non-clinical folks taking over key clinical admin positions, but I did of course notice the disconnect between what we said we do versus what clinicians are actually able to do given the resources/time they auth to do it. There was not as much focus as I would have wanted (during my time there) on time-limited treatment and various access and resource management issues. When I heard the VA actually had utilization management nurses and other associated staff, I was floored! Where are these people?! VA healthcare, as currently practiced, desperately needs widespread utilization review by someone. More care is not "better care" and it is actually mostly harmful (i.e., poorer outcomes). This is NOT new and is well-known across non-VA healthcare settings. When I brought this up at various meetings, you might have thought I had just ran over a bus full of Catholic Nuns or something because we happened to be working with "veterans" as opposed to "normal" people. Seriously, how infantalizing is that?!
I also started to become skeptical of the PCMHI model (both its fundamentals AND how we had to practice it given the first issue I mentioned). They mandated a group therapy program for wife beaters when I was there. They used a euphemism for this, of course. I did not like having to constantly treat "doc, I can't sleep." I did not like treating, "I feel fine but my wife thinks I'm "different" after deployment." I did not like having to be "at a work station" from 8-4:30 no matter what.

MRT, right?

I'd be down for a VA venting thread. Or maybe a group PM. 🙂 Preview of my issues: if I never, ever hear the word "access" again, it will be too soon.

I think one difference about me vs. other VA providers is that I don't view Veterans (on that note, how silly is that we capitalize that word?) as gods or some incredible group of people. I just view them as normal people.
 
MRT, right?

I'd be down for a VA venting thread. Or maybe a group PM. 🙂 Preview of my issues: if I never, ever hear the word "access" again, it will be too soon.

I think one difference about me vs. other VA providers is that I don't view Veterans (on that note, how silly is that we capitalize that word?) as gods or some incredible group of people. I just view them as normal people.

What is MRT?
 
MRT, right?

I'd be down for a VA venting thread. Or maybe a group PM. 🙂 Preview of my issues: if I never, ever hear the word "access" again, it will be too soon.

I think one difference about me vs. other VA providers is that I don't view Veterans (on that note, how silly is that we capitalize that word?) as gods or some incredible group of people. I just view them as normal people.
Viewing them as normal people is appropriate and therapeutic. Holding people accountable for their behavior is a form of respect and is humanizing. I, too, dislike the taste of theKool-Aide, LOL.
 
This is the first I've heard of this supposedly evidence-based miracle. Will need to explore more but--I gotta say--the following sentence characterizing the approach seems a bit...naive and utopian/parental: "The therapist strives to reeducate clients socially, morally, and behaviorally and to instill appropriate goals, motivation and values."
 
This is the first I've heard of this supposedly evidence-based miracle. Will need to explore more but--I gotta say--the following sentence characterizing the approach seems a bit...naive and utopian/parental: "The therapist strives to reeducate clients socially, morally, and behaviorally and to instill appropriate goals, motivation and values."

MRT may be...ambitious but that quote is actually not that ridiculous in many therapeutic frameworks on careful consideration, especially in treating "personality disorders". Some day I will actually write rather than just ramble about wanting to write a philosophical paper examining third-wave/radical behaviorist theories and how they instantiate aspects of Aristotelian virtue ethics.

Learning to act successfully from wise mind is hard to distinguish in many respects from "being a better person."
 
Last edited:
MRT may be...ambitious but that quote is actually not that ridiculous in many therapeutic frameworks on careful consideration, especially in treating "personality disorders". Some day I will actually write rather than just ramble about wanting to write a philosophical paper examining third-wave/radical behaviorist theories and how they instantiate aspects of Aristotelian virtue ethics.

Learning to act successfully from wise mind is hard to distinguish in many respects from "being a better person."

That HIGHLY depends on the individual’s idea of “better”. Teaching the rules to someone only works if they are motivated to abide by them. There will always be a subset of people who look to exploit. It is a mistake to think otherwise.

Someone looked at the rules and decided Eddie Gaedel was a wise move. I don’t know if that was “better”.

Kent Kiehl has regularly said that many of the individuals high on trait psychopathy know to say they feel remorse, but are unable to describe the emotion, and admit they just know they are supposed to say it.
 
That HIGHLY depends on the individual’s idea of “better”. Teaching the rules to someone only works if they are motivated to abide by them. There will always be a subset of people who look to exploit. It is a mistake to think otherwise.

Someone looked at the rules and decided Eddie Gaedel was a wise move. I don’t know if that was “better”.

Kent Kiehl has regularly said that many of the individuals high on trait psychopathy know to say they feel remorse, but are unable to describe the emotion, and admit they just know they are supposed to say it.

Virtue ethics is predicated on the idea that there is, for some specified place, time, culture, and social role, a describable moral paragon. It is intrinsically teleological so yes, you're right, it depends a lot on the details of "better", but the values of DBT definitely overlap with Aristotelian conceptions of virtue.

Your point about being able to say it without really feeling it is also highly relevant. Virtue ethicists tend to argue that actual virtues involve taking the right actions for the right reasons. How bad faking it is depends on the virtue in question, but certainly for some plausibly faking it when you don't feel it is arguably vicious (i.e. the opposite of virtuous).
 
Yeah, Baudrillard is interesting and all.

But this is about a psychotherapy administered to individuals that have already broken the social contract.
 
Yeah, Baudrillard is interesting and all.

But this is about a psychotherapy administered to individuals that have already broken the social contract.

Nothing to do with Baudrillard or continental philosophy, I think you'll find.

Dunno who you work with clinically but 80% of the people in my clinic have broken the social contract in some significant way. I'm not a Szaszian but I do take the point that regulating behavior violating the social order is a major function of the mental health system. I do not really see how that is specific to IPV.

Also like I said, not trying to defend MRT at all, my point was much more about DBT and co. Thought "wise mind" gave that away, sorry I was not more clear.
 
Nothing to do with Baudrillard or continental philosophy, I think you'll find.

Dunno who you work with clinically but 80% of the people in my clinic have broken the social contract in some significant way. I'm not a Szaszian but I do take the point that regulating behavior violating the social order is a major function of the mental health system. I do not really see how that is specific to IPV.

Also like I said, not trying to defend MRT at all, my point was much more about DBT and co. Thought "wise mind" gave that away, sorry I was not more clear.

I think the concept of meaning being defined by the individual, to include virtue, was a clear reference to Baudrillard.


I highly disagree with your idea that mental health systems functions to regulate social contracts. I do believe that it is an enormous fallacy to assume that all desire to abide by the social contract. Many will violate that in awful ways without a second thought to the social contract. Many of those will feign as if they actually feel bad afterwards. But that’s just me.
 
I think the concept of meaning being defined by the individual, to include virtue, was a clear reference to Baudrillard.


I highly disagree with your idea that mental health systems functions to regulate social contracts. I do believe that it is an enormous fallacy to assume that all desire to abide by the social contract. Many will violate that in awful ways without a second thought to the social contract. Many of those will feign as if they actually feel bad afterwards. But that’s just me.

Nope, no relativism in anything I was talking about. The idea that different virtues are appropriate to different social roles in different contexts is not at all the same as saying that the individual has free reign to define meaning for themselves. A parent requires different virtues than, say, a soldier, but that is very much the opposite of saying that things are a free for all. Indeed, to the extent that some social roles are not entirely or at all voluntary, sometimes virtue/character ethics is going to require things of you whether you like it or not. The point is that universally applicable rules per se are going to fail as a way of trying to specify morality.

Yeah, no arguments here that plenty of people are not hugely invested in the current social order. What you are describing is absolutely vicious behavior in the vast majority of cases. My point was that many psychotherapies basically aim to instill virtue properly understood. If there existed a therapy that could actually do this reliably, it would correct the problem you identify quite precisely.

I am not saying MRT is that or that any therapies can do this. Please don't mistake a conceptual analysis for an accounting of facts on the ground or a brief for this particular therapy. Simply that the rhetoric of MRT quoted earlier actually correctly identifies the ultimate goal (irrespective of whether it can ever accomplish this) and that this rhetoric is not out of line with the implicit (and sometimes explicit) goal of other therapeutic modalities on my analysis.

f we are both clear on that not sure where our point of disagreement is, really..

(As an aside, I rather think Aristotle and Thomas Aquinas get intellectual priority over anyone French if we are going to be assigning credit for these things!)
 
Last edited:
Nope, no relativism in anything I was talking about. The idea that different virtues are appropriate to different social roles in different contexts is not at all the same as saying that the individual has free reign to define meaning for themselves. A parent requires different virtues than, say, a soldier, but that is very much the opposite of saying that things are a free for all. Indeed, to the extent that some social roles are not entirely or at all voluntary, sometimes virtue/character ethics is going to require things of you whether you like it or not. The point is that universally applicable rules per se are going to fail as a way of trying to specify morality.

Yeah, no arguments here that plenty of people are not hugely invested in the current social order. What you are describing is absolutely vicious behavior in the vast majority of cases. My point was that many psychotherapies basically aim to instill virtue properly understood. If there existed a therapy that could actually do this reliably, it would correct the problem you identify quite precisely.

I am not saying MRT is that or that any therapies can do this. Please don't mistake a conceptual analysis for an accounting of facts on the ground or a brief for this particular therapy. Simply that the rhetoric of MRT quoted earlier actually correctly identifies the ultimate goal (irrespective of whether it can ever accomplish this) and that this rhetoric is not out of line with the implicit (and sometimes explicit) goal of other therapeutic modalities on my analysis.

f we are both clear on that not sure where our point of disagreement is, really..

(As an aside, I rather think Aristotle and Thomas Aquinas get intellectual priority over anyone French if we are going to be assigning credit for these things!)

I think I am focused upon MRT or whatever andyou are focused upon more general psychotherapies.

Wittgenstein > Aquinas.
 
Top