PhD/PsyD Is cognitive testing for ADHD pseudoscience?

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
Joined
Jul 15, 2014
Messages
1,890
Reaction score
2,411
So, I'm learning all kinds of things today. I'd like to hear more about what is known both from clinical judgement and the literature on the current state of cognitive testing for ADHD. Thanks, -R.M.

Members don't see this ad.
 
To make a quick reply: I wouldn't say it's pseudoscience, in that research does show that some people with ADHD perform more poorly (often relative to controls) on some neuropsychological measures. But I liken it to autism spectrum disorders in that there's no well-defined profile for ADHD, just like there's no particularly well-defined neuropsychological profile for diagnosis autism spectrum disorders. As a diagnostic tool, it's very limited.

Edit: Now, if research were able to consistently show that, for example, individuals with ADHD who perform poorly on neuropsychological testing have poorer outcomes/worse prognosis, are more likely to benefit from pharmacotherapy or cognitive/behavioral techniques, are more in need of academic and/or testing accommodations, etc., relative to individuals with ADHD who do not perform poorly on those tests, that could be clinically meaningful.
 
  • Like
Reactions: 3 users
We have already done this, at least twice in the past 2 years. Maybe we can combine threads so we don't have to repeat??? I will repeat some general summaries here:
____________________________________________________________________________________________________________________________________________________________

The most reliable and valid way to diagnose most psychiatric conditions is by using a structured clinical interview/examination (as opposed to one's own idiosyncratic clinical interviewer session...which is prone to lack of follow-up, distraction, laziness, poor question wording, lack of adherence to DSM criteria, and not attending to the "not otherwise accounted for by" clause of DSM), observation during session, and collateral information that is then supplemented by some normative/rating scale data. And yes, this might take more than 60 minutes. Sorry. This is not new, surprising, or controversial to anyone who is even the least bit up-to-date on the matter. Many psychologists, for whatever reason, seem not to understand this, even though it is well spelled out in the literature. The "test boundness" of some psychologists is, frankly, embarrassing.

Additional information is needed for some things and situations/cases though...of course.
________________________________________________________________________________________________________________________________________________________
Although it is possible for researchers to learn quite a bit about ADHD from small-to-modest mean differences in test performance, attention tests do not improve individual diagnostic accuracy because problems with attention (or any other cognitive function) are not part of the DSM criteria. And there is a reason for that.

It is common for clinical psychology supervisors not too care much about what is needed, efficient, discriminatory, or actually translatable to a treatment plan, etc when training grad students. They aren't billing for it and you need to learn how to give instruments anyway. So, why not? I dont really agree with this philosophy of training, but its the predominate one out there.
________________________________________________________________________________________________________________________________________________________
There are a quite a view myths out there in assessment-land. At least to the point that they are not as discriminatory as many psychologists seem to believe, or that they do not really translate into substantially changing the treatment that will ultimately be rendered to the patient. Testing that is needed for accommodations is just that. For educational accommodations/recommendations. Its not for diagnosis or for medical/psychiatric treatment planing.

If you are already confident (in either direction) about whether ADHD is the correct diagnosis, there are no cognitive test results in the world that should undermine your confidence. The evidence simply does not exist for this. And the DSM criteria reflect this evidence base, as there is no measurable deficits in attention or other cognitive domain as part of the diagnostic criteria.
_______________________________________________________________________________________________________________________________________________________
Not the most up to date, but it's a start. We can all cherry-pick studies of course...but we need to keep in mind the idea of preponderance of the scientific/empirically evidence and literature here. There is a difference between clinically significant vs statically significance differences. Research vs Clinical application, etc.. And there is also what is "needed/necessary and pragmatically useful" vs what is "curiosity" on the part of the provider or patient.

Your tax dollars pay for psycho-educational evaluations. FPL 94-142 requires school systems to identify and provide adequate services for children with disabilities. This does not have to come "out- of-pocket" (all my children are in Catholic school and are still required to be covered by this, by the way), nor should it be the be responsibility of a health insurance company. Someone is already paying for it (i.e., you).

Lastly, doing a large (or even a small) battery of tests/testing that includes symptom validity testing for every self-reported ADHD case is cost and resourceful prohibitive and is not a viable solution to the clinical examination of this disorder.

People can fake the developmental history of ADHD. People can fake schizophrenia and depression too, right? But we don't "test" them ad naseum to prove it, do we? And what do we do about that? We articulate our clinical evidence and clinical opinion for such. It happens every day.

There is no "lower bar" for educational testing. This is already covered by a payor source. That paryor source is you/tax payer. Thus, these questions should first be assessed and triaged by said payor source (the school system) before being given to a provider billing health insurance policies.

-

References

Doyle, A.E., Biederman, J., Seidman, L.J.,Weber,W., & Faraone, S.V. (2000). Diagnostic efficiency of neuropsychological test scores for discriminating boys with and without ADHD. Journal of Consulting and Clinical Psychology, 68, 477–488.

Elwood, R. W. (1993). Clinical discriminations and neuropsychological tests: An appeal to Bayes’ theorem. The Clinical Neuropsychologist, 7(2), 224−233.

Frazier, T. W., Demaree, H. A., & Youngstrom, E. (2004). Meta-analysis of intellectual and neuropsychological test performance in Attention Deficit Hyperactivity Disorder. Neuropsychology, 18, 543−555.

Goldstein, S., & Kennemer, K. (2009). Neuropsychological aspects of attention-deficit hyperactivity disorder. In C. R. Reynolds & E. Fletcher-Janzen (Eds.), Handbook of Clinical Child Neuropsychology (3rd ed.; pp. 617−633). New York, NY: Springer.

Grodzinsky, G. M., & Barkley, R. A. (1999). Predictive power of frontal lobe tests in the diagnosis of attention deficit hyperactivity disorder. The Clinical Neuropsychologist, 13(1), 12−21.

Schwean, V. L., & Saklofske, D. H. (2005). Assessment of Attention Deficit Hyperactivity Disorder with the WISC-IV. In A. Prifitera, D. H. Saklofske, & L. G. Weiss (eds.), WISC-IV clinical use and interpretation: Scientist-practitioner perspectives (pp. 235–280). San Diego, CA: Elsevier Academic Press.

Woods, S. P., Lovejoy, D. W. & Ball, J. D. (2002). Neuropsychological characteristics of adults with ADHD: A comprehensive review of initial studies. The Clinical Neuropsychologist, 16(1), 12−34.
 
Last edited:
  • Like
Reactions: 5 users
Members don't see this ad :)
There is ample research supporting the use of cognitive testing as part of the assessment process for ADHD. I'm surprised that anyone would consider it a pseudoscience.
 
  • Like
Reactions: 3 users
The issue arises as @AcronymAllergy said, when people try to discern an ADHD profile to "confirm" diagnosis. What they are doing is using group statistics and applying them to an individual, when in fact the individual predictive power is....not good. Mainly because of heterogeneity within the clinical group itself. Testing is fine to look at strengths and weaknesses and to rule out other possible disorders that involve cognition, but it's pretty limited in what it adds to teh diagnosis of ADHD in and of itself.

So, it's far from pseudoscience, rather the misapplication of science.
 
  • Like
Reactions: 4 users
Neuropsych for ADHD can be helpful to gather some data about basic abilities, but it really isn't very good for differentiating a diagnosis of ADHD. It's not a good use of time or resources in most cases. I don't take ADHD referrals for a number of reasons, but a big one is that it usually doesn't add much to a case ONLY interested in ADHD.
 
  • Like
Reactions: 1 users
Neuropsych for ADHD can be helpful to gather some data about basic abilities, but it really isn't very good for differentiating a diagnosis of ADHD. It's not a good use of time or resources in most cases. I don't take ADHD referrals for a number of reasons, but a big one is that it usually doesn't add much to a case ONLY interested in ADHD.

Another reason to avoid ADHD referrals, generally can't bill insurance for them.
 
  • Like
Reactions: 1 user
Another reason to avoid ADHD referrals, generally can't bill insurance for them.
The problem is that this doesn't get in the way of shady providers looking to make a buck off of patients paying out of pocket. The two predominant scenarios I've seen are (1) desperate parents willing to do and pay whatever ever they have to help their kid even if it isn't necessary or demonstrably helpful (thereby being taken advantage of by the unscrupulous) and (2) patients looking to buy a diagnosis for some reason (e.g., accommodations on high stakes testing.
 
We have already done this, at least twice in the past 2 years. Maybe we can combine threads so we don't have to repeat??? I will repeat some general summaries here:
____________________________________________________________________________________________________________________________________________________________

The most reliable and valid way to diagnose most psychiatric conditions is by using a structured clinical interview/examination (as opposed to one's own idiosyncratic clinical interviewer session...which is prone to lack of follow-up, distraction, laziness, poor question wording, lack of adherence to DSM criteria, and not attending to the "not otherwise accounted for by" clause of DSM), observation during session, and collateral information that is then supplemented by some normative/rating scale data. And yes, this might take more than 60 minutes. Sorry. This is not new, surprising, or controversial to anyone who is even the least bit up-to-date on the matter. Many psychologists, for whatever reason, seem not to understand this, even though it is well spelled out in the literature. The "test boundness" of some psychologists is, frankly, embarrassing.

Additional information is needed for some things and situations/cases though...of course.
________________________________________________________________________________________________________________________________________________________
Although it is possible for researchers to learn quite a bit about ADHD from small-to-modest mean differences in test performance, attention tests do not improve individual diagnostic accuracy because problems with attention (or any other cognitive function) are not part of the DSM criteria. And there is a reason for that.

It is common for clinical psychology supervisors not too care much about what is needed, efficient, discriminatory, or actually translatable to a treatment plan, etc when training grad students. They aren't billing for it and you need to learn how to give instruments anyway. So, why not? I dont really agree with this philosophy of training, but its the predominate one out there.
________________________________________________________________________________________________________________________________________________________
There are a quite a view myths out there in assessment-land. At least to the point that they are not as discriminatory as many psychologists seem to believe, or that they do not really translate into substantially changing the treatment that will ultimately be rendered to the patient. Testing that is needed for accommodations is just that. For educational accommodations/recommendations. Its not for diagnosis or for medical/psychiatric treatment planing.

If you are already confident (in either direction) about whether ADHD is the correct diagnosis, there are no cognitive test results in the world that should undermine your confidence. The evidence simply does not exist for this. And the DSM criteria reflect this evidence base, as there is no measurable deficits in attention or other cognitive domain as part of the diagnostic criteria.
_______________________________________________________________________________________________________________________________________________________
Not the most up to date, but it's a start. We can all cherry-pick studies of course...but we need to keep in mind the idea of preponderance of the scientific/empirically evidence and literature here. There is a difference between clinically significant vs statically significance differences. Research vs Clinical application, etc.. And there is also what is "needed/necessary and pragmatically useful" vs what is "curiosity" on the part of the provider or patient.

Your tax dollars pay for psycho-educational evaluations. FPL 94-142 requires school systems to identify and provide adequate services for children with disabilities. This does not have to come "out- of-pocket" (all my children are in Catholic school and are still required to be covered by this, by the way), nor should it be the be responsibility of a health insurance company. Someone is already paying for it (i.e., you).

Lastly, doing a large (or even a small) battery of tests/testing that includes symptom validity testing for every self-reported ADHD case is cost and resourceful prohibitive and is not a viable solution to the clinical examination of this disorder.

People can fake the developmental history of ADHD. People can fake schizophrenia and depression too, right? But we don't "test" them ad naseum to prove it, do we? And what do we do about that? We articulate our clinical evidence and clinical opinion for such. It happens every day.

There is no "lower bar" for educational testing. This is already covered by a payor source. That paryor source is you/tax payer. Thus, these questions should first be assessed and triaged by said payor source (the school system) before being given to a provider billing health insurance policies.

-

References

Doyle, A.E., Biederman, J., Seidman, L.J.,Weber,W., & Faraone, S.V. (2000). Diagnostic efficiency of neuropsychological test scores for discriminating boys with and without ADHD. Journal of Consulting and Clinical Psychology, 68, 477–488.

Elwood, R. W. (1993). Clinical discriminations and neuropsychological tests: An appeal to Bayes’ theorem. The Clinical Neuropsychologist, 7(2), 224−233.

Frazier, T. W., Demaree, H. A., & Youngstrom, E. (2004). Meta-analysis of intellectual and neuropsychological test performance in Attention Deficit Hyperactivity Disorder. Neuropsychology, 18, 543−555.

Goldstein, S., & Kennemer, K. (2009). Neuropsychological aspects of attention-deficit hyperactivity disorder. In C. R. Reynolds & E. Fletcher-Janzen (Eds.), Handbook of Clinical Child Neuropsychology (3rd ed.; pp. 617−633). New York, NY: Springer.

Grodzinsky, G. M., & Barkley, R. A. (1999). Predictive power of frontal lobe tests in the diagnosis of attention deficit hyperactivity disorder. The Clinical Neuropsychologist, 13(1), 12−21.

Schwean, V. L., & Saklofske, D. H. (2005). Assessment of Attention Deficit Hyperactivity Disorder with the WISC-IV. In A. Prifitera, D. H. Saklofske, & L. G. Weiss (eds.), WISC-IV clinical use and interpretation: Scientist-practitioner perspectives (pp. 235–280). San Diego, CA: Elsevier Academic Press.

Woods, S. P., Lovejoy, D. W. & Ball, J. D. (2002). Neuropsychological characteristics of adults with ADHD: A comprehensive review of initial studies. The Clinical Neuropsychologist, 16(1), 12−34.
Too much to gain for malingerers who want highly addictive stimulant medication.
 
  • Love
Reactions: 1 user
The problem is that this doesn't get in the way of shady providers looking to make a buck off of patients paying out of pocket. The two predominant scenarios I've seen are (1) desperate parents willing to do and pay whatever ever they have to help their kid even if it isn't necessary or demonstrably helpful (thereby being taken advantage of by the unscrupulous) and (2) patients looking to buy a diagnosis for some reason (e.g., accommodations on high stakes testing.


Unfortunately, people are gonna pay for junk science because it gives them some hope, or thy know they can pay their way to a desired outcome. I can do what's right for my practice and patients with a clear conscience. About all you can do these days, especially when there is no real oversight and regulation of the people who are willing to do these things, and it isn't likely to happen anytime soon. Heck, as a nation we can't even outlaw practices that we know are very harmful (conversion therapy), how are we going to regulate expensive placebos?
 
  • Like
Reactions: 1 users
Too much to gain for malingerers who want highly addictive stimulant medication.
Can't you provide non-stimulant medication?

Also, as I mentioned in another thread, the functional outcomes for stimulants alone are very poor. If people are reporting difficulty at work/school then I would be requiring behavioral strategies along with the medication. Meaning, I would prescribe the meds only in conjunction with therapy. This also allows the therapist to monitor the medication use. If they person complains that non-stimulant meds do not work, make the stimulant medication contingent on a trial of non-stimulant along with 3 months of behavioral intervention. Of course, good luck finding someone that is willing to do CBT for ADHD that takes insurance and has a spot.
 
  • Like
Reactions: 1 user
Can't you provide non-stimulant medication?

Also, as I mentioned in another thread, the functional outcomes for stimulants alone are very poor. If people are reporting difficulty at work/school then I would be requiring behavioral strategies along with the medication. Meaning, I would prescribe the meds only in conjunction with therapy. This also allows the therapist to monitor the medication use. If they person complains that non-stimulant meds do not work, make the stimulant medication contingent on a trial of non-stimulant along with 3 months of behavioral intervention. Of course, good luck finding someone that is willing to do CBT for ADHD that takes insurance and has a spot.
Of course. But the typical patient comes in with I tried my friends Adderall and it worked. I know I have ADHD! I won't try anything else!
And then they list off all the criteria they found on the net.
The few people who will try behavior intervention most always say they fail so they can get stims. They don't get it from me.
I only prescribe if clinically indicated, including non stims.
I prescribe stimulants very rarely. So they leave. Which is fine.
People want shortcuts
 
Here's my question. For those of us not in neuropsych, who do psych assessment but can't turn down ADHD evals, what is the recommended battery? We use the Barkley but it's very face valid.
 
CAARS has a couple of SVTs embedded. I'd use that, along with a solid clinical interview and chart review to look for consistencies/inconsistencies

Lol, we switched to the Barkley from the CAARS on the recommendation of our neuropsych dept. We give a personality measure (e.g., PAI, MMPI) for a validity check though.

Then, of course, the question becomes: what do you conclude if the validity indicators suggest there's overreporting?
 
Last edited:
  • Like
Reactions: 1 user
Then, of course, the question becomes: what do you conclude if the validity indicators suggest there's overreporting?

Still depends on what the other data say, but generally that you cannot make a reliable diagnosis at this time due to evidence of symptom magnification. Just as in the neuro data, failing PVT/SVTs does not necessarily mean that there is no deficit/diagnosis, just that you cannot have faith in the validity of the data.
 
  • Like
Reactions: 2 users
This is a topic I've spent a lot of time studying and am particularly interested in. Feel free to PM if interested in having a back-channel conversation.
 
This is a topic I've spent a lot of time studying and am particularly interested in. Feel free to PM if interested in having a back-channel conversation.
Why can't that conversation happen here?
 
  • Like
Reactions: 4 users
This is a topic I've spent a lot of time studying and am particularly interested in. Feel free to PM if interested in having a back-channel conversation.

I’ll second that. I started this thread because I want to hear from people in the know on this topic. If you have info, please consider sharing it!
 
  • Like
Reactions: 1 users
I've found these articles pretty helpful to summarize various camps' views:

Barkley, R. A. (2019). Neuropsychological testing is not useful in the diagnosis of ADHD: Stop it (or prove it)!. The ADHD Report, 27(2), 1-8.

Mapou, R. L. (2019). Counterpoint: Neuropsychological testing is not useful in the diagnosis of ADHD, but…. The ADHD Report, 27(2), 8-12.
 
Last edited:
  • Like
  • Love
Reactions: 2 users
Top