Sensitivity of Executive-Function Neuropsychological Tests

[PETRAN]

So, yesterday i was in a round-table with some psychiatry folks talking about neuropsychological evaluation for schizophrenia and i was astounded (yes, for one more time ) about the devaluation of neuropsychological procedures. Overcoming various statements such as "neuropsych evaluation for schizophrenia is easy and can be performed in 10 minutes by physicians" (!) an accomplished psychiatrist stated that "for him neuropsych evaluation for schiz. is useless because it has low sensitivity". He went on by saying that "the the sensitivity of executive-function tests is usually lower than 60% and hence these tests could not differentiate from chance". I asked him whether he meant it for the majority of disorders or specifically for schizophrenia and he said that he meant executive-function tests in general and that he had no trust in them at all.

I was surprised that many other doctors agreed with him, despite the fact that what he said was purely anecdotal and no specific research was provided to support his statement. I wanted to say that this could not be true but i wasn't sure of of that myself and about whether what he said had some truth in it (plus i didn't have any data myself other than a gut feeling 😛). Since i'm still in my doctorate, i was wondering if the sensitivity of EF tests is that low in general (for the majority of conditions)? I am knowledgeable of a few studies that question many psychometric aspects of e.g. the WCST and the Tower of London but i'm still not sure if these tests are of any significant clinical value in the end of the day. Also, what do you think about neuropsych. evaluation for schizophrenia/psychosis? Do any of you have any experience with that?

---

Members don't see this ad.

 

[erg923]

So, wait... is the question whether NP tests are sensitive to: frontal lobe damage/lesion, "executive functionings," or schizophrenia? The answer for all of those is different.

While I dont support the idea that even basic neuropsych assessment should be done by those not trained properly, I do think we need to look hard at the uselfullness (clinically) of what we do.

Obviously, npsych assessment can help determine if cognitive complaints by the patient are consistent with schizophrenia...or whether they represent the presence of other cognitive disorders/dementia. And of course, mapping out cognitive strengths and weaness can always be utilized in psychosocial and medical treatment planning. However, if the treating psychiatrist is not really interested in those questions, then yes, I can easily see why they would view it as essentially useless.

I would assume that by "low sensitivity," the psychiatist meant that there is no typical neuropsychological profile of schizophrenia..at least not one that would be anywhere near diagnostic. He is indeed correct about that. A neuropsychological exam is in no way needed to diagnosis schizophrenia or differntially diagnosse it from other psychiatric disorders.

I think the tests you mentioned (WCST, TOL) are indeed useful tests, but much more for what they say about a persons functional problem solving abilities rather than their ability to locate damage/degeneration neuroanatomically. Poor performance on these measures can be due to a variety of problems or conditions.

 

[PETRAN]

So, wait... is the question whether NP test are sensitive to: frontal lobe damage/lesion, "executive functionings," or schizophrenia? The answer for all of those is different.

While I dont support the idea that even basic neuropsych assessment should be done by those not trained properly, I do think we need to look hard at the uselfullness (clinically) of what we do.

Obviously, npsych assessment can help determine if cognitive complaints by the patient are consistent with schizophrenia...or whether the represent other presence of other cognitive disorders/dementia. And of course, mapping out cognitive strengths and weaness can always be utilized in psychosocial and medical treatment planning. However, if the treating psychiatrist is not really interested in those questions, then yes, I can easily see why they would view it as essentially useless.

I would asume that by "low sensitivity," the psychiatist meant that there is no typical neuropsychological profile of schizophrenia..at least not one that would be anywhere near diagnostic. He is indeed correct about that.

I think the tests you mentioned (WCST, TOL) are indeed useful tests, but much more for what they say about a persons functional problem solving abilities rather than their ability to locate damage/degeneration neuroanatomically.

I see, so you say that these tests are better measures of frontal functionality rather than as diagnostic tools (e.g. for clinical differentiation). Thats what i knew about these tests as well, i never thought though about their diagnostiv power/sensitivity/specificity etc.


The conversation was about the assessment and possible remediation of cognitive deficits in schizophrenia/psychosis. So, yeah, it started about the diagnostic usefulness of these tests in schizophrenia, but that specific psychiatrist generalized to the diagnostic efficiency of executive tests in general (possibly in their ability to differentiate frontal from non-frontal function? It was never made explicit).

 

[Therapist4Chnge]

That is like saying that all anti-depressants don't work for all depression, without actually look at the data, specifics of the case, etc. Don't let one speciality's slant on things generalize across all physicians. Psychiatry as a speciality has a long and complicated history with neuropsychology. In the forensic world there are some very different views about objective measures, clinical judgment, etc. Darn data getting in the way of clinical judgement! 😱

Neuropsychological assessment can be very helpful if utilized appropriately. Unfortunately, many professionals don't understand the nuances involved, and then they get frustrated when they don't get the outcome they seek. In a purely psychiatic case, it probably won't be that helpful, though it can be quite useful in other cases (CVA, TBI, AD, etc). Neuropsychology assessment definitely has limitations, but when utilized by a properly trained neuropsychologist in an appropriate setting, there are clear benefits.

 

[IT514]

There is a lot of variability depending on which executive function you want to measure. It seems as though we have tests that do a fair job of assessing "dorsal" executive functions like attention, set shifting, working memory, problem solving and planning to a degree, but we lack formal tests of orbital and medial EFs like impulse control, emotional regulation, initiation, goal-directed persistance, and self-regulation. Essentially, the structured environment of the testing setting becomes a prosthetic frontal lobe, and unless you overcome the ethical and psychometric problems that go with provoking people to rage, you wont get a test for emotional regulation. However, as psychologists, we have other ways to assess this through collateral interview and rating scales, if you know what questions to ask and patterns to look for. Most neuropsychs have a solid understanding of behavioral neurology as related to the orbital PFC, so we dont always need formal tests to do the job. We also have the EXIT-25 (which can do the job of provoking some patients to rage) and the Frontal Assessment Battery, which are well validated for EF questions in dementia, stroke, and perhaps TBI; not so sure about schizophrenia though.

One issue I have seen is that on some measures of EF, (WCST, for example) an impaired score can be due to two causes – EF dysfunction due to neuropathology, but low scores can also be due to executive skills that were never developed in the first place.EFs are far different than many other cognitive abilities in that way. Assuming sufficient effort, if you have a hippocampus and midline diencephalon, you have a memory ability. Having a frontal lobe that "looks like" every other frontal lobe on an MRI is not a guarantee of good executive skills.

Point being, a lack of naturally developed executive functions may exacerbate cases in which neuropathology is also present, and this is the point at which a neuropsych eval might be appropriate in schizophrenia; a functional contribution, not so much a diagnostic one.

 

[IT514]

The conversation was about the assessment and possible remediation of cognitive deficits in schizophrenia/psychosis.

To the question of remediation of cognitive deficits, here is a pretty good study: Bell, M., Fiszdon, J., Greig, T., Wexler, B., & Bryson, G. (2007). Neurocognitive enhancement therapy with work therapy in schizophrenia: 6-month follow-up of neuropsychological performance. Journal of Rehabilitation Research & Development, 44(5), 761-770.

Studies like this are a good reminder of two things, among others: 1. sometimes you need to isolate processing deficits and selectively train them to see improvement in broader domains of functioning (I see this in kids all the time); 2. In studies of cognition, you need norms with enough statistical variance to see improvement from baseline - many physicians are numb to this fact which is why you will see MMSEs used in studies even though it is useless for this purpose. Its this line of thinking that contributes to ideas that "a neuropsych eval can be done by anyone in 10 minutes." Differences in training focus likely contribute to this and folks think you can assess cortical areas the same way you test cranial nerves, but its just not true. Most physicians I've met understand this (probably b/c I work mostly with neurologists), some do not.

A caveat to the above study is that its in a VA setting with a bunch of resources. In your setting, it might be out of the question to administer a neuropsych battery to every patient with schizophrenia to isolate deficits and track improvement.

 

[coldsweat]

On a related topic, what do people think about Russell Barkley's claim that executive function tests have little utility for diagnosing ADHD?

 

[IT514]

On a related topic, what do people think about Russell Barkley's claim that executive function tests have little utility for diagnosing ADHD?

I dont entirely disagree with Barkley. However, the DSM diagnosis is a behaviorally driven system, not an anatomically driven one, so to say that rating scales and informant report is better than lab-like tests when the diagnosis is based upon rating scales and informant report shouldnt be real surprise.

Barkely seems to be all for rating scales and opposed to continuous performance tests. However, both have inherent limitations. Try to find a normal six year old who isnt impulsive on any CPT, and try to find any kid with ODD or CD who doesnt meet criteria on ADHD scales, despite the fact that those disorders often require much different methods of treatment. Both CPTs and rating scales have higher sensitivity than specifivity to ADHD, and its easy to understand why.

The approach that Barkley takes - the review of records, interviewing, consulting with the childs physician, and using instruments when necessary in the assessment of ADHD is not contradictory to a neuropsychological approach to the assessment of ADHD, in my opinion.

James Hale has produced some interesting research suggesting that impairment in response inhibition (on a CPT, I would imagine) is the best predictor of successful response to medication vs. those who are diagnosed only through rating scales. There are definitely some limitations to the study (low n among them). Heres the citation if anyone is interested:

Hale, J. B., Reddy, L. A., Semrud-Clikeman, M., Hain, L. A., Whitaker, J., Morley, J., & ... Jones, N. (2011). Executive Impairment Determines ADHD Medication Response: Implications for Academic Achievement. Journal of Learning Disabilities, 44(2), 196-212.

 

[IT514]

Something else I'd like to add is that you need something objective in the assessment of ADHD. If all you do is rating scales, at least pick ones with validity scales. The reasons why Johnny is driving mom and dad and teachers nuts can be varied and complex, and may be related more to the executive weaknesses of the adults in Johnny's environment than to Johnny himself...

 

[coldsweat]

I dont entirely disagree with Barkley. However, the DSM diagnosis is a behaviorally driven system, not an anatomically driven one, so to say that rating scales and informant report is better than lab-like tests when the diagnosis is based upon rating scales and informant report shouldnt be real surprise.

Barkely seems to be all for rating scales and opposed to continuous performance tests. However, both have inherent limitations. Try to find a normal six year old who isnt impulsive on any CPT, and try to find any kid with ODD or CD who doesnt meet criteria on ADHD scales, despite the fact that those disorders often require much different methods of treatment. Both CPTs and rating scales have higher sensitivity than specifivity to ADHD, and its easy to understand why.

The approach that Barkley takes - the review of records, interviewing, consulting with the childs physician, and using instruments when necessary in the assessment of ADHD is not contradictory to a neuropsychological approach to the assessment of ADHD, in my opinion.

James Hale has produced some interesting research suggesting that impairment in response inhibition (on a CPT, I would imagine) is the best predictor of successful response to medication vs. those who are diagnosed only through rating scales. There are definitely some limitations to the study (low n among them). Heres the citation if anyone is interested:

Hale, J. B., Reddy, L. A., Semrud-Clikeman, M., Hain, L. A., Whitaker, J., Morley, J., & ... Jones, N. (2011). Executive Impairment Determines ADHD Medication Response: Implications for Academic Achievement. Journal of Learning Disabilities, 44(2), 196-212.

Thanks for the reply and the citation. I agree with you that you need an objective measure when assessing ADHD, particularly with adults. Although tests of executive functioning are not immune to malingering, it appears that rating scales are more susceptible to faking.