Psychiatry and psychological tests

[whopper]

Someone asked me this question, and I think it's something I should share with everyone.

Hi whopper,

I've read a lot of your posts on sdn and use you as a reference for a lot of my forensic psychiatry questions and in one of your posts you mentioned psychologists make use of psychometric testing, such as testing for malingering, and psychiatrists generally don't.

What is stopping psychiatrists from learning to use these tests? Is this just a general trend -- most psychiatrists dont bother? Are there forensic psychiatrists who take the time to learn these psychometric tests so they can better assess their clients? Is there anything explicitly prohibiting anyone but a psychologist from administering it, or will it just look odd if a psychiatrist starts using tools out of a psychologists tool box?

1) Several psychometric tests require a knowledge of statistics not taught in medical school or residency:
Yes it's true. We physicians, while being required to know statistics, don't know it to the degree needed to master the MMPI among other psychological tests. Psychologists, for the overwhelming most part, have a knowledge of statistics not expected of physicians.
2) Why isn't psychometric testing taught more in residency?
Honestly I don't know. Here's what I suspect, it doesn't make money. Hospitals have problems billing for it. What economically drives hospitals to want residents is they can do billable work and that bad attendings love to use them to do their grunt work for them, neither of which contribute to opening the gates for psychometric testing to play a bigger part.

(Notice that psych residents learn almost completely by clinical experience? Why is that? Why is it that psychology students tend to get more classes and not as much clinical experience? It's all about the money IMHO).

3) Several psychometric tests, even if a psychiatrist can appropriately use them, have written instructions that only a licensed psychologist can use it.
A lot of these has to simply do with the union/trade loyalty that some people adopt. A psychologist developed a test and wanted to keep it for psychologists only because he sees other providers are competitors. I completely agree that the MMPI should not be done by a psychiatrist with our current level of training. Other tests, however, can be competently done by psychiatrists but even they are considered off-limits because their instruction manuals say so.

4)

Are there forensic psychiatrists who take the time to learn these psychometric tests so they can better assess their clients?

Yes, in fact the last AAPL president, Charles Scott, emphasized that psychiatrists need to know several tests that most psychiatrists don't. Despite this, I've seen several forensic psychiatrists and fellowships not teach about several of the tests available. I do know for a fact that UC-Davis, U. Mass,. Case Western, and U. of Cincinnati teach the tests. Most fellowships don't. It's a shame.

5) When should a psychiatrist not use a psychological test? a) if the instructions say we aren't supposed to use it b) if the testing and/or evaluation requires a level of expertise we do not possess.

During residency, I was doing a forensic rotation, sat through a court case and the expert witness I was doing the rotation with had no testing. He did not know how to do testing and only had his clinical skills. He gave an opinion the guy was truly mentally ill and could tell the guy's mental state despite not having much to go on other than interviewing the defendant long after this guy confessed he was guilty.

A psychologist was hired by the prosecutor and that person did malingering testing, very good testing by the way, showing the guy was malingering. Given that the test used was objective, highly valid and reliable, and the guy I was rotating with really had little real evidence, I asked him what made him a better witness.

The guy got mad at me (I could tell) and told me that it was his $5000 suit, that he was charismatic and the other guy stuttered. No, he did not mention anything to the effect that he had something truly scientific he could offer. He also ridiculed the defendant's mother, calling her a "loser" and that he happily took a large fraction of her yearly salary (she was something to the effect of a waitress not making much) for his evaluation, and that his testimony likely broke her savings and he was happy about it.

---

Members don't see this ad.

 

[heartsink]

Thank you for taking the time to post this. Just out of curiosity, in the case you described at the end, did the psychologist with the test or the expert witness win out?

 

[whopper]

Unfortunately for me the case required more time and I was only there one day.

I'm not going to name the psychiatrist in the case but he is faculty as a forensic psychiatric fellowship. Since he wasn't young and seemed pretty set in his ways I believe he's probably still the same hired-gun.

Just to let you know this, Phil Resnick says this, and it's backed by evidenced based data, psychiatrists are no better in detecting lying than laymen. For this reason, someone comes in and only has clinical opinion, it's really should not be considered acceptable as an expert opinion. If it's a clinical opinion based on some objective data that's different (e.g. patient develops rash and is on lamotrigine, and there's data showing that med could cause Stevens-Johnson Syndrome, now that should be admissible as the med being a possible cause of the rash), but if it's just opinion, it's hogwash.

Many psychiatrists, unfortunately, have a history of making expert witness opinions with people's lives in the balance not really having any science behind their statements, and when questioned, they back their credentials with bogus comments like "I'm a psychiatrist with decades of experience. This is what we do" even though psychiatry for the overwhelming most part does no training in detecting lies.

 

[heartsink]

Have you used, or do you know of any malingering tests that are not exclusive to psychologists offhand? Can a psychiatrist learn and train themselves on tests like these on their own? Or do they need a fellowship or board certification to be permitted to administer them?

 

[notdeadyet]

Most aren't the exclusive domain of psychologists. A forensic trained psychiatrist can administer the M-FAST and SIRS (two of the most common), as well as the PCL-R, HCR-20, VRAG, SORAG, SVR-20, static-99, and TOMM. Some of these are 15 minute structured interviews, some require some practice learning how to rate and chart review appropriately, but most aren't rocket science. They require a fellowship program committed to doing a little training and giving the opportunity to use the tests.

I think the only reason it's not more common in forensic fellowships is that psychiatry has taken a lasses faire attitude towards letting psychologists carve out their niche. Don't get me wrong, clinical psychologists have a better background for often designing these tests, but implementing them is not outside of a forensic psychiatrists scope. You just have to have a fellowship willing to play ball.

 

[whopper]

Agree with the above. I'm willing to teach a general psychiatry resident about these tests but what often happens is they're already up to their eyeballs in work, I usually only tell them they exist. Of course the resident, when an attending, could learn about them but the first few times they use them, they'll be doing it with no real mentor giving them some direction and instruction, and they will, for some time, be in a sophomore mode where they'll make some rookie mistakes using something they've never done before.

And if you're in a situation where you actually want to use these tests (e.g. a STATIC-99 on a repeat violent sex offender), you're likely not in a situation where a rookie mistake is something you want.

 

[gibits]

Due to a number of circumstances I ended up doing all of my 4th year in the state hospital. I tried to go to psychology lectures they offered and was given the runaround. Then I just asked to observe the testing and was declined (something about confidentiality on a patient I was managing). The only time I was able to observe a test (the WISC and the MMPI) was with a PhD candidate. And later I found out she got in trouble for letting me observe that.

There is defiantly some kind of turf war going on. I even said that I just wanted to see what the test entailed and had no real interest in administering the test in the future but I still didn't get a chance to really get my hands wet.

 

[notdeadyet]

There is defiantly some kind of turf war going on. I even said that I just wanted to see what the test entailed and had no real interest in administering the test in the future but I still didn't get a chance to really get my hands wet.

Probably fairly institutional dependent. I had the opposite experience. If you show ANY interest in psychological testing, the PhD's get very excited! The one exception has been the MMPI.

That said, the majority of testing that is useful and psychiatric practice, can be learned if you show initiative. If you're so inclined, there are courses that can be taken, typically open to MDs or PHDs.

 

[whopper]

There is defiantly some kind of turf war going on. I even said that I just wanted to see what the test entailed and had no real interest in administering the test in the future but I still didn't get a chance to really get my hands wet.

There is, but there are also reasons for this. Most psychological tests, and a practice among most psychologists is to be somewhat "wizardly" about doing tests. In that I mean that they do not want to explain or educate people on tests because if they became common knowledge, it'd be like everyone having an answer key. If you're a Trek fan, imagine how cadets would do on the Kobayashi Maru test if they knew it was a test of character in the face of defeat. It was an unwinnable test where the cadet would always assume they were defeated with their ship captured, and the real test would be to see how they reacted to this.

Take for example the MOCA test. There's three versions of it. Reason being if you take it more than once the score loses accuracy. With most psychological tests, there's some rule in the instructions that if the person takes it, screws up in the middle of it, and you try to do it again, the results might not be accurate.

Another example, all tests of malingering I've seen involve the person being tricked into revealing they are faking. E.g. the evaluator asks the person if they ever experienced symptoms of mental illness that are never known to happen in truly mentally ill people. If someone knew how the test operated, they would know how to beat it.

Some of it, however, definitely is just turf war. I can cite a few tests that stated only a psychologist could administer them despite the statistics behind the exam being simple enough for a medical doctor or other mental health professional to administer.

 

[phonyreal98]

Wow I didn't know this....and I know that if I continue on the path I'm on in terms of my research interests, I will need to have some knowledge of psychological testing.

How might I learn about this topic, even if my institution may not be friendly toward physicians learning about psychological testing? I'm not so interested in administering the tests myself, but I would like to see them administered, learn about how they were validated, and how to interpret the results. Might institutions be more willing to teach me about psychological testing (moreso than most physicians) if I'm involved in research which requires an understanding of certain tests?

 

[gibits]

Hey whoop, what was this malingering test you are talking about? I only had one experience with a malinging test and found it to be lacking.

I had a patient that had refractory paranoia and command hallucinations and was tried several antipsychotic with little relief. My senior, who gave me the patient after she graduated referred him to a PsyD to test for malingering and it said he was malingering. He was applying for SSI and maybe they felt he had motive to even do the test in the first place. By the time I get him he is on 3 antipsychotics, got his SSI and was still complaining of paranoia and CAH.

I start him on Clozaril and he shows rapid improvement. Was fine with all the blood work and I had him on only clozaril within 2 months. Did I just have a bum test or is this test a bit more hocus pocus?

 

[Therapist4Chnge]

Due to a number of circumstances I ended up doing all of my 4th year in the state hospital. I tried to go to psychology lectures they offered and was given the runaround. Then I just asked to observe the testing and was declined (something about confidentiality on a patient I was managing). The only time I was able to observe a test (the WISC and the MMPI) was with a PhD candidate. And later I found out she got in trouble for letting me observe that.

There is defiantly some kind of turf war going on. I even said that I just wanted to see what the test entailed and had no real interest in administering the test in the future but I still didn't get a chance to really get my hands wet.

No, it's not a turf war. Being able to observe the testing can introduce 3rd Party Observer Effects into the mix, which are a threat to the validity of the exam.

In regard to training...it isn't just about reading a manual and administering the assessment a few times. Administration of various tests can be learned, but the interpretation of the data takes more than a weekend course. The underlying principles of many of these tests require a much broader knowledge base, in addition to a sound foundation in statistics and psychometric theory to inform the practice of evaluation.

Since this is about forensic assessment:
1. How do you make heads or tails of 2 passed SVTs and 1 failed one? Do you give all of the data equal weighting?
2. How does brain injury (and other special populations) impact performance on each assessment measure?
3. What are the cultural implications when you are tasked with assessing a recent immigrant? What if they don't speak the language? Do you use an interpreter?
4. What are the salient issues when considering a fixed v. flexible battery approach? How do you support each approach during a cross examination?
5. How does each test's specificity, reliability, validity, etc. impact the data?
6. When should alternative norms be utilized? Why?
7. Why are certain cutoff scores utilized while others are not?
8. What is the likelihood of a false-positive score for [insert assessment] for [insert special population]? How does sensitivity of a particular measure impact this? What if there is a near pass for a less sensitive measure?

And so on.

 

[gibits]

No, it's not a turf war. Being able to observe the testing can introduce 3rd Party Observer Effects into the mix, which are a threat to the validity of the exam.

In regard to training...it isn't just about reading a manual and administering the assessment a few times. Administration of various tests can be learned, but the interpretation of the data takes more than a weekend course. The underlying principles of many of these tests require a much broader knowledge base, in addition to a sound foundation in statistics and psychometric theory to inform the practice of evaluation.

Since this is about forensic assessment:
1. How do you make heads or tails of 2 passed SVTs and 1 failed one? Do you give all of the data equal weighting?
2. How does brain injury (and other special populations) impact performance on each assessment measure?
3. What are the cultural implications when you are tasked with assessing a recent immigrant? What if they don't speak the language? Do you use an interpreter?
4. What are the salient issues when considering a fixed v. flexible battery approach? How do you support each approach during a cross examination?
5. How does each test's specificity, reliability, validity, etc. impact the data?
6. When should alternative norms be utilized? Why?
7. Why are certain cutoff scores utilized while others are not?
8. What is the likelihood of a false-positive score for [insert assessment] for [insert special population]? How does sensitivity of a particular measure impact this? What if there is a near pass for a less sensitive measure?

And so on.

It wasn't just observing a test, I was stonewalled from the weekly lectures offered by the psychologists.
Okay good point on the observer effect. I had been treating this patient for almost a year so maybe there is some of that. Personally I only feel like I need to know what the test is for. I have psychologists to refer to for all that other stuff tbh. I want to know what i can test for, which is better for which diagnosis. Sure you bring up interesting points but all I want to know is what resources are open to me.

 

[whopper]

Agree with Therapist4Change, and during the few unfortunate times someone has tried to start a turf war between our two professions, T4C has always acted above that gutter mentality.

But IMHO there is somewhat a turf war in having tests that are specified in the instructions as only being able to be done a psychologist despite that the statistics could be done by other providers. As I've said several psychological tests should stay within psychology, others, no.

Some tests, however, do allow for non-psychologists to perform the test if they've done a test in psychometrics and entry level statistics at an accredited college. IMHO that's completely appropriate. That said, all psychiatrists have taken the statistics but hardly any have done the psychometrics.

I only had one experience with a malinging test and found it to be lacking.

There are several but the SIRS is the best.

 

[Therapist4Chnge]

But IMHO there is somewhat a turf war in having tests that are specified in the instructions as only being able to be done a psychologist despite that the statistics could be done by other providers. As I've said several psychological tests should stay within psychology, others, no.

Some tests, however, do allow for non-psychologists to perform the test if they've done a test in psychometrics and entry level statistics at an accredited college. IMHO that's completely appropriate. That said, all psychiatrists have taken the statistics but hardly any have done the psychometrics.

This is where I have significant reservations about non-psychologists (or psychologists who do not have extensive experience/fellowship training involving formal assessment) attempting to use and interpret various assessment measures. Most physicians do not have sufficient training in statistics, though some do. However, of those that do, how many actually have training in psychometrics and enough knowledge in related areas to competently and ethically utilize the assessment as intended? Anyone can be taught to administer an assessment measure in isolation, but the interpretation and integration of that data is the important part. I touched on some of the psychometric concerns I have with my questions above, but it really makes a difference in the forensic arena.

ps. I actually just ran into an example today where the data from an assessment measure was being incorrectly interpreted. Scores on a particular set of scales were being collapsed down in a way that was not intended by the author, so the interpretation that followed was completely inaccurate.

 

[heartsink]

It's beginning to sound like psychiatrists are virtually incapable of completely handling any forensic case by themselves if there is such great difficulty in attaining the competence to administer and correctly interpret psychometric tests, especially malingering tests. If the average (non-psychometric trained) psychiatrist is ethically bound, then, to refer out all tests for malingering to forensic psychologists, this seems unconventional.

I'm not trying to entice a turf war, but how far off am I in this description?

 

[AcronymAllergy]

Hey whoop, what was this malingering test you are talking about? I only had one experience with a malinging test and found it to be lacking.

I had a patient that had refractory paranoia and command hallucinations and was tried several antipsychotic with little relief. My senior, who gave me the patient after she graduated referred him to a PsyD to test for malingering and it said he was malingering. He was applying for SSI and maybe they felt he had motive to even do the test in the first place. By the time I get him he is on 3 antipsychotics, got his SSI and was still complaining of paranoia and CAH.

I start him on Clozaril and he shows rapid improvement. Was fine with all the blood work and I had him on only clozaril within 2 months. Did I just have a bum test or is this test a bit more hocus pocus?

We'd obviously need to know the test and/or see the report to be sure, but keep in mind that just because a test flags an individual as over-endorsing and/or feigning in their symptom report, that's not to say the person doesn't truly have some form of mental illness. If the person was going for SSI, I'd say (based on base rates alone) it's very possible they might have been exaggerating their symptoms for whatever.

 

[gibits]

We'd obviously need to know the test and/or see the report to be sure, but keep in mind that just because a test flags an individual as over-endorsing and/or feigning in their symptom report, that's not to say the person doesn't truly have some form of mental illness. If the person was going for SSI, I'd say (based on base rates alone) it's very possible they might have been exaggerating their symptoms for whatever.

He had already his SSI and was still distressed by his symptoms. My thinking was that he wasn't malingering because most malingerers would drop the pretense once they got there primary gain. Also he was receptive to Clozaril even with all the bloodwork, which I felt was a commitment towards treatment (of a real mental alignment).
I honestly have no idea what test was done or if it was just an interview, I just skimmed the report's summary. Sorry I can't give more info. Whoop's statement that the malingering test conflicted with my personal experience so I wanted to see where the deal was.

 

[whopper]

It's beginning to sound like psychiatrists are virtually incapable of completely handling any forensic case by themselves if there is such great difficulty in attaining the competence to administer and correctly interpret psychometric tests, especially malingering tests

No. There's plenty of areas where both professions need each other. Take for example a case of substance-induced psychosis. We psychiatrists as a whole tend to be better in this area. We're much better in answering to a judge the likely effect of a medication on someone and that is often critical in court cases.

Forensics just isn't about malingering. Even in malingering, clinical opinion does matter but IMHO testing, which a psychiatrist can do if properly trained, is what sets us above simple psychobabble that any psychiatrist without a sense of ethics can do for the right amount of money.

And just like any doctor who needs help outside his/her field, if you need a psychologist, you have one ready just like you have an IM doctor if you need one on the hospital floor. Cultivate good relationships with psychologists. Likewise, they'll ask you to help them on cases outside their field too.

 

[heyjack70]

Rekindling an older thread.

I'm interested in buying sets of the TOMM, M-FAST and SIRS for forensic work, but there's a roadblock from the publisher because I'm not a psychologist. Anyone else experience this or know how to proceed.

 

[whopper]

I never had that problem but I never had to buy those tests. When I worked for the state, the hospital had them available to me. Now that I work for the university, the department has them available, but in each place, this was handled through psychology.

A reason why they may be making it difficult is the authors and producers of those tests don't want it out in the open because if anyone could get it, it'd be like having the answer key to an exam before you take the exam. That's understandable. In addition, several psychological tests have specific requirements that the evaluator have a background that's appropriate. E.g. some of them require that the evaluator be a Ph.D. or Psy.D. psychologist, or a psychiatrist that at least completed a course in psychotmetric testing.

Another factor on my end is often times, I simply had the psychologist do the test for me and we worked together as a team.

My point being that even if you get a copy of some of these tests, you might not be an appropriate person per the producer to use it, and that has been a subject in some court cases. Sometimes a psychiatrist uses the test and then the lawyer on the other side asks you to read the section aloud explaining who is appropriate to use the test and who isn't.

 

[notdeadyet]

One challenge I've found is finding a specific listing of what constitutes sufficient training in psychometrics for the sake of a psychiatrist conducting and interpreting the results of those tests. Do you have any idea?

 

[heyjack70]

Any idea what Charles Scott has to say about the limitations of psychiatrists doing basic testing, I know he is an advocate for psychiatrists incorporating it into forensics? The tomm sirs and mfast aren't complicated. Granted, training in psychometric testing should be required, but what is the standard, certainly not a phd. For these more basic tests you could probably become proficient via an online training or a few day work shop. Of course I'm not talking about mmpi or neuropsychology batteries.

 

[AcronymAllergy]

The thing is, they can become complicated when you start factoring in the contexts of the evaluation and the psychometric principles underlying how you can be certain (or at least reasonably certain) that your validity testing data are, in fact, valid. Do you need a doctorate to administer one of the measures? No, of course not. But to interpret them within the context of the larger evaluation can certainly require a couple years' worth of training in psychometrics, statistics, and research (for the sake of keeping up with relevant recent findings in the literature) in addition to the typical clinically-based knowledge a psychiatrist or psychologist has.

 

[heyjack70]

The thing is, they can become complicated when you start factoring in the contexts of the evaluation and the psychometric principles underlying how you can be certain (or at least reasonably certain) that your validity testing data are, in fact, valid. Do you need a doctorate to administer one of the measures? No, of course not. But to interpret them within the context of the larger evaluation can certainly require a couple years' worth of training in psychometrics, statistics, and research (for the sake of keeping up with relevant recent findings in the literature) in addition to the typical clinically-based knowledge a psychiatrist or psychologist has.

2 years?

 

[AcronymAllergy]

2 years?

It's honestly a tough question to answer. Personally, I'd be wary of even a psychologist independently administering these measures after something like a few-day workshop or online training if they weren't already familiar with the intricacies of validity testing, and that's with the assumption that the psychologist has the statistical and psychometric background I mentioned above. In some ways, it'd be akin to asking how much training would someone need to administer ECT, or read an EEG, etc. Part of the answer depends on the purpose of the treatment/evaluation, but a large part also depends on the pre-existing training and knowledge base. I'm sure the answer to my analogy would vary greatly for a psychiatrist vs. a psychologist, given that with the psychiatrist you can assume there's a base level of medical knowledge and expertise due to years of classroom and clinical training.

Again, I'm not saying that it definitively requires one, two, three (etc.) years' worth of training to be able to physically administer the measures; that part really could likely be done after a workshop. I'm also not trying to try to overstate the complexity of the issue. But the interpretation really is where things get sticky, and misinterpretation can of course have significant effects, particularly in the forensic arena. I still regularly have to work through issues related to it myself, and I've been administering and conducting research with/on these types of measures for nearly a decade at this point.

 

[notdeadyet]

It's honestly a tough question to answer.... In some ways, it'd be akin to asking how much training would someone need to administer ECT, or read an EEG, etc ...

Or prescribe medications?

I keed, I keed. This is a helpful post. I think the thing that I find a little frustrating, and even a little irritating, is the fact that there are no standards. If something is to be restricted just to psychologists, I'm fine with that. But all of these tests that I've seen, the publishers indicate that they can be administered by doctorate level providers with sufficient psychometric testing training and experience. But the standards bodies seem to very intentionally never define what that is. I find that a little disingenuous.

 

[Therapist4Chnge]

Granted, training in psychometric testing should be required, but what is the standard, certainly not a phd. For these more basic tests you could probably become proficient via an online training or a few day work shop. Of course I'm not talking about mmpi or neuropsychology batteries.

I completely disagree that these are "basic tests", as the interpretation of the data within the context of the case is where it gets complicated. Again, administration of the assessments is not hard. Scoring is not hard. However, having to interpret the data in the context of the case will be problematic. Referring back to my questions earlier in the thread….what happens if there are 2 passes and 1 failure? How do you interpret that data? The answer won't be in the administration manuals. How would you fair when cross examined about the sensitivity and validity of each measure compared to other effort measures/SVTs or similar? Does the defendant having [insert organic/neurologic condition here] impact the interpretation?

 

[Therapist4Chnge]

...the publishers indicate that they can be administered by doctorate level providers with sufficient psychometric testing training and experience. But the standards bodies seem to very intentionally never define what that is. I find that a little disingenuous.

Most (but not all) publishers want the flexibility to sell more assessments to more people who are willing. The standards bodies are many, so it can be difficult to corral cats.

 

[AcronymAllergy]

Or prescribe medications?

I keed, I keed. This is a helpful post. I think the thing that I find a little frustrating, and even a little irritating, is the fact that there are no standards. If something is to be restricted just to psychologists, I'm fine with that. But all of these tests that I've seen, the publishers indicate that they can be administered by doctorate level providers with sufficient psychometric testing training and experience. But the standards bodies seem to very intentionally never define what that is. I find that a little disingenuous.

I agree, it's frustrating, and it'd likely just be easier to say, "this is restricted to XX professionals." If a publisher is going to mention "those with sufficient psychometric testing training and experience," and they're open to selling to non-psychologists, then you ideally should be able to define what "sufficient" training entails.

Although like T4C mentioned, there's likely more of a profit motive behind that than perhaps anything else (e.g., they don't want to immediately disqualify anyone if they're willing and able to pay).

 

[heyjack70]

I completely disagree that these are "basic tests", as the interpretation of the data within the context of the case is where it gets complicated. Again, administration of the assessments is not hard. Scoring is not hard. However, having to interpret the data in the context of the case will be problematic. Referring back to my questions earlier in the thread….what happens if there are 2 passes and 1 failure? How do you interpret that data? The answer won't be in the administration manuals. How would you fair when cross examined about the sensitivity and validity of each measure compared to other effort measures/SVTs or similar? Does the defendant having [insert organic/neurologic condition here] impact the interpretation?

I want to restate that I'm talking about a couple of specific tests for forensic purposes that would help tease out feigning. I don't think the MMPI or PAI (or Rorshach testing) is going to be learned after a brief workshop. Learning the specificity/sensitivity is taught in medical school; this could be learned by a psychiatrist for specific test measures.

If I'm using a test like the SIRS because of suspicions of malingering, why would I throw 2 or 3 other tests at the person? I've seen this in psychological evaluations where they administer IQ testing, MMPI, and SIRS, and structured competency testing. And often none is even needed.

 

[erg923]

If I'm using a test like the SIRS because of suspicions of malingering, why would I throw 2 or 3 other tests at the person? I've seen this in psychological evaluations where they administer IQ testing, MMPI, and SIRS, and structured competency testing. And often none is even needed.

Because, despite what the Slick criteria have unintentionally propagated, malingering is not a psychometrically defined construct. We don’t give tests and then conclude "malingering" or "feigning." We use it as (additional) "evidence of." Its the objective butress, of sorts.

And, I think what you are talking about above is something quite different anyway, although I am not quite sure. 1.) the notion of assessing a domain multiple times is standard in neuropsychological assessment practice. A comprehensive npsych evaluation doesn’t just give one memory test. 2.) Many test have embedded indices that can be derived/calculated in order to further examine and possibility substantiate symptom distortions. 3.) Something like the MMPI (or PAI) helps us one to assess psychopathology and personality across a broad spectrum. SIRS does not. Similarly, general intellectual functioning is important for establishing and ruling out other conditions that could be contributing to the clinical picture.

 

[heyjack70]

Because, despite what the Slick criteria have unintentionally propagated, malingering is not a psychometrically defined construct. We don’t give tests and then conclude "malingering" or "feigning." We use it as (additional) "evidence of." Its the objective butress, of sorts.

And, I think what you are talking about above is something quite different anyway, although I am not quite sure. 1.) the notion of assessing a domain multiple times is standard in neuropsychological assessment practice. A comprehensive npsych evaluation doesn’t just give one memory test. 2.) Many test have embedded indices that can be derived/calculated in order to further examine and possibility substantiate symptom distortions. 3.) Something like the MMPI (or PAI) helps us one to assess psychopathology and personality across a broad spectrum. SIRS does not. Similarly, general intellectual functioning is important for establishing and ruling out other conditions that could be contributing to the clinical picture.

Maybe it's just poor word choice on my part, but to avoid confusion, I'm not talking about "neuropsychological tests." I'm asking about psychological tests that can be used as additional data points whether a person is feigning symptoms. I never said a test should be used in a vacuum, of course it shouldn't. It's another piece of information to use in the overall formulation. I think that psychologists are trying to make some of these tests sound more complicated than they are. Again, I'm not suggesting a psychiatrist should be administering and interpreting an MMPI.

 

[erg923]

Maybe it's just poor word choice on my part, but to avoid confusion, I'm not talking about "neuropsychological tests." I'm asking about psychological tests that can be used as additional data points whether a person is feigning symptoms. I never said a test should be used in a vacuum, of course it shouldn't. It's another piece of information to use in the overall formulation. I think that psychologists are trying to make some of these tests sound more complicated than they are. Again, I'm not suggesting a psychiatrist should be administering and interpreting an MMPI.

No. The test is not complicated. The 10,000 different reasons for its score (interpretation) is. And much of what makes this complex is both the statistics and variables that influence cognition and human decision making. Contrary to what you may believe, the TOMM is not just a test that gives us a dichotomous answer. Results below the cut-off, even significantly below the cut-off, do not automatically mean malingering of memory impairment. Many variables, patient and test, have to be known and accounted for in order to make that conclusion in a truely informed manner.

 

[whopper]

I haven't read all of the posts since my last one, so bear with me while I address each of the questoins I've seen in order.

What are the requirements for these tests? It depends on the test. Each specific test will mention their recommendations.
There are two ways to look at this
1-The requirement is valid and real. For example, I believe one should have graduate level statistics to do an MMPI. The mathematical equations used in it are far above the material in an MD/DO training program.
2-The requirement is more for psychologists to corner the market. Some tests, such as the CAST-MR (From my memory, I haven't done that test for a few years) are on a level where IMHO a psychiatrist can do them, but they have requirements that leave most if not all psychiatrists out.

Several in both fields of psychiatry and psychology openly acknowledge that some of the requirements are simply to keep more business for psychologists. That said, some tests really do require someone with a graduate level in psychology. It all depends on the specific test.

While a psychiatrist here could try to do a test where they don't meet the requirements, they are doing so at their own peril. A lawyer on the other side could easily identify that the psychiatrist is not deemed appropriate by the test's authors and producers and bring this up to the court, in effect possibly destroying that doctor's credibilty not only for that case but in future cases where simliar court officers are involved.

The way I've worked around that is I either use a psychologist and I have a bachelor's degree in psychology and did take a psychomotrics course. Some of the tests that allow for a non-psychologist have rules where the evaluator needs to be a mental health professional that completed a course in psychometrics at an accredited university.

Again, don't think you have to be a psychologist to do forensic work. There's things in both fields where we need each other and the other guy doesn't have the needed expertise. I recently worked on case where a woman was questioned by police fainted during that process, and was rushed to the hospital. Her O2 saturation was dangerously low, and she met the criteria of neuroglycopenia --> to a degree where most with that level wouldn't have been able to meet the legal requiements of knowingly, intelligently and voluntarily giving information. One of my mentors is a forensic psychologist, and he needed me, a physician, not himself to go over that data.

 

[whopper]

Any idea what Charles Scott has to say about the limitations of psychiatrists doing basic testing, I know he is an advocate for psychiatrists incorporating it into forensics? The tomm sirs and mfast aren't complicated.

You are correct about Scott. I don't know if he specifically wrote about the limitations and it's likely because his general tone has been to incorporate more testing. Several old-school forensic psychiatrists have based their evaluations on very unobjective measures, using some forms of psychiatric assessment with hardly any foundation in measurable science such as psychoanalsysis. Courts and doctors have accepted this (often times inappropriately) because the doctor was a, ahem, a doctor, thinking it to be true.

(Just like those doctors that tried to treat stuttering by filling marbles on a person's mouth because they read about it in a Greek myth).

The M-FAST, while not complicated, is considered more for screening. the SIRS is considered the real meat and potatoes test. The TOMM is not complicated either (it's actually quite ingenious in it's simplicity). I forgot, however, if that test requires a psychologist or not.

The issue of testing limitations has been brought up several times by others aside from Scott. I've read some books by Scott on this topic and while I don't know if he wrote any specific articles where the limitations are the main topic, I do know he's well-aware of them.

 

[whopper]

How might I learn about this topic, even if my institution may not be friendly toward physicians learning about psychological testing? I'm not so interested in administering the tests myself, but I would like to see them administered, learn about how they were validated, and how to interpret the results. Might institutions be more willing to teach me about psychological testing (moreso than most physicians) if I'm involved in research which requires an understanding of certain tests?

If you are part of an organization that has a Psy.D or Ph.D psychologist, you can partner with that doctor. Other options include (aside from enrolling in a forensic psychiatric fellowship), going to a CME that teaches these things. Your best shot is an AAPL course or perhaps one at the APA but check to see the availability of such a CME before you attend.

Other CMEs may not be centered on the tests but will include them in their material. Phil Resnick, for example, does mention the appropriateness of some malingering testing in his malingering lecture.

If your institution does at least some research, they more than likely have psychologists that will work there and help you in this area. The bottom line here is research requires numbers and psychologists get trained better in measuring mental phenomenon than we ever do. They need a psychologist doing the numbers for research unless it's unobjective research.

For those of you still in residency that want to learn more about psychological testing, forensic psychiatry covers this area the most out of all the fellowships I've seen but some programs will not cover it much if at all. You have to inquire the specific program. For a fact I know that Case Western, U. of Cincinnati, U. Mass, and UC-Davis will cover these topics very well. I don't know about other programs.

Or you could do it the hard way. Learn up on graduate level mathematics, maybe take some grad level psychology courses. I know a psychiatrist that did this, on his own time. Let's just say that guy is a braniac.

 

[whopper]

I've read the posts from the psychologists (AcronymAllergy and Therapist4Change) and I completely agree with them.

A problem here is this. Psychiatry isn't about finding out if someone is lying, their reactions on a lie detector, etc. It's core is supposed to be about treating someone with mental illness using medical science. Our level of training in statistics is only scratching the surface compared to what psychologists go through.

And for several reasons, we psychiatrists deal with people where we often times don't believe what they're telling us but they don't give us good training in figuring out what is really going on, only at prescribing a medication for a specific pathology.

While it would really take a superficial level of training to administer, say an MMPI, to interpret one is very complicated. The real genius on the part of the psychologist that the psychiatrist cannot do unless he/she had equivalent training is applying the numbers to the specific person.

Two people, let's just say for argument could have for all intents and purposes the same exact MMPI scores (yeah I know, next to impossible), but figuring out what the test means could be very different depending on the test subject. That's where an MD cannot simply walk in, see the test results and come up with a conclusion.

Same thing goes on with several medical tests. A skip lesion biopsy for temporal arteritis being negative doesn't mean the person doesn't have it. When you read the results you interpret it as they either have it, they may or may not have it but the test didn't find anything wrong. The MMPI (among other tests) are far far far more grey because they're dealing with an entire spectrum of mental states and pathologies.

 

[heyjack70]

No. The test is not complicated. The 10,000 different reasons for its score (interpretation) is. And much of what makes this complex is both the statistics and variables that influence cognition and human decision making. Contrary to what you may believe, the TOMM is not just a test that gives us a dichotomous answer. Results below the cut-off, even significantly below the cut-off, do not automatically mean malingering of memory impairment. Many variables, patient and test, have to be known and accounted for in order to make that conclusion in a truely informed manner.

You're not really explaining anything. This post just serves to paint these tests in a magical light that couldn't possibly be comprehended by non-psychologists.

 

[erg923]

I discussed statistics/psychometrics and variables that effect cognition. Hardly magic.

What part/point in that post do you disagree with, exactly?

 

[Therapist4Chnge]

While a psychiatrist here could try to do a test where they don't meet the requirements, they are doing so at their own peril. A lawyer on the other side could easily identify that the psychiatrist is not deemed appropriate by the test's authors and producers and bring this up to the court, in effect possibly destroying that doctor's credibilty not only for that case but in future cases where simliar court officers are involved.

Agreed.

Even if they are "legally" allowed to administer the assessments, whether or not they know what they are doing is a separate matter. A lawyer with experience in the area (and/or a savvy psychologist on their side) can skewer the opposing expert if that expert is shaky on the underpinnings of the assessments used. I'd welcome this type of expert if I was advising the opposing counsel because credibility is easily undermined in that scenario.

 

[heyjack70]

I discussed statistics/psychometrics and variables that effect cognition. Hardly magic.

What part/point in that post do you disagree with, exactly?

"10,000 different reasons"--wow. The argument about psychiatrists not understanding statistics is very weak. This is something that could be easily learned.

 

[erg923]

I see. Well, a multitude of factors contribute to the sample of data that you are getting. Doesn't matter if you believe that or not. It is what it is.

I am not sure what argument you are refering to, as I have made no such claim. I think an auto mechanic could learn the proper statistics it if given the proper background and training/supervision. There is certainly nothing about being a psychiatrist that lends you unable to understand these concepts and then applying them in practice, given the proper background and training/supervision.

I would agree with T4C that you would be opening yourself up to a proverbial firing squad on the stand if you just decieded to go ahead with this, because, in your opinion, it aint that complicated. Both your training/experience in psychometric assessment and your knowledge of these tests and what they are purporting to measure would be torn into. And I would assume we can count on these evals being at an increased risk for being dragged into court, no? Thus, having some kind of formal training and supervision in this area is to your benefit. Not too mention to the benefit of the patients you are seeing.

 

[heyjack70]

I see. Well, a multitude of factors contribute to the sample of data that you are getting. Doesn't matter if you believe that or not. It is what it is.

I am not sure what argument you are reffering to, as I have made no such claim. I think an auto mechanic could learn the proper statistics it if given the proper background and training/supervision. There is certainly nothing about being a psychiatrist that lends you unable to understand these concepts and then applying them in practice, given the proper background and training/supervision.

I would agree with T4C that you would be opening yourself up to a proverbial firing squad on the stand if you just decieded to go ahead with this, because, in your opinion, it aint that complicated. Both your training/experience in psychometric assessment and your knowledge of these tests and what they are purporting to measure would be torn into. And I would assume we can count on these evals being at an increased risk for being dragged into court, no? Thus, having some kind of formal training and supervision in this area is to your benefit. Not too mention to the benfit of the patients you are seeing.

Will you define proper background and supervision?

We're all professionals trained in mental healthcare, I don't see how a psychologist has a monopoly on understanding the "multitude of factors that effect the sample of data" of a psychological test. Is it that big of a leap for a psychiatrist to learn the statistics and how to administer and interpret some basic tests; it certainly isn't comparable to an auto mechanic learning to do it. I guess where we differ is you seem to be saying that it requires a PhD to give these tests, which sounds exclusionary for no good reason.

 

[erg923]

Will you define proper background and supervision?
We're all professionals trained in mental healthcare, I don't see how a psychologist has a monopoly on understanding the "multitude of factors that effect the sample of data" of a psychological test. Is it that big of a leap for a psychiatrist to learn the statistics and how to administer and interpret some basic tests; it certainly isn't comparable to an auto mechanic learning to do it. I guess where we differ is you seem to be saying that it requires a PhD to give these tests, which sounds exclusionary for no good reason.

My personal opinion would be a class in psychometrics, a class in symptom validity assessment/issues and supervision of your reports based on these measures for a specified period of time.

The universal "proper background and supervision" standard that you want so badly to exist, does not. The closest that there is would be the standard curriculum and supervised training offered in clinical psychology doctoral programs. Since this is obviously a cumbersome thing to obtain for a nonpsychologist, I think the elements above would be the bare minimum to suffice for ethical and truely competent use and practice.

I think part of the communication difficulty we are having in this conversation is a fundamental disagreement that these are "basic tests." Most educated in this subspecialty are well aware that the field of symptom validity testing (which is relatively new, btw) is fraught with controversy and disagreements that range from psychometric properties/analysis, to the very philosophical meaning and implications of test data. I am not sure if he has published much on the topic, but Rod Swenson brings up many extremely relevant arguments about the quality of the research and research design (sample bias, underlying assumptions, lack of true FP, TP, FN and FP for most of these tests, inherent problems in treating malingering as a disease entity rather than the transcient behavior that it is, etc.) that many of these tests are based upon, as well as the whole tautaology inherent to this issue (any test, medical or otherwise, that purports to diagnose something has to be able to demonstrate that base rates of the condition can be empirically known if we are going to be certain that the number of times the test is wrong can be known. In the absence of that derivation, the claims are tautological). I would also suggest reading Richard Rogers and his groups work on this issue. He is at the University of North Texas, I believe.

I am curious about your response to some of the question therpist4change asked you in previous posts in this thread?

 

[whopper]

The argument about psychiatrists not understanding statistics is very weak. This is something that could be easily learned.

The level psychologists learn regarding statistics is far above our own. They learn psychometrics on a level far above our own. The overwhelming majority of psychiatric training if clinically based. When we get lectures, half the time we fall alseep in them because we're post call and we're not tested on those lecture except for the board exam.

Psychologists, on the other hand, have to take classes in statistics (close to a graduate mathematical level) and psychometrics and they have to pay attention because they get tested in these courses just like we did in medschool.

Several of the tests psychologists administer are very sophisticated.

 

[erg923]

Whopper, I know its all good, but I think this conversation would actually be helped by refocusing from the "statistics" aspect, to more the nuanced aspect of interpretation (which is best learned via supervsion of ones work) involved even in seemingly "basic tests." A point that the psychiatrists and psychologists (Including you and I, i think) seem to be in disagreement here. Please see my post, # 45 for details about this.

 

[PsyDr]

"Doctor, thank you for joining us today. Can you outline your education and training for us?"
"That's a pretty impressive training. You would agree that being proper education is important for a professional, correct?"
"For example, you wouldn't have your taxes prepared by someone with no education, correct?"
"Doctor, what is a t score??"
"How is it calculated?"
"what is the formula for a t score?"
"where was this first published?"
"Isn't that story fairly famous?"
"so you don't know how the MMPI is scored?"
"Doctor, can you tell me the formula for the binomial theorem?"
"Doctor, isn't it true that the manual for XYZ test calls for training in statistics, psychometrics, etc"
"Can you show me where in your education, you received any formal education in statistics , psychometrics, ?"
"What is a confidence interval?"
"So Doctor, you administered a variety of these standardized tests which you used to make this opinion. How did you select these tests?"
"What is the error rate for this particular battery of tests?"
"Can you cite some peer reviewed studies that support your opinion and technique?"
"How did you administer these tests?"


"So Doctor, you used a battery of tests, which you no education, which you are unable to cite the error rate for, the literature of which you are unfamiliar, which were administered not according to standards, to make this opinion?"

"I'd like to call a Daubert hearing and move to strike"

 

[heyjack70]

My personal opinion would be a class in psychometrics, a class in symptom validity assessment/issues and supervision of your reports based on these measures for a specified period of time.
The universal "proper background and supervision" standard that you want so badly to exist, does not. The closest that there is would be the standard curriculum and supervised training offered in clinical psychology doctoral programs. Since this is obviously a cumbersome thing to obtain for a nonpsychologist, I think the elements above would be the bare minimum to suffice for ethical and truely competent use and practice.

I think this is fair. I think a forensic psychiatry fellowship should be able to accomodate this type of training. The question would be how much time would this take (forensic fellowship is only 1 year), and are there be psychologists willing to supervise psychiatrists as they learn some of the easier, but still sophisticated, tests?

 

[erg923]

I think this is fair. I think a forensic psychiatry fellowship should be able to accomodate this type of training. The question would be how much time would this take (forensic fellowship is only 1 year), and are there be psychologists willing to supervise psychiatrists as they learn some of the easier, but still sophisticated, tests?

I dont know. I honestly have never come across a psychiatrist with an interest in doing any meaningful psychometrically based assesssment. The few I've seen that have employed SVTs into their work obviously did not put much critical thought into the issue beyond the cut score cited in the publisher's manual. Emabarrisng, if you ask me. And anything worth doing, is worth doing right, IMHO. I would assume the forensic patients who lives hang in balance would agree with that practice philosophy as well.