Death of the Rorschach (and others)?

[Anasazi23]

---

Members don't see this ad.

I knew this day would come when I was in graduate school. In fact, it helped influence my decision to leave the field, predicting that one day, all testing would be hopelessly tainted by its publication on the internet. Soon, all versions of the WAIS will be out, and the results will be essentially meaningless to those internet savvy enough to do a google search.

http://www.thedailyinquirer.net/rorschach-inkblot-test-cheat-sheet/072071

 

[whopper]

I was thinking the same could happen.

However in most (at least as far as I'm aware), the test taker wants to get an honest answer, and therefore will not seek to cheat on it. The same cannot be said of malingerers. That's a reason why I'm glad most of the information on testing for malingering is not very easy to find.

 

[Therapist4Chnge]

Unfortunately the general public doesn't understand the implications of having this kind of information out there. It starts with this, and then personality, IQ, etc.

 

[Sneezing]

I've thought about such things, too. Even general medicine for instance is all on the internet and in theory some savy enough could write their own dialysis orders. However, the thing I think that still gives all of medicine its merit, especially the differences between physicians and midlevels is experience.

Experience conveys the authority that yes, the results, the diagnosis, the treatment, etc. is indeed correct. It is encapsulated in the information that isn't in the text book. I find myself now wanting to find answers to certain questions and text books or other online media just don't have them. Yet, older seasoned physicians are always rattling off little kernels of knowledge that just can't be found or are in only one journal article published obscurely years ago.

The point is, even though all knowledge is now pretty close to freely obtainable, people will still need and seek the value and authority of some one with experience.

 

[Solideliquid]

I sure am glad Night Owl didn't get killed.

 

[billypilgrim37]

I sure am glad Night Owl didn't get killed.

👍

 

[chameleonknight]

I sure am glad Night Owl didn't get killed.

Forget Nite Owl, what about Silk Spectre?? Sheesh! 😎

 

[whopper]

Despite the plethora of data out there, I'm still surprised with the lack of data. E.g. I read how many books of Internal Medicine, and I still can't find many sources on when exactly to switch from IV to IM insulin. I looked in about 5-6 books. Then I pop open a Kaplan question book, and see it in the answer section.

Or in our own field--antidepressants. Nowhere do I see a userfriendly book that cross compares the various antidepressants like the CATIE trial did with antipsychotics. My own opinions on antidepressants are a culmination of years of training & lectures. Yet someone could've written an easy to understand guide that summed up the side effects (weight gain), protein binding, cost, etc in a head to head manner in one nicely written chapter. I'm not talking about those guides that just put a (+) or (+++), that's way too simple, and they don't show their sources. I'm talking something that's easy to use, but at the same time backing its data so you know you're actually being a real doctor by recommending an antidepressant instead of seeing some chart with 1-4 + signs next to the med.

There's still so much in our field that we do get an opinion on that no one is willing to write about because its not PC or there's too much liability or not enough data--such as at what point should a psychiatrist throw someone out of an ER for malingering? Or how about a definitive way to deal with factitious disorder? Is Schizoaffective Disorder really just schizophrenia combined with a mood disorder or is it really a distinct disorder from schizophrenia? What do you do with the psychiatrists that you know are readily engaging in polypharmacy, deals with drug reps, etc? How many psychiatrists are diagnosing someone with Depressive DO NOS, Mood DO NOS, Impulse Control DO NOS, Anxiety DO NOS when they actually believe the person is malingering because they need to bill for something from medicaid? How many of those people end up in the revolving door system because of that initial diagnosis?

 

[kugel]

Whopper,
I'm sure Stahl will come up with a cute cartoon that explains it all very succinctly. And it will all appear to be taking place within a knee joint.

 

[maybepsych]

So does the insurance not pay if you "diagnose" someone as malingering?

Do they not understand that it takes skill and time to come to this conclusion? If this is the case, the system is even more broken than I realized before.

 

[kugel]

So does the insurance not pay if you "diagnose" someone as malingering?

Do they not understand that it takes skill and time to come to this conclusion? If this is the case, the system is even more broken than I realized before.

In general, that's true. Insurance companies and Medicare/Medicaid are basically set up to pay for treatment of an illness/injury. If no illness is present, they are unlikely to pay. This isn't completely ridiculous, since it would be foolish for them to pay for treatment of someone who has no pathology. However, there is no system in place to pay for the exam/tests designed to determine that - except in the emergency arena. It is understood that patients come into an emergency setting with complaints - not diagnosed illnesses.

However, it can be hard to get the 9th-grade educated insurance claims clerk to understand this, so claims for a workup on a patient that turns out to have no serious illness/injury often get denied on the first try.

Years and years ago, when I had a sudden onset of dizziness for no apparent reason, I called my PCP and he told me to have someone drive me to the Emerg Dept. At the ED, they examined me and found an acute otitis, Rx'd medication, and recommended that I reschedule my first skydive to another week. (really, I was enough of a goof to argue that it would be fine since I can't mistakenly go any direction but down.) The insur company denied the claim for the ED exam and tx since otitis is not an emergency condition. It took about 6 months of phone calls, letters, getting the ED notes sent to the insur company, getting the MD to interpret his illegible handwriting to the insur co. (PLEASE write legibly on charts!), etc. to get the claim paid. I spent 50% as much in long-distance calls and stamps/envelopes as I would have paid for the entire bill.

In psychiatry, if the pt complains of SI or HI, or there has been an incident of suicidality or violence in the past 24 hrs, the payor will usually pay to get the pt admitted (pay for the ED care and 1 day of inpt care), but then each subsequent day has to have documentation of exactly why the pt could not be safely discharged - no matter what the original complaint or the current diagnosis; no matter how manic or depressed or psychotic. You need specific data to prove the patient would hurt self/others or be completely unable to care for self. You need specifics like...
"Still states he intends to kill himself" or
"Continues to fantasize cutting his own throat" or
"Still states his family would be better off if he was dead" or
"Cont's to cut L arm w/ pencil b/c he believes it is not his own" or
"Still plans to kill his elderly female neighbor because he believes she is from another planet" or
"Will not eat unless prevented from returning to her room until she has eaten 30% of her meal and 8 oz fluid, and even then needs several prompts over 30 minutes" or
"Noted attempting to eat non-food objects 3 times in last 24 hrs" or...
you get the idea.

 

[Shalom77]

From the NYT

Dr. Heilman, the man who originally posted the material, compared removing the plates to the Chinese government’s attempt to control information about the Tiananmen massacre. That is, it is mainly a dispute about control, he said. “Restricting information for theoretical concerns is not what we are here to do,” Dr. Heilman said, adding that he was not impressed by the predictions of harm from those who sought to keep the Rorschach plates secret. “Show me the evidence,” he said. “I don’t care what a group of experts says.”
To illustrate his point, Dr. Heilman used the Snellen eye chart, which begins with a big letter E and is readily available on the Wikipedia site.
“If someone had previous knowledge of the eye chart,” he said, “you can go to the car people, and you could recount the chart from memory. You could get into an accident. Should we take it down from Wikipedia?”
And, Dr. Heilman added, “My dad fooled the doctor that way.”
http://forums.studentdoctor.net/newreply.php?do=newreply&noquote=1&p=8488821

👍 Totally approve of his actions. There is nothing wrong with posting material already in the public domain. I also agree that the impact is negligible and there is unlikely to be any harm since he didn't post anything that wasn't already publicly available.

A book was published in 1985, (William Poundstone, Big Secrets) which outlines of the plates, discussing common responses, good responses and responses to avoid along with other factors the examiner is evaluating. Available used from Amazon - along with a review which describes how useful the reviewer found it in preparing for his court ordered eval.

Given the serious challenges to the Rohrshach - it's just wrong anyways that it would be used in a legally connected eval.

I agree it might be more problematic for researchers.... But researchers always have to deal with the issue of contamination. So researchers will have to assess peoples familiarity with the test - which they probably should have been doing anyway given that the info was already widely available. I see the potential for a research study comparing responses of those who read Wikipedia to those who claim no prior exposure to the Rorschach. 🙂

As for malingering - one can find whole textbooks about the topic online. Which of course names all the tests used - from there it wouldn't be too hard to obtain them. In fact in one textbook (Neuropsychology Malinger Casebook, in one chapter they discuss the problem of tests (including the Rorschach being available on the internet. They give this example

One of us (WvG) once encountered an elderly woman who had for years been declared by the Court incompetent to manage her own finances. When asked to undergo a neuropsychological evaluation
as part of her challenge to this issue, the woman indicated she knew which tests would be administered because she had read many of the neuropsychologist’s articles and saw what tests he used in his
studies. “How did you do that?” she was asked. “I hired a college student to find your articles and he did,” she replied. If a person who had been found legally incompetent by the Court performed this due diligence, what about an examinee with thousands (even hundreds of thousands or millions) of dollars at stake, largely related to the evaluation? pg.15

 

[erg923]

From the NYT


👍 Totally approve of his actions. There is nothing wrong with posting material already in the public domain. I also agree that the impact is negligible and there is unlikely to be any harm since he didn't post anything that wasn't already publicly available.

A book was published in 1985, (William Poundstone, Big Secrets) which outlines of the plates, discussing common responses, good responses and responses to avoid along with other factors the examiner is evaluating. Available used from Amazon - along with a review which describes how useful the reviewer found it in preparing for his court ordered eval.

Given the serious challenges to the Rohrshach - it's just wrong anyways that it would be used in a legally connected eval.

I agree it might be more problematic for researchers.... But researchers always have to deal with the issue of contamination. So researchers will have to assess peoples familiarity with the test - which they probably should have been doing anyway given that the info was already widely available. I see the potential for a research study comparing responses of those who read Wikipedia to those who claim no prior exposure to the Rorschach. 🙂

As for malingering - one can find whole textbooks about the topic online. Which of course names all the tests used - from there it wouldn't be too hard to obtain them. In fact in one textbook (Neuropsychology Malinger Casebook, in one chapter they discuss the problem of tests (including the Rorschach being available on the internet. They give this example

Um yea, well this is a cogent argument for why it is not IILEGAL, however, it doesnt address the ethical quandry that it can put examiners in. Yes, its out there already, but, why make it MORE readily available than it already is (ie., Wiki is often the first thing thsat pops up on a google search). Who does it benefit exactly?

Nevertheless, the exposure of blots themselves is not really a big deal. No one is gonna be given the rorshach alone to see if they are disturbed. It is not used for differential diagnosis and it not something you would give in a workers comp, tort, or criminal forensic case. Results from a rorshach are never gonna overrule my diagnosis. Eseically, if it the only test in a batery that is descrpant with my diagnostic impression. In would weigh other tests heavier in this process.

This is more about the issue of psychological tests in general being pirated on the internet. This represents a risk to the integrity of tests and is copyright infrigment in many examples (MMPI, WAIS, etc). Ask the UM Press if you can make a copy of the score sheet from the MMPI for your practice and see how that goes. Its all copyrighted, the questions, the scoring, even the norms.

I recall a neurologist who published a book last year with the Rey Complex figure on the front cover. If he thought about the issue of test security (which I would wager he didnt), he probably rationalized that the risk was minimal at best. Well, yea, but whats the benefit of having it on the cover? If there is no benefit, then why create even minimal risk?

 

[Shalom77]

Unfortunately the general public doesn't understand the implications of having this kind of information out there. It starts with this, and then personality, IQ, etc.

I think the general public understand that the information is already available - either in textbook form (ie Exner's textbook) or in various popular redacted forms such as the book mentioned above or was already on the internet.

If the tests alone with brief explanations were so valuable - we really wouldn't need psychologists or neuropsychologists. It's the experience, expertise and skill of the examiner and their ability to analyze and synthesize responses to any instrument in context of other aspects of the evaluation that makes it worthwhile to go (or refer) to a neuropsychologist. Wikipedia can't replace that.

No matter what - people can and will game the system. Given the complexity of testing and interpretation - even having access to tests/answers - it still would be quite challenge to complete game an exam - although perhaps not the Rorshach alone (it should not be used alone anywys. At the same time, publicly available information is important to people who need to defend themselves against the inappropriate use of these tests. Which unfortunately is a common and more serious problem.

In this day and age patient ignorance can not be assumed.

In general I support anything that empowers patients and decreases the power differential beween patients and professionals. As they say - "Knowledge is Power."

In any case a patient wanting an honest eval has no incentive (and actually has a disincentive) to try to game a test - posting the images are irrevelant. People who have incentive to game the system - will have incentives to look a bit deeper than the first link that pops up on google. Those who are being harmed by the inappropriate use of the test will be helped by having easier access to the info. So all in all the balance of his actions is beneficial.

It's not unethical for an ER doc to post the images to the Web. There's no code of ethics for docs that prohibits that. He isn't a psychologist and isn't being forced to live in the pre information age by the APA.

 

[erg923]

In general I support anything that empowers patients and decreases the power differential beween patients and professionals. As they say - "Knowledge is Power."

First, I'm not sure how exposing psychological test items (and again, I'm not really fired up over the Rorschach issue) "empowers" anyone, especially patients. If anything, it can compromise the integrity and validity of neuropsych test findings. For example, I have seen kids whose parents have coached them on the WISC (using various pirated subtests) so they can obtain inflated IQs and get placed in advanced/accelerated classes. When the kid learn that he/she is fact not gifted, because the classes and material is overwhelming, this then causes all kinds of psychosocial, adjustment, and self-esteem issues. Not to mention when they get tested in their teens (well after the practice effects and coaching have worn off) and learn for certain via feedback from my assessment that they are not gifted and that it was all an illusion caused by undue familiarity with the items. So, in this common scenario, I'm not sure who this has "empowered." However, I do know who it has hurt.

I understand that part of the rationale is so one can become "educated," and thus defend themselves from improper test use and interpretation. While this may be a valid concern, reading and becoming familair with a test via info posted on the internet does not give a person a deep enough undertanding of the material to play "amateur psychologist." That is, it in no way qualifies them to form opinions about appopriate use and interpretation. If it was that easy, you wouldnt need a doctorate for this profession. Even if it did, people are notoriously biased regarding their own behaviors, thus, their judegments of their own test results would be severly biased anyway.

Lastly, the therapeutic relationship is built upon an inherent and naturally occurring power differential between patient and doc/therapist. There is no denying this and I see no reason to change it, frankly. This power differential is partly responsible for the professional rapport and bond that occurs during the therapeutic hour. Not only that, but it also helps to set the naturally occurring boundaries that are necessary in this professional relationship. I also feel that it prevents the session from resembling, or evolving, into a friendly chat between 2 "friends." The value of what we do as therapists is that we offer something that most "friends" do not. That is-a totally one sided relationship, where the sole focus is on the other person, not oursleves.

 

[erg923]

Is Schizoaffective Disorder really just schizophrenia combined with a mood disorder or is it really a distinct disorder from schizophrenia?

Ira Glick's office is down the hall from our groups lab. I dont know if he actually writes about it in pubs or reviews, but he is famous arouund here form proclaiming that there is "no such thing as Schizoaffective Disorder."

I do about 2-3 SCIDs per week for research studies and one involves schizoaffective (examing cortisol levels and the HPA axis system). It really never gets easier to diagnose it and parse it all out. The timeline seems to be the definitive marker for this diagnosis, and it is nearly impossible to get that timeline from patients. And even if you can, you never really know. People are so bad about retrospectively examining their behaviors and mind states that I never fully trust anyones self report 100%.

 

[whopper]

First, I'm not sure how exposing psychological test items (and again, I'm not really fired up over the Rorschach issue) "empowers" anyone, especially patients. If anything, it can compromise the integrity and validity of neuropsych test findings

It does empower someone if that person has an agenda in acheiving a specific result that is not based on good faith. E.g. if getting diagnosed with ADHD allows one to get amphetamines, with the intent on abusing them.

It can also empower people who have have been misdiagnosed because a test was not properly done. Finding out more about the test can allow them the person to understand better if they were correctly tested, and if the results were properly diagnosed.

In short if helps if the person has an agenda that's not the same as the test giver (who might've been lazy and not given the test right or interpreted it correctly), but it can also empower a malingerer or someone with factitious disorder.

As for malingerers, IMHO insurance companies would save more money if they allowed doctors to have up to 4 days to diagnose someone as a malingerer & still bill. Reason being is several doctors are billing anyway for a bogus diagnosis because its either get in trouble with the insurance company or medicaid or with their employer--the hospital. By diagnosing someone as a malingerer-it stops the revolving door, or prevents it from opening & turning round & round for years. Once that first incorrect diagnosis of Depressive DO NOS is started--the next doctor in the ER when they see the person again faking suicidal ideation is now put under even more pressure to admit that person-discharging someone claiming to be suicidal with a diagnosis of depression is too much of a risk for that ER doctor who only seens the patients for a few hours.

Someone could rightfully ask--how do I know they're incorrectly billing? This is based on what I see on an everyday basis--but this is one of those things I mentioned that no one is willing to confront in studies. To do so damns yourself & your colleagues. However not doing so creates the current revolving door situation we have with this issue.

 

[erg923]

It does empower someone if that person has an agenda in acheiving a specific result that is not based on good faith. E.g. if getting diagnosed with ADHD allows one to get amphetamines, with the intent on abusing them.

It can also empower people who have have been misdiagnosed because a test was not properly done, or used in the right context. E.g. several tests are only for screening purposes, and that may have been used to incorrectly diagnose someone with schizophrenia.

Hmm....That sounds like "manipulating" to me. The previous poster was referring to the construct as a postive goal, as in they are "empowering" their health for the better via "education." This is how I was using the term as well.

I address your second example in my post as well. I would argue that reading about a test on the internet does not give someone the base knowledge to know why that test was chosen as appopriate. Similarilty, it does not educate them on proper interpreation, or perhaps more importantly, proper integration with the rest of the findings (that may actually be how the interpretation was reached). If they suspect foul play, they have every right to seek a second opinion. However, I dont think they are qualified to come up with one on their own.

 

[whopper]

Sorry, changed & edited my post while you were responding to it.

"Empower"-to give power or authority to. How is that first example "empowering." That sounds like "manipulating" to me.

Empower has these meanings...
http://dictionary.reference.com/browse/empower
1. to give power or authority to; authorize, esp. by legal or official means: I empowered my agent to make the deal for me. The local ordinance empowers the board of health to close unsanitary restaurants.
2. to enable or permit: Wealth empowered him to live a comfortable life.

Learning how to beat a test empowers people in the 2nd sense of the word, definitely not the first.

I'm not saying its right. It is though something that happens.

A phenomenon I've encountered in the forensic unit setting is whenever someone was found to be malingering, that person from jail or prison often times calls up the other people in the hospital, and informs them of every single detail they could remember that got that nabbed. That person has been on the unit for a several weeks while they were monitored & evaluated. They often times developed friendships with the other people there on the unit for similar reasons. I've nicknamed it the "patient intranet." They've developed quite an economy of information, cigarettes & sexual favors.

 

[MoodDoc]

Interesting discussion. I have a few comments.
1. A dependence on standardized testing in psychiatry is very limiting. I have never in close to 30 years made a diagnosis based solely on results or a projective test or an IQ test. We are dependent on good clinical interviewing skills.
2. We use a chinese menu means of diagnosing psychiatric illness which leaves us with lots of patients who do not fall into any diagnostic category (not enough symptoms from column B to qualify for a mixed state).
3. Our treatment strategies are often not able to be based on evidence as most of our data from clinical trials comes from folks without co-morbidities who have not already failed multiple treatments. As well the intrusions of the legal profession into our care of patients does not help the communication of ideas. When I was being sued, the opposing counsel made great efforts to find articles I had writen or grand rounds I had given to use them against me as evidence I was not following my own standard of care. It does give me pause before I commit recommendations to the printed page.

 

[erg923]

1. A dependence on standardized testing in psychiatry is very limiting.

Very true, but I dont think anyone suggested that this was the case. Moreover, this seems to be an issue that is most directly salient for clinical neuropsychologists, rather than psychaitrists. Psychologists, and neuropsychologists specifically, are the ones performing standardized psychometric assessments of cognitive and emotional functioning. Insuring the integrity and security of out tests is of high priority for us. These instruments are important for us because the objective data that they breed form the basis for our clinical opinions. Although most psychiatrists I have met find psychometric assessment intellectually interesting, they are generally not at all envious that they are not the one doing it.

I have never in close to 30 years made a diagnosis based solely on results or a projective test or an IQ test. We are dependent on good clinical interviewing skills.

And as neuropsychologists, we try never to base a impressions or diagnosis about cogntive fucntioning based soley on an clinical interview.

 

[Shalom77]

First, I'm not sure how exposing psychological test items (and again, I'm not really fired up over the Rorschach issue) "empowers" anyone, especially patients. If anything, it can compromise the integrity and validity of neuropsych test findings. For example, I have seen kids whose parents have coached them on the WISC (using various pirated subtests) so they can obtain inflated IQs and get placed in advanced/accelerated classes.

That would be an example of a misuse of a test. The WISc should not be used for classroom placement. In any case IQ tests are hardly objective measures regardless of whether the items are known or not.

The problem of parents pushing their kids into advanced classes that they are not equipped to handle will always be an issue regardless of the testing/procedures for determination. Exposure of the test items doesn't change that fact. It may make it less likely that such tests will be used - a positive change. It may lead to more objective and robust means of placing kids. Maybe.l...

Understanding the basis and the strength of such basis upon which testing and diagnoses are based does empower patients. A sophisticated understanding of how to administer and interpret the test is not needed. A rudimentary understanding can still be helpful.

Like I said most patients want unbiased evaluation and testing and will not have an "incentive" to game the test. In any case the most positive effect of Dr. Heilman's actions is the controversy which certainly has brought to my awareness the limitations of the Rorshach, the areas where it lacks validity, and the ways that it is misused often to great detriment. It will be harder to misuse from now on. And legitimate use will hardly be affected.

 

[erg923]

That would be an example of a misuse of a test. The WAIS should not be used for classroom placement. In any case IQ tests are hardly objective measures regardless of whether the items are known or not.

What are you basing this on? Opinion? That is patently false. And I said WISC (Wechler Intelligence Scale for Children), not WAIS. In order for child to qualify as "gifted" for clasroom placement there has to be objective evidence of superior IQ (among a couple other qualifiers). Part of the definition of "gifted" is having a superior IQ. How else would one make this termination besides using standardized IQ test such as WISC or Stanford-Binet, etc? How would you suggest it be done? Clinical judgement? And you think they are not objective measures because.....? I dont follow.

I would also add that the psych community has known about the problems with the Rorschach since the time of Klopfer in the 1940s and when Exner refused to release his norming data in the 80s after discovery of a multitude of errors. And most recently, the work of Lillienfeld and Wood et al in the late 90s and early 2000s. This is nothing new, but im glad to see that you're catching up with the lit.

 

[Shalom77]

I just wanted to add that it's not that "learning to how to beat a test" that empowers people - but understanding the basis for diagnosis and treatment that is empowering.

We could try to keep the DSM a secret so no patients could ever see it (except physician and psychologist patients 😉 ) Certainly the DSM has its failings and can be misused by malingers (as well as psychologists/psychiatrists 😉 ) alike. But some patients do benefit from understanding their diagnosis and the basis on which it was made. Certainly not everyone. That doesn't mean they have to be qualified in applying the DSM. Similarly it can be comforting and enhance trust to pick up a book on how a condition is treated and find that yes indeed ones' physician is following a recommended course of care. Not to mention provide insight into some of the why behind certain actions.

I would oppose any code of ethics the prevents me from sharing the DSM, or the ASIA exam or whatever with patients because there certainly times where it is appropriate and useful.

Certainly all these items can be used by malingerers to malinger.

In any case in this day and age it's better to assume that everything is accessible and go from there. Having a false sense of secrecy leading to a false sense of objectivity is clearly an error.

 

[erg923]

Fair enough. But please answer my question from my post above. Im curious about how you reached the (false) conclusion that that is "misuse of a test," and that IQ tests are not objective measures. What is your propose model for assessing "giftedness" in school age children that is more objective?

 

[Shalom77]

I know it's nothing new, but these tests are still being inappropriately used. That is the primary benefit of the wikipedia article and the ensuing controversy to bring what is commonly known by qualified psychologists to the general public to end such misuse. I would hope that psychologists wouldn't be learning about the rorshach from wikipedia....

I don't have time to get in an argument about IQ and classroom placement. Needless to say there are methods of placing kids in gifted classrooms that don't include the WISC.

 

[erg923]

I don't have time to get in an argument about IQ and classroom placement. Needless to say there are methods of placing kids in gifted classrooms that don't include the WISC.

Thats correct,there are other ways. Stanford-Binet, Woodcock-Johnson, K-BIT, etc. So is your beef with the Wechsler Scales then? I dont follow. How would one determine "giftedness" without an objective intelligence measure. Superior intelligence is part of the definition of the word for goodnes sake! This is standard of practice and an objective IQ measures is of course required by the schools for this process, since it is what the word itself means. Your argument would be akin to formally diagnosing a dementia without demonstrating any objective evidence of global cogntive decline from a previous level. Obviuosly, you would need a IQ measure, or an abbreviated IQ estimate, (ie., WTAR) to demonstarate this. Similarly, you need some kind of objective and quantifiable evidence of superior intellectual functioning in order to assign the "giftedness" label. How would you propose getting around this?

If you are going to say a collegue is misusing a test, thats a serious accusation to make, it and deserves to be accompanied by evidence for how and why it qualifies as "misuse".....as well as stating the appopriate ethics code or practice guideline that it violates. Ethics violation accusations is not something I take lightly.

 

[Shalom77]

I have no particular beef with the wechsler scales. I do not think IQ tests are "objective." They measure certain types of knowledge and abilities and fail to measure others.

As was explained to me in high school (and your example of the cramming kid also exemplifies - even without exposure to the actual questions though - coaching can improve scores) much of what they measure is exposure to knowledge. Thus kids who has grown up in circumstances where they are more likely to be exposed to the base of knowledge (including language) implicitly tested on the WISC (and others) will do better than one who has not.

That's why there are court cases which argue that IQ testing as used in American schools leads to discrimination since minorities, those from different ethnic groups, and ESL students are disproportionaltely affected (common base of knowledge being different that that assumed by the IQ tests being a large part of the problem. Falsely labeling of certain groups of kids as "******ed" which is certainly harmful (more harmful than falsely being labeled as gifted.

Another problem with IQ tests is they propogate the misconception that IQ is something that is fixed - when that is also not true. This is something even Binet aparently was concerned about - the original purpose of his scale was to identify special needs students not to identify the gifted from the normal.

Another issue is that they only measure intelligence in certain areas - think howard gardner and multiple intelligences.

Seriously if you are going to use such tests you should at least be familiar with the century long controversy, controversy with regards to their validity and reliability, and the many arguments against their use as they are used in schools.

I am not alone in thinking that the use of IQ tests as they are used in schools is a misuse of such tests.

 

[erg923]

I am well aware of these controversies and the limits of assessing such an abstract construct. The point was to claify, or get you to admit rather, that "misuse" was simply your personal opinion. By using the term "misuse," you presented the issue as one where someone is violating an ethics code or standard of practice/care somehow. The term "misuse" suggests malpractice. If it is standard practice to utilizes these instruments for this purpose (which it is), then by defintition, it is not "misuse." The phrase "I disagree with that approach" would have been better fit here IMHO. Without derailing too much further, lets clarify some things.

1.Yes, Gardners theory is nice and all (and probably better), but as of today, we dont have a way to measure and quantify it. Therefore, it is of little practical utillity outside the textbook. That is-when a school request in eval for giftedness, I have no way of utilizing and quantifying Garnders model.

2. Of course enriched enviroments can inflate IQ. So called "crystalized intelligence" such as information learned in school, vocabulary, and social judegment/social comprehension are part of almost everyones theoretical model of intelligence. I (and most other researchers) see no reason to pull them out of the model, as these are obviously valued skills that contrbute to an organisms adaptability in the modern world. Not including them as facets of the larger construct of intelligence is silly and I can not think of scientifically justifiable reason for removing them from the current model. So, since IQ tests follow these theoretical models, then naturally, these skills would be assessed by these instruments. Yes, they can be inflated temporarily. This is just the nature of the beast and another example of how primitive our understanding and conceptualization of intelligence is at the present time.

3. Binet did not design his orginal test with a high ceiling, because he was not seeking to assess giftness, you are correct. However, this does not mean the test can not be revised to assess for this. Changing the ceiling does not change make the theory its based on somehow wrong.

4. Language is a big part of IQ tests, you are correct. Fortunately there are well validated measures of nonverbal intelligenece that can give us statistically equivalent estimates of Full scale IQ. The Test of Nonvebal Intelligence (TONI) is one such example. In general, nonverbal assessments attempt to remove language barriers in the estimation of an individuals intellectual apptitude. This is a wonderful alternative when assessing deaf individuals, individuals with aquired expressive language disorders and aphasias, as well as those from verbally impoverished enviornments.

5. No, IQ is not fixed, but its not all that flexible either (except in the case a a dementing ilness). A one standard deviation rise over a lifetime would be lucky, any higher would probably be statistical artifact.

6. Yes, California had cases of this for years in the 70s. However, this was primarily incompentent examiners. One of whom gave the test in english to a girl (or boy, not sure) who only spoke spanish. As far as their scores being lower than their white counterparts, yes, this is true and it is largely a product of impoverished social enviorments that you spoke of. Demographically corrected normative data almost always has minorities scoring lower than their aged matched caucasion counter parts on IQ measures. This is fact. And I would argue that this shows that the tests are infact picking up real differences between the 2 groups. But only the foolhardy man would interpret this to mean that whites are "smarter" than minorities. Unfortunately, thats exactly what Herstein and Murray did in 1994s "The Bell Curve"

So, as you see, all these issues are indeed real and indeed challeging. However, until we have a better theory from which to devise quatifiable measurements, we are forced to rely on what we have availble today. And as much as you may not like it, IQ as meaured by the WAIS and others is still the single best predictor of academic success that we have. And yes, its a better predictor than even SES. These are large correlation coeffcients and there is no denying that. They are real. They are there. We are tapping into something meaningful, that is for sure. Hence, our defintion of "giftedness" still hinges on having superior IQ and, hence, why schools require a validated IQ measure for placement in these classes. This is the way things are and its the best we can do at the moment. I have yet to hear an altenative model to replace this that is both valid and reliable, as well as objective. What would you suggest?

So, as you can see, intelligence testing plays a pivitol role in this process whether you like it not, or whether you agree with it or not. Your opinion that there are "other methods" does not stop the unitended harm and contamination that can take place by compromising the integrity of test items because we are not using "other methods"..........we are using "these methods." 🙂

 

[Anasazi23]

To me, the Rorschach is only the beginning. And, I agree with Erg above in saying that there is a natural power differential between doctor and patient that facilitates rapport, and has shown in research to increase compliance. I'll try and look up that reference later.

I think that the posting of the complete rorschach is only the beginning. What will you say next when all the 2-point answers for the information subtest are released, and when the picture arrangement section becomes a java-based game on i-am-bored.com?

It's naive to think that all 'patients' want a completely unbiased and clean clinical experience. No one likes to feel stupid, and it should be mentioned again that many, many entities (fire, police, teachers, etc) are given at least some of these subtests by psychologists and private testing companies under the guise of employee selection.

 

[BabyPsychDoc]

I've thought about such things, too. Even general medicine for instance is all on the internet and in theory some savy enough could write their own dialysis orders. However, the thing I think that still gives all of medicine its merit, especially the differences between physicians and midlevels is experience.

Experience conveys the authority that yes, the results, the diagnosis, the treatment, etc. is indeed correct. It is encapsulated in the information that isn't in the text book. I find myself now wanting to find answers to certain questions and text books or other online media just don't have them. Yet, older seasoned physicians are always rattling off little kernels of knowledge that just can't be found or are in only one journal article published obscurely years ago.

The point is, even though all knowledge is now pretty close to freely obtainable, people will still need and seek the value and authority of some one with experience.

Just a couple of points.
1) Many midlevels have experience far exceeding that of many physicians. In fact, many physicians would be well-served to learn from their midlevels in that sense.

2) Some older seasoned physicians who are "always rattling off little kernels of knowledge that can only be found in an old journal article" may not have read much after they have read that article in that ancient journal. Don't get me wrong, I admire Shea and think Sullivan is awesome, but even with Sullivan you would not take everything you read at its face value (for example).

In summary - experience is just one arm of the knowledge, albeit a valuable one.

 

[BabyPsychDoc]

And, I agree with Erg above in saying that there is a natural power differential between doctor and patient that facilitates rapport, and has shown in research to increase compliance. I'll try and look up that reference later.

Shea talks a lot about it in his "Psychiatric Interview: the Art of Understanding".

 

[Shalom77]

Empowering the patient does not eliminate the power differential. To the extent that for certain patients it increases trust - in trusting the physician the patient may give the physician(psychologist) even more power - but because of the patient centered/empowering stance of the physician - have greater trust that it will not be abused.

Personally I am much more likely to follow the advice of a physician who understands that they are my consultant (afterall ultimately it's my health and I'm the one who lives with the consequences of decisions) than someone who takes an authoritarian approach, particularly if they don't even bother to explain what they are doing or their reasoning.

Of course some people like the authoritarian approach. That's fine too - because a truly patient centered approach will take into account that preference.

In any case with regard to tests being available and the effect this will have, there's a bit of inconsistency regarding the effects of releasing the tests - either they are so complicated and sophisticated that only a neurospsych phd can properly administer and interpret them in which case just knowing the questions and a brief wikipedia description will really not enable most malingers. Those who are brilliant enough to figure it out would probably be successful malingers regardless of the availability of test questions. Even a patient who wishes to manipulate the physician /psychologist towards a diagnosis will likely have difficulty doing so particularly in an evaluation that uses several different evaluation methods or lengthy tests.

On the other hand someone who has true deficits may not have the capacity to fake the test even with prior knowledge of the questions. Having just watched a former clinical psychologist do poorly on the MMSE - apriori knowledge of the test (admittedly an assumption based on his past profession) could not mask his level of cognitive dysfunction.

Someone with poor short term memory for example will likely show a deficit in that area no even if they know what will be tested.

That's probably what IQ tests were initially primarily designed to reveal deficits - since distinguing average from superior is fraught with difficulty and problems of contamination.

In terms of IQ tests being used for employment purposes. Just as its not wrong to prep for the mcat or gre there's nothing wrong with obtaining the test questions/reading about what the tests are measuring for the purpose of obtaining a job. Afterall since the tests are primarily knowledge tests and are not really measuring intelligence - an intelligent person will of course work to increase their knowledge in a way that maximizes their employability. Similarly all job applicants try to present themselves in such a way to try to convince employers that they have the right "personality" or fit for the job.

 

[erg923]

Afterall since the tests are primarily knowledge tests and are not really measuring intelligence - an intelligent person will of course work to increase their knowledge in a way that maximizes their employability.

What are you basing this statement on? Personal opinion, again?

I think you may be confusing "Intelligence" with "achievment," no? Achievement, (what one has learned and can accuratley perform) seems to be the contruct you are referring to here. Achivement tests (eg. WIAT) are knowledge tests. IQ tests are "intelligence" tests. The two are very seperate, and can be very discrepant within the same individual. But dont take my word for it, the lit on the difference between these two contructs goes back to the time of Thorndike. Simillarly, literature that has accumulated since the 1950s clearly shows that WIAT and WAIS/WISC are infact measuring two very differernt things.

WAIS, Stanford-Binet, etc are all measuring intelligence as it is currently defined and agreed upon in the scientific community. It it perfect? No. Will it change as function of societal demands? Yes, most certainly. Will it ever be perfectly understood and comprehensively measured by one test? Probably not. You have got to get it out of your head that intillegence is some universal, tangible enitity. Its not. Its a man made construct and it always will be. This methodds we have are the closest apporiximation at the moment. And the fact that they correlated highly with education attainment and are statistically divergent from "achivement" speaks highly to the fact that we are indeed measuring a unique construct that is a combination of fluid and crystalized abilties. No one would argue that Martix Reasoning and Block design are measures of previously aquired knowledge for example.

So my final question would be, what is your proposed model for quantitatively meauring "intelligence." Can you demostrate that your methods are more valid and more psychometrically superior to existing methods?

 

[Shalom77]

You have got to get it out of your head that intillegence is some universal, tangible enitity. Its not. Its a man made construct and it always will be.

that's exactly my point. on this we 100% agree. I've never had this in my head - i though that is what you thought. All i'm was saying is that if such tests will be used for employment purposes then it is fair game for potential employees to obtain as much information as possible about these tests to maximize their performance.

The reason I said I didn't want to get into this is that I don't have the time to carefully construct my answer. I'm doing the best I can in the 1 minute I don't have. In any case the only point that I really wanted to make is that 100% approved of the doc's actions who posted the rorschah blots and that I believe more benefit than harm ensued from his actions.

And that intelligence tests are a relatively biased unobjective construct of intelligence not a true measure of intelligence whatever that is. Yes I do think there are serious problems with the tests construct validity - as many others have also pointed out. No I don't have a better idea - it's totally not my field and I have no desire to contribute to the idea that intelligence can be measured by a test(s) or signified by a single number.

 

[Therapist4Chnge]

Since it isn't perfect you'd rather throw the baby out with the bathwater? Combined with other assessments, a clinical interview, etc.....an "IQ assessment" can be pretty darn accurate. It isn't a static thing, but at that moment....it gives a pretty decent snapshot of that person's current level.

 

[erg923]

signified by a single number.

Again, incorrect. There are many layers of summary scores.

Verbal intelligence quotient
Performance intelligence quotient
Full scale intelligence quotient

4 index Scores:
verbal comprehension index
perceptual organization index
working memory index
processing speed index

I count more than one here quantifiable summary score here. I am unsure why you have this mistaken notion, although my hypothesis would be that its because you have no experience in this area and all your arguments thus far seem to be personal opinion based on an uniformed and misinformed knowledge of intelligence, achievement, (and the difference between those 2 constructs) psychometrics, assessment theory, and how "numbers" are interpreted and integrated with other clinical information by psychologists. 🙄 this is just my personal opinion...... I will cease the derailment now.

 

[Shalom77]

I know there are many layers of summary scores. I didn't put the (s) as I did with tests. Mea culpa for writing too fast. If I had the time I could cite the literature which agrees with my opinion. Unfortunately I don't.

I definitely know the difference between achievement and intelligence.

Just because I don't think such tests should be used for advanced placement in schools (many others agree with me on this one!) or for employment doesn't mean that I don't think they have their uses. When I went through school (or at least in the schools I went to) advanced placement was based on achievement not intelligence testing. There are clearly some disadvantages to that approach as well.

There is no instrument that has perfect construct or face validity. And yes instruments that have good reliability in the hands of researchers often can have much poorer reliability in the field. Intelligence testing is not alone in that regard. Obviously though when instruments are used in ways that are not validated (like the Rorshach unfortunately has been), particularly when the socioeconomic stakes are high, the potential for harm is much greater than just having an imperfect measure in the research setting. Then greater public awareness of the content, limitations, and misuse of the tests is clearly of value.

My main point was that I don't think Wikipedia and greater knowledge about the content of the tests is a disaster or makes them meaningless. That's all. In some cases (like the Rorshach - I actually think it is a positive thing.) Isn't that what this thread was about?

(IQ testing in schools definitely deserves its own thread.)

Instruments are constantly being revised. Researchers constantly have to deal with contamination. Clinicians have to make judgments about malingering, etc. The complexities of measuring function, and the complexity of the human body and mind, however, do making it challenging to fake with perfect consistency even if the person has read the book - or stolen a copies of the tests. For those with a true impairment probably doesn't make much difference.

 

[hkusp]

This is a very interesting discussion.

One of the best ways to get around any kind of "coaching" is to have a fluid and variable testing process that is based on the individual to be tested. For instance, if I were testing a child on the WISC, and they seemed to know the answers before I asked them, or they had a perfected surface knowledge of the test, I would casually discontinue the test and move on the say the Stanford-Binet, or I may test them on achievement using the WRAT. I may also test them with a multitude of lesser known tests but nevertheless dear to our profession. But, regardless of anything, I can always choose to end testing on the clinical judgment that the child was coached. Once you test frequently you get a feel for "intelligence" based upon talking to them and hearing their vocabulary, answers to random hypothetical questions, and other bits of information that can be gained from a simple conversation with them and their parents.

Any psychologist worth their salt would get a feel for something not quite right, and make a decision to discontinue testing. Of course, any clinical report based upon a single assessment alone is worthless and not even ethical.