Rorschach

[MeghanHF]

Does anyone know much about the Rorschach? Particularly about its reliability and validity when the Exner system is used? Would you trust a study based primarily on Rorschach data?

---

Members don't see this ad.

 

[Logic Prevails]

Does anyone know much about the Rorschach? Particularly about its reliability and validity when the Exner system is used? Would you trust a study based primarily on Rorschach data?

I did nearly half a course on the Rorschach - its been a while though. My impression is that it would be okay to use as a 'probing' measure, or to supplement with more accepted/standardized measures, but I wouldn't trust it (or any projective measure) as a diagnostic tool by itself.

... and I think using projective measures with children is especially problematic

 

[MeghanHF]

I did nearly half a course on the Rorschach - its been a while though. My impression is that it would be okay to use as a 'probing' measure, or to supplement with more accepted/standardized measures, but I wouldn't trust it (or any projective measure) as a diagnostic tool by itself.

... and I think using projective measures with children is especially problematic

Makes sense, thanks! ...why do you think using projective measures with children is especially problematic?

 

[Therapist4Chnge]

I use the Rorschach (and other projective tests) frequently with my patients. I think LP is right about "probing". I am not a fan of the Extner system, though that has more to do with how I use the assessment. I do not usually give all of the cards....which makes the Extner system pretty much a moot point. I like to use a more traditional psychodynamic interpretation of the answers because I find it gives me a better idea of what is going on, and it is more efficient. It is definitely important to know how to use the Extner system, but whether you use it or not when you get out.....I think that depends on your preference.

-t

 

[Logic Prevails]

Good comments from T4Change

Makes sense, thanks! ...why do you think using projective measures with children is especially problematic?

We'll, one of the requirements for this test is that the examinee needs to understand that this is a test of what they "see" and NOT a test of immagination, creativity, etc. - I think kids are less likely to grasp this. Attention/concentration is also an issue with kids - they may give you some strange responses just because they are tired or wired. Development is hugely variable at an early age - personality is still forming - it is hard to draw any kind of clinical conclusions.

This is not to say that you can't give projectives to children, it means that you have to bee a damn good clinician to be able to find any use out of them. Actually, I think you'd need to be an above average clinician to be able to use the Rorschach properly in the first place.

 

[50960]

THE COMPREHENSIVE SYSTEM FOR THE RORSCHACH:
A Critical Examination
By James M. Wood,1 M. Teresa Nezworski,2 and William J. Stejskal3
The Comprehensive System (Exner, 1993) is widely accepted as a reliable and valid approach to Rorschach interpretation However, the present article calls attention to significant problems with the system First, contrary to common opinion, the interrater reliability of most scores in the system has never been demonstrated adequately Second, important scores and indices in the system are of questionable validity Third, the research base of the system consists mainly of unpublished studies that are often unavailable for examination Recommendations are made regarding research and clinical use of the Comprehensive System



There is alot of concern for the statiscal properties of the Rorschach...

 

[Therapist4Chnge]

I haven't done research in regard to the reliability of traditional interpretations.....but in my own experience, it is a worthwhile tool. Though don't rely on me, please do your own research before you rely on my experiences.

The rorschach (and pretty much all of the projectives, as far as I know) should not be used as a primary assessment tool, but they can be great in a battery assessment. I wouldn't go to court with one (not admissible), but they can be great for individual use.

I'd be curious to hear of other people's experiences with the use of the Rorschach (and other projectives) in their work.

-t

 

[MeghanHF]

Thanks LP, T4Change, and psisci! I personally think the Rorschach is useful and unique in what it is able to access and I'm aware of debates about its psychometric properties. I'm considering using it paired with other more psychometrically sound instruments in 8-11 yr olds. Does anyone have thoughts/experience with using it with young children? I know it is normed for kids as young as five yrs but don't have first hand experience. LP I agree with your previous comments about giving projectives to kids.

 

[Dr.JT]

The Rorschach is psychometrically weak (at best) and lacks evidence for its validity. It continues to be used by individuals who insist, based on their "opinion", that it provides them with good clinical information, although this has never resulted in a body of published empirical evidence. This type of pseudoscience weakens the field of clinical psych. Many prominent professional agree that is should not be used clinically until there is research to address the many issues that plague it.

 

[Therapist4Chnge]

I think the youngest i've used a Rorschach was 12 / 13, though that has more to do with my population. I have used House Tree Person with some younger kids, and that was very useful. I've found doing a couple different types of art therapy (drawing, clay, etc) was also helpful, though it very much depends on what you want to get from the work. I found some of the drawing and clay exercises worked well with lower functioning children.

-t

 

[MeghanHF]

I appreciate the diversity of "opinions"...

Journal of Personality Assessment 2001, Vol. 77, No. 1, Pages 1-15
Weighing Evidence for the Rorschach's Validity: A Response to Wood et al. (1999)
Ronald J. Ganellen*
Department of Psychiatry and Behavioral Sciences , Northwestern University Medical School

Wood, Nezworski, Stejskal, Garven, and West (1999) challenged Ganellen's (1996) characterization of the revised Rorschach Depression Index (DEPI; Exner, 1991) as a promising psychometric marker of depression that deserves serious attention by researchers and clinicians. To the contrary, however, a careful examination of existing studies indicates that no compelling empirical evidence exists indicating that Ganellen's conclusions should be modified at the present time, although no firm conclusions about the DEPI can be reached until further evidence accumulates. Furthermore, although Wood et al. (1999) suggested that evidence supporting the reliability and validity of the Rorschach in general is weak, ample evidence exists demonstrating that the Rorschach can be scored reliably (Meyer, 1997), that Rorschach variables in general have respectable levels of criterion-related validity (Bornstein, 1996; Hiller, Rosenthal, Bornstein, Berry, & Brunnel-Neuleib, 1999), and that the Minnesota Multiphasic Personality Inventory (MMPI; Hathaway & McKinley, 1943) and Rorschach have comparable levels of criterion-related validity, with the MMPI outperforming the Rorschach in certain respects and the Rorschach outperforming the MMPI in others (Bornstein, 1999; Hiller et al, 1999).

 

[MeghanHF]

THE COMPREHENSIVE SYSTEM FOR THE RORSCHACH:
A Critical Examination
By James M. Wood,1 M. Teresa Nezworski,2 and William J. Stejskal3

Journal of Personality Assessment 2000, Vol. 75, No. 1, Pages 46-81

On the Science of Rorschach Research Gregory J. Meyer*

I describe problems in an article by Wood, Nezworski, Stejskal, Garven, and West (1999b). These include (a) claims that researchers found or said things they did not, (b) an assertion that my data did not support the incremental validity of the Rorschach over the MMPI-2 when the opposite was true, (c) complications with their recommended incremental validity procedures, (d) unwarranted criticism of Burns and Viglione's (1996) statistical analyses, (e) oversimplifying issues associated with extreme groups research, (f) misleading criticisms of composite measures, and (g) faulty criticisms of Burns and Viglione's composite scale that overlooked relevant evidence. Wood et al. also asserted that Burns and Viglione's primary Rorschach variable was faulty and created a formula that seemed to show how Burns and Viglione's scores were "incompatible" and "not ... even very close" to those obtained from the proper formula. These criticisms were made even though Wood et al. had been told that their formula was incorrect and shown that it was almost perfectly correlated with the proper formula in 8 large samples (rs > .998). Sound criticism of Rorschach research will advance science and practice, but the Wood et al. article did not provide sufficient guidance.

There is alot of concern for the statiscal properties of the Rorschach...

...indeed!

 

[MeghanHF]

The Status of the Rorschach in Clinical and Forensic Practice: An Official Statement by the Board of Trustees of the Society for Personality Assessment (2005)

This statement is intended for psychologists, other mental health professionals, educators, attorneys, judges, andadministrators. Its purpose is to present a summary of the issues and evidence concerning the Rorschach. This statement affirms that the Rorschach possesses reliability and validity similar to that of other generally accepted personality assessment instruments, and its responsible use in personality assessment is appropriate and justified.

Full article - www.personality.org/SPA Rorschach White Paper Final.pdf

 

[Therapist4Chnge]

Interesting. I'll have to give that a read this weekend.

-t

 

[Dr.JT]

Don't you think it is a little suspicious that this organization exists for the sole purpose of promoting and defending a single test, yet the research they site rarely come from peer reviewed journals. The majority of their criticisms of the Wood article rely on evidence from unpublished research that comes out of Exner's workshops. This type of evidence is scientifically unacceptable. The Rorschach is pseudoscience with a group of cult-like defenders who support it. Why do you think it does not hold up in court? Answer: there is no evidence to support its validity.

 

[psychwhy]

Dr.JT: Don't you think it is a little suspicious that this organization exists for the sole purpose of promoting and defending a single test, yet the research they site rarely come from peer reviewed journals. The majority of their criticisms of the Wood article rely on evidence from unpublished research that comes out of Exner's workshops. This type of evidence is scientifically unacceptable. The Rorschach is pseudoscience with a group of cult-like defenders who support it. Why do you think it does not hold up in court? Answer: there is no evidence to support its validity.

As I read this discussion, I was struck by the perception that psychology is a profession at war with itself. Even if we granted the assumption that we all seek validity, there doesn't seem to be a universally accepted definition of what constitutes validity.

Is this what gives rise to the public's perception that psychology as a whole is an unverifiable, "fuzzy" science?

Then I spent some time lurking on the other SDN boards and found the same sorts of debates happening with nearly equal frequency (and passion).

So the new question is: If every health profession seems to succumb to these sorts of "my approach is better than your approach" in-fighting, with questionable veracity to the claims, why does psychology still get tagged with the "soft science" "gang that can't shoot straight" reputation?

[Disclosure: My graduate program provided minimal training in projectives. I then had an externship with a passionate Rorschacher (who kept a photo of he and Exner on his desk!). My internship required at least one test battery be conducted with a Rorschach. I'd say I fall in the midrange between "it is totally worthless" and "it is the wonder test." ]

 

[Dr.JT]

So the new question is: If every health profession seems to succumb to these sorts of "my approach is better than your approach" in-fighting, with questionable veracity to the claims, why does psychology still get tagged with the "soft science" "gang that can't shoot straight" reputation?
QUOTE]

Good question. Here is my opinion.

In other profession, when individuals cannot back up their claims with science they are often shunned by the mainstream. In our profession they just ignore the science and keep doing things based on their beliefs and this is considered acceptable to some degree by the mainstream. There are many examples of this such as recovered memories and EMDR to name a few. In our field these unsupported, and often refuted, clinical practices not only fail to be marginalized, but they often flourish. How could we not be viewed as "soft science" when so many of our practitioners refuse to be guided by research and prefer their own "opinions" to empirical data?

It is interesting that I get into these types of debates regularly with psychologists at different conferences, but I have never been involved in a forensic case where these issues came up due to another psychologist’s use of the Rorschach. The reason is they do not have a scientific leg to stand on.

 

[MeghanHF]

Seems there are overstatements, understatements, and misstatements on both sides of the debated Woods article and yes the fact that the Society for Personality Assesment used to be called the Rorschach Institute is decidedly suspect. (However, I do find SPA's statement comprehensive and their position justifiable.) A lot of the Rorschach debate parallels the historical development of the field of psychology in the US, which like the history of every other profession is fraught with egos and politics. I agree wholeheartedly that our practices need to be backed up with evidence and I'm quick to get the willies when people start talking about EMDR or I read a psych report in which the interpretations are so far extrapolated from the data they seem to be entirely made up. However, I also don't condone throwing the baby out with the bathwater. Certain scales of the Rorschach, such as the Oral Dependency Scale are quite well validated (see Bornstein, 1996 for a start), and useful, tapping into aspects of personality that may not be accessed by other measures. When used by someone well trained and in proper context, the Rorschach offer valuable information that should not be thrown to the wayside because of a forced choice "either/or" of the Rorschahch debate.

 

[50960]

It can be useful as a clinical tool, but not as a research instrument...

 

[MeghanHF]

It can be useful as a clinical tool, but not as a research instrument...

How so?

 

[50960]

If you use it clinically you can make the judgement if its lack of validity and even moreso relaibility is acceptable based upon what clinical information you need. To be used in research you either need to know it has easily defedable statistical properties, or be judged by everyone who reads it to the contrary.

 

[MeghanHF]

To be used in research you either need to know it has easily defedable statistical properties, or be judged by everyone who reads it to the contrary.

True...if one uses a particular index of the rorschach with proven reliability and validity then...would it not be a research instrument and possibly useful?

 

[50960]

Maybe, but the only evidence I havee seen showing this is heavily political research that is mainly me-search. The point is we can argue all day whether it is good, useful etc..., but many in the field have dismissed it and you will be judged accordingly.

 

[MeghanHF]

Maybe, but the only evidence I havee seen showing this is heavily political research that is mainly me-search. The point is we can argue all day whether it is good, useful etc..., but many in the field have dismissed it and you will be judged accordingly.

Grrrrrrrrrrrrrrrrr...all research is political and mesearch, blatant or not. I do however get your point and know that many would judge...often based on limitted knowledge on the Rorschach as well as their "opinion" (that arguement runs both ways). Let's say I don't want to wear a scarlet R across my chest for the rest of my career, what is a better measure of reality testing in kids, one that can capture paranoia, psychotic anxiety, and primitive defences??

 

[Dr.JT]

all research is political and mesearch, blatant or not. QUOTE]

While politics unfortunately does influence research, your statement is absolutely wrong. There is a huge difference between well conducted research that is published in peer reviewed journals versus the kind of research cited by, for example, Rorschach advocates. Look at any one of Exner's books. It will have a reference section that appears impressive until closer inspection. As I said before, the majority of Rorschach research that is cited comes from the workshops. It is never published in peer reviewed journals because it does not hold up to scientific scrutiny. Compare this to the literature on the MMPI-2. A world of difference. Also, Exner presented normative data for his Rorschach data and many years later acknowledged that the normative sample was half the size of what was reported. Better yet, the data for each subject was entered twice which led to the sample size being doubled. Can you imagine this happening to the WAIS?

 

[Logic Prevails]

what is a better measure of reality testing in kids, one that can capture paranoia, psychotic anxiety, and primitive defences??

I don't think you are going to find a single measure that assesses what I think you are looking for (a seemingly "psychodynamic-ish" interpretation of symtomatology in children). If you can flesh-out exactly what you mean, then I might be able to point you in the right direction. As I had mentioned, I would stay away from projectives.

If by...

Paranoia <you mean> exessive worrying, and you can get away with measures of anxiety, atypicality, depression, etc... then you might be able to use the BASC-2; it is quick, has good norms, validity etc.

I think you are going to run into problems assessing defense mechanisms and "psychotic anxiety" in children. You might spot some of these issues by doing a lit search to see what is already out there.

 

[MeghanHF]

Thanks, LP. You're right I am trying to look at symptomatology psychodynamically and part of the appeal for me is that there's not a lot of "evidence" for psychodynamic approaches. Most analytic types write theoretical or case study type articles. I see the problems with using a projectives but run up against a wall with objective measures because they don't necessary access unconscious process. I think you're right the BASC-2 might work and there's also a Reality Distortion scale on the Personality Inventory for Kids - 2.