Huge WJ III/WISC-IV discrepancy

[strad48]

I used the processing speed subtests of the WISC-IV to confirm an unexpected outcome on the processing speed subtests of the WJ-III, and the resulting index score differed by 39 points. No changes in testing environment or subject behavior. The tests were administered a week apart. I followed standard administration procedures for both instruments, and had a colleague double check the scoring. In case an error in timing had affected the WJ-III results, I double-tripled checked timing during the WISC - IV, and those results were even more discrepant than the unexpected WJ-III results.

Any ideas what else could contribute to a difference of this magnitude, or how it's even possible?

---

Members don't see this ad.

 

[JockNerd]

First thought is to check the clinical interviews for something major (major traumatic life event, etc.) or something weird happening in the weaker administration (no sleep, etc).

 

[IT514]

I'm not familiar with that portion of the WJ-III, so I cant comment on the task demands or norms as compared to the WISC-IV, but that is where I would look first. Are there more working memory demands on the WJ subtests vs the WISC? If so, was coding lower than symbol search on the WISC? If so, working memory could be the culprit. Dont take my word though...again, I'm not familiar with that part of the WJ-III.

Also, I would look at medications. Being cold and flu season, I wouldnt be surprised if he or she was on a cold medication which can result in frontal hypoarousal and general moodiness, which is always fun. A temporary medication would affect scores on processing speed but leave more "g" loaded measures largely unaffected. Antiepileptic medications also reduce performance on tests of processing speed.

 

[PsychScience]

I used the processing speed subtests of the WISC-IV to confirm an unexpected outcome on the processing speed subtests of the WJ-III, and the resulting index score differed by 39 points. No changes in testing environment or subject behavior. The tests were administered a week apart. I followed standard administration procedures for both instruments, and had a colleague double check the scoring. In case an error in timing had affected the WJ-III results, I double-tripled checked timing during the WISC - IV, and those results were even more discrepant than the unexpected WJ-III results.

Any ideas what else could contribute to a difference of this magnitude, or how it's even possible?

A couple of thoughts-

1- do a quick lit review on the reliability of the processing speed index score, it tends to be less reliable on test-retest than other index scores, especially for certain clinical populations.

2- I know this seems silly, but you are comparing standard scores, right?

3- Any possibility of malingering?

 

[FadedC]

I believe that the tests on the WISC for processing speed are significantly more complicated then the ones on the WJ, and as a result they may be testing more then one area of ability. If the scores on the WISC are lower, you might consider if there is another area that the subject is having difficulties with that might influence their low score.

 

[PsychScience]

I should add that I'm not very familiar with the WJ IIII, and so the comment I made about the reliability of the processing speed index score was in reference to the WISC.

The other thing I was wondering about is the idea of using separately normed intelligence tests to make comparisons. What is the reliability between the WJ-III processing speed and the WISC processing speed? If its less that .65, it wouldn't be that surprising to have a large discrepancy (although, the discrepancy you found is exceptionally high). You might want to get the reliabilities of the two tests and calculate the standard error of the difference to get a sense of the significance of this difference.

 

[IT514]

The other thing I was wondering about is the idea of using separately normed intelligence tests to make comparisons. What is the reliability between the WJ-III processing speed and the WISC processing speed? If its less that .65, it wouldn't be that surprising to have a large discrepancy (although, the discrepancy you found is exceptionally high). You might want to get the reliabilities of the two tests and calculate the standard error of the difference to get a sense of the significance of this difference.

This is a good point. Better than good, its probably the first thing to consider when comparing two tests. Even if the task demands were exactly the same, you still have two tests that are normed on different populations. If you have access to the Compendium of Neuropsychological Tests, they have a very concise chapter on this and related issues. Generally I would not try to validate one processing speed score by using another test. I might even repeat the same test a few weeks later if I were that puzzled about it.

Also, there is an article by Ingram (sp?) and Aiken describing the chance of having a score in the impaired range, given a certain number of tests administered. Its worth looking at.

Was the original low score in the impaired range, or just significantly lower than their est. general ability? I've often seen reports that make too much of low scores compared to ones FSIQ (i.e. reading comp scores in the high 90s with a 130 IQ). If its not in the impaired range I wouldnt make much of it, other than a relative weakness.

 

[FadedC]

To clarify what I had said earlier, one of the big critiques of the WISC 4 is that the tests are not very focused. While the tests on the WJ usually test one specific area of intellectual functioning, tests on the WISC 4 frequently bleed out into other areas. So their tests of processing speed for example might also contain strong elements of fluid reasoning and short term memory. This makes it hard to know exactly what area an individual is bad at when they score poorly on a subtest or index.

So if someone appears to have a good processing speed based on their WJ 3 scores, but scores poorly on the WISC 4 processing speed tasks, then one of the first things to consider is what other areas those tasks test for that they might be bad at. This can often be done by considering their other scores, but may require additional supplemental tests.

Of course it's also possible that there may be a more mundane solution, such as the subject being tired and less focused the second time around.

 

[Existenz]

I used the processing speed subtests of the WISC-IV to confirm an unexpected outcome on the processing speed subtests of the WJ-III, and the resulting index score differed by 39 points. No changes in testing environment or subject behavior. The tests were administered a week apart. I followed standard administration procedures for both instruments, and had a colleague double check the scoring. In case an error in timing had affected the WJ-III results, I double-tripled checked timing during the WISC - IV, and those results were even more discrepant than the unexpected WJ-III results.

Any ideas what else could contribute to a difference of this magnitude, or how it's even possible?

The technical manual states an overall .70 relationship with other measures like the Wechsler tests. Additionally, Spreen and Strauss wrote

Although this is not explicitly discussed in the Technical
Manual, all the studies presented in the manual for validity
purposes indicate that WJ III Standard GIA scores are usually
lower than those of other intelligence tests.

could this account for a huge difference? dunno.

 

[midwesty]

hi, i'm late to the party but i'd add that some processing speed subtests are more LANGUAGE RETRIEVAL SPEED (rapid naming, etc.) whereas some are more symbol scanning (nonverbal) & some of the latter add fine-motor/paper-pencil issues to the mix. Even tho they all correlate, you can get some folks (say someone with dyslexia) that have difficulty retrieving language quickly but can do silly little worksheet scanning things, etc. Unfortunately, the WISC's processing speed is all symbol scanning which is less relevant to school purposes than language processing speed. (but they luck out, b/c the nonverbal processing speed & verbal processing speed correlate - but that doesn't mean they can't be very discrepant in some - for SOME people those are 2 different things - for some - meh, it's all mixed together). also, some of the WJ-III subtests seem to be normed TOO hard (some of its subtests give pretty low scores depending on age/ability levels) ...also, we could help better if we knew which subtests were so high & so low, etc.