Help Cronbach's alpha

[parto123]

I completed collecting responses for the 2 measures used in my study. One of the measures had high alphas (.90+), but the other measure had very very low alphas (.20) including a couple subscales with negative alphas.

Anybody have any idea how this could have happened on the second measure? In all previous studies I looked at this measure had very high alphas, and I checked my data again to see if there was any coding or entrée calculations, but couldn’t find any. Anyone have any idea what could have caused it?

---

Members don't see this ad.

 

[cmuhooligan]

I completed collecting responses for the 2 measures used in my study. One of the measures had high alphas (.90+), but the other measure had very very low alphas (.20) including a couple subscales with negative alphas.

Anybody have any idea how this could have happened on the second measure? In all previous studies I looked at this measure had very high alphas, and I checked my data again to see if there was any coding or entrée calculations, but couldn’t find any. Anyone have any idea what could have caused it?

Were there reverse scored items? I know that in the past I've forgotten to reverse score items before and that really played havoc with the alpha levels.

 

[Cosmo75]

My first reaction was there are reversed scored items on the measure that weren't properly coded. But it sounds like you checked that already.

I assume you have a decent sample size? Did you run correlations on the individual scale items to see what those look like? What about a factor analysis?

More information on the scale and your sample would be helpful.

 

[JockNerd]

You can get a negative alpha from having many negatively correlated items, and if item covariance is larger than the sum of individual covariance. You must have forgotten to reverse code. Or, the measure is terrible.

If you're sure you coded right, run a principle axis factor analysis on your data to see how the data pulls itself apart into subscales. If your measure was never factor analyzed in development it might just have terrible psychometrics.

You said "subscales." You shouldn't be calculating a total alpha for a multidimensional scale.

 

[parto123]

My first reaction was there are reversed scored items on the measure that weren't properly coded. But it sounds like you checked that already.

I assume you have a decent sample size? Did you run correlations on the individual scale items to see what those look like? What about a factor analysis?

More information on the scale and your sample would be helpful.

Im actually currently looking into my scoring/reverse scoring with much greater detail. How do I find the alpha level for each individual subscale on spss? Thanks

Oh, the measure is the BFI

 

[parto123]

You said "subscales." You shouldn't be calculating a total alpha for a multidimensional scale.

How do I calculate it for individual subscales on spss?

 

[JockNerd]

How do I calculate it for individual subscales on spss?

Go into reliability analysis and enter only the items on that subscale. Also ask it to report "alpha if item deleted" (indicates the anticipated alpha if you delete an item. Item deletion should not improve cronbach's alpha, and losing any single item should not destroy alpha) and "item-total correlation," (correlation of the item with the total score. Should be high) and output the correlation matrix (should all be positive and about .20-.60ish.

 

[parto123]

Go into reliability analysis and enter only the items on that subscale. Also ask it to report "alpha if item deleted" (indicates the anticipated alpha if you delete an item. Item deletion should not improve cronbach's alpha, and losing any single item should not destroy alpha) and "item-total correlation," (correlation of the item with the total score. Should be high) and output the correlation matrix (should all be positive and about .20-.60ish.

Thanks. Im not sure I understand. SPSS wont let me enter in only one subscale in the reliability analysis (it needs at least 2 items into the item box).

Also, on the output page, where do I find the alpha for the individual item (is it "corrected item total correlation"?)

 

[JockNerd]

Thanks. Im not sure I understand. SPSS wont let me enter in only one subscale in the reliability analysis (it needs at least 2 items into the item box).

Also, on the output page, where do I find the alpha for the individual item (is it “corrected item total correlation”?)

Clarification:
You have ITEMS and SUBSCALES.
-Items are the individual units participants responded to, e.g. "I feel good about myself most of the time" (0 = very unlike me, 6 = very like me).
-Subscales are multiple items grouped together by theme (e.g. a six-item subscale measuring self-love).
-Scales, measures, inventories, etc., are groups of subscales that all have a common theme (e.g. a scale of self-worth contains subscales of self-love, self-respect, self-esteem, whatever).

I think a few things might be happening:
1. You (or whoever) entered the data not at the item level, but at the subscale level (i.e. summing some scores on paper or in your head and entering that sum, not the individual item responses). This is bad, because it would make you unable to calculate subscale alphas at all.
2. You have single-item subscales. This is terrible.
3. You're misunderstanding how spss wants the data. Remember, this is ITEM-level analysis. It can't pull apart only a total subscale score (i.e. it needs all the items on the self-love subscale to give you the alpha, not the self-love total score).

For pt 2 of your Q: Individual items do not have an alpha. Alpha is a property of a set of items. It measures intercorrelations between those items (essentially, "do people respond, in general, the same to all these items in this set?").

 

[parto123]

Clarification:
You have ITEMS and SUBSCALES.
-Items are the individual units participants responded to, e.g. "I feel good about myself most of the time" (0 = very unlike me, 6 = very like me).
-Subscales are multiple items grouped together by theme (e.g. a six-item subscale measuring self-love).
-Scales, measures, inventories, etc., are groups of subscales that all have a common theme (e.g. a scale of self-worth contains subscales of self-love, self-respect, self-esteem, whatever).

I think a few things might be happening:
1. You (or whoever) entered the data not at the item level, but at the subscale level (i.e. summing some scores on paper or in your head and entering that sum, not the individual item responses). This is bad, because it would make you unable to calculate subscale alphas at all.
2. You have single-item subscales. This is terrible.
3. You're misunderstanding how spss wants the data. Remember, this is ITEM-level analysis. It can't pull apart only a total subscale score (i.e. it needs all the items on the self-love subscale to give you the alpha, not the self-love total score).

For pt 2 of your Q: Individual items do not have an alpha. Alpha is a property of a set of items. It measures intercorrelations between those items (essentially, "do people respond, in general, the same to all these items in this set?").

Thanks. That was very helpful.

 

[parto123]

Clarification:
You have ITEMS and SUBSCALES.
-Items are the individual units participants responded to, e.g. "I feel good about myself most of the time" (0 = very unlike me, 6 = very like me).
-Subscales are multiple items grouped together by theme (e.g. a six-item subscale measuring self-love).
-Scales, measures, inventories, etc., are groups of subscales that all have a common theme (e.g. a scale of self-worth contains subscales of self-love, self-respect, self-esteem, whatever).

I think a few things might be happening:
1. You (or whoever) entered the data not at the item level, but at the subscale level (i.e. summing some scores on paper or in your head and entering that sum, not the individual item responses). This is bad, because it would make you unable to calculate subscale alphas at all.
2. You have single-item subscales. This is terrible.
3. You're misunderstanding how spss wants the data. Remember, this is ITEM-level analysis. It can't pull apart only a total subscale score (i.e. it needs all the items on the self-love subscale to give you the alpha, not the self-love total score).

For pt 2 of your Q: Individual items do not have an alpha. Alpha is a property of a set of items. It measures intercorrelations between those items (essentially, "do people respond, in general, the same to all these items in this set?").

Yep, it works this way. I was trying to enter it in the subscale level. Hooray!

 

[JockNerd]

Yep, it works this way. I was trying to enter it in the subscale level. Hooray!

Woot!

I think everyone tries to do that the first time they try to calculate alpha.

Does that resolve the problem you initially posted for (low alphas for subscales and negative alphas)?

And, now is as good a time as any to remember to not calculate alpha for total scores on multidimensional scales!

 

[parto123]

Woot!

I think everyone tries to do that the first time they try to calculate alpha.

Does that resolve the problem you initially posted for (low alphas for subscales and negative alphas)?

And, now is as good a time as any to remember to not calculate alpha for total scores on multidimensional scales!

Yep, it resolves the problem. Now all my alphas are above .80. Thanks again!

 

[somedaypsych]

Hi all. Reviving an old thread. I'm using two dichotomous measures, and am wondering what the best reliability analysis is to use. In my mind, Cronbach's alpha just does not make sense (both scales are quite short, as well), and yet the authors report it for both measures. Perhaps I am missing something? I also know that some suggest the Kuder-Richardson's coefficient, yet it yields pretty much the same result. So, what do you do when you need to report reliability for a dichotomous measure? Thanks!

 

[erg923]

If you message me, we can discuss. Need more info.

 

[somedaypsych]

Thanks, sent!

 

[WisNeuro]

For dichotomous measures, Kuder-Richardson is the traditional reliability coefficient. Just make sure your variables are coded the right way. Even T/F q's can have reverse scored items.

 

[somedaypsych]

Hi, WisNeuro. Thanks! Yes, the scale definitely has reverse-coded items, and they were coded correctly. It's also a short scale (13 items), which is also likely impacting my super low alpha (around .57).

 

[WisNeuro]

Ah, what is the scale? And, do you have a low n? that reliability coefficient is not good.

 

[somedaypsych]

Marlowe-Crowne Social Desirability Scale, Short Form C, n=143.

 

[WisNeuro]

Well, for the MCSD, still on the low end, but not necessarily off other validated alphas, although I thought it was generally in the low .70's range. I would just double/triple check the reverse scoring procedures.

 

[somedaypsych]

Thanks, Wisneuro. Will def check again!