Power analysis

[Ollie123]

Bit thrown off by my latest sample size calculation so hoping someone can explain it to me.

Say you have a mixed model design (A x B) where A is a between-subjects factor and B is a within subjects factor. Why do you need a larger sample size to detect a between subjects main effect, than the interaction term, all other things (alpha, effect size, etc.)? Or is my power analysis software broken? My primary hypothesis is the interaction (main effect unnecessary and meaningless), and I apparently need half the sample size to detect an interaction as I do the main effect.

Conceptually, I can't wrap my brain around why this might be. I've always heard interactions are harder to detect, but that might be from folks who do purely between-subjects designs. If this is correct, I'm going to have to power my analyses for the post-hocs rather than the primary outcomes, because there is no way I can justify running a study with less than 20 people per group, as nice as it would be to finish data collection in a month.

---

Members don't see this ad.

 

[futureapppsy2]

Can I hop in here with another power question (sorry, Ollie!). For my honors thesis, I'm dong two multiple hierarchical regressions (testing the same four variants of the IV on two separate DVs)--how do I account for the different variants of the IV (the steps of the regressions) in the power analysis?

Thanks.

 

[thewesternsky]

Bit thrown off by my latest sample size calculation so hoping someone can explain it to me.

Say you have a mixed model design (A x B) where A is a between-subjects factor and B is a within subjects factor. Why do you need a larger sample size to detect a between subjects main effect, than the interaction term, all other things (alpha, effect size, etc.)? Or is my power analysis software broken? My primary hypothesis is the interaction (main effect unnecessary and meaningless), and I apparently need half the sample size to detect an interaction as I do the main effect.

Conceptually, I can't wrap my brain around why this might be. I've always heard interactions are harder to detect, but that might be from folks who do purely between-subjects designs. If this is correct, I'm going to have to power my analyses for the post-hocs rather than the primary outcomes, because there is no way I can justify running a study with less than 20 people per group, as nice as it would be to finish data collection in a month.

Hey Ollie! I'm pretty sure what you describe is correct, though I don't have my textbooks on hand at the moment to provide details. I've always been impressed by the (sometimes dramatic) advantage that within-subject designs give you in terms of power, especially when the two measures are fairly highly correlated (i.e., before and after treatment, etc.) I've a hunch that the power for the interaction is getting a boost from that within-subject variable.

 

[JockNerd]

I think it's a combo of two things:
-TWS is right, I think, about the power boost from the within-sub factor in the interaction
-powering withing-subject designs is a bit of voodoo anyhow

 

[Ollie123]

-powering withing-subject designs is a bit of voodoo anyhow

Can you elaborate on this at all? I'm not sure why it would be any different from between-subjects power analysis.

 

[irish80122]

Futureapp, what software are you using to run your power analysis? I have done this in Sample Power (SPSS's power program) and it isn't too hard to do if you have access to that software. Just let me know if that is the case and I will tell you where to go. I don't currently have G-Power but I am happy to download it if you still need help and that is what you are using.

Ollie, I agree with the others, I think the interaction is getting a boost from your within-subjects measure. You said the expected effect size is the same for the interaction and the between-subjects main effect, correct?

 

[futureapppsy2]

Hey,

Thanks. I'm using SPSS (v15 or 16--I have access to both--if it makes a difference) , and any help would be much appreciated, as this is the last thing I have to clear with my chair before getting the go-ahead to schedule my prospectus defense.

 

[futureapppsy2]

Tactful bump...

 

[Psychmeout]

Can you elaborate on this at all? I'm not sure why it would be any different from between-subjects power analysis.

Theoretically, a within subjects power analysis would require less subjects to detect a meaningful difference because there would be less error in the "error term" (because you are using the same people) as opposed to a between subjects analysis where more error is assumed. Because the error term in the between subjects analysis is presumed to be greater, and consequently dampens the actual difference between means, you need more subjects to overcome the error.

Hopefully this makes sense.

 

[Ollie123]

Theoretically, a within subjects power analysis would require less subjects to detect a meaningful difference because there would be less error in the "error term" (because you are using the same people) as opposed to a between subjects analysis where more error is assumed. Because the error term in the between subjects analysis is presumed to be greater, and consequently dampens the actual difference between means, you need more subjects to overcome the error.

Hopefully this makes sense.

Appreciate the reply, but I was asking JN why he considers within-subjects power analysis "voodoo". I understand the design differences and how within-subjects is more powerful (though I'm still unclear how an interaction can be easier to find than a main effect, but several sources have confirmed so I'm just going with it at this point).

 

[Psychmeout]

Appreciate the reply, but I was asking JN why he considers within-subjects power analysis "voodoo". I understand the design differences and how within-subjects is more powerful (though I'm still unclear how an interaction can be easier to find than a main effect, but several sources have confirmed so I'm just going with it at this point).

Woops....no problem, I think I read the correspondence between you and JN too quickly.

At any rate, I am interested in your question as well. This question that I am about to pose may make absolutely no sense, but, I am going to throw it out there anyway.....I am wondering if the reason why the interaction is easier to detect compared with main effects has anything to do with the fact that you are examining whether changes in DV scores occur when one level of the IV is considered in the presence of one of the levels of the second IV, rather than examining differences when both levels of the IV's are collapsed together?? In other words, the interaction may allow for a more "fine grained" analysis that is likely to detect any differences compared to when you consider the entire variable (i.e., main effect). I would love to hear someone else's opinion who is more savvy in this area.