Stats Question

[jennigold]

---

Members do not see this ad.

I posted this on a statistics forum last night, and while 60+ people have read the question, I haven't gotten any advice - and I'm starting to lose sleep!!! Any advice or insight would be much appreciated!

I have data and am at a loss for how to analyze it to get at what I'm interested in. I will try and explain what I have and maybe someone can point me in the right direction.

My collaborators published that an expressive writing paradigm decreased BDI scores over time from baseline to 4-week follow up. I feel that sleep plays a role, but I'm not sure how to prove this.

I ran a moderation and although the sleep measure I have (SleepQuality) explains sig. more variance than just the writing condition, alone, the interaction of condition and SleepQuality did not improve the model fit.

I also ran a repeated measures anova looking at BDI score among four groups:
Expressive Writing with Low sleep Quality
Expressive Writing with High sleep quality
Control Writing with Low sleep quality
Control Writing with High sleep quality
There was a significant interaction. But, one of the groups only had a n=4, and I really don't want to lose all the SleepQuality data by using this median split.

Can anyone point me in the right direction to try to show that sleep improved BDI score above and beyond the writing paradigm (or something similar)?

I hope I've provided enough information, but please let me know of any questions!!
Thank you everyone!!!!

 

[WisNeuro]

Just curious, if one cell n=4, what are the others? You may be immensely underpowered to find any meaningful effect from well....anything, especially with 4 conditions.

 

[jennigold]

Agreed! Which is why I don't think the repeated measures ANOVA is the way to go.
n=13, n=9, n=7 were the other 3 groups

 

[WisNeuro]

Wow, that's rough. With the small n's and unequal cells, it's going to be tough going. The error variance would likely swallow up any effect even if it were there. And, because you are dichotomizing sleep quality, even the within group variance is going to be very large. To be honest, you'd need a monster effect size to show a truly significant difference here.

 

[Icculus8]

I don't have any specific analytic advice other than to agree that most analyses and especially moderation with such small and unequal cell sizes is likely to be underpowered. Did you do any kind of a priori power analysis to estimate needed sample size? I also find it curious how you are discussing what you are trying to do, I would be careful about simply seeking out analyses to "prove" your "feeling" rather than testing planned hypotheses and letting the data speak for itself, while understanding the limitations of the limited sample

 

[EmotRegulation]

First of all, if you want to know if sleep quality improves the BDI score above the writing condition, then you're talking about a regular old regression here--that's what you already found. If the sleep quality variable is significant, it means it's explaining significant variability in the outcome.

A moderation is essentially saying that the writing condition worked differently for folks of different sleep quality....that's a different question entirely.

You could run a moderation between sleep quality (full continuous variable) and condition on the BDI change scores if you want to look for an interaction, but your sample size is still pretty small. If you do, I highly recommend Andrew Hayes's PROCESS macro--it makes moderations and mediations so much easier.

 

[Ollie123]

Agree with everything EmotRegulation just said, but I'm also somewhat unclear what the question you are actually asking is (i.e. "Does sleep play a role" is not a testable model on its own). This is a shot in the dark but it sounds like you might be talking about mediation rather than moderation (i.e. improvements in BDI from the intervention are DUE to improved sleep). Running a regression with both things in the model just shows that sleep is associated with depression, which isn't news - the question is whether you think the intervention is MORE effective for people with a certain sleep pattern (moderation) or whether you think the intervention affects people BY improving their sleep(mediation). Or something else entirely I'm not clear on from your description.

If mediation, you'll need to use bootstrapping procedures with that small a sample, though even there you will likely be wildly underpowered. Quite honestly, even the original results are questionable since you really don't even see stabilization til around 20 people per cell and current results could be driven by a single person, so my main recommendation would be to collect more data if at all possible.

 

[jennigold]

Thanks so much Emot and Ollie!!! Let me give you some background:
The way this came about was that I was told about the original findings and inquired whether they had collected sleep data. They had, and so I wondered if their findings weren't only due to the writing, but that the writing made participants sleep better, and feel better. I also considered that sleeping better might be allowing the writing to have more of an effect. I understand that these are two different questions, and that this is exploratory and not planned, but I still think its an important concept to explore. Since it is a study that was already completed, I can't collect more data, unfortunately.

With regard to the regression, I did run it initially with the Hayes PROCESS and found, like I mentioned, that although the sleep measure adds explained variance above and beyond the writing paradigm, but the interaction of sleep x writing didn't. I believe this means there is no moderation. Am I right?

From your description, it does sound like mediation would be a good option. I've tried to run it using these instructions which also uses the Hayes PROCESS (http://orsp.kean.edu/documents/Moderation_Meditation.pdf). Here's the rub, I don't think the writing condition (which is a categorical variable, either 1 or 2) is not correlated with the sleep measure (a continuous variable) and I * think * you must have correlated variables to run a mediation. Is that right?

Thanks for taking the time to think about this! I truly appreciate it!

 

[EmotRegulation]

When running any regression with categorical variables, it's best to code them as 0 and 1 (rather than 1 and 2), FYI.

Yes, if the interaction of sleep x writing wasn't significant, then your analysis found no moderation. As pointed out by everyone else, it doesn't mean that writing might not interact with sleep, but it's not detected in this analysis (which could be power and not lack of a significant effect, though it's impossible to tell).

You certainly don't have to have correlated variables to *run* a mediation--the statistics will run regardless of what you put in there. But you won't find a significant *effect* if your X variable (condition) is not related to your M variable (sleep quality). It used to be thought that mediation required a significant relationship between the X and Y variable (per Baron & Kenny's rules) although newer approaches to mediation have, in my opinion, adequately argued why the X-Y relationship doesn't need to be there for significant mediation to occur.

 

[jennigold]

Thanks so much!!!!