Significance of moderating effect

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

maxy

New Member
10+ Year Member
Joined
May 2, 2012
Messages
1
Reaction score
0
Hi there,

I have 3 moderators. Following regression analysis only one moderator is significant. However, the R square change is just 0.02. I have decided not to include it in my model because of the following reason:


"The significance of the model and the increase in R square was expected due to the large sample size. Furthermore, none of the moderators entered into the equation in Model 2 were significant apart from ONE at the 0.05 level. However, the decision was taken not to include ONE into the model due to the small amount of variance (B = 0.08) it contributed to the model and because there was no change in the adjusted R square result following the addition of the moderating variables (including ONE) in Step 2."

My supervisor is not convinced if this is completely true. Does anyone know any reference to back up my decision not to include this moderator? Or is there a better way of rephrasing it? As far as I know, Aguinis and Gottfredson (2010, p. 784) do not particularly make any particularly useful conclusion in this regard:

"Reporting interaction effect size estimates such as f square and R square change does not necessarily provide information on the practical importance of a given effect. In many contexts, small effect sizes are very meaningful for practice (Aguinis et al., 2009). Conversely, in other contexts a large effect size may not be very meaningful and impactful. Thus, they proposed a ‘‘customer-centric’’ approach to reporting research results, which involves conducting a qualitative study that describes the importance of the results for specific stakeholder groups in specific
contexts."

Thank you.

Members don't see this ad.
 
Hi there,

I have 3 moderators. Following regression analysis only one moderator is significant. However, the R square change is just 0.02. I have decided not to include it in my model because of the following reason:


"The significance of the model and the increase in R square was expected due to the large sample size. Furthermore, none of the moderators entered into the equation in Model 2 were significant apart from ONE at the 0.05 level. However, the decision was taken not to include ONE into the model due to the small amount of variance (B = 0.08) it contributed to the model and because there was no change in the adjusted R square result following the addition of the moderating variables (including ONE) in Step 2."

My supervisor is not convinced if this is completely true. Does anyone know any reference to back up my decision not to include this moderator? Or is there a better way of rephrasing it? As far as I know, Aguinis and Gottfredson (2010, p. 784) do not particularly make any particularly useful conclusion in this regard:

"Reporting interaction effect size estimates such as f square and R square change does not necessarily provide information on the practical importance of a given effect. In many contexts, small effect sizes are very meaningful for practice (Aguinis et al., 2009). Conversely, in other contexts a large effect size may not be very meaningful and impactful. Thus, they proposed a ‘‘customer-centric'' approach to reporting research results, which involves conducting a qualitative study that describes the importance of the results for specific stakeholder groups in specific
contexts."

Thank you.

I think we need to know more about what type of research you're doing/ what the main effects are and what your moderators are. Presumably you have a rationale as to why you are looking at those specific moderators in the first place. Anything in the literature about previous studies/ outcomes of those specific variables as moderators, or anything on the variables as main effects that might explain why they didn't moderate? Also, any reason methodologically? How's your variability in your scales? Lots of stuff you could use to potentially explain why your variables didn't moderate; I would see if there's anything in the previous literature first, rather than trying to provide a statistical rationale as to why you shouldn't include them. I would first try to go for theory with respect to those variables in your research field to explain why they didn't work.

Also, are you familiar with HLM (Hierarchical linear modeling)? Much better to use than Stepwise-regression, and may provide an explanation as to why your moderators aren't working out if your data are nested within levels. Could be helpful to run the model there.
Good luck to you!!
 
Last edited:
Top