Comprehensive Meta Analysis

[aly cat]

---

Members don't see this ad.

Has anyone used Comprehensive Meta Analysis? This is my first time conducting a meta and using the program, and I have some questions about what some settings and columns mean, but there are no answers in their tutorial or embedded within the program.

The majority of studies I'm looking at are Pre/Post design with a control and treatment group.

When I select the options to report pre and post mean, pre and post SD, and N they ask me for a pre/post correlation. This feels like it should be a "duh" thing for me, but I don't know what they mean. Is this something I can calculate or does it need to be reported?

They also ask for an effect direction. In many cases, it is positive for control and negative for the treatment group. Which one do they want? I assume in these cases that "Auto" is the best choice.

Finally, there is a "standardize by" option and choices are "Post Score SD" and "Change Score SD." Any suggestions for some reading so that I can decide which is better?
​

Thanks in advance!

 

[Ollie123]

Haven't really used it but...

1) This is a major problem with meta-analysis. For within-subjects designs, you technically need to account for that correlation as it makes a WHOPPING difference in your outcomes, p values, etc. However, it is never, ever, ever reported. Ever. I'm convinced most people totally ignore it and calculate it just like for between-subjects research. Unfortunately, this pretty much invalidates any results - especially if one is combining between and within subjects research. I think if you have all other information (F, ps, means, SDs) you can step backwards and estimate it - not sure if the software can do that or you'd have to do so by hand (which would be miserable for multi-factor designs).

2) Not sure what you mean, but I'd be very reluctant to pick "auto" in a situation like that. If this is for each study, I'm guessing they just want you to specify which group is favored (i.e. if the effect size is .2 its important to confirm that is in favor of the treatment group!).

3) This is difficult to answer without knowing details on the study and questions of interest. Comparison of post-scores versus changes scores are answering two very different questions (in an oversimplified way - akin to main effects vs. interactions).

 

[aly cat]

Thanks for the help, Ollie.

You are absolutely right, NO ONE publishes it. It's so annoying. (Well, while lit searching to learn what I can, it seems like the I/O literature publishes it often for training programs). About 3/4 of my 60something articles do not provide sufficient information to calculate ES in CMA. This is going to be fun 🙂.