Dissertation stuck due to power analysis

[psydstudent2020]

I’m not sure if this is allowed on here but I hope so because I’ve received solid advice on this site before. My dissertation isn’t moving forward until I do a power analysis. However, my chair does not know how to do it for the statistical tests I am running. The other member of my committee said I don’t need one, and my third is the one who said I NEED to do it but he is off campus and when I emailed about it he asked me questions like he didn’t remember my study even though he wrote most of the proposal revisions. I asked psychologists/statisticians outside my committee but they didn’t know either. My study is on the development of a self report inventory. It has 7 items but may have more or less because step one after submitting to the IRB is to send it to psychologists for review based on their clinical experience. The statistical tests that will be run: 1) split half reliability, 2) internal consistency using cronbachs alpha, 3) test retest reliability, 4) differential validity (giving a well known personality questionnaire along with my instrument). Would there need to be one power analysis done for all this, or one for each test? Also, how do I actually do the power analysis? I know what G power is but that is where everyone gets stuck because they say you can’t do my specific statistical tests on there. Please message me if you can help. Thank you and I’m sorry if this isn’t allowed!

---

Members don't see this ad.

 

[Seihai]

I’m not sure if this is allowed on here but I hope so because I’ve received solid advice on this site before. My dissertation isn’t moving forward until I do a power analysis. However, my chair does not know how to do it for the statistical tests I am running. The other member of my committee said I don’t need one, and my third is the one who said I NEED to do it but he is off campus and when I emailed about it he asked me questions like he didn’t remember my study even though he wrote most of the proposal revisions. I asked psychologists/statisticians outside my committee but they didn’t know either. My study is on the development of a self report inventory. It has 7 items but may have more or less because step one after submitting to the IRB is to send it to psychologists for review based on their clinical experience. The statistical tests that will be run: 1) split half reliability, 2) internal consistency using cronbachs alpha, 3) test retest reliability, 4) differential validity (giving a well known personality questionnaire along with my instrument). Would there need to be one power analysis done for all this, or one for each test? Also, how do I actually do the power analysis? I know what G power is but that is where everyone gets stuck because they say you can’t do my specific statistical tests on there. Please message me if you can help. Thank you and I’m sorry if this isn’t allowed!

I just PMed you. Doing power analyses for these seems fairly simple (assuming I understand what you're doing properly). Feel free to ask more questions if you need to.

Do keep in mind that I'm just an undergrad with a propensity for stats analyses! I would verify everything I said in the PM with some literature review.

 

[pazzer2]

A power analysis (or sample size calculation) is needed when doing any type of prospective research. The methods to calculate power will vary depending on what your primary endpoint is. Here are some examples:

comparing 2 means: T test (paired or unpaired)
comparing 2 proportions: binomial distribution
comparing 2+ means: ANOVA
compare correlation among 2 variables: Pearson correlation coefficient
compare 2 survival rates: kaplan meier

You don't explain whether your instrument is continuous or not, as this will have an impact. When designing any type of scale, it is always better to have something that can be summed into a continuous variable. The first step you need to do is decide how this instrument will be analyzed. Will each of the items be treated separately? Will you have a likert type score? Will the scale be ordinal? or continuous.

 

[751866]

A power analysis (or sample size calculation) is needed when doing any type of prospective research. The methods to calculate power will vary depending on what your primary endpoint is. Here are some examples:

comparing 2 means: T test (paired or unpaired)
comparing 2 proportions: binomial distribution
comparing 2+ means: ANOVA
compare correlation among 2 variables: Pearson correlation coefficient
compare 2 survival rates: kaplan meier

You don't explain whether your instrument is continuous or not, as this will have an impact. When designing any type of scale, it is always better to have something that can be summed into a continuous variable. The first step you need to do is decide how this instrument will be analyzed. Will each of the items be treated separately? Will you have a likert type score? Will the scale be ordinal? or continuous.

Why do you write your post as if ordinal and continuous are mutually exclusive? Variables can be either discrete or continuous; measurement levels are one of the following nominal, ordinal, interval, or ratio scaled measurements. Discrete vs continuous and measurement level are two separate properties.

 

[751866]

I just PMed you. Doing power analyses for these seems fairly simple (assuming I understand what you're doing properly). Feel free to ask more questions if you need to.

Do keep in mind that I'm just an undergrad with a propensity for stats analyses! I would verify everything I said in the PM with some literature review.

A power analysis is estimating power from fixed sample sizes whereas a sample size estimate is telling you an estimated sample size required to test a specific hypothesis at a given power under other assumptions. They're related but different targets. In general, sample size calculations aren't simple in the real world. If you're not looking at a range of sample sizes under varying reasonable estimates of inputs, you're doing it wrong. You should never get a single number as your answer; you should have a range and you need to use subject matter expertise to decide which number is the bare minimum, assuming all of your assumptions were correct.

 

[751866]

my third is the one who said I NEED to do it but he is off campus and when I emailed about it he asked me questions like he didn’t remember my study even though he wrote most of the proposal revisions.

He's probably talking a big game but doesn't know how to execute it and hopes you figure it out before he returns...

I asked psychologists/statisticians outside my committee but they didn’t know either.

I guarantee the "statisticians" you asked don't have an MS/PhD in statistics or biostatistics because this would be incredibly unusual for a real statistician not to be able to assist with-- a psychologist or other non statistician, this is not so surprising. Trainings are very different.

My study is on the development of a self report inventory. It has 7 items but may have more or less because step one after submitting to the IRB is to send it to psychologists for review based on their clinical experience. The statistical tests that will be run: 1) split half reliability, 2) internal consistency using cronbachs alpha,

You should look at Krippendorff's alpha for data reliability as well (and look at the original papers and subsequent by Hayes at OSU).

3) test retest reliability, 4) differential validity (giving a well known personality questionnaire along with my instrument).

Why are you insisting on splitting for testing/retest? Why not use a resampling/cross-validation technique that is far better in terms of information use?

Would there need to be one power analysis done for all this, or one for each test?

If you have a limited sample size already known, a power analysis will tell you, based on your other assumptions about the distribution, the power estimated for this particular test. If you are trying to determine how many participants you need to achieve a certain power, this is a sample size estimation-- these are different like measuring MAP and blood pressure-- related but different. You should at least do a sample size calculation to test your primary hypothesis that needs to be very precisely defined-- i.e. the mean systolic blood pressure for all patients on losartan is 10 mm hg lower than the mean systolic bp for all patients on nifedipine (rather than "outcomes are better for losartan compared to nifedipine" which is a common "hypothesis" in medicine, poorly defined).

Also, how do I actually do the power analysis? I know what G power is but that is where everyone gets stuck because they say you can’t do my specific statistical tests on there. Please message me if you can help. Thank you and I’m sorry if this isn’t allowed!

Go to Stat UCLA IDRE Go here, and then read the other parts of the seminar fully, too!