Power vs Validity

[betterfuture]

Advertisement - Members don't see this ad

What is the difference between statistical power and validity? Validity refers to how a data could prove causality? And could someone give me an example of both? Thank you!

 

[ProspectiveKidd]

What is the difference between statistical power and validity? Validity refers to how a data could prove causality? And could someone give me an example of both? Thank you!

I would think its the other way around. Statistical power would be more likely to prove causality as it implies significance(magnitude) and accuracy of the testing methods. I wouldn't necessarily say that validity refers to how data could prove causality directly. I mean, you would need valid and scientifically sound data to do that, and so it is also concerned with the accuracy of testing methods. but Id say that validity determines how much that data actually applies to what were concerned with more than proving anything in itself.

 

[betterfuture]

Would you mind giving an example of both?

 

[ProspectiveKidd]

What is the context? In the most lay terms, validity is accuracy of the data collected. Validity is also concerned with relevance of that data. Statistical power is the measure of effect and gives a stronger indication of a significant and further applicable finding.

If you have an accurate test your data may be valid, but if you only did that test on 5 people you have a low statistical significance. Is this what you're asking?

 

[aldol16]

From my uneducated viewpoint I would think its the other way around. Statistical power would be more likely to prove causality as it implies significance(magnitude) and accuracy of the testing methods. I wouldn't say that validity refers to how data could prove causality either. Id say that validity determines how much that data actually applies to what were concerned with.

Neither of these can prove causality from a scientific viewpoint simply because only experiments can prove causality, not correlations drawn from data.

OP, validity has to do with how well the test(s) you're doing actually measure what you're claiming to measure. There are multiple types, including construct and face validity. For example, if you're trying to measure whether someone has skin cancer and you're measuring the number of moles they have, that's not exactly a valid test. Statistical power, in layman's terms, has to do with how good your test is statistically. It doesn't have anything to do with the idea you're trying to measure but rather whether your test sufficiently measures something. So if you're trying to do that same test above with skin cancer and do it with a sample of 100,000 patients, your test might have statistical power but not validity. If you're doing it on three patients, it's got no power or validity.

 

[ProspectiveKidd]

Neither of these can prove causality from a scientific viewpoint simply because only experiments can prove causality, not correlations drawn from data.

See my edited posts and tell me if it makes sense to you. I believe I'm saying the same thing in post #3

 

[ProspectiveKidd]

And no statistical power does not prove causality, but a causal finding with low statistical significance diminishes the effect. Same idea with the sample size.

 

[betterfuture]

So this is what these terms mean:

Validity: how well an assessment measures what one is trying to measure.

Example: Testing patients for how they each react to stress during different situations by measuring cortisol levels at baseline and during testing.

Power: how well an experiment is setup to ensure results aren't by chance

Example: randomized groups, control groups, and large sample size to measure patients reaction to stress during different situations.

??

 

[ProspectiveKidd]

So this is what these terms mean

Validity: how well an assessment measures what one is trying to measure.

Example: Testing patients for how they each react to stress during different situations by measuring cortisol levels at baseline and during testing.

Power: how well an experiment is setup to ensure results aren't by chance

Example: randomized groups, control groups, and large sample size to measure patients reaction to stress during different situations.

??

Yes.

The example of validity provides an accurate way of measuring stress.

The example of power provides assurance that this data is not coincidental and thus carries more weight.

 

[betterfuture]

Thank you!! @ProspectiveKidd @aldol16

 

[ProspectiveKidd]

Np. Again, you could have measured those stress levels accurately, and had perfectly VALID data. You know that the stress level has gone up or down. However, if you don't randomize the group to minimize confounding variables, or have enough people for that data to actually mean something significant, it has less power.

 

[aldol16]

See my edited posts and tell me if it makes sense to you. I believe I'm saying the same thing in post #3

And no statistical power does not prove causality, but a causal finding with low statistical significance diminishes the effect. Same idea with the sample size.

Yes, sorry, I wrote my post before your new one came up. I just didn't like how you used the word "prove." The new post is great!

 

[ProspectiveKidd]

I just didn't like how you used the word "prove."

Meh, I agree. Should have said reinforce or something similar.