Research Question

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
JackD

JackD

-
10+ Year Member
15+ Year Member
Joined
Nov 8, 2007
Messages
498
Reaction score
3
Members do not see this ad.
This is a specific question about something from one of my classes. Sorry if this is against forum rules, we will see soon enough if it is.

Anyway, i am just starting my psychological research methods class. While interesting, it is far from straight forward material, so i am a little confused about a concept. The teacher put us in groups yesterday and gave us a sheet with three proposed research studies. They are all flawed and we are supposed to point out what is wrong with them. Among other information, one of the hypothesis for one of the experiment is stated "that men and women have similar attitudes towards homeless individuals."

A member of my group strongly insists that hypothesis is attempting to confirm the null hypothesis. He is ardently supporting this position and says we have to put that as part of our answer for what is wrong with this study, among other things. I don't think it is trying to confirm the null hypothesis at all. I don't think it has anything to do with that. I thought i had a grasp of what that concept was but his passion towards this issue gives me some doubt. So am i right or is he right?
 
Thanks. I was a little concerned for a second there. My thread got moved and the only person to respond is a moderator. 9 times out of 10, that would signal the locking of the thread.
 
I am a little confused by your question....but a general rule about hypothesis testing is that the null hypothesis can never be confirmed, it can only be rejected (i.e. if one finds a single piece of evidence to counter the null, the null is rejected; but nobody can be 100% certain that no counter-evidence exists, thus the null can never be accepted).
 
JackD said:
This is a specific question about something from one of my classes. Sorry if this is against forum rules, we will see soon enough if it is.

Anyway, i am just starting my psychological research methods class. While interesting, it is far from straight forward material, so i am a little confused about a concept. The teacher put us in groups yesterday and gave us a sheet with three proposed research studies. They are all flawed and we are supposed to point out what is wrong with them. Among other information, one of the hypothesis for one of the experiment is stated "that men and women have similar attitudes towards homeless individuals."

A member of my group strongly insists that hypothesis is attempting to confirm the null hypothesis. He is ardently supporting this position and says we have to put that as part of our answer for what is wrong with this study, among other things. I don't think it is trying to confirm the null hypothesis at all. I don't think it has anything to do with that. I thought i had a grasp of what that concept was but his passion towards this issue gives me some doubt. So am i right or is he right?
Click to expand...

Erg, that's weird. It would be odd to make a hypothesis that there's no significant difference, but I can think of situations where you would do it (e.g. prior research finds a difference, but you think there's something wrong with sampling or whatever, so you predict that your change to the protocol would eliminate the gender difference).

If it's just a conjecture (e.g. "We anticipate no significant gender effects") then it's certainly not wrong. That's just stating that the researcher wants to use gender as a potential covariate (maybe some research suggests it might have an effect) but doesn't expect anything to come of it.
 
After the research is conducted regarding this question (probably using some type of assessment to measure attitudes about homeless people?) the null hypothesis would be somehting like "there are no sig. differences in men and women's attitudes about homeless people." So, like the previous two posters, I too am a little confused about the question.

Is the hypothesis the only piece of the puzzle you are given? Because on it's face, while not the most mind blowing concept, it is a totally valid question to ask.

If I were assigning this kind of project to students, I would most likely include some methodological problem that makes it difficult to generalize to "men and women" since that is a huge group of people. (Everyone on earth). For example, does your study use all college students? Is there a restricted age range or geographic location or culture? See what I am getting at?

Also, the construct "attitude" can be manipulated to mean quite a few things. If your professor is really trying to stump you, look into that.
 
Ok, i'll just post the entire thing.

Directions: Identify and explain the problems (e.g., internal validity, external validly, construct validity, arguing for the null hypothesis) and redesign the study to fix the problems.

Study #2

You are investigating the hypothesis that men and women share similar attitudes towards homeless individuals. What you plan to do is survey an equal number of men and women who work in area homeless shelters as volunteers about their attitudes towards the homeless.

I can pretty well figure out most of the problems with it but someone in my group is saying that this is arguing for the null hypothesis. I think don't think it is and no one else in the group has any clue.
 
JackD said:
Ok, i'll just post the entire thing.

Directions say: Identify and explain the problems (e.g., internal validity, external validly, construct validity, arguing for the null hypothesis) and redesign the study to fix the problems.

Study #2

You are investigating the hypothesis that men and women share similar attitudes towards homeless individuals. What you plan to do is survey an equal number of men and women who work in area homeless shelters as volunteers about their attitudes towards the homeless.

I can pretty well figure out most of the problems with it but someone in my group is saying that this is arguing for the null hypothesis. I think don't think it is and no one else in the group has any clue.
Click to expand...

My not so subtle hint about what I think the professor wants you to look at is in red. This presents itself as a threat to the external validity of the study.
 
73BARMYPgsp said:
My not so subtle hint about what I think the professor wants you to look at is in red.
Click to expand...

The one thing we were able to agree on is that part of it was an external validity problem since the sampled group was not representative of all men and women.
 
Aside from the obvious problem, I think your classmate is right. Trying to prove the null is listed as one of the prof's things to watch for, so it's clearly what he/she wants you to write.
 
JockNerd said:
Aside from the obvious problem, I think your classmate is right. Trying to prove the null is listed as one of the prof's things to watch for, so it's clearly what he/she wants you to write.
Click to expand...

Agreed. If the professor wants you to analyze all of the possible threats listed, you could come up with at least one way each of those might be a problem.

Take them one at a time:

Internal Validity is another way of saying "reliability." Can you think of some ways this study might not be reproduceable?

Construct Validity: What instrument is being used to measure the construct of "attitude." Is it valid?

Confirming the null hypothesis: As mentioned, there is a bit of confirmation bias built in by the way the hypothesis is worded, which is sometimes unavoidable and may be a function of the a priori nature of the question. (Assuming you will find or not find something)
 
JockNerd said:
Aside from the obvious problem, I think your classmate is right. Trying to prove the null is listed as one of the prof's things to watch for, so it's clearly what he/she wants you to write.
Click to expand...

There are two other studies that i didn't list here. I would have to really look at them to see if it were possible for either of those hypotheses.
 
73BARMYPgsp said:
Internal Validity is another way of saying "reliability." Can you think of some ways this study might not be reproduceable?
Click to expand...


I don't think that's what internal validity means at all... validity and reliability are two different things. Internal validity refers to whether you are sure that a causal relationship really exists.
 
JackD said:
It basically says that aggressive children aren't as good at coming up with several solutions when presented a problem, when compared to "normal" children.
Click to expand...

That's not arguing the null. Arguing for the null is trying to prove that there's no difference or no relationship.
(Well, not necessarily, but I'm sure that's what the prof means)

This scenario is suggesting a relationship; aggression is correlated negatively with problem solving.
 
psy19 said:
I don't think that's what internal validity means at all... validity and reliability are two different things. Internal validity refers to whether you are sure that a causal relationship really exists.
Click to expand...

Right. And you can never really be sure, but if you can reproduce the results, you get "more sure." Reliability/Internal Validity/reproduceability are roughly the same thing in this context.

However, you are right--reliability usually has more to do with subject design and concepts like internal validity and internal consistency are reserved for describing instruments and their ability to measure the same thing over and over again. But conceptually, they are comparable.
 
JockNerd said:
That's not arguing the null. Arguing for the null is trying to prove that there's no difference or no relationship.
(Well, not necessarily, but I'm sure that's what the prof means)

This scenario is suggesting a relationship; aggression is correlated negatively with problem solving.
Click to expand...

I agree. I looked at it again and realized it wasn't. I don't think any of them are arguing for the null hypothesis . The final one says that children with strong social support will be less likely to develop depression and/or anxiety when bullied compared to those with little social support. The problem with that study is he says that the way of determining who has strong social support vs. weak is simply asking children how strong their social support is.

I am thinking that each of the three examples has two problems because the assignment is apparently worth six points.
 
73BARMYPgsp said:
Right. And you can never really be sure, but if you can reproduce the results, you get "more sure." Reliability/Internal Validity/reproduceability are roughly the same thing in this context.
Click to expand...


Hm, I'd have to disagree. You can have reliability/reproduceability without any internal validity (ie, doing the same experiment 100 times and getting the same results... but you still could have very low internal validity if you don't know if x is really causing y). To my knowledge, having a very reliable study has no effect on the validity.

But I'm not a stats expert, so I guess I could be wrong 🙄


*oops I didn't read your edit
 
psy19 said:
Hm, I'd have to disagree. You can have reliability/reproduceability without any internal validity (ie, doing the same experiment 100 times and getting the same results... but you still could have very low internal validity if you don't know if x is really causing y). To my knowledge, having a very reliable study has no effect on the validity.

But I'm not a stats expert, so I guess I could be wrong 🙄


*oops I didn't read your edit
Click to expand...

Oh believe me, if you watch this tread for about a week, there will be 50 different opinions about this. My dissertation is all about internal consistency/internal validity/reliability of an instrument my professor developed so I am learning the nuances of these concepts and how they interact as we speak. For something really crazy, check out Item Response Theory. 🙂

Poor OP. He just wanted help with his Research Methods project.
 
I should probably leave now before i get more confused. My grasp of this subject is too weak to sort out the facts in this debate that is guaranteed to follow. I will most likely have problems understanding this later. It is a lot like trying to teach a four year old calculus. The more you try, the more confused they are going to get. I'm the four year old here, so i should run off before my frail knowledge is damaged beyond repair.

Thank you all for your help. I believe i got my answered my question, anything said beyond that can only hurt me at this point.
 
JackD said:
I don't think any of them are arguing for the null hypothesis .
Click to expand...

I disagree; the first one you posted had a hypothesis that there was no relationship. That's arguing for the null. 🙂
 
73BARMYPgsp said:
Oh believe me, if you watch this tread for about a week, there will be 50 different opinions about this. My dissertation is all about internal consistency/internal validity/reliability of an instrument my professor developed so I am learning the nuances of these concepts and how they interact as we speak. For something really crazy, check out Item Response Theory. 🙂
Click to expand...

Internal validity? Of a measure?

Internal validity is about making sure a manipulation is the cause of some change in an experimental design. It's usually only important in experimental designs. I think you're thinking of construct validity.
 
JockNerd said:
Internal validity? Of a measure?

Internal validity is about making sure a manipulation is the cause of some change in an experimental design. It's usually only important in experimental designs. I think you're thinking of construct validity.
Click to expand...

Yep. I don't know how that got in there. I was probably harkening back to the short cuts I used in order to pass the research comp.😀
 
disagree; the first one you posted had a hypothesis that there was no relationship. That's arguing for the null. 🙂
Click to expand...
Well now, it turns out you are correct. Thankfully class was canceled yesterday and i didn't try to stop our group from putting that down.

I finally got around to reading the section of the book that talks about null hypotheses and it turns out there are two parts two it. First, the one i knew about, is that you can't hypothesize that your treatment or whatever will have no effect. Second, it means that you can't say that two treatments will have the same effect. So in other words, finding no difference between men and women on their attitudes towards homeless people doesn't mean there is no difference, only that you didn't find a difference, and therefore you have a hypothesis that can never be proved correct, also known as "null hypothesis".

I yield to your superior knowledge of research methodology.
 
Re: Jon Snow's comments

I think you're totally right JS, and that was the reason behind my first posting that the question seemed a little weird.

JackD (and others who might be in stats), maybe keep in mind that your lecturer might not be totally clear about a lot of topics in research design. Sometimes grad schools just throw a grad student to the wolves, teaching a stats course that they don't have a good hang of. Check out what your prof WANTS you to answer.... in this case, arguing the null was one of your choices for a "problem" in design, so that's what the prof wants you to answer.😉
 
I think the professor's issue is just with the wording. You couldn't ever say "this study proves that men and women share similar opinions." You can only say that it does not provide evidence that there is a difference.

The same argument came up in the discussion of whether the death penalty detours crime. You can't prove that it doesn't; you can only state that there is no evidence that it does.
 
amy203 said:
The same argument came up in the discussion of whether the death penalty detours crime. You can't prove that it doesn't; you can only state that there is no evidence that it does.
Click to expand...


Technically it detours that person's ability to commit a crime (being dead and all), though it is harder to say if it detours others. 😀
 
amy203 said:
The same argument came up in the discussion of whether the death penalty detours crime. You can't prove that it doesn't; you can only state that there is no evidence that it does.
Click to expand...

Therapist4Chnge said:
Technically it detours that person's ability to commit a crime (being dead and all), though it is harder to say if it detours others. 😀
Click to expand...

All this time I thought the argument was that it deterred crime. Not sent it on a slightly longer route. Say, through the country?

😀
 
JockNerd said:
All this time I thought the argument was that it deterred crime. Not sent it on a slightly longer route. Say, through the country?

😀
Click to expand...

I thought it. You wrote it.
 
Oops! Oh spellcheck, you are a blessing and a curse! I'll leave my comment unedited for posterity's sake.
 
Here is a new one for you. I must say, this research stuff is confusing. Is construct validity the ultimate, overarching validity concept? Is that the final goal that a test is trying to achieve is construct validity? Is that the holy grail if you will? If a test is valid in every other way, then that is when you achieve construct validity?

It seems to me that if a test has construct validity, that tells you everything you need to know about its validity.

Right, wrong, close?
 
JackD said:
It seems to me that if a test has construct validity, that tells you everything you need to know about its validity.
Click to expand...

I think I see what you're saying and you'd be right; if you knew 100% that you had construct validity, then you are, by definition, measuring the thing you want to measure. So, yeah, that would be all you need. But, the problem is that there's no test of construct validity; it's pretty subjective. Does the WAIS measure intelligence? No, the WAIS measures how well you do on the WAIS. In contrast, for example, you can test divergent validity (people who score low on your measure should score high on measure x) and convergent validity (scores on your measure should be correlated positively with measure y).
 
you'd be right
Click to expand...
Yes! I finally figured something out in this convoluted and dry subject! Thank you Jesus!
 
Another for you. What is the difference between a "moderator variable" and a "confounding variable"?
 
JackD said:
Another for you. What is the difference between a "moderator variable" and a "confounding variable"?
Click to expand...

Great question. A moderator variable changes (moderates) when and for whom the independent variable causes the dependent variable. A confounding variable is quite different-- it represents a factor that causes both the "independent variable" and the dependent variable. In the case of a confounding variable, the proposed IV does not cause the DV at all-- rather, the confounding variable causes both! Only the confounding variable represents a threat to internal validity. I work better with examples, so let's use one (completely made up, I might add).

Imagine I am conducting a study to determine whether depression causes cognitive impairment. I run depressed and nondepressed people through a battery of cognitive tasks. I find that depression DOES influence cognitive performance, but only for the female subjects. Gender here is a moderating variable-- it moderates the effect of age on cognitive performance. Depression causes cognitive impairment, but only for females.

A confounding variable, by contrast, is one that might cause both depression and cognitive impairment and give rise to a spurious association between the two. Age, for example, could represent a confounding variable in this study.

Does that help?
 
as i understand it:

they are very different constructs. a moderator variable moderates. gender could be a moderator variable, as in "the intervention resulted in a signifigant reduction in depression, and the results were moderated by gender ; men using the intervention improved more than women".

a confounding variable confuses the issue. so looking at how percieved racism may impact recovery from stroke, another variable, such as low socio-economic status, may be a confounding variable, confusing us at to whether the poor stroke recovery was due to the stress of percieved racism (and chronic stress can result from repeated expereince of racism and of course stress is bad for the body) or due to other things linked to percieved racism, like less $$$ resources, or instutitional racism (like poorer care in the hospital), or different cultural values regarding help seeking, diet, whatever.
 
thewesternsky said:
Great question. A moderator variable changes (moderates) when and for whom the independent variable causes the dependent variable. A confounding variable is quite different-- it represents a factor that causes both the "independent variable" and the dependent variable. In the case of a confounding variable, the proposed IV does not cause the DV at all-- rather, the confounding variable causes both! Only the confounding variable represents a threat to internal validity. I work better with examples, so let's use one (completely made up, I might add).

Imagine I am conducting a study to determine whether depression causes cognitive impairment. I run depressed and nondepressed people through a battery of cognitive tasks. I find that depression DOES influence cognitive performance, but only for the female subjects. Gender here is a moderating variable-- it moderates the effect of age on cognitive performance. Depression causes cognitive impairment, but only for females.

A confounding variable, by contrast, is one that might cause both depression and cognitive impairment and give rise to a spurious association between the two. Age, for example, could represent a confounding variable in this study.

Does that help?
Click to expand...

what westernsky said! 😛
 
I think so, let me just see if i got this.

A moderator variable works with the independent variable to have an effect on the strength of a relationship to the dependent variable? It kind of gets between the IV and DV to effect exactly how they relate to each other without actually causing the dependent variable. So the independent variable is causing the dependent variable but a moderating variable sort of tweaks how they relate? Let me try to make an example here. Putting a CD into your CD player will cause music and a volume knob will change the intensity of the music. So the CD is the independent variable, the music is the dependent variable, and the volume knob is the moderator variable. Yes? No?

A confounding variable makes it look like A causes B, when in fact C (the confounding variable) causes A and B.

I hope i at least ballparked that one.
 
Ok, so what's a mediating variable, or a mediator? I see that in the literature a lot.
 
empathiosis said:
Ok, so what's a mediating variable, or a mediator? I see that in the literature a lot.
Click to expand...

i picture the mediator allowing two other variables to talk to each other (i'm doing work exploring both mediator and moderatior relationships among variables related to our student adjustment to college).

in our preliminary, correlational study, moderator variables were gender, for instance, whereas the impact of percieved social support (the perception that there are people there for you to support you if you need them) on how well students do in college was mediated by depression. so percieved social support reduced feeling of depression, and reduced feelings of depression improved how well adjusted students felt to college. social support, by itself did not however have a signifigant impact on student adjustment. so social support does impact student adjustment, but not directly. instead, it's impact on adjustment is mediated by depression.

hows that?
 
empathiosis said:
Ok, so what's a mediating variable, or a mediator? I see that in the literature a lot.
Click to expand...

here you go: http://www.gseis.ucla.edu/courses/ed230a2/mediator.html

That site has visuals for the best ways of describing the relationships; a path diagram for mediation and a line chart for moderation. Couldn't be more clear!

Mediator Y explains part of all of the relationship between variable X and outcome Z. Basically, if you wanted to do a study on mediation you know that X and Z are related, but you think they're related because of Y.

Moderators just alter the strength of relationships. Random made-up illustrative example: So, income as a 2-level moderator of depression and social support: Below subsistence level, there's no relationship btwn depression and social support (everyone is depressed even when they have social support because they can't afford food), but above subsistence level increased social support is associated with lower levels of depression.

Good question. So many people mix those up. I've seen them get mixed up in journal articles.
 
thanks jocknerd and psybee. i think you've explained it well. and thanks to jackd for starting the thread.
 
okay, now I have a question -- what is an abc correlation?

thanks!
 
I had my first research test today and since i am always paranoid (very paranoid) about these things before i get the results, there is one question i want to make sure i got right. I think i did but let me run it by you guys.

The question was: Which of these shows the strongest correlation?

A. +.23
B. +.45
C. +1.25
D. -.90

Let me just make 100% sure. The answer is D because there is no such thing as a 1.25 correlation, yes? I don't think two variables can have 156% overlap. I know the standard correlational studies use the -1 to 1 (the pearson correlation i think they call it), in which case the answer is obvious. However, i just want to make sure there isn't one i didn't know about that actually goes beyond 1.0 Maybe he was asking about some alternate design that i was not aware of. The 1.25 thing seemed so wrong that it kind of looped back around and made me wonder, "wait, is there something i am not getting here?" Can we say with a high level of certainty, that the answer is D?

There isn't correct?
 
JackD said:
There isn't correct?
Click to expand...

Yes again. You're becoming quite the stats guru.

edit:
I don't think two variables can have 156% overlap
Click to expand...

And good catch on using the R^2 to check your hypothesis. The stats is strong with this one.
 
JackD said:
I had my first research test today and since i am always paranoid (very paranoid) about these things before i get the results, there is one question i want to make sure i got right. I think i did but let me run it by you guys.

The question was: Which of these shows the strongest correlation?

A. +.23
B. +.45
C. +1.25
D. -.90

Let me just make 100% sure. The answer is D because there is no such thing as a 1.25 correlation, yes? I don't think two variables can have 156% overlap. I know the standard correlational studies use the -1 to 1 (the pearson correlation i think they call it), in which case the answer is obvious. However, i just want to make sure there isn't one i didn't know about that actually goes beyond 1.0 Maybe he was asking about some alternate design that i was not aware of. The 1.25 thing seemed so wrong that it kind of looped back around and made me wonder, "wait, is there something i am not getting here?" Can we say with a high level of certainty, that the answer is D?

There isn't correct?
Click to expand...

D is right; you're fine. 🙂
 
kyril said:
okay, now I have a question -- what is an abc correlation?

thanks!
Click to expand...

I've never heard of this. A google search turned up stuff about engineering and an "Abortion-Breast Cancer correlation."

Where did you run across this? If it was just in lecture maybe it's just a funky way of saying "Variable A is related to B, B is related to C," although that's useless information for a mediational analysis (i.e. unless A and C are related, there's no mediation).
 
You must log in or register to reply here.

Similar threads

postbaccpsychology
Post-Bacc Research Experience
Replies
3
Views
1K
chicandtoughness
chicandtoughness
EKS9
PhD/PsyD SEEKING PARTICIPANTS: IRB Approved Research Study
Replies
0
Views
309
EKS9
EKS9
N
  • Locked
Question
Replies
20
Views
1K
psych.meout
P
B
Bad research mentorship?
Replies
17
Views
2K
beantownpsych
B
MCParent
Paid, remote research intern position
Replies
0
Views
833
MCParent
MCParent
Top