About the ads

Psychology to come undone?

Discussion in 'Psychology [Psy.D. / Ph.D.]' started by perhaps11, Apr 25, 2012.

  1. SDN is a nonprofit organization. Services are made possible through the generous support of SDN members and sponsors. Thank you.
  1. perhaps11

    perhaps11

    Joined:
    Jan 11, 2012
    Messages:
    98
    Status:
    Psychology Student
    SDN 2+ Year Member

    SDN Members don't see this ad. (About Ads)
    I just happened to come by a very interesting (and possibly disturbing) article posted on the popular website, RealClearScience, regarding new efforts in replicating psychological experiments published in prestigious psychological journals (e.g. Journal of Social and Personality Psychology) beginning with articles published in 2008. While I personally welcome the idea of increased replication of psychological studies to ensure validity and reliability, the article and its inspiration, the newly formed Reproducibility Project, seems to project familar doubts that psychology is a sound science. Indeed, the article is literally titled "Psychology to become undone?" and mentions the so-called File Drawer Effect several times. :rolleyes:

    Here is the article:http://chronicle.com/blogs/percolator/is-psychology-about-to-come-undone/29045

    What do you guys make of this article, The Reproducibility Project, and its (possible) impact on research in psychology and psychological science as a whole?
  2. Ollie123

    Ollie123

    Joined:
    Feb 19, 2007
    Messages:
    3,825
    Status:
    Psychology Student
    SDN 7+ Year Member
    Been a hot topic in my department the last couple weeks.

    The overarching idea is nothing new and I've been following discussion of it for years. Personally, I couldn't be happier with the fact that it is occurring though obviously the devil is in the details. Probably the most disheartening aspect of graduate school for me was realizing the relative sloppiness of most research, and the fact that doing "good science" (i.e. careful, conservative analytic approaches, rigorous procedures for checking data) not only takes more work it almost inevitably leads to more convoluted and or null findings, and "worse" manuscripts (worse in quotations to indicate the interpretive ambiguity). Some of the most successful folks I know have actively argued against doing things like checking for normality, outliers, etc.

    My suspicion is that some effects will replicate, some won't. Depending on the particular methodology, I'd estimate the overall will be moderate with some variance across research areas. Not to pick on social, but I suspect they will likely be in the worst shape based on what I've heard from folks in the field, with a number of INCREDIBLY sketchy practices being commonplace and likely more variability attributable to the sample by nature of the things they study (e.g. it seems reasonable that views of gender and sexuality would be affected by geographic location/political climate relative to say...neural processing of vertical vs. horizontal lines).

    The main concern I've heard is that it will make psychology look bad. That's a possibility, but one I can live with. I don't think we are much worse than other fields (if at all). At the very least, medical research is also largely a train wreck! If anything, the fact that we are the driving force behind such a movement might open new avenues for psychologists to pursue as an authority on the "Science of Science" so to speak. Now is not the time, but if I was at a point I was running my own lab I would likely be throwing myself full force into the project and other related studies.
  3. Psychadelic2012

    Psychadelic2012 PhD Student

    Joined:
    Sep 17, 2011
    Messages:
    824
    Status:
    Psychology Student
    SDN 2+ Year Member
    Why is this? Is it because they find significance only when the data is raw? And you are saying this is a bad thing, right?
  4. Ollie123

    Ollie123

    Joined:
    Feb 19, 2007
    Messages:
    3,825
    Status:
    Psychology Student
    SDN 7+ Year Member
    I am definitely saying its a bad thing - at least ethically. Like I said, the disheartening thing is unless the system changes, its probably good for your career. For the record, I should have said "successful early career folks" (i.e. other grad students and young faculty). Have yet to have a senior person say that, though I'm sure many do.

    And yes, it basically boils down to the results looking better that way and the relative ambiguity about how to handle data. I also know several people who have admitted they check for outliers only when hypotheses don't come out...not when they do. I'm sure the prevalence of that is just astronomical.
  5. Psychadelic2012

    Psychadelic2012 PhD Student

    Joined:
    Sep 17, 2011
    Messages:
    824
    Status:
    Psychology Student
    SDN 2+ Year Member
    Gasp. Sigh. Belch. Ugh.

    Seriously, though, it is very disappointing. And it is, of course, directly related to the fact that publishing null findings is...well, non-existent. I've never heard it said explicitly, but it is implied that whatever gets you a significant finding is what needs to be done. "Fishing? No, we don't fish. Fishing is bad. But keep looking, we'll find something."
  6. Ya Ya

    Ya Ya

    Joined:
    Jul 24, 2011
    Messages:
    101
    Status:
    Psychology Student
    SDN 2+ Year Member
    Seems like a really great direction to go in. The disparaging comments about our discipline was surprising. We're a useless field apparently. Sigh.
  7. AcronymAllergy

    AcronymAllergy Neuropsychologist Moderator

    Joined:
    Jan 7, 2010
    Messages:
    5,323
    Status:
    Psychologist
    Psychologist SDN 2+ Year Member
    Well, in response to the negative tone of the article (and probably some of the comments), Dr. Nosek posted the following as his own comment, so it looks like he agrees with others' disappointments at the snarkiness of the blog:

  8. cara susanna

    cara susanna Predoctoral Intern

    Joined:
    Feb 10, 2008
    Messages:
    4,526
    Location:
    Midwest
    Status:
    Psychology Student
    SDN 5+ Year Member
    I'm reading the comments and, wow, I'd forgotten how much people hate and disrespect our field. Wasn't there just something published recently about how ALL scientific fields are having this problem?
    Last edited: Apr 26, 2012
  9. AcronymAllergy

    AcronymAllergy Neuropsychologist Moderator

    Joined:
    Jan 7, 2010
    Messages:
    5,323
    Status:
    Psychologist
    Psychologist SDN 2+ Year Member
    I honestly wouldn't place too much credence in the comments as reflective of society's views psychology. Anonymous comments on pretty much anything online, as we've all probably seen, more often than not fall into the excessively-negative and critical variety. Just look through some of the things people "say" in response to news stories on CNN.com, for example.

    I have a feeling that if the subject were sociology, economics, law, or history instead of psychology, the general gist of the comments would be the same. Although yes, psych does have a history of sometimes being described in a less-than-favorable light when compared to the natural/"hard" sciences.
  10. cara susanna

    cara susanna Predoctoral Intern

    Joined:
    Feb 10, 2008
    Messages:
    4,526
    Location:
    Midwest
    Status:
    Psychology Student
    SDN 5+ Year Member
    That's true, but it's also a Chronicle article where the comments tend to be more educated. At least, from my experience anyway.
  11. IT514

    IT514

    Joined:
    Oct 14, 2007
    Messages:
    274
    Status:
    Post Doc
    SDN 5+ Year Member
    Some of those comments made me laugh.

    I remember having a conversation with a physics grad student who liked to sit in on psych thesis presentations. He remarked that all psych studies are weak because we can get away with calling a .6 correlation "high." My comment to him was, "So what you are saying is that we should study humans EXACTLY same way we study rocks and planets?" His reply was something to the effect of "yeah, and what would be the problem with that?" I guess he wasnt paying attention during the lecture on closed vs open physical systems.
  12. wigflip

    wigflip

    Joined:
    Oct 18, 2010
    Messages:
    1,555
    SDN 2+ Year Member
    Sorry to say it, but I think this (hatin' on psych) is pretty commonplace in academia. When I started my current grad program, I wasn't aware of how negatively psych was regarded. To be fair, the humanities folks I've been exposed to seem to revile all the social sciences (especially where quantitative methods dominate), but I've heard profs and grad students in a variety of social science and interdisciplinary fields denigrate psych in particular.

    Also to be fair, at least some psychology folk seem to be complicit in upholding the traditional disciplinary hierarchy (with psych nestled close to the STEM fields on top, other social sciences below, humanities at the bottom). I know my undergrad psych profs and TAs didn't exact gild the lily when it came to discussing their views of other social science disciplines.

    Though I think there are concrete reasons for these cross-disciplinary pissing contests (epistemological, theoretical, methodological differences, as well as the reality of differential funding from universities), cross-disciplinary rivalry and hostility isn't collegial or productive. I'm interested in what other disciplines have to offer in terms of the types of questions they are uniquely suited to answer and their approach to doing so.
  13. roubs

    roubs Ph.D. Student

    Joined:
    Sep 15, 2006
    Messages:
    1,084
    Status:
    Psychology Student
    SDN 7+ Year Member
    Before I read Dr. Nosek's comment I was formulating a reply along those lines, of course he put it better than I could have :)

    Saying psych research will come undone is a bit dramatic.

    And to be clear, realclearscience aggregates stories. It is a blog post and aren't we shocked a blog post is overselling its point?

Share This Page


About the ads