Student Doctor Network Forums - View Single Post

Member 138116 · 04-25-2012, 03:50 PM

Been a hot topic in my department the last couple weeks.

The overarching idea is nothing new and I've been following discussion of it for years. Personally, I couldn't be happier with the fact that it is occurring though obviously the devil is in the details. Probably the most disheartening aspect of graduate school for me was realizing the relative sloppiness of most research, and the fact that doing "good science" (i.e. careful, conservative analytic approaches, rigorous procedures for checking data) not only takes more work it almost inevitably leads to more convoluted and or null findings, and "worse" manuscripts (worse in quotations to indicate the interpretive ambiguity). Some of the most successful folks I know have actively argued against doing things like checking for normality, outliers, etc.

My suspicion is that some effects will replicate, some won't. Depending on the particular methodology, I'd estimate the overall will be moderate with some variance across research areas. Not to pick on social, but I suspect they will likely be in the worst shape based on what I've heard from folks in the field, with a number of INCREDIBLY sketchy practices being commonplace and likely more variability attributable to the sample by nature of the things they study (e.g. it seems reasonable that views of gender and sexuality would be affected by geographic location/political climate relative to say...neural processing of vertical vs. horizontal lines).

The main concern I've heard is that it will make psychology look bad. That's a possibility, but one I can live with. I don't think we are much worse than other fields (if at all). At the very least, medical research is also largely a train wreck! If anything, the fact that we are the driving force behind such a movement might open new avenues for psychologists to pursue as an authority on the "Science of Science" so to speak. Now is not the time, but if I was at a point I was running my own lab I would likely be throwing myself full force into the project and other related studies.

04-25-2012, 03:50 PM	#2
Member 138116 3K Member Status: Psychology Student Join Date: Feb 2007 Posts: 3,544	Been a hot topic in my department the last couple weeks. The overarching idea is nothing new and I've been following discussion of it for years. Personally, I couldn't be happier with the fact that it is occurring though obviously the devil is in the details. Probably the most disheartening aspect of graduate school for me was realizing the relative sloppiness of most research, and the fact that doing "good science" (i.e. careful, conservative analytic approaches, rigorous procedures for checking data) not only takes more work it almost inevitably leads to more convoluted and or null findings, and "worse" manuscripts (worse in quotations to indicate the interpretive ambiguity). Some of the most successful folks I know have actively argued against doing things like checking for normality, outliers, etc. My suspicion is that some effects will replicate, some won't. Depending on the particular methodology, I'd estimate the overall will be moderate with some variance across research areas. Not to pick on social, but I suspect they will likely be in the worst shape based on what I've heard from folks in the field, with a number of INCREDIBLY sketchy practices being commonplace and likely more variability attributable to the sample by nature of the things they study (e.g. it seems reasonable that views of gender and sexuality would be affected by geographic location/political climate relative to say...neural processing of vertical vs. horizontal lines). The main concern I've heard is that it will make psychology look bad. That's a possibility, but one I can live with. I don't think we are much worse than other fields (if at all). At the very least, medical research is also largely a train wreck! If anything, the fact that we are the driving force behind such a movement might open new avenues for psychologists to pursue as an authority on the "Science of Science" so to speak. Now is not the time, but if I was at a point I was running my own lab I would likely be throwing myself full force into the project and other related studies.