Should studies be retracted for methodological weaknesses?

[futureapppsy2]

This author is making a big deal over how his study is statistically unsound and should be retracted, but tbh, I don't see this as retraction-worthy--it's not fraudulent, misreported, or faked data. It's not necessarily even "bad" data/results, just questionable ones. This is why we have replication, quality ratings, and meta-analysis, but I don't know if it's retraction material, unless (as some are speculating) he never got IRB approval, which is an actual ethical issue on his part, seeing as how he actually ran the study.

http://retractionwatch.com/2015/05/...for-some-hours-email-however-says/#more-28700

http://io9.com/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800

---

Members don't see this ad.

 

[Ollie123]

Well, if you read the full article they did some sketchy things (submitted to 20 journals simultaneously). The publisher was not a "real" academic publisher and was just a pay-to-publish journal with a semi-legit-sounding name. The debate doesn't seem to be around retraction but whether it ever should have been published in the first place.

For what its worth, I don't know that the research methodology per se warrants retraction as that just seems silly, unless the were straight up dishonest about the methods (which is a different story). Even then, there is a lot of grey in this process. If we're going to retract every paper that didn't include every single outcome measure collected, I'd guess > 95% of larger-scale studies are getting retracted. Beginning with the entire field of epidemiology and any/all longitudinal projects I've ever heard of. People will always disagree about the relative strengths of a particular study, but I don't think its a good idea to open the door for folks to say "Eh...actually this wasn't so good after all so we're retracting it" as its just too wishy washy.

In this particular case, it seems a goof but they paid a pay-to-publish journal to publish it so what did they think would happen? I don't think the study should be retracted but I do think the journal should be shut down.

 

[futureapppsy2]

Yeah, there's no doubt that the journal is very likely predatory and should be shut down, but this seems more like a weak/pilot study than a retractable one, and I'm not sure that pilot studies are so meaningless as to be universally rejected, especially because having a solid pilot study done can really help in securing funding for a larger, more focused replication or extension study. Tbh, now I want someone to see if they can replicate it with a larger n.

 

[PsyDr]

There is always something to learn from such articles, be it the limitations of a methodology,the way it is written, or as a signifier that a reader is too stupid to understand (E.g., the dead salmon fmri study's true purpose and methodology). I was always taught that the purpose of an article is to push science forward. Sometimes that is a 99 yard touchdown, sometimes a simple yard, sometimes a loss of a yard to get an understanding of the defense's strategy.

 

[deleted343839]

This person was trying to find out whether science journalism is as thoughtless, uncritical, knee-jerk, and clickbait-oriented as he thought it might be, and it looks like his worst suspicions were confirmed. The question of retraction seems peripheral in this context. The only difference between this guy and thousands of other authors who have published in sketchy "journals" is that he knew how to get his paper publicized.