The Ethics of Data Interpretation

[DendWrite]

---

Members do not see this ad.

I know that this is a touchy subject around my lab, but how do you guys feel about the way some data is interpreted? You look at Nature papers with Western blots that have the backgrounds bleached out to completely white, nearly every paper shows a "representative" sample (which we all know is them picking out the best ones and running them on the same gel) of their results. But from my experience, the real results NEVER look like that. Some of the samples are good, some are bad -- variation in treatment effects, etc. Yet everyone in my lab who I've worked with still does this. I'm not saying it's lying, but it makes me uneasy. Why is there this hesitation in scientific papers to admit the slight possibility that you might be wrong...that your n=3-4 in vivo sample is just something that occurs under your hands and not universally?

This has just been a problem for me among people I work with: it seems like all too often we are working backward from the hypothesis to come up with data to support it (and rejecting data that contradicts) until we end up supporting our hypothesis (p<0.05 of course). And, similarly, people with the other side of the story could take the data that WE threw out and support THEIR hypothesis.

I'm not saying that research is subjective, because I really don't believe that. But have any of you encountered stuff like this, and do you think it is one of those things that you accept because "everyone else is doing it?" Maybe I was naive before doing research, but I quickly realized that the whole "open mind to any new discoveries" is all-too-quickly supplanted by "open mind to data that will fund my next grant." Any experiences?

 

[Neuronix]

Why is there this hesitation in scientific papers to admit the slight possibility that you might be wrong...that your n=3-4 in vivo sample is just something that occurs under your hands and not universally?

Because otherwise your paper will be rejected. I would have agreed with you five years ago. But then I realized that putting any little scrap of data or even words that your data isn't 100% solid in a manuscript means that your paper will be rejected. My PI always called it "giving the reviewers the rope to hang you with." My own experience has shown that he was right, over, and over, and over again!

Maybe I was naive before doing research, but I quickly realized that the whole "open mind to any new discoveries" is all-too-quickly supplanted by "open mind to data that will fund my next grant." Any experiences?

No money = no lab = no job.

The biggest question is, how do you trust anyone's research? The answer unfortunately is, you can't. And, with very rare exceptions, you can't publish verification or rebuttal of someone's else's work.

 

[Shifty B]

By the end I didn't even trust my own research...

 

[DendWrite]

Do you guys think that it is still worthwhile to pursue research?

I _really_ enjoy science. But so often I hear things like the "golden age of research is over" and that soon we'll see the "death of the physician-scientist" as research funds dwindle and clinical productivity outranks problem-solving and search for new therapies.

Are you finding that this is the case? I want to make an impact on patients, and I feel like with my skills and talents, the best way to do that is through research. But if the system is broken, maybe no one can do that, and the best way is to become the best clinician possible and treat one at a time...

 

[Build]

The decision to pursue research is ultimately a personal one. That's because even if you are a stunningly blind optimist, you will inevitably encounter challenges in your career. Challenges that will make you question whether you're going in the right direction. I think there is a happy medium somewhere between being willing to just go for it and being aware of what you're signing up for. Look to successful mentors to inspire you. Read the research coming out of top journals and you'll see that there is tangible progress in the advancement of medical science and patient care published on a weekly basis. Talk with people at different phases in a research career to make sure you know what it's like and it's something you want to sign up for.

Personally, I don't buy it that the golden age of research is over. I think things change and it's up to you to adapt and figure out what you need to do to get from point A to point B in your career. If you want to become a physician scientist and use your work to benefit patients, you'll find a way to do it.