Research is so hard...

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
D

deleted887473

Recently I have been given more trust and responsibility at my research lab, and with that also came some challenges and struggles... I am processing and analyzing a very large batch of data by myself. I was blind to the groups during the processing, paid attention to detail and made sure I did everything right. Now that I have a better picture of the data, it is quite messy and I’m not sure what they mean anymore. I am anxious to whether that’s my fault or just the way the data turned out. I’ll talk to my mentors about it, but I’m worried they’ll doubt my ability.
When I read literature, it always seem like the scientist published just magically end up with super neat data that they can make clear statements about, and I am quite intrigued how they do that.

Members don't see this ad.
 
Recently I have been given more trust and responsibility at my research lab, and with that also came some challenges and struggles... I am processing and analyzing a very large batch of data by myself. I was blind to the groups during the processing, paid attention to detail and made sure I did everything right. Now that I have a better picture of the data, it is quite messy and I’m not sure what they mean anymore. I am anxious to whether that’s my fault or just the way the data turned out. I’ll talk to my mentors about it, but I’m worried they’ll doubt my ability.
When I read literature, it always seem like the scientist published just magically end up with super neat data that they can make clear statements about, and I am quite intrigued how they do that.

A prerequisite to being published is super neat data with clear statements. Not all research is publishable. Talk to your mentors about what they would like to do with the data, attempt to do that, and if it doesn't work out, that's science. Your PI (or you) will most likely have more ideas and ways to analyze it, or may scrap the project entirely.
 
  • Like
Reactions: 1 users
Members don't see this ad :)
Recently I have been given more trust and responsibility at my research lab, and with that also came some challenges and struggles... I am processing and analyzing a very large batch of data by myself. I was blind to the groups during the processing, paid attention to detail and made sure I did everything right. Now that I have a better picture of the data, it is quite messy and I’m not sure what they mean anymore. I am anxious to whether that’s my fault or just the way the data turned out. I’ll talk to my mentors about it, but I’m worried they’ll doubt my ability.
When I read literature, it always seem like the scientist published just magically end up with super neat data that they can make clear statements about, and I am quite intrigued how they do that.
Welcome to the club.
 
  • Like
Reactions: 12 users
A prerequisite to being published is super neat data with clear statements. Not all research is publishable. Talk to your mentors about what they would like to do with the data, attempt to do that, and if it doesn't work out, that's science. Your PI (or you) will most likely have more ideas and ways to analyze it, or may scrap the project entirely.
Will surely do. And yes I am aware only good stuff get published; just taken aback by the amount of not-so-good stuff there can be for every good stuff that comes out.
 
Talking with my stats hat on. I have been doing research for years. I am also usually the stats person in the groups. So, yes, most of the time it would be me handling data, overseeing the quality of data, analyzing the data, and interpreting the data to other group members.

From my experiences, usually it is the data cleaning that will take away most of my time. Yes, you read it correctly, data cleaning. Not modeling, not model checking, not writing the reports, but data cleaning. Ask any statistician, they will tell you that they hate data cleaning. It takes tremendous efforts to clean messy data. You have to know why there are missing values. You have to guess why you see some out-of-range data. You have to be mindful about potential strange patterns as they may arise from systematic biases, from some known, or unknown sources. Extreme values? Rare values? non-compliance? vague documentations? Coding errors? But, your efforts will worth every minute you put in. Hey, just think this way, after you clean the data, maybe you can use them for more than 1 paper!

So, yeah, take a deep breath, smile, and keep on data cleaning - this is one of the most important steps during the research process yet quite often overlooked.
 
  • Like
Reactions: 6 users
Talking with my stats hat on. I have been doing research for years. I am also usually the stats person in the groups. So, yes, most of the time it would be me handling data, overseeing the quality of data, analyzing the data, and interpreting the data to other group members.

From my experiences, usually it is the data cleaning that will take away most of my time. Yes, you read it correctly, data cleaning. Not modeling, not model checking, not writing the reports, but data cleaning. Ask any statistician, they will tell you that they hate data cleaning. It takes tremendous efforts to clean messy data. You have to know why there are missing values. You have to guess why you see some out-of-range data. You have to be mindful about potential strange patterns as they may arise from systematic biases, from some known, or unknown sources. Extreme values? Rare values? non-compliance? vague documentations? Coding errors? But, your efforts will worth every minute you put in. Hey, just think this way, after you clean the data, maybe you can use them for more than 1 paper!

So, yeah, take a deep breath, smile, and keep on data cleaning - this is one of the most important steps during the research process yet quite often overlooked.
Thank you! I talked to my mentors and we are chugging along some more. It actually might still be promising from the look of things. Will keep working on it!
 
When I read literature, it always seem like the scientist published just magically end up with super neat data that they can make clear statements about, and I am quite intrigued how they do that.

Lol, I can't even describe how many hours it takes to get to reputable scientific publication in a high impact journal.

I spent several thousand hours in chemistry graduate school optimizing a reaction and "data cleaning" before it became all tidy and nice-looking in the publication/thesis.
 
  • Like
Reactions: 1 user
Sounds like you're frustrated that your data is not delivering publishable results.
 
  • Like
Reactions: 1 user
Yeah if only everybody could publish and get funding and be happy, butterflies and rainbows

Ohhhh do you mean I am actually doing research by producing messy data? ;) Good to know
Yes, actually you are. Science isn't pretty.

The wise DrMidlife on research: “you've preferably had some exposure to research so you can be convinced that Wakefield used malicious dirtbag methods and is not the savior of the world's children.”

The wise Crayola227 on research: So tired about the whining over the foundation of knowledge that is expected in a physician. We're applied scientists ffs. Own that. If you can't own it and take pride in it, gtfo.
 
  • Like
Reactions: 1 user
Ohhhh do you mean I am actually doing research by producing messy data? ;) Good to know

Now you're getting it! I used to tell my undergrad. researchers (many went to medical school) that research is when you "aren't" getting good data. This should be 90+% of your experiments.

My star researcher once said when starting on research in the process of being close to a publication, after he wasn't having any luck on his project "wow, its so nice to get data doing these reactions". My response was "yes, its not really research since I already optimized the reaction".

Basically, he was just helping me polish up already publishable data for a last spot publication.

OP, research is the messy part.
 
Last edited:
  • Like
Reactions: 1 user
Yes, actually you are. Science isn't pretty.

The wise DrMidlife on research: “you've preferably had some exposure to research so you can be convinced that Wakefield used malicious dirtbag methods and is not the savior of the world's children.”

The wise Crayola227 on research: So tired about the whining over the foundation of knowledge that is expected in a physician. We're applied scientists ffs. Own that. If you can't own it and take pride in it, gtfo.

Gawd stop encouraging me to go on! I've got a future ahead of me!

Now you're getting it! I used to tell my undergrad. researchers (many went to medical school) that research is when you "aren't" getting good data. This should be 90+% of your experiments.

My star researcher once said when starting on research in the process of being close to a publication, after he wasn't having any luck on his project "wow, its so nice to get data doing these reactions". My response was "yes, its not really research since I already optimized the reaction".

Basically, he was just helping me polish up already publishable data for a last spot publication.

OP, research is the messy part.

Those college lab classes where all the experiments work and we can say everything about the data spoiled me into thinking that science is actually clear cut and dry... There's actually soooo much room for interpretation and I love it.
 
  • Like
Reactions: 1 user
Members don't see this ad :)
Basically, he was just helping me polish up already publishable data for a last spot publication.

I am working with a young PhD student, so we are both still trying to figure out the best way to do things, and I suppose that's where the real learning lies :)
It's great that you were able to optimize your methods!
 
I am working with a young PhD student, so we are both still trying to figure out the best way to do things, and I suppose that's where the real learning lies :)
It's great that you were able to optimize your methods!

Took a solid 1.5 years before I went from virtually no publishable data to massively useful data. It happened one night, after my millionth reaction.

It is a unique feeling to know that at one point in time, I was the only person in the world or history of mankind, that knew how this reaction worked.

It's like being Isaac Newton for a day.
 
  • Like
Reactions: 1 users
Took a solid 1.5 years before I went from virtually no publishable data to massively useful data. It happened one night, after my millionth reaction.

It is a unique feeling to know that at one point in time, I was the only person in the world or history of mankind, that knew how this reaction worked.

It's like being Isaac Newton for a day.

Man, scientists should really be able to publish more about their struggles to getting published. I would love to read about how they fail and not just how they succeed. My PI says it's completely normal for a PhD student to graduate with the data they produced in their last few months and now I get why.

And feeling like that must be great... like Thomas Edison.

"In view of this immense amount of thought and labor, my sympathy got the better of my judgment, and I said: 'Isn't it a shame that with the tremendous amount of work you have done you haven't been able to get any results?' Edison turned on me like a flash, and with a smile replied: 'Results! Why, man, I have gotten lots of results! I know several thousand things that won't work!'" -- Walter S. Mallory
 
  • Like
Reactions: 1 user
Took a solid 1.5 years before I went from virtually no publishable data to massively useful data. It happened one night, after my millionth reaction.

It is a unique feeling to know that at one point in time, I was the only person in the world or history of mankind, that knew how this reaction worked.

It's like being Isaac Newton for a day.
We live for bands on a gel.
 
  • Like
Reactions: 2 users
On a similar note, how would you recommend going about learning to write academically and scientifically?
I am writing a report, and I feel intimidated by the heavy, specific jargons, the obligation to organize every idea logically, the difficulty of using precise yet revealing language, and my general lack of more profound knowledge (and thus confidence).
 
On a similar note, how would you recommend going about learning to write academically and scientifically?
I am writing a report, and I feel intimidated by the heavy, specific jargons, the obligation to organize every idea logically, the difficulty of using precise yet revealing language, and my general lack of more profound knowledge (and thus confidence).

Send drafts to you PI.
 
Talking with my stats hat on. I have been doing research for years. I am also usually the stats person in the groups. So, yes, most of the time it would be me handling data, overseeing the quality of data, analyzing the data, and interpreting the data to other group members.

From my experiences, usually it is the data cleaning that will take away most of my time. Yes, you read it correctly, data cleaning. Not modeling, not model checking, not writing the reports, but data cleaning. Ask any statistician, they will tell you that they hate data cleaning. It takes tremendous efforts to clean messy data. You have to know why there are missing values. You have to guess why you see some out-of-range data. You have to be mindful about potential strange patterns as they may arise from systematic biases, from some known, or unknown sources. Extreme values? Rare values? non-compliance? vague documentations? Coding errors? But, your efforts will worth every minute you put in. Hey, just think this way, after you clean the data, maybe you can use them for more than 1 paper!

So, yeah, take a deep breath, smile, and keep on data cleaning - this is one of the most important steps during the research process yet quite often overlooked.


Data cleaning? That sounds unethical
 
Data cleaning? That sounds unethical

Data cleaning is an important step in research, as the process will help you to have better idea what is really going on in your data.

It is a jargon we use - to turn your messy dataset into a usable one. In fact it would be problematic if you don't clean your data and take as they are - because you can't evaluate and guarantee the overall quality of your data.

I think you may refer to "data massaging" - cherry picking what you want to see and discard others. That is unethical.

By the way, no matter what, documentation is very important, and you should make all of your documentations and codes available for other people.
 
  • Like
Reactions: 1 users
Update: my PI and mentor are actually very pleased with the hard work I have put in! They really appreciated my efforts to clearly represent everything and communicate what I have done and see.
I guess in the end, at the undergraduate level at least, the most important takeaway from research is how to do clean, good science, how to troubleshoot when things don't work out, and how to discuss science in an understandable and engaging manner. I will strive to do my best in those regards!
 
  • Like
Reactions: 7 users
Top