Systematic Review Publications - what is their worth?

[workaholic007]

Hello everyone - due to coronavirus I was not able to secure a good research position. I however am going to be a second-author on a systemic review. I am curious how much weight this holds for residency applications for fields like ENT and neurosurgery. It took me a very long time to finish the project but i heard from few people review papers are not viewed as anything. Any insight on the heiarchacy of research and how beneficial a review paper? I can assist with another review as well but im debating if its worth it. Hard to get other research

---

Members don't see this ad.

 

[Frogger27]

I wouldn’t worry about the “hierarchy” of publications as a medical student. A systematic review is a legit publication and you should jump on the opportunity

 

[Lawpy]

Reviews count and are sure as hell far better than crappy school posters

 

[AnatomyGrey12]

A systematic review is a legit publication. Far more impressive than a case report

 

[workaholic007]

what about being a second author? I wanted to get involved in retrospective studies and other clinical studies but just do not have the opportunity. I feel as if a systemic review is not sufficient to determine an interest in research since it is not necessary novel

 

[AnatomyGrey12]

what about being a second author? I wanted to get involved in retrospective studies and other clinical studies but just do not have the opportunity. I feel as if a systemic review is not sufficient to determine an interest in research since it is not necessary novel

I have multiple pubs, all of them are meta-research and review-y type papers, and most of them 2nd-4th author with one 1st. Every interview has brought them up very, very favorably. Many applicants have research, but the gist I get from these interviewers is a lot don’t have actual publications.

A systematic review pub will be looked at favorably and in no way will be a negative.

 

[aldol16]

Systematic review is a specific type of review paper that applies the scientific method (in some regards) to the process. That's what makes it systematic. If you're interested in hierachy, I would say original article is best and then meta-analysis, with systematic review behind. But people probably won't care.

 

[Lawpy]

Systematic review is a specific type of review paper that applies the scientific method (in some regards) to the process. That's what makes it systematic. If you're interested in hierachy, I would say original article is best and then meta-analysis, with systematic review behind. But people probably won't care.

What's your complete ranking, out of curiosity?

 

[workaholic007]

Systematic review is a specific type of review paper that applies the scientific method (in some regards) to the process. That's what makes it systematic. If you're interested in hierachy, I would say original article is best and then meta-analysis, with systematic review behind. But people probably won't care.

what about authorship number, is co-author with 5-10 authors on retrospective study worth anything

 

[workaholic007]

I don’t know bro... I’ll trade ya a holographic Charizard for it.

im down, as long as its first edition.

 

[IMG69]

Bare in mind i'm not applying US but the research hierarchy for where i'm heading is at least, case reports < review article < systematic review article < retrospective chart-review paper < prospective study paper < experimental paper.

 

[aldol16]

what about authorship number, is co-author with 5-10 authors on retrospective study worth anything

In terms of authorship, it's position that matters. First author > second author > other authors (other than senior author spot which a med student will not fill).

 

[aldol16]

What's your complete ranking, out of curiosity?

I would say original article > meta-analysis > systematic review > review > case study/report.

 

[aldol16]

Bare in mind i'm not applying US but the research hierarchy for where i'm heading is at least, case reports > review article > systematic review article > retrospective chart-review paper > prospective study paper > experimental paper.

I think you meant to reverse the inequalities

 

[workaholic007]

in regards to time commitment whats the difference between original article or retrospective chart review and a Systemic review

 

[aldol16]

in regards to time commitment whats the difference between original article or retrospective chart review and a Systemic review

It really depends. Retrospective chart review is probably the least time commitment but if you're reviewing hundreds of charts, then obviously that becomes a huge time suck. It's also mindless so it's not exactly time spent doing something fun. Systematic review also depends on scope. So if you define a big problem that has hundreds of articles published on the topic and you catch say, 100 of them, you need to review all of those articles.

Original articles have varying time commitments depending on the project. Basic science papers take a long time because of experimental time. Other sorts of analyses can take shorter but you also run the risk of not finding a significant result. There's a time commitment for design, data collection, and data analysis.

 

[GoSpursGo]

what about authorship number, is co-author with 5-10 authors on retrospective study worth anything

In short, everything is "worth something." First/senior author are worth most, then second author, then everyone else, but you don't get to be an author without having done something meaningful.

I think the main problem with a systematic review is that it is a HUGE time sink for only one publication, and you're generally not contributing any new information to the medical literature, so it is true to some extent that these publications are relatively low-yield. That said, you take what you can get, and if the alternative is that you're sitting around twiddling your thumbs then by all means do it.

 

[workaholic007]

In short, everything is "worth something." First/senior author are worth most, then second author, then everyone else, but you don't get to be an author without having done something meaningful.

I think the main problem with a systematic review is that it is a HUGE time sink for only one publication, and you're generally not contributing any new information to the medical literature, so it is true to some extent that these publications are relatively low-yield. That said, you take what you can get, and if the alternative is that you're sitting around twiddling your thumbs then by all means do it.

thank you for the information. ENT applicants or other subspecialties, applicants typically have what type of publications? I know they group abstracts, posters, etc together

 

[Dr G Oogle]

Systematic review and meta-analysis of RCTs is the highest in hierarchy of papers and is what guidelines are best on. One of the reasons they are high in the hierarchy is that they get cited far more than all but the most ground breaking RCTs. I serve on a couple of committees for our major organizations and our job is basically to synthesize the evidence of existing studies, publish reviews and create guidelines. In general a good systematic review is worth more than its parts so, even if all it shows is lack of good evidence, that’s still valuable information. A review is generally considered of higher value by most acdemicians than the majority of retrospective studies. Of course if the the residency faculty is composed of predominantly Clinical rather than academic docs, then they may not make the distinction. So I guess the answer is systematic review > retrospective study based on your target audience. Note that animal studies are not the lowest form of research (case reports are) it’s just that they are generally not used to make clinical judgements. In many cases a high value animal study will supersede most other types of paper.

 

[Lawpy]

Systematic review and meta-analysis of RCTs is the highest in hierarchy of papers and is what guidelines are best on. One of the reasons they are high in the hierarchy is that they get cited far more than all but the most ground breaking RCTs. I serve on a couple of committees for our major organizations and our job is basically to synthesize the evidence of existing studies, publish reviews and create guidelines. In general a good systematic review is worth more than its parts so, even if all it shows is lack of good evidence, that’s still valuable information. A review is generally considered of higher value by most acdemicians than the majority of retrospective studies. Of course if the the residency faculty is composed of predominantly Clinical rather than academic docs, then they may not make the distinction. So I guess the answer is systematic review > retrospective study based on your target audience. Note that animal studies are not the lowest form of research (case reports are) it’s just that they are generally not used to make clinical judgements. In many cases a high value animal study will supersede most other types of paper.
View attachment 327263

Where do you place invited reviews at? Because that's what i usually think of wrt systematic reviews - senior attendings being invited by a major journal to summarize and update recent findings on a topic

 

[GoSpursGo]

thank you for the information. ENT applicants or other subspecialties, applicants typically have what type of publications? I know they group abstracts, posters, etc together

I've never applied to ENT, but again... everything counts. Everyone takes whatever kind of publications they can get.

Put another way... recall that the main downside to doing a systematic review is that it takes a long time. So I think rather than asking, "Is a systematic review valuable?" the better question would be, "What would I do if I DIDN'T do this systematic review?" Because it sounds like you looked around, and this is what you got, so if your choice is do this or keep looking then you should just do this. When you're first joining a group, you don't get to choose what paper you work on, and you don't normally get the most exciting one. Take whatever you get, do a good job, and they'll probably give you something more exciting next time.

Systematic review and meta-analysis of RCTs is the highest in hierarchy of papers and is what guidelines are best on. One of the reasons they are high in the hierarchy is that they get cited far more than all but the most ground breaking RCTs. I serve on a couple of committees for our major organizations and our job is basically to synthesize the evidence of existing studies, publish reviews and create guidelines. In general a good systematic review is worth more than its parts so, even if all it shows is lack of good evidence, that’s still valuable information. A review is generally considered of higher value by most acdemicians than the majority of retrospective studies. Of course if the the residency faculty is composed of predominantly Clinical rather than academic docs, then they may not make the distinction. So I guess the answer is systematic review > retrospective study based on your target audience. Note that animal studies are not the lowest form of research (case reports are) it’s just that they are generally not used to make clinical judgements. In many cases a high value animal study will supersede most other types of paper.

While technically true, 1) I highly doubt that this is a practice-changing systematic review for a high impact journal as that just isn't something that gets farmed out to a med student; 2) While a review may be very clinically useful, professionally it is generally not seen as valuable to your CV as original research as you didn't generate your own evidence. Again, not that this is something that a med student should inherently care about.

 

[Dr G Oogle]

Systematic review and meta-analysis of RCTs is the highest in hierarchy of papers and is what guidelines are best on. One of the reasons they are high in the hierarchy is that they get cited far more than all but the most ground breaking RCTs. I serve on a couple of committees for our major organizations and our job is basically to synthesize the evidence of existing studies, publish reviews and create guidelines. In general a good systematic review is worth more than its parts so, even if all it shows is lack of good evidence, that’s still valuable information. A review is generally considered of higher value by most acdemicians than the majority of retrospective studies. Of course if the the residency faculty is composed of predominantly Clinical rather than academic docs, then they may not make the distinction. So I guess the answer is systematic review > retrospective study based on your target audience
View attachment 327263

Where do you place invited reviews at? Because that's what i usually think of wrt systematic reviews - senior attendings being invited by a major journal to summarize and update recent findings on a topic

There’s usually a separate section on CV for invites articles: editorials, topic review, expert review series etc different then systematic reviews which are analytic methods for combining multiple research studies. For a med stud I think it just goes under “ publication other than peer reviewed” or something like that.

 

[Dr G Oogle]

While technically true, 1) I highly doubt that this is a practice-changing systematic review for a high impact journal as that just isn't something that gets farmed out to a med student; 2) While a review may be very clinically useful, professionally it is generally not seen as valuable to your CV as original research as you didn't generate your own evidence. Again, not that this is something that a med student should inherently care about.

I sometimes will have a particularly adapt student help either with screening articles or even extrActing data; the latter I’ve done exactly 2 times and both students had masters in epidemiology. Either or both of the above could easily garner a 2nd author spot.

can’t really tell from OP the nature or quality of review they performed.

the professional viewpoint of how valuable a review is relative to “original research” really depends on review

for example Surgery Versus Radiotherapy for Clinically-localized Prostate Cancer: A Systematic Review and Meta-analysis - PubMed
The above review might be more useful than the individual studies; But it would also not exist Without the individual studies

 

[GoSpursGo]

I sometimes will have a particularly adapt student help either with screening articles or even extrActing data; the latter I’ve done exactly 2 times and both students had masters in epidemiology. Either or both of the above could easily garner a 2nd author spot.

can’t really tell from OP the nature or quality of review they performed.

the professional viewpoint of how valuable a review is relative to “original research” really depends on review

for example Surgery Versus Radiotherapy for Clinically-localized Prostate Cancer: A Systematic Review and Meta-analysis - PubMed
The above review might be more useful than the individual studies; But it would also not exist Without the individual studies

I mean yes, there are some very strong and impactful systematic reviews in the literature. Based on the OP saying he was "not able to secure a good research position," I would say the odds are slim that he fell backwards into this kind of publication and doesn't realize it. It would be extremely rare for a medical student to be involved in these kinds of publications at all.

Again, not saying a review is "bad!" As a medical student, anything where you're able to contribute to a project that is publishable is great. I would say as a general rule for medical students, it is fair to consider review articles to be something of a time suck, which may make them low-yield relative to other projects that can be turned around more quickly. There are always exceptions to the rule.

 

[workaholic007]

I mean yes, there are some very strong and impactful systematic reviews in the literature. Based on the OP saying he was "not able to secure a good research position," I would say the odds are slim that he fell backwards into this kind of publication and doesn't realize it. It would be extremely rare for a medical student to be involved in these kinds of publications at all.

Again, not saying a review is "bad!" As a medical student, anything where you're able to contribute to a project that is publishable is great. I would say as a general rule for medical students, it is fair to consider review articles to be something of a time suck, which may make them low-yield relative to other projects that can be turned around more quickly. There are always exceptions to the rule.

how do I gain the skillsets to assist with other projects? People mention background in stats or epidemiology but how can I independently learn. I took stats in college but that is it. Any self-selected resources? Is this worth my time?

 

[Lawpy]

how do I gain the skillsets to assist with other projects? People mention background in stats or epidemiology but how can I independently learn. I took stats in college but that is it. Any self-selected resources? Is this worth my time?

I learned stats online but it's mainly from when i'm working. Try Khan Academy or other youtube stats tutorial. Also learn R

 

[GoSpursGo]

how do I gain the skillsets to assist with other projects? People mention background in stats or epidemiology but how can I independently learn. I took stats in college but that is it. Any self-selected resources? Is this worth my time?

You do some smaller projects and you can learn the basics on the fly. Realistically, anything more involved than Excel or Prism and you probably need formal training for it to actually be an asset that a research group could depend on (ie the two particularly adept students @Dr G Oogle both had masters degrees). There's a reason any major university has an entire biostats department to support major research endeavors.

 

[Dr G Oogle]

It’s hard for students to learn new hi-if skills while in medical school. There just isn’t enough time in the day and in all honesty I don’t recommend trading precious free time to learn how to code unless this is something you are super passionate about and brings you joy. If there the case then resources above are helpful. Also if you create a wire frame of your study design mapping out what stats tests you need to do then it’s relatively easy to use YouTube and other internet resources to run the analysis. Projects where the average student can be helpful are prospective studies where you can help with sample collection, patient recruitment, questionnaire administration etc. If you had unique experiences in prior to med school like learning how to do perform qualitative research or engineering and have contacts then try to think of a way to apply your skills to research.

 

[dienekes88]

what about being a second author? I wanted to get involved in retrospective studies and other clinical studies but just do not have the opportunity. I feel as if a systemic review is not sufficient to determine an interest in research since it is not necessary novel

Any publication is good, but i will say that review articles have a different return on investment than other publications. Review articles are usually invited but still peer reviewed. So it has the upside of high probability of publication and if it's a slightly niche topic then a high chance of being cited as long as its in a decent journal (very good for your eventual h-index). The annoying part is that it is a lot of work (as much if not more than an original clinical science manuscript but higher probability of acceptance), but at this stage you don't really have that many other options to be picking and choosing which projects you do.

If I have 3 original clinical science projects and 2 case reports on my to-do list, and someone sends me a review article... then I'm probably not going to accept the project and I would try to pass that off to someone else. This is not your situation as research, by your admission, is hard to come by.

 

[Dr G Oogle]

I just want to clarify that there is a difference between an invited review and systematic review with the former being correctly described above and the latter providing actual scientific value. That being said I don’t think a review article is a complete waste of time like say a book chapter is solely because it is a lot less work than a book chapter but will demonstrate that you have a grasp of the clinical aspect of the surgery.

I also want to clarify what value research has on residency apps. For the vast majority of medical students most of the publications can be lumped into one group and provide the following information about a student: they are able to finish specific tasks and work in a team environment. Most medical students do not have the skill set or knowledge base to perform important research that contributes to scientific community and knowledge in an obvious way. The ones that do that sort of research are easy to spot. They have things like a PhD and first author publications in Nature and Science, etc. this is not a realistic expectation for most students. Also having a ton of “ research“ And abstracts on your application and no publications Looks bad. And I realize that is often out of the hands of the student, so even more important than the kind of research you do is the kind of PI you select, and I recommend picking one that will get your name on a pub before eras is due.

when students come to me I can usually guarantee a publication if they come early enough (year 1 or 2) if I can’t guarantee one than I am upfront about it. If we are going to have this ridiculous research “requirement” on ERAS and in med school than I think it is well within the rights of a student to expect to work with PIs that will guarantee them a publication (assuming of course the student demonstrates Effort to aid with project completion). I also recommend students ask specifically what boxes need to be checked for them to get a publication, since often this is a somewhat nebulous aspect of research.

 

[GoSpursGo]

I just want to clarify that there is a difference between an invited review and systematic review with the former being correctly described above and the latter providing actual scientific value. That being said I don’t think a review article is a complete waste of time like say a book chapter is solely because it is a lot less work than a book chapter but will demonstrate that you have a grasp of the clinical aspect of the surgery.

I also want to clarify what value research has on residency apps. For the vast majority of medical students most of the publications can be lumped into one group and provide the following information about a student: they are able to finish specific tasks and work in a team environment. Most medical students do not have the skill set or knowledge base to perform important research that contributes to scientific community and knowledge in an obvious way. The ones that do that sort of research are easy to spot. They have things like a PhD and first author publications in Nature and Science, etc. this is not a realistic expectation for most students. Also having a ton of “ research“ And abstracts on your application and no publications Looks bad. And I realize that is often out of the hands of the student, so even more important than the kind of research you do is the kind of PI you select, and I recommend picking one that will get your name on a pub before eras is due.

when students come to me I can usually guarantee a publication if they come early enough (year 1 or 2) if I can’t guarantee one than I am upfront about it. If we are going to have this ridiculous research “requirement” on ERAS and in med school than I think it is well within the rights of a student to expect to work with PIs that will guarantee them a publication (assuming of course the student demonstrates Effort to aid with project completion). I also recommend students ask specifically what boxes need to be checked for them to get a publication, since often this is a somewhat nebulous aspect of research.

This is a great post. I just want to emphasize a couple of points.

1) Agree that there's a difference between a systematic review vs. an invited review. And book chapter is the absolute worst. Again, something is better than nothing.

2) Also strongly agree that students really shouldn't care about what kind of publication they get, so much as that they get a publication period. The ERAS applicant pool is littered with students who have "dabbled" in research with an experience or two, maybe an abstract, but were ultimately unable to turn it into a publication. If you were able to put in the effort and synthesize your work into a story that passed through peer-review, that is notable even for low-impact publications. It means that if you can deliver when handed a task--and if publishing is a "thing" that your residency wants, then they know you can do that going forward.

3) To this day, when I sit down to discuss a new project I hash out authorship expectations in the first meeting explicitly. Much easier down the line so that everyone knows exactly what they will get out of the project (first/second/middle author), and what they will be expected to put in. It's uncomfortable, but I would suggest students bring that question up if the faculty does not so that it is crystal clear what they are hoping to get out of the experience.

 

[Lawpy]

I just want to clarify that there is a difference between an invited review and systematic review with the former being correctly described above and the latter providing actual scientific value. That being said I don’t think a review article is a complete waste of time like say a book chapter is solely because it is a lot less work than a book chapter but will demonstrate that you have a grasp of the clinical aspect of the surgery.

I also want to clarify what value research has on residency apps. For the vast majority of medical students most of the publications can be lumped into one group and provide the following information about a student: they are able to finish specific tasks and work in a team environment. Most medical students do not have the skill set or knowledge base to perform important research that contributes to scientific community and knowledge in an obvious way. The ones that do that sort of research are easy to spot. They have things like a PhD and first author publications in Nature and Science, etc. this is not a realistic expectation for most students. Also having a ton of “ research“ And abstracts on your application and no publications Looks bad. And I realize that is often out of the hands of the student, so even more important than the kind of research you do is the kind of PI you select, and I recommend picking one that will get your name on a pub before eras is due.

when students come to me I can usually guarantee a publication if they come early enough (year 1 or 2) if I can’t guarantee one than I am upfront about it. If we are going to have this ridiculous research “requirement” on ERAS and in med school than I think it is well within the rights of a student to expect to work with PIs that will guarantee them a publication (assuming of course the student demonstrates Effort to aid with project completion). I also recommend students ask specifically what boxes need to be checked for them to get a publication, since often this is a somewhat nebulous aspect of research.

This is a great post. I just want to emphasize a couple of points.

1) Agree that there's a difference between a systematic review vs. an invited review. And book chapter is the absolute worst. Again, something is better than nothing.

2) Also strongly agree that students really shouldn't care about what kind of publication they get, so much as that they get a publication period. The ERAS applicant pool is littered with students who have "dabbled" in research with an experience or two, maybe an abstract, but were ultimately unable to turn it into a publication. If you were able to put in the effort and synthesize your work into a story that passed through peer-review, that is notable even for low-impact publications. It means that if you can deliver when handed a task--and if publishing is a "thing" that your residency wants, then they know you can do that going forward.

3) To this day, when I sit down to discuss a new project I hash out authorship expectations in the first meeting explicitly. Much easier down the line so that everyone knows exactly what they will get out of the project (first/second/middle author), and what they will be expected to put in. It's uncomfortable, but I would suggest students bring that question up if the faculty does not so that it is crystal clear what they are hoping to get out of the experience.

How important is 2nd vs 3rd author? I always thought they're the same, and since if i'm not the 1st author, i usually aim to be 3rd and let someone else (usually more senior than me) be 2nd author. Is this a bad idea?

 

[GoSpursGo]

How important is 2nd vs 3rd author? I always thought they're the same, and since if i'm not the 1st author, i usually aim to be 3rd and let someone else (usually more senior than me) be 2nd author. Is this a bad idea?

I mean at that point you're splitting hairs. Outside of first/senior author, I usually don't specify that level of authorship until the manuscript is getting written up.

The second author should be whoever contributed more to the publication. If that's you, then you should be able to say so, but I also wouldn't fight over it.

 

[Lawpy]

I mean at that point you're splitting hairs. Outside of first/senior author, I usually don't specify that level of authorship until the manuscript is getting written up.

The second author should be whoever contributed more to the publication. If that's you, then you should be able to say so, but I also wouldn't fight over it.

So to clarify, it's unlikely that 2nd authors will be weighted more by PDs vs 3rd/middle authors? Because my strategy is to aim for at least a few 1st authors and everything else 3rd/middle even if i contributed a lot to the papers similar to 2nd author.

However if PDs for whatever reason do value 2nd author papers and use that to rank highly, i'd need to change this strategy.

As you mentioned, it's splitting hairs and i don't understand why anyone would care because 1st/senior authors are most important for obvious reasons. But seeing how 2nd author is repeatedly separated from other middle authors in its own category makes me a bit cautious

 

[GoSpursGo]

So to clarify, it's unlikely that 2nd authors will be weighted more by PDs vs 3rd/middle authors? Because my strategy is to aim for at least a few 1st authors and everything else 3rd/middle even if i contributed a lot to the papers similar to 2nd author.

However if PDs for whatever reason do value 2nd author papers and use that to rank highly, i'd need to change this strategy.

As you mentioned, it's splitting hairs and i don't understand why anyone would care because 1st/senior authors are most important for obvious reasons. But seeing how 2nd author is repeatedly separated from other middle authors in its own category makes me a bit cautious

So, not all 2nd authorships are created equally. In one case I'm the second author but put in almost as much work as the first author as they required some extensive revisions. In many cases there's no appreciable difference between the 2nd author and the other middle authors.

PDs have no way of knowing what work actually went into the project just by reading your CV. So I would generally not worry much about where you land on the author list if you're not first. Realistically the first/last authors should decide the author order in an equitable manner.

 

[Dr G Oogle]

How important is 2nd vs 3rd author? I always thought they're the same, and since if i'm not the 1st author, i usually aim to be 3rd and let someone else (usually more senior than me) be 2nd author. Is this a bad idea?

there probably is a hierarchy, it’s mostly for promotion once you’re faculty , but for a student anything Between first and last is middle author. Again don’t worry about where you are on the list of authors, just that 1) there’s a publication and 2) you can speak intelligently about the study

I actually make it a point to ask people to describe their study to me; and I am much more likely to respond favorably to a second to last author who can speak about their study intelligently than a second author who can’t

 

[GoSpursGo]

there probably is a hierarchy, it’s mostly for promotion once you’re faculty , but for a student anything Between first and last is middle author. Again don’t worry about where you are on the list of authors, just that 1) there’s a publication and 2) you can speak intelligently about the study

I actually make it a point to ask people to describe their study to me; and I am much more likely to respond favorably to a second to last author who can speak about their study intelligently than a second author who can’t

Yep I agree--and not just explaining the study, but what you specifically did. Having the lines of pubs on your CV is nice and may get you interviews, but it's a bad look if you can't answer those questions well.

 

[Dwan]

How important is 2nd vs 3rd author? I always thought they're the same, and since if i'm not the 1st author, i usually aim to be 3rd and let someone else (usually more senior than me) be 2nd author. Is this a bad idea?

Typically, 1st/2nd/last are the most highly valued in academics, with everything else being similar after that. 1st obviously much bigger than 2nd, but 2nd is still a valuable spot normally. Definitely do not give it away if you earned it.