advice for retrospective/chart review research

[Raygun77]

Hey Guys,

I'm almost finishing up two semesters of dedicated (kind of..) research time, so I figured I'd quickly share with you a few things I've gleaned.
Hopefully this encourages other med students who've done retrospective research to talk about their experience and whether they found it fruitful.

First of all- I'm a big supporter of retrospective clinical research! In the right system with the right topic the possibilities are endless. A few months can see you with a paper worth publishing in a 5+ impact factor journal (what i'd describe as GREAT!). All my friends who chose to spend but a year in a lab...well...good luck getting any meaningful, publishable basic science research done in that short time-scope.

But a few things to consider before setting up a research project

1) ALWAYS do a quick literature review BEFORE choosing a topic. So you find a hundred published papers already answering your research question? Guess what- your research would have been useless and only a bogus, low quality journal would have taken it anyway.
This is where it's GREAT having a good mentor/supervisor guide you in your choice- it needs to be for something novel and UNANSWERED- a gap in the literature. You find these all the time however. A good place to start is by reading CLINICAL PRACTICE GUIDELINES or EVIDENCE BASED REVIEWS in the topic of your choice. Pretty soon you'll read things like "current practice is based on a few large trials/history but no one has ever proved the efficacy of...(e.g. routine imaging, whatever)". Bingo.

2) Read through a few eligible patient notes to get a good grasp of what type of data you'll be able to collect. Ask your supervisor what sort of info was routinely collected or if you can see a patient chart before ethics goes through. No point listing in your protocol that you'll compare ECOG performance status scores against OS if it's never listed!!!

3) Keep your objectives simple! Don't try to overdo it- usually retro research is just "hypothesis-forming" anyway- in terms of practice-changing research, a good retrospective study would lead to a phase 2, then a phase 3 (RCT), then practice gets changed (simplistically speaking).

4) Pray that your chosen institution has electronic records. What a boon they are....

Thats it for the moment. If anyone else wants to chime in i'd be very interested to read what makes for successful clinical retrospective research in your opinion!

---

Members don't see this ad.

 

[Raygun77]

one other thing i forgot to mention-

I honestly thing it's better to do SINGLE ARM COHORT STUDIES for retrospective chart reviews. Confounders and selection bias (as well as bogus comparisons!) become too big a deal when you compare one arm to another retrospectively. Not only do you often find one arm has a vastly greater supply and inclusion frequency, but often the groups aren't equal.

Your single cohort study will become a great historical control to compare phase II trials (e.g. your historical control is based on the current best treatment, a phase II is a prospective cohort study for experimental treatment), and gives useful information in its own right.

 

[tbo]

Kudos on the topic, since I think there's a lot of misconceptions and psychological barriers to getting a project done. I'll add some thoughts of my own.

On reading a few eligible patients' charts before you go "whole hog" in your data collection, I can't agree more. Rarely do the set of variables you start out to collect end up being your entire data set. Diving into the first few charts will make the rest of your data collection more efficient. Often times this helps you refine how you answer your clinical question.

On not over-complicating the study, I also agree - but this only holds true for medical students and possibly residents. I think the barriers to getting something published are lower than most med students think. The clinical question and the results you provide do not have to be Nobel-prize winning material but does have to be some incremental contribution to the topic of interest. I think the expectations as a resident or fellow are to tease out more nuanced clinical questions, so they necessarily need to be more complex studies and study designs.

On electronic medical records, I also completely agree. Don't let the lack of an EMR deter you, however. I recently completed a project that was entirely paper-based but it was highly facilitated by having standardized intake forms that had all the information I needed. Only in less than 10% of cases did I have to search elsewhere for the data. But let's not kid ourselves - ALL retrospective chart review is tedious. This probably discourages the relative few who attempt it. Find the right topic, one that interests you enough that you will feel compelled to see it through.

Which gets me to identifying the right study/topic. Raygun77 suggests an approach that implies you bring an idea to a mentor who helps you refine it. I don't know if this is necessary, or even smart. I think it's great to feel like you've "rolled your own" or that it's YOUR idea. We don't see patients, however, so we wouldn't be expected to know what's an important clinical question to investigate and test. I don't think anyone later in your career will ever ask you whether you dreamt the study up yourself or not. I don't think it really matters. Most of the clinical research I've been involved in was an attending who 1) thought of the clinical question and 2) thought the question could be answered via retrospective review. How the question is answered can be highly independent, and arguably where the bulk of the science actually is. This, to me, is the most stimulating part as well. I'm sure with more advanced medical training, identifying interesting and relevant questions will be something we can take responsibility for later.

On limiting oneself to single cohort design, I think that's too broad a statement for my taste. While it's true from a practical standpoint, single cohort studies are the most straightforward retro reviews, there may also be some areas of medicine that are ripe for a comparative review without adding too much complexity. I think all discerning physicians reading retrospective reviews know that bias still exists in any of these papers, whether that's a single cohort study or a well-designed comparative multi-center retrospective study. So if your institution can support a comparative study in a particular field, go for it. It will make for a stronger paper, likely in a stronger journal.

On journal impact factors, who the heck cares. Unless we're talking about NEJM, JAMA or maybe Lancet (or Nature, Science, Cell), I think the key is to do a project you're excited about, have a good mentor that will guide you through the ins and outs of clinical research, hopefully present it somewhere (a national or regional meeting) or get it accepted as an abstract, and hopefully write it up as a paper (even if you get screwed on first authorship). At the end of the day, what will add value to your career is to speak eloquently on how you contributed to the field and shows (not just tells) others that you've invested time and energy towards furthering the field.

 

[Raygun77]

Thanks so much for your input tbo! I'm no expert having had experience in all of two literature reviews so its great to hear different opinions!

You're very right in saying that 'how' the question may be answered is really important in itself- that is, designing a methodology that will prove one way or another on the issue.
I had the experience of formulating my own research question though, and don't think it's exactly a bad one. I was lucky enough to have a really supportive supervisor who thought along the same lines as me- unfortunately our prospective study fell through (funding issues) so i was the one who suggested a retrospective look as an alternative. I'd already read the literature then by myself so noticed the gaps, so to speak.

And you're right, i was limiting my opinion to the n = 2 studies that have concerned me when i made the broad statement that single-arm retrospective research may actually be better. Case-control studies are inherently two-arm and retrospective, and they do add a wealth of knowledge! I was only speaking of my own experience (my studies both are single cohorts)

Finally, I don't know any surrogate other than impact factors to gauge how "read" journals are. I think its better to write a paper up and then decide which journal it should go into, however it's a good idea knowing what you can expect. Read a few editions of the journal before wasting your time in the lengthy process and just getting rejected!
For example, things like J. Clin Oncol or Lancet Oncol (my research is in oncology!) seem like they wouldn't accept retrospective research at all!

 

[BlitzSleep]

I'm not a medical student, but I have been working on a retrospective study for the local hospital and I wish you posted this sooner!!!

I would also like to add, SRC members who are a FACP are (in my experience) very willing to help you. With my undergrad status they know that I am a noobie at this and during the SRC meeting, they offered their help without me even having to ask (which was nice cause they ripped me a new one with their suggested revisions lol).

 

[dude1344]

So I've spent the summer (incoming MS2) doing a retro chart review project for an attending. I've got some good data and he thinks I can put together a paper and wants me to start writing. So... any good tips/pointers? 😀

 

[loveoforganic]

Read a bunch of similar published articles. Imitate them