A few years out...so how do I know if I'm doing a good job?

[hippopotamusoath]

---

Members don't see this ad.

So does anyone have a good strategy they use to measure their outcomes, and their general performance as a psychiatrist?

I'm a few years out. I mainly work outpatient. I read, I follow treatment guidelines, I think I'm attentive and responsive to my patients. I still am not sure who gets better because I'm smart and good at psychiatry, and who would have gotten better showing up to an office with a piece of driftwood on an office chair and a bowl of Prozac on the desk.

Surgeons have lots of metrics--infections, revision rates, etc.

What do we have? I could do scales for everything (YBOCS, PHQ-9, GAD-7), but that consumes a ton of clinical time and apart from YBOCS, hasn't seemed all that more beneficial than just talking to the patient, and the scales can sometimes have a negative effect on rapport-building efforts. Hospitalization rates doesn't seem so useful--it's a semi-random and relatively rare event that is heavily dependent on one's particular patient mix, their community resources, the attitude of the particular social worker they interact with in the ED, etc. Suicide is too rare to be useful, and we don't have a good way of finding out who would have died but didn't because of our intervention.

I have read some papers on psychotherapy outcomes that seemed to indicate that the patients actually did worse as the therapist gained experience and confidence, when compared to the patients who saw an earnest rookie. So I don't really think it's even fair for me to trust my general impressions of how good I am or how the patients are doing. I could be completely wrong, and just achieving more unfounded confidence as I go along.

I'm driven to improve, but I want to spend my time wisely and actually have some sense that my efforts are doing something for the patients.

Curious to hear how others have approached this, and maybe I just have to accept that our work is inherently uncertain.

 

[splik]

Patients vote with their feet. If your patients come back and you're not prescribing them candy, that is a positive sign. If you successfully discharge patient from your practice because they no longer need you, that is a positive sign. If patients go from crisis to stable, that is a positive sign. If the frequency of visits necessary with patients decreases over time, that is a positive sign.

I was trained in a system that was very big on measurement based care so I do use measures for most patients. It doesn't have a negative effect on rapport at all. However, some patients won't complete them (which is to be expected). Although PHQ scores etc only tell you so much, it is one metric that helps track progress, another piece of data, means I don't have to ask a bunch of questions about symptoms and can focus in on things flagged on on the questionnaires, and it can be helpful for patients (and you) to graphically see their progress over time. For example, some patients may think they haven't improved when actually their general trend is in the right direction with some blips here and there. That is helpful information. Also, insurance companies like this and it can be helpful for justifying medical necessity, negotiating higher rates (if you take insurance) or supporting OON claims.

 

[hippopotamusoath]

Patients vote with their feet. If your patients come back and you're not prescribing them candy, that is a positive sign. If you successfully discharge patient from your practice because they no longer need you, that is a positive sign. If patients go from crisis to stable, that is a positive sign. If the frequency of visits necessary with patients decreases over time, that is a positive sign.

I was trained in a system that was very big on measurement based care so I do use measures for most patients. It doesn't have a negative effect on rapport at all. However, some patients won't complete them (which is to be expected). Although PHQ scores etc only tell you so much, it is one metric that helps track progress, another piece of data, means I don't have to ask a bunch of questions about symptoms and can focus in on things flagged on on the questionnaires, and it can be helpful for patients (and you) to graphically see their progress over time. For example, some patients may think they haven't improved when actually their general trend is in the right direction with some blips here and there. That is helpful information. Also, insurance companies like this and it can be helpful for justifying medical necessity, negotiating higher rates (if you take insurance) or supporting OON claims.

Thanks, exactly the kind of feedback I was hoping for. The patient return rate is an incredibly pragmatic way to assess whether patients feel like you're worth it or not. Although, I suppose I could make the counter-argument that I have some patients who see an NP for an hour or so every two weeks for "chronic Lyme" antibiotics. They love it, and feel better, but it's hard to say that the treatment is moving the ball forward in terms of their health.

Using more scales is probably prudent. What better option do we have..maybe have the nurse do it before the visit with the patient or something.

 

[FlowRate]

I was talking a little about this in our consultation group recently. Until recently, my organization used "art of medicine" surveys (similar to press ganey surveys). I never found them all that helpful, although there is probably some sort of meaning to them, as there is a wide variance between psychiatrists in our system as to average scores and subscores. But the actual data isn't granular enough to know if it's each item is something I'm actually good/bad at. (Or to discriminate angry patient vs genuine feedback.) We also have "goodness of fit" as a scale on our pre-appointment questionnaire. Most of the time it seems like patients have questionnaire fatigue and are just rushing through at that point. A couple of times it has helped me identify a break in the patient-doctor relationship with patients who weren't otherwise very forthcoming about what was amiss, exactly. (Although, generally, most of those end up being pts with very poor histories for ADHD who don't like that I'm not focusing on ADHD as "the diagnosis.")

 

[annoyedpsychiatrist]

The problem with satisfaction surveys is if you think about it, when you have a great doctor visit how often do you think after "Hey, i really need to go out and leave a review!". People are more inclined to leave bad reviews when they're upset than good ones when they're happy. And then when you dont give some people their party mix of adderall ir/xanax/ambien, they're more inclined to leave a bad review. I try to turn them down gracefully to reduce the probability of resentment.

Remember this; you will remember your failures more than your sucess patients. You dont see your sucess patients as often because they're well/stable, but your failures (or works in progress) you may hear more from due to more frequent f/us/messages. BUT on that note, as stated above, if they keep coming/trying with you, thats always a good sign.

you will have days that just suck and you question if you're making a difference. That is inevitable. You will have great days. When you have bad days it becomes easy to snowball down the negativity slope, but when this happens i always ask myself "Did I do what I felt was right at the time?". If you feel you did the right thing (no matter the outcome) then its hard to fault yourself.

 

[Merovinge]

Surgeons have lots of metrics--infections, revision rates, etc.

I think this is a lot less clear for other specialties than you might expect (as a psychiatrist). Many surgical complications are rare and their defining likelihood is just based on how many of that procedure that you do. Many surgeons also hide or intentionally obfuscate any complications they have "I've done 1000 of these cases and never had a complication" is actually a very common thing for surgeons to say. Many of the metrics they do track are heavily influenced by patient factors (e.g. infection rates) or hospital staff factors. Other numbers can be influenced by cherry picking cases. A significant amount of value they add is knowing when not to operate (just like a psychiatrist knowing when not to prescribe or how to de-prescribe), no one gets points for not taking a patient to the OR that would die on the table (as understanding counterfactuals is very difficult for the human brain).

It is not straightforward at all to determine how good you are at many fields/professions and generally the wrong pursuit anyway. There is always someone smarter, faster, or better than you at everything. The only reflection I think is necessary or helpful is A) am I doing everything I can for my patients and B) does that meet or exceed the standard of care. If I am not meeting A and B, what barriers are in the way and what can I do to remove them. If you are meeting A and B, sleep well at night and of course keep up with your education and training where you can as well as live a life outside of medicine.

 

[PikminOC]

So does anyone have a good strategy they use to measure their outcomes, and their general performance as a psychiatrist?

I'm a few years out. I mainly work outpatient. I read, I follow treatment guidelines, I think I'm attentive and responsive to my patients. I still am not sure who gets better because I'm smart and good at psychiatry, and who would have gotten better showing up to an office with a piece of driftwood on an office chair and a bowl of Prozac on the desk.

Surgeons have lots of metrics--infections, revision rates, etc.

What do we have? I could do scales for everything (YBOCS, PHQ-9, GAD-7), but that consumes a ton of clinical time and apart from YBOCS, hasn't seemed all that more beneficial than just talking to the patient, and the scales can sometimes have a negative effect on rapport-building efforts. Hospitalization rates doesn't seem so useful--it's a semi-random and relatively rare event that is heavily dependent on one's particular patient mix, their community resources, the attitude of the particular social worker they interact with in the ED, etc. Suicide is too rare to be useful, and we don't have a good way of finding out who would have died but didn't because of our intervention.

I have read some papers on psychotherapy outcomes that seemed to indicate that the patients actually did worse as the therapist gained experience and confidence, when compared to the patients who saw an earnest rookie. So I don't really think it's even fair for me to trust my general impressions of how good I am or how the patients are doing. I could be completely wrong, and just achieving more unfounded confidence as I go along.

I'm driven to improve, but I want to spend my time wisely and actually have some sense that my efforts are doing something for the patients.

Curious to hear how others have approached this, and maybe I just have to accept that our work is inherently uncertain.

Pls link the psychotherapy papers you are referring to.

 

[tr]

I use a rating scale at most visits. I don't think it interferes with rapport at all and many patients appreciate having a quantifiable marker of improvement.

 

[hippopotamusoath]

Pls link the psychotherapy papers you are referring to.

Here’s one paper, and it alludes to some of the other research on this topic in the body of the paper:

Do psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a clinical setting - PubMed

Implications of these findings for the development of expertise in psychotherapy are explored. (PsycINFO Database Record

pubmed.ncbi.nlm.nih.gov

I don’t think I can post the full paper because of copyright issues, but it shouldn’t be hard to find…pretty fascinating outcomes.

 

[Candidate2017]

I read, I follow treatment guidelines, I think I'm attentive and responsive to my patients. I still am not sure who gets better because I'm smart and good at psychiatry, and who would have gotten better showing up to an office with a piece of driftwood on an office chair and a bowl of Prozac on the desk.

There's only one way to find out. Treat half your patients as usual, and assign the other half to a driftwood SSRI treatment arm. Be sure to hit up your state's NP association and hospital exec association for a few million in grant money for this study.

 

[J ROD]

Google reviews from your patients.
Especially the ones you see once and refuse to give benzos after they have been to several other providers including PCP who have documented multiple abuse episodes. LOVE those 1 star reviews. Even better is the fact part of my bonus is derived from patient feedback and this is the first quarter for me having outpatient feedback. First time, not making my full bonus due to mainly patients wanting benzos and stimulants inappropriately and a few borderline patients who do not like being told therapy is the best option and next best option. Still got like 90 percent but still....the principle.

 

[shoomer]

For those who use rating scales routinely, can anyone shed light on which scales they're using? And do you use different scales within the same disorder or is one sufficient for each?

 

[Merovinge]

Google reviews from your patients.
Especially the ones you see once and refuse to give benzos after they have been to several other providers including PCP who have documented multiple abuse episodes. LOVE those 1 star reviews. Even better is the fact part of my bonus is derived from patient feedback and this is the first quarter for me having outpatient feedback. First time, not making my full bonus due to mainly patients wanting benzos and stimulants inappropriately and a few borderline patients who do not like being told therapy is the best option and next best option. Still got like 90 percent but still....the principle.

I recently saw a restaurant that posted all the 1 star reviews in small frames inside the restaurant. I love this way of going about, would be great to post up the 1-star reviews from patients pissed about lack of controlled substance prescriptions. I know no one on Earth will ever do this, but I would love seeing this when I went into an MD office in the waiting room.

 

[clausewitz2]

If you have patients who you have been seeing for a while and have good rapport with come to their next appointment and struggle to come up with anything to talk about, you're doing good. Not needing to see a psychiatrist regularly is winning at life.

 

[hippopotamusoath]

If you have patients who you have been seeing for a while and have good rapport with come to their next appointment and struggle to come up with anything to talk about, you're doing good. Not needing to see a psychiatrist regularly is winning at life.

I'm not sure if you're joking or not, but I actually love this. When things are dull, and I start to wonder why I'm seeing the patient at all anymore, that's a pretty good outcome in my opinion! And this kind of "test" is really pragmatic and simple to deploy in the real world. If you're bored, and they're bored, and the rapport is good, you're probably doing something right. Beautiful.

That being said, it's probably not perfect. There are also those patients where I am sure they will never get "better," exactly, but I have to assume the monthly meds + supportive therapy has been useful to them and possibly prevented back-sliding.

 

[comp1]

How is your own emotional wellbeing? Are you meeting your financial goals? Do you have a life outside of work? If all of that is going well, you're a lot more likely to be a good psychiatrist.

 

[splik]

For those who use rating scales routinely, can anyone shed light on which scales they're using? And do you use different scales within the same disorder or is one sufficient for each?

I use self report measures primarily as screeners, but some for tracking progress as well.
All new pts complete PHQ-9, GAD-7, GSAQ, DAST and AUDIT and SCID-5-SPQ
depending on their problems may also have them complete PCL-5 with LEC-5, CECA.Q, DES-II, TAS-20, PHQ-15, HCL-32, Neurobehavioral Symptom Inventory, McLean Borderline Questionnaire, ASRS, BDEFS, Ritvo, AQ, EQ, YBOCS

There are some others, but I think PHQ, GAD, PCL-5, ASRS are probably going to be the main ones for general psych practice

 

[clausewitz2]

I use self report measures primarily as screeners, but some for tracking progress as well.
All new pts complete PHQ-9, GAD-7, GSAQ, DAST and AUDIT and SCID-5-SPQ
depending on their problems may also have them complete PCL-5 with LEC-5, CECA.Q, DES-II, TAS-20, PHQ-15, HCL-32, Neurobehavioral Symptom Inventory, McLean Borderline Questionnaire, ASRS, BDEFS, Ritvo, AQ, EQ, YBOCS

There are some others, but I think PHQ, GAD, PCL-5, ASRS are probably going to be the main ones for general psych practice

I'm surprised that you use the ASRS. It's so obviously face valid that I have always been suspicious of it as a self-report measure, but I suppose if you have told them up front you're not going to give them stimulants you are less concerned about feigning. I want to second the McLean, I never cease to be surprised at the people who actually score really high on it who manage to come across as very put-together and buttoned-down for the first few appointments.

 

[splik]

I'm surprised that you use the ASRS. It's so obviously face valid that I have always been suspicious of it as a self-report measure, but I suppose if you have told them up front you're not going to give them stimulants you are less concerned about feigning. I want to second the McLean, I never cease to be surprised at the people who actually score really high on it who manage to come across as very put-together and buttoned-down for the first few appointments.

I don't find it anymore problematic than say the PHQ-9 or PCL-5. Obviously I'm not like Cerebral or one of those other companies who uses it as their entire 1 minute ADHD eval. but it gives a data point and can track progress. It gives you some sense of their report of inattentiveness and hyperactivity (neither of which are specific to ADHD). If patients are scoring highly on it and you see no observable evidence of ADHD on your exam, that inconsistency is useful to identify. Similarly, lots of borderline pts score 27 on the PHQ-9, doesn't mean we think they have severe MDD based on that alone but it gives you a sense of their subjective sense of things. Or lots of patients score highly on PCL-5 and they never had a criterion A event or they're just malingering hard. Can't rely on it to make a dx of PTSD and need to get good concrete examples, but once you have made the dx, using PCL-5 weekly during CPT or PE can be incredibly useful at tracking progress.

 

[mistafab]

I use self report measures primarily as screeners, but some for tracking progress as well.
All new pts complete PHQ-9, GAD-7, GSAQ, DAST and AUDIT and SCID-5-SPQ
depending on their problems may also have them complete PCL-5 with LEC-5, CECA.Q, DES-II, TAS-20, PHQ-15, HCL-32, Neurobehavioral Symptom Inventory, McLean Borderline Questionnaire, ASRS, BDEFS, Ritvo, AQ, EQ, YBOCS

There are some others, but I think PHQ, GAD, PCL-5, ASRS are probably going to be the main ones for general psych practice

After buying the SCID-5-SPQ, does the APA allow you to use the measure clinically as much as you want? Or is there a cost per X tests.

 

[tr]

I'm surprised that you use the ASRS. It's so obviously face valid that I have always been suspicious of it as a self-report measure, but I suppose if you have told them up front you're not going to give them stimulants you are less concerned about feigning. I want to second the McLean, I never cease to be surprised at the people who actually score really high on it who manage to come across as very put-together and buttoned-down for the first few appointments.

I use the ASRS too. I don't really like it but the problem is there are no good alternatives. I don't use it for screening, mostly just symptom tracking. I haven't found anything else that's short enough to use regularly and gives at least some kind of quantitative indicator of level of functioning.

I actually don't use any of these instruments as screeners, even though that is mostly what they are designed for. I do a very thorough Scid-type clinical interview at intake and that is my screener.