Metrics that actually matter

[Tiger26]

---

Members don't see this ad.

To build on the topic of most metrics being useless and in some cases harmful for patients (see satisfaction, patient), what do you all think would be beneficial to advancing the practice of emergency medicine and providing better care for patients:

Some initial ideas:
-72 hour mortality after ED discharge (exclude unrelated issues, i.e. getting hit by a bus)

-72 hour return to ED for major deterioration in original condition (i.e. not just worsening sniffles, but discharged with a cold-->return to ED requiring intubation)

-Survival to ED discharge if presenting with some degree of stability (i.e. not in extremis upon arrival)

-Transfers to higher level of care for inpatient admissions for actual clinical deterioration in condition (i.e. not for a heart rate that went from 100-->101)

-Some basic clinical benchmarks (early goal directed therapy, pain medication for long-bone fractures)

-Patient dissatisfaction (just kidding . . . kind of . . . . although it might improve mortality

 

[Tiger26]

I'd also add physician satisfaction and longevity

 

[glorfindel]

I like these suggestions. If you could combine some of these with a metric that would identify over-ordering of tests (e.g. CBC on everyone!) then you'd be in the sweet spot.

Metrics that matter should measure efficient saving of lives.

 

[RustedFox]

I like these suggestions. If you could combine some of these with a metric that would identify over-ordering of tests (e.g. CBC on everyone!) then you'd be in the sweet spot.

Metrics that matter should measure efficient saving of lives.

I like the 'over-ordering' thing, too. 1-2 midlevels at my shop got a d-dimer on everyone who ate mexican food last night until I told him/her about the PERC rule and Wells Criteria. That fixed that.

 

[deleted109597]

-72 hour mortality after ED discharge (exclude unrelated issues, i.e. getting hit by a bus)
Too many lost to followup, so wouldn't be statistically sound. Good idea though.

-72 hour return to ED for major deterioration in original condition (i.e. not just worsening sniffles, but discharged with a cold-->return to ED requiring intubation)
Too many failure of outpatient management due to whatever reason.

-Survival to ED discharge if presenting with some degree of stability (i.e. not in extremis upon arrival)
Uh, do you have a lot of people walk in and die?

-Transfers to higher level of care for inpatient admissions for actual clinical deterioration in condition (i.e. not for a heart rate that went from 100-->101)
Yeah, lots of places auto review the RRT stuff within 72 hours of admission

-Some basic clinical benchmarks (early goal directed therapy, pain medication for long-bone fractures)
CMS follows this to a degree


What every place really needs is not "statistical analysis" based on whatever random stat you want. What they really need is someone (or a panel) of reasonable medical doctors who are up to date to review cases like those mentioned above, and discern if adequate medical care was taken. And if a physician has a routine of not performing great care, and instead routinely performs minimal care, then they need to either train or fire that physician. We need to practice good medicine, and not worry about the frail people dying. They're going to die. We also need to not defend bad medicine. Controversial medicine can get a pass for now.

 

[WilcoWorld]

...

 

[Arcan57]

-72 hour mortality after ED discharge (exclude unrelated issues, i.e. getting hit by a bus)
Too many lost to followup, so wouldn't be statistically sound. Good idea though.

-72 hour return to ED for major deterioration in original condition (i.e. not just worsening sniffles, but discharged with a cold-->return to ED requiring intubation)
Too many failure of outpatient management due to whatever reason.

-Survival to ED discharge if presenting with some degree of stability (i.e. not in extremis upon arrival)
Uh, do you have a lot of people walk in and die?

-Transfers to higher level of care for inpatient admissions for actual clinical deterioration in condition (i.e. not for a heart rate that went from 100-->101)
Yeah, lots of places auto review the RRT stuff within 72 hours of admission

-Some basic clinical benchmarks (early goal directed therapy, pain medication for long-bone fractures)
CMS follows this to a degree


What every place really needs is not "statistical analysis" based on whatever random stat you want. What they really need is someone (or a panel) of reasonable medical doctors who are up to date to review cases like those mentioned above, and discern if adequate medical care was taken. And if a physician has a routine of not performing great care, and instead routinely performs minimal care, then they need to either train or fire that physician. We need to practice good medicine, and not worry about the frail people dying. They're going to die. We also need to not defend bad medicine. Controversial medicine can get a pass for now.

I agree that the bad outcomes should be rare enough that statistics are not terribly useful in differentiating performance. But if you're not collecting data like the above, then you are going to have a very skewed group of cases going before your (peer review, M&M, etc). I would disagree that we should be let off the hook for 72 hr return admits. Not every return admit is a failure. There are a lot of docs that don't really care if their outpatient plan of care is actually workable., as long as it looks good on paper. At the very least, having a working idea about abx cost is useful or not prescribing zofran to an self-pay patient. Differences in return admits would probably be statistically different among docs, although then the question is asked "Do doc A's metrics look good because they are a great doc, or do they just admit everyone that walks in the door?" As soon as metrics are measured, they start being gamed.

 

[Tiger26]

All good points above--although unfortunately I think metrics are here to stay. They're not just in EM, but pretty much every aspect of society (sports, politics, finance, etc.)

Regardless of what we use, I think we as a specialty have to start owning this. It's easy to let others dictate how we're measured, but we need to come up with some clinically relevant things that matter to patients.

Sure, any time you start measuring things, people are going to find a way to game it, but patient satisfaction or length of stay can't fully communicate the fact that we do some pretty amazing things every day to take care of patients.

Of course, maybe we just prefer to let Press Ganey continue to do their thing . . . .

 

[deleted109597]

Of course, maybe we just prefer to let Press Ganey continue to do their thing . . . .

It doesn't matter. We don't decide what metrics to use, hospitals do. We often aren't there for the decisions. Patient satisfaction is part of HCAPS, so it isn't going anywhere. The hospital still uses Press Ganey, and you can argue with the C suite until you're blue in the face, they don't care. They want us faster, more efficient, with less staff, while doing less tests than necessary, but satisfying everyone....
The only thing we can do is strive to be the best physicians we can be.

I can say that the hospital doesn't actually perform the QI for my ED, we do our own. And we look at many of these metrics, plus others(inpatient mortality 72 hours after admission, return admissions within 72 hours of discharge, all the CMS core measures). We police ourselves, because if we don't, someone else will.

 

[docB]

To build on the topic of most metrics being useless and in some cases harmful for patients (see satisfaction, patient), what do you all think would be beneficial to advancing the practice of emergency medicine and providing better care for patients:

Wow. Great topic. The more I think about it the more convinced I am that we are using the worst, most counter productive metrics we could but it's also hard to think of a really good metric too.

I'd also add physician satisfaction and longevity

That would be nice but no one on the hospital end would care at all about that.

I like these suggestions. If you could combine some of these with a metric that would identify over-ordering of tests (e.g. CBC on everyone!) then you'd be in the sweet spot.

Metrics that matter should measure efficient saving of lives.

We occasionally get push back on "utilization" (the C suite term for ordering stuff). They never look at any specific instance. It would be too time consuming. They just look at who orders the most. If you're in the top three look out. There's no science to it. Just an unshakable belief on the part of administrators that every mean is golden and all outliers are deviant.

There are a lot of docs that don't really care if their outpatient plan of care is actually workable., as long as it looks good on paper. At the very least, having a working idea about abx cost is useful or not prescribing zofran to an self-pay patient. Differences in return admits would probably be statistically different among docs, although then the question is asked "Do doc A's metrics look good because they are a great doc, or do they just admit everyone that walks in the door?" As soon as metrics are measured, they start being gamed.

Good points. But will the hospital care? Does your hospital want you to admit a non-hypoxic, uncomplicated pneumonia because he can't afford his meds (I routinely have people tell me they can't even afford things on the $4 list)? Or would they rather you d/c him and roll the dice that he lands elsewhere when he gets worse. We all know the moral and medical thing to do but in this case that's exactly the opposite of what the hospital would want.

You're also right on about gaming the system. If the metric is about 72 hour bounce backs then everyone who is borderline gets admitted. If the metric is patients who crump on the floor (which is usually measured by RRTs and level of service upgrades) then everyone gets admitted to the unit. I bet if a hospital initiated those two metrics their units would be critically overloaded in a week.

 

[RustedFox]

Bump for relevance.

 

[The White Coat Investor]

Door to admission matters to me. Door to discharge matters to me too. I see nothing wrong with measuring those things. I think we see significant improvement when we do so.

How about length of time from ordering labs until they return? Same with x-rays. I see nothing wrong with door to pain meds for fractures.

Quality stuff can be okay too, but less useful most of the time.

Things like checking a pulse ox for patients with respiratory complaints are also low hanging fruit. Or blood types for vag bleeders. Maybe ratio of CT scans to number of patients seen? Might encourage fewer CTs, for better or worse.

We track RVUs per patient and patients per hour. That gives us some information about our various practice styles.

 

[coachB]

Door to admission matters to me. Door to discharge matters to me too. I see nothing wrong with measuring those things. I think we see significant improvement when we do so.

How about length of time from ordering labs until they return? Same with x-rays. I see nothing wrong with door to pain meds for fractures.

Quality stuff can be okay too, but less useful most of the time.

Things like checking a pulse ox for patients with respiratory complaints are also low hanging fruit. Or blood types for vag bleeders. Maybe ratio of CT scans to number of patients seen? Might encourage fewer CTs, for better or worse.

We track RVUs per patient and patients per hour. That gives us some information about our various practice styles.

A bit off track here, but perhaps evidence that metrics need to be thought about carefully and evidence based and not just "because." Why do you suggest checking a blood type for vag bleeders? I'm guessing you are talking about first trimester vag bleeders. There is not even a theoretical risk of alloimmunization before 8 weeks and there is no evidence to suggest that this occurs at all in any threatened miscarriage. In fact, the UK College of OB-GYN clearly states that this does not need to be evaluated in the first trimester. Many of us are still practicing based on a non evidence-based guideline from ACOG to "consider" testing.

 

[MSmentor018]

oh I didn't even see this thread. I had started something of my own

I've been saying this $hit for years.....

http://www.kevinmd.com/blog/2016/07/youll-surprised-ers-meeting-metrics.html

 

[Hercules]

Door to admission matters to me. Door to discharge matters to me too. I see nothing wrong with measuring those things. I think we see significant improvement when we do so.

How about length of time from ordering labs until they return? Same with x-rays. I see nothing wrong with door to pain meds for fractures.

Quality stuff can be okay too, but less useful most of the time.

Things like checking a pulse ox for patients with respiratory complaints are also low hanging fruit. Or blood types for vag bleeders. Maybe ratio of CT scans to number of patients seen? Might encourage fewer CTs, for better or worse.

We track RVUs per patient and patients per hour. That gives us some information about our various practice styles.

I would argue that this recent set of CMS metrics is the most useful from an EM perspective ever. Seriously. BS blood cultures before Abx? Useless. These new metrics actually address the universal ED problem of boarding patients. These new metrics now mean that a hospital will get its reimbursement decreased if it takes too long for an admitted patient to get upstairs. That is freaking awesome. Because of these changes, I now have the CEO's full attention when discussing delays in getting admitted patients upstairs. Because of these new metrics I finally got the green light to implement a no delay nurse report where the patient goes up within 30 mins of bed assignment whether the floor nurse feels like they are ready or not. We have pushed for that for years, but now all of the sudden C-suite is all ears and thinks it's a fabulous idea. In my medical career, I can honestly say this is the first time a CMS metric has had a positive impact on patient care as well as my practice.

I'm all for righteous indignation over useless metrics. I just think these are some of the most relevant metrics we've seen from CMS.


Sent from my iPad using SDN mobile

 

[GeneralVeers]

I don't think CMS should have metrics. Any organization involved in payment/non-payment should not be setting standards or metrics because they have a vested interest to establish guidelines that are either irrelevant or unrealistic.

 

[deleted109597]

I would argue that this recent set of CMS metrics is the most useful from an EM perspective ever. Seriously. BS blood cultures before Abx? Useless. These new metrics actually address the universal ED problem of boarding patients. These new metrics now mean that a hospital will get its reimbursement decreased if it takes too long for an admitted patient to get upstairs. That is freaking awesome. Because of these changes, I now have the CEO's full attention when discussing delays in getting admitted patients upstairs. Because of these new metrics I finally got the green light to implement a no delay nurse report where the patient goes up within 30 mins of bed assignment whether the floor nurse feels like they are ready or not. We have pushed for that for years, but now all of the sudden C-suite is all ears and thinks it's a fabulous idea. In my medical career, I can honestly say this is the first time a CMS metric has had a positive impact on patient care as well as my practice.

If only. I can't imagine I'm the only one who, when the red alarm for door to admission starts ringing, it gets all the suits downstairs so they can get in the way while asking me and everyone else "what can we do to help". Well, you see all these admitted patients? Put them upstairs. "No, I meant what else can we do to help, because we aren't doing that"
Sigh

 

[WilcoWorld]

I would argue that this recent set of CMS metrics is the most useful from an EM perspective ever. Seriously. BS blood cultures before Abx? Useless. These new metrics actually address the universal ED problem of boarding patients. These new metrics now mean that a hospital will get its reimbursement decreased if it takes too long for an admitted patient to get upstairs. That is freaking awesome. Because of these changes, I now have the CEO's full attention when discussing delays in getting admitted patients upstairs. Because of these new metrics I finally got the green light to implement a no delay nurse report where the patient goes up within 30 mins of bed assignment whether the floor nurse feels like they are ready or not. We have pushed for that for years, but now all of the sudden C-suite is all ears and thinks it's a fabulous idea. In my medical career, I can honestly say this is the first time a CMS metric has had a positive impact on patient care as well as my practice.

I'm all for righteous indignation over useless metrics. I just think these are some of the most relevant metrics we've seen from CMS.


Sent from my iPad using SDN mobile

It sounds like you are using your situation very wisely - kudos. Yours is an example to follow. Unfortunately, there are still a lot of places like those where @Dr.McNinja works that just don't (or won't) get it. The way I see it, we have two options:

1 - Apply for a job with Hercules' or WCI's group.
2 - Work within our systems to help the administration get it.

My wife likes where we live now, so I'm working on #2.

 

[Boatswain2PA]

I like the 'over-ordering' thing, too. 1-2 midlevels at my shop got a d-dimer on everyone who ate mexican food last night until I told him/her about the PERC rule and Wells Criteria. That fixed that.

I've got one doc at my shop who tried to refuse signing my chart because I didn't do a D-dimer on an old lady with resolved belly pain with a Wells of zero because her O2 sats were at 90....despite "she's supposed to be wearing oxygen at home".

 

[RustedFox]

I've got one doc at my shop who tried to refuse signing my chart because I didn't do a D-dimer on an old lady with resolved belly pain with a Wells of zero because her O2 sats were at 90....despite "she's supposed to be wearing oxygen at home".

She didn't PERC-out.

 

[Boatswain2PA]

Correct....but neither did the little old lady with a stubbed toe. Neither one of which had a PE.

 

[RustedFox]

So why was he wanting the d-d-d-d-dimer?

 

[Boatswain2PA]

So why was he wanting the d-d-d-d-dimer?

R/o PE, same reason your 2 mlps use it. He was worried about slight hypoxia which was well explained away.

 

[Boatswain2PA]

Rusted - I'm not arguing with you...just pointing out that sometimes us MLPs over order because of the physician oversight. Some docs directly tell us to over-order, other times we over-order to clean up our charts so that the doc who reads the chart will feel better about it.

And...sometimes we simply over-order. I know I order stuff all the time that I don't think is going to come back positive.....but then find that I'm glad I did.

 

[RustedFox]

The way your first sentence was written, it seemed like he wanted a dimer, but wasn't concerned about PE. Maybe I read it wrong.

I'm not arguing either. The d-d-d-dimer is like c-c-c-c-combo breaker. Poorly implied joke on my part. Google it if you don't get it.

Our MLPs are extraordinarily lazy when it comes to "I ordered this; but now it's your problem." Its a systems issue here between our two hospitals.

Maybe people could chime in on this.

The same MLP crew works at hospital A and hospital B.

At hospital "A", there's a "SuperTrack" section, where they start the chart and workup, but then the patient is "identified" as one that needs to be graduated to MD/DO care. There are 3-4 docs on at any time, so those docs can always "pick up the SuperTrack graduates".

At hospital "B", there is no such "SuperTrack" - the MLPs are expected to see their cases thru and finish the charting....but instead, they treat mostly all level 3's (and some level 4's) as "SuperTrack" patients - despite there only being one MD/DO on a time, who has their hands full. Nevermind the fact that the cases generally don't need oversight. They just love to dump a half-workup/half-finished chart in your lap. The justification/argument that we hear is "Well, that's how we do it at hospital A". Thus, the number of bogus D-dimers is high, because they can always "dump it to the doc".

Yeah, we know that's hospital "A"s flow. They're higher volume, higher acuity, higher coverage. Hospital "B" is lower acuity, higher pts/hour. Doesn't work.

There's probably more to it than I can parse out right now - will come out when asked for.

 

[deleted109597]

RustedFox isn't wrong here. I've found the fastest way to decrease X over-utilized test by midlevel is to make them responsible for it. That EKG you ordered for leg pain? Read it. And the one for chest pain. And the one for shortness of breath. If you don't know how to interpret results of a test, you shouldn't be ordering it. I don't expect them to get nuances here, but damned if they can even look at the damn things. Sure, have the doc sign it, but take it back up front for them to deal with. Same as d-dimer or whatever. Suddenly they're less likely to do something when the default isn't "make it someone else's problem".
That being said, a lot of it has to do with the "us vs them" mentality at all of the shops I've been at. They're up front, not talking with the docs for pretty much the whole shift. We don't do the hiring, the chief PA does. It's a bizzaro world.

 

[Boatswain2PA]

The way your first sentence was written, it seemed like he wanted a dimer, but wasn't concerned about PE. Maybe I read it wrong.......

He never saw, or was briefed on, the patient. Just saw on chart that a LOL had mild hypoxia and wasn't going to sign the chart because it "could've been" a PE and I missed it cause I didn't do a D-dimer.

As to the rest....wow, that's a mess. I want to handle my patients because too many cooks ruin the pot. I use my Doc as a consultant and QA/QI.

RustedFox isn't wrong here. I've found the fastest way to decrease X over-utilized test by midlevel is to make them responsible for it. That EKG you ordered for leg pain? Read it. And the one for chest pain. And the one for shortness of breath. If you don't know how to interpret results of a test, you shouldn't be ordering it. I don't expect them to get nuances here, but damned if they can even look at the damn things. Sure, have the doc sign it, but take it back up front for them to deal with. Same as d-dimer or whatever. Suddenly they're less likely to do something when the default isn't "make it someone else's problem".
That being said, a lot of it has to do with the "us vs them" mentality at all of the shops I've been at. They're up front, not talking with the docs for pretty much the whole shift. We don't do the hiring, the chief PA does. It's a bizzaro world.

Absolutely the MLP should be responsible for every aspect of the patient....and call you for help when needed, but never just dump on your lap.

Regarding the "us vs them"....I don't see that, but I'm in rural America.

 

[Psai]

RustedFox isn't wrong here. I've found the fastest way to decrease X over-utilized test by midlevel is to make them responsible for it. That EKG you ordered for leg pain? Read it. And the one for chest pain. And the one for shortness of breath. If you don't know how to interpret results of a test, you shouldn't be ordering it. I don't expect them to get nuances here, but damned if they can even look at the damn things. Sure, have the doc sign it, but take it back up front for them to deal with. Same as d-dimer or whatever. Suddenly they're less likely to do something when the default isn't "make it someone else's problem".
That being said, a lot of it has to do with the "us vs them" mentality at all of the shops I've been at. They're up front, not talking with the docs for pretty much the whole shift. We don't do the hiring, the chief PA does. It's a bizzaro world.

Maybe this chief PA can start looking at their charts.