Calling Bullsh*t in the Age of "Big Data"

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.
drusso

drusso

Full Member
Moderator Emeritus
Lifetime Donor
Verified Member
Physician
Verified Expert
Gold Donor
Joined
Nov 21, 1998
Messages
13,273
Reaction score
7,911
Members do not see this ad.
"Bergstrom and West are longtime scientific collaborators and have spent years grumbling about the inflated claims, manipulated algorithms, and twisted interpretations of scientific research that they see not just in the popular press, but also in grant applications and scientific papers. “We just thought, instead of griping, we could make a class — a really fun class,” Bergstrom said

https://www.statnews.com/2017/02/17/science-fights-alternative-facts/

"But one thing is certain: If policymakers want clearer direction about how to prioritize limited resources, then they can stop paying for meta-analysis. Instead, taxpayers would be best served by having publicly supported scientists spend their time conducting better experiments and getting dirty again with original data."

http://www.bizjournals.com/portland...lumn-big-data-the-danger-in-knowing-less.html

The world is watching, Roger Chou...
 
So, you you think Bergstom and West think medical meta-analyses are BS? You realize that all big published meta-analyses have
one or more statisticians involved in vetting the final results.
 
101N said:
So, you you think Bergstom and West think medical meta-analyses are BS? You realize that all big published meta-analyses have
one or more statisticians involved in vetting the final results.
Click to expand...

An oldie, but a goodie: In other words, GIGO.

J Accid Emerg Med. 1996 Nov;13(6):373-8.
The use and misuse of meta-analysis.
Lecky FE1, Little RA, Brennan P.
Author information

Abstract
OBJECTIVE:
To demonstrate how the results of a meta-analysis can confuse rather than clarify therapeutic dilemmas if clinical heterogeneity among trials is ignored. Then to further discuss the qualities emergency physicians should expect from published meta-analyses if they are to affect clinical practice.

SUBJECTS AND METHODS:
The data and results were examined from 23 randomised controlled trials of selective decontamination of the digestive tract (SDD), which have been combined in a previous meta-analysis. These were reviewed to take account of clinical heterogeneity, particularly with regard to severity of patient illness.

RESULTS:
Severity of patient illness predicts degree of reduction in mortality with SDD in a regression analysis: log odds ratio (OR) of death with SDD = -0.0074 - (0.0035 x control group mortality rate), P = 0.017. This is also true when trials are stratified into more and less severely ill patients: pooled OR (a) for CMR > 41% = 0.69 (0.54 to 0.89), with (b) CMR < 3% = 1.02 (0.86 to 1.21). This difference was not suggested by the original meta-analysis result.

CONCLUSIONS:
Failure to take account of clinical heterogeneity between trials can mean a meta-analysis result ignores important differences in the effect of a treatment on different groups of patients. The discussion indicates how emergency physicians might guard against basing clinical practice on misleading meta-analysis results.
 
So all meta-analyses are BS, garbage. That is your assessment. Just trying to be clear🙂
 
drusso said:
"Bergstrom and West are longtime scientific collaborators and have spent years grumbling about the inflated claims, manipulated algorithms, and twisted interpretations of scientific research that they see not just in the popular press, but also in grant applications and scientific papers. “We just thought, instead of griping, we could make a class — a really fun class,” Bergstrom said

https://www.statnews.com/2017/02/17/science-fights-alternative-facts/

"But one thing is certain: If policymakers want clearer direction about how to prioritize limited resources, then they can stop paying for meta-analysis. Instead, taxpayers would be best served by having publicly supported scientists spend their time conducting better experiments and getting dirty again with original data."

http://www.bizjournals.com/portland...lumn-big-data-the-danger-in-knowing-less.html

The world is watching, Roger Chou...
Click to expand...

Yeah except Roger Chou has zero real clinical experience and is paid an inflated salary to essentially do nothing.

People don't give up rackets like that without a fight.
 
101N said:
So all meta-analyses are BS, garbage. That is your assessment. Just trying to be clear🙂
Click to expand...

He posted clearly how "meta analyses" can be misused with a very convincing article.

Furthermore, when Chou recommends "Chiropractic" care as "high yield" despite zero studies confirming its efficacy, I find it amusing.

Chou seems to think that patients never see Chiropractors before going to their PCP. If he was out in the real world, he would realize the a good number of patients go to the Chiropractor FIRST before going to their PCP about their pain state.
 
101N said:
So all meta-analyses are BS, garbage. That is your assessment. Just trying to be clear🙂
Click to expand...

I think that I make myself pretty clear about my values and positions on these topics, but I'll take another run at it:

"The interpretation of a meta-analysis is potentially subject to an author’s bias by what inclusion and exclusion criteria is selected, the type of statistical evaluation performed, decisions made on how to deal with disparities between the trials, and how the subsequent results are presented. Whether the conclusions of a meta-analysis are broad reaching or limited can be affected by the inherent bias that the author of the meta-analysis brings to the study.

Human nature dictates that each of us tends to find it more satisfying to confirm a previously held opinion, particularly a published opinion, rather than create an analysis that refutes our own prior conclusions. Hence, interpretive bias is even more likely to occur when a meta-analysis is conducted by an author with a strong particular viewpoint in an area of controversy. When the meta-analysis is conducted by a strong advocate of a particular position, it is more likely to be biased in concordance with the author's previously advocated opinion."

In other words, in order to believe in Roger's analysis of opioids (the APS/AAPM guidelines, the PROP/CDC Redeux, etc), injections (OHA's HERC), or Moo-Shu pain treatments (OHSU/Kaiser Foundation), one *MUST* believe that Roger Chou is an honest broker of information and not tainted by influence from who is writing checks, signing off on promotions, or matching his PERS/retirement contributions: That's debatable.

I would trust @ampaphb's, @lobelsteve, or @algosdoc experience in pain/spine care over Roger's experience any day.
 
drusso said:
I think that I make myself pretty clear about my values and positions on these topics, but I'll take another run at it:

"The interpretation of a meta-analysis is potentially subject to an author’s bias by what inclusion and exclusion criteria is selected, the type of statistical evaluation performed, decisions made on how to deal with disparities between the trials, and how the subsequent results are presented. Whether the conclusions of a meta-analysis are broad reaching or limited can be affected by the inherent bias that the author of the meta-analysis brings to the study.

Human nature dictates that each of us tends to find it more satisfying to confirm a previously held opinion, particularly a published opinion, rather than create an analysis that refutes our own prior conclusions. Hence, interpretive bias is even more likely to occur when a meta-analysis is conducted by an author with a strong particular viewpoint in an area of controversy. When the meta-analysis is conducted by a strong advocate of a particular position, it is more likely to be biased in concordance with the author's previously advocated opinion."

In other words, in order to believe in Roger's analysis of opioids (the APS/AAPM guidelines, the PROP/CDC Redeux, etc), injections (OHA's HERC), or Moo-Shu pain treatments (OHSU/Kaiser Foundation), one *MUST* believe that Roger Chou is an honest broker of information and not tainted by influence from who is writing checks, signing off on promotions, or matching his PERS/retirement contributions: That's debatable.

I would trust @ampaphb's, @lobelsteve, or @algosdoc experience in pain/spine care over Roger's experience any day.
Click to expand...

Also where is all the "meta analysis" of all the crap Chou is pushing like acupuncture or Chiropractic care or "tai chi" for spinal stenosis/herniated discs?

Seems like his opinion on these subjects bear far less scrutiny for these "treatment" modalities.

If "injections" only provide "short term" pain benefit, is he arguing that Chiropractic, acupuncture and "tai chi" offer longer acting benefits? Where's the evidence for all that crap he's pushing?

Are you honestly trying to say that tai chi and acupuncture are "high yield" compared to a laminectomy surgery for a spinal stenosis that is considered "low yield"?

Should be we be sending all these patients to Chinese alternative practitioners and Chiropractors for these conditions now? Why even see a physician then?

Why Chou's never cut cost in the system either: https://fee.org/articles/the-chart-that-could-undo-the-us-healthcare-system/ These "administrators" cost far more than they "save" in terms of cost.
 
So not all meta-analyses are suspect. Just those by Roger Chou because of his implicit bias about opioids and injections?
 
101N said:
So not all meta-analyses are suspect. Just those by Roger Chou because of his implicit bias about opioids and injections?
Click to expand...

Results from network meta-analysis are even more suspect. They're like the Collateralized Debt Obligations of GIGO science...I wonder if you know that Roger has done any network meta-analyses?

Ann Intern Med. 2013 Jul 16;159(2):130-7. doi: 10.7326/0003-4819-159-2-201307160-00008.
Conceptual and technical challenges in network meta-analysis.
Cipriani A1, Higgins JP, Geddes JR, Salanti G.
Author information

Abstract
The increase in treatment options creates an urgent need for comparative effectiveness research. Randomized, controlled trials comparing several treatments are usually not feasible, so other methodological approaches are needed. Meta-analyses provide summary estimates of treatment effects by combining data from many studies. However, an important drawback is that standard meta-analyses can compare only 2 interventions at a time. A new meta-analytic technique, called network meta-analysis (or multiple treatments meta-analysis or mixed-treatment comparison), allows assessment of the relative effectiveness of several interventions, synthesizing evidence across a network of randomized trials. Despite the growing prevalence and influence of network meta-analysis in many fields of medicine, several issues need to be addressed when constructing one to avoid conclusions that are inaccurate, invalid, or not clearly justified. This article explores the scope and limitations of network meta-analysis and offers advice on dealing with heterogeneity, inconsistency, and potential sources of bias in the available evidence to increase awareness among physicians about some of the challenges in interpretation.

PMID:

23856683

DOI:

10.7326/0003-4819-159-2-201307160-00008
 
101N said:
So not all meta-analyses are suspect. Just those by Roger Chou because of his implicit bias about opioids and injections?
Click to expand...

Once again, I'd like to see how Chou did the "analysis" in Chiropractic care or Acupuncture for spinal stenosis or degenerative disc disease and long term benefits considering he is strongly recommending those.

Cochrane analysis of "Chiropractic care" has shown there are zero studies proving its efficacy yet Chou recommends it as "high yield".

Something doesn't seem right in Denmark here.

I am suspect about alot of big Pharma stuff on Statin drugs as well and many other topics.
 
algosdoc said:
Chou lives in an alternate universe. That explains everything.
Click to expand...

Just from an intuitive standpoint, if sticking needles into the spine to inject medication around a disc herniation doesn't help long term enough so that its considered "low yield" by Chou, how the F does sticking needles into the skin throughout the body (and possibly muscles) with zero medication that isn't even close to the spine give "high yield" results for long term pain benefit?

Here's the results I find for Chiropractic care and spinal stenosis:

https://www.ncbi.nlm.nih.gov/pubmed/19646390

https://health.spectator.co.uk/the-evidence-shows-that-chiropractors-do-more-harm-than-good/

https://sciencebasedmedicine.org/top-10-chiropractic-studies-of-2013/

Have there been a bunch of "randomized" trials in the last few years im missing for these modalities?
 
Last edited:
Ducttape said:
looking at the literature, there are a few articles on tai chi for nonspecific back pain.

https://www.ncbi.nlm.nih.gov/pubmed/22034119
https://www.ncbi.nlm.nih.gov/pubmed/24926131
https://www.ncbi.nlm.nih.gov/pubmed/27062950
not great articles.

but nothing online disparaging tai chi for chronic nonspecific back pain, either. unlike some other treatments...
Click to expand...

True but isn't Chou's algorithim for all causes of back pain including stenosis/herniated discs/spondylosis/etc? Not just nonspecific back pain of unknown etiology?

How does that make it "high yield"? Acupuncture and Chiropractic is even worse (see above)
 
Last edited:
101N said:
So not all meta-analyses are suspect. Just those by Roger Chou because of his implicit bias about opioids and injections?
Click to expand...

http://www.newyorker.com/magazine/2017/02/27/why-facts-dont-change-our-minds

"One way to look at science is as a system that corrects for people’s natural inclinations. In a well-run laboratory, there’s no room for myside bias; the results have to be reproducible in other laboratories, by researchers who have no motive to confirm them. And this, it could be argued, is why the system has proved so successful. At any given moment, a field may be dominated by squabbles, but, in the end, the methodology prevails. Science moves forward, even as we remain stuck in place."
 
drusso said:
http://www.newyorker.com/magazine/2017/02/27/why-facts-dont-change-our-minds

"One way to look at science is as a system that corrects for people’s natural inclinations. In a well-run laboratory, there’s no room for myside bias; the results have to be reproducible in other laboratories, by researchers who have no motive to confirm them. And this, it could be argued, is why the system has proved so successful. At any given moment, a field may be dominated by squabbles, but, in the end, the methodology prevails. Science moves forward, even as we remain stuck in place."
Click to expand...

Chou isn't a scientist either since he never does original research since he has no clinical experience or real world knowledge.

Ergo, Chou's whole career is to run meta analyses for his employer to determine "efficiency" in any manner he would like.

That is why this clown argues that Chiropractic, Tai Chi and Acupuncture are "high yield" for spinal stenosis/DDD which is literally impossible to show using current literature.

I think alot of this has to do with jealousy by medicine people towards specialists as well.

Any type of argument you want to make that an injection only provides "short term" relief can be triple said for those above modalities.
 
101N, are you a fan of meta-analysis?

I have, on several occasions, read a lot of articles on a subject, then read the meta-analysis that included most of the articles I read, and the meta-analysis leaves a distinctly different impression then what I got from reading the articles.

Medicine, more then lab or bench work, is very nuanced - and meta-analysis washes all this away.

I'll give a short example. If you read the meta-analysis that Apfel did recently on post dural puncture headaches, you conclude - hardly anything works at all. But if you read the articles, you get a very different impression. For example, leaving an intra-thecal catheter in place has often been touted as useful after a known wet tap. So if you read those articles that say this - you get an impression they actually work. They may not cut down on the incidence, but the headaches are MUCH less, and usually don't need any further treatment, and the time the catheter is in place maters, etc, etc. However, if the meta-analysis just looks at incidence and doesn't go into detail - it concludes "NO, it doesn't work" which is complete bull**** if you read the articles. It is 100% a lie - but Apfel can defend his position because of data bla bla bla.

There is not a single hierarchy of data scale that I am aware of that includes meta-analysis. They all start with case reports, then observations and retrospective, then small randomized controlled, then multiple, large scale, randomized trials. Meta-analysis is usually not listed.
 
epidural man said:
101N, are you a fan of meta-analysis?

I have, on several occasions, read a lot of articles on a subject, then read the meta-analysis that included most of the articles I read, and the meta-analysis leaves a distinctly different impression then what I got from reading the articles.

Medicine, more then lab or bench work, is very nuanced - and meta-analysis washes all this away.

I'll give a short example. If you read the meta-analysis that Apfel did recently on post dural puncture headaches, you conclude - hardly anything works at all. But if you read the articles, you get a very different impression. For example, leaving an intra-thecal catheter in place has often been touted as useful after a known wet tap. So if you read those articles that say this - you get an impression they actually work. They may not cut down on the incidence, but the headaches are MUCH less, and usually don't need any further treatment, and the time the catheter is in place maters, etc, etc. However, if the meta-analysis just looks at incidence and doesn't go into detail - it concludes "NO, it doesn't work" which is complete bull**** if you read the articles. It is 100% a lie - but Apfel can defend his position because of data bla bla bla.

There is not a single hierarchy of data scale that I am aware of that includes meta-analysis. They all start with case reports, then observations and retrospective, then small randomized controlled, then multiple, large scale, randomized trials. Meta-analysis is usually not listed.
Click to expand...

Exactly. This hits the nail on the head.

Bottom line: Roger Chou is entitled to his own opinions, but not his own facts. Moreover, no one is entitled to tax-payer paid/government handouts for grants to do GIGO science perpetuating an air of "truthiness"--- especially performed by someone with little or no real content expertise or experience in the field.
 
drusso said:
Exactly. This hits the nail on the head.

Bottom line: Roger Chou is entitled to his own opinions, but not his own facts. Moreover, no one is entitled to tax-payer paid/government handouts for grants to do GIGO science perpetuating an air of "truthiness"--- especially performed by someone with little or no real content expertise or experience in the field.
Click to expand...

Do you even think Chou even knows how to put in an iv or perform an LP?

Dude is a paid ***** for special interests and fake administrative people who offer zero value to the System.

He is the reason why administrators have gone up 3000 percent in the last two decades and the real reason for high healthcare costs.
 
http://www.medscape.com/viewarticle/875175

Lies, lies, COI, and more lies.

Reviews done by SFSBM (Crislip, Novella) show otherwise.

Am J Emerg Med. 2016 Oct 5. pii: S0735–6757(16)30690–8. doi: 10.1016/j.ajem.2016.10.001. Complementary therapies in addition to medication for patients with nonchronic, nonradicular low back pain: a systematic review.. "In conclusion, for patients with nonchronic, nonradicular LBP, available evidence does not support the use of spinal manipulation or exercise therapy in addition to standard medical therapy. There is insufficient evidence to determine if yoga or massage is beneficial."
 
lobelsteve said:
http://www.medscape.com/viewarticle/875175

Lies, lies, COI, and more lies.

Reviews done by SFSBM (Crislip, Novella) show otherwise.

Am J Emerg Med. 2016 Oct 5. pii: S0735–6757(16)30690–8. doi: 10.1016/j.ajem.2016.10.001. Complementary therapies in addition to medication for patients with nonchronic, nonradicular low back pain: a systematic review.. "In conclusion, for patients with nonchronic, nonradicular LBP, available evidence does not support the use of spinal manipulation or exercise therapy in addition to standard medical therapy. There is insufficient evidence to determine if yoga or massage is beneficial."
Click to expand...

Interesting stuff.

Did some Chiros pay off chou or some thai chi dudes?

We have our own acupuncture person in our practice and I dont see these great results Chou is reporting. Doesnt seem to be high yield either since they need the acupuncture basically weekly. Chiro often runs thousands of dollars per three month interval at 5 min "adjustment" sessions. Price is even higher when they use the DRX 9000 or some other crap machine. Dont see this "high yield" in terms of pricing either.

I've had multiple patients be offered 5-10K chiropractic "deals" that lasted for approximately 3 months with adjustments 2-3X per week and usage of a DRX 9000 machine.

Don't see how this is going to "save" money. These patients ALL come back to my practice approximately 3-6 months after going to Chiropractor.

We encourage our patients to use thai chi, acupuncture, CBT and Chiropractic all the time. In fact, most of have done this before coming to our practice.

Haven't been short for patients.
 
Ducttape said:
So why still have this acupuncturist in your practice?

Oh wait... It's about $$$?


Sent from my iPhone using SDN mobile
Click to expand...

He makes the money, I don't get nada from it.

Dude trained in acupuncture and is a PMR physician as well. Does it on the side. Took the requirements you need to obtain certification in that stuff.

If patients are willing to pay for the service from him, why should I fight against it?

Shouldn't you like that since its "high yield" treatment as per Chou?
 
depends on what you are mentioning. hong kong style chou mein, yes.

oh, that chou from Washington...



where did you ever get the impression that i thought he was a "contributor" to healthcare policy, regardless of whether he is wrong or just plain wrong?
 
Ducttape said:
depends on what you are mentioning. hong kong style chou mein, yes.

oh, that chou from Washington...



where did you ever get the impression that i thought he was a "contributor" to healthcare policy, regardless of whether he is wrong or just plain wrong?
Click to expand...

According to 101N, Chou is beyond reproach.

Ergo, I am going to further add thai chi, tae kwon doe, yoga, shaman soothers and ancient chinese herbal medicine to my pain management regimen. That stuff is pretty high yield. I think I'll make the combination into 300/month installations of high yield treatment patterns for spinal conditions.

Got to charge a fair market price for high yield stuff.
 
im willing to bet good money that anyone that can do any of these yoga poses doesnt have back pain that requires MED>90.
8b2bce9fb527ca073f5e6a5a2a74adee.jpg

advance-postures.jpg

o-KINO-MALLORCA-900.jpg
 
Ducttape said:
i was going to post some of the other more, well, fat shamed individuals, but didnt want to get banned...
Click to expand...

show me any mainstream fat shaming sources.

I just gave you a video of feminists demanding people find obese women attractive. Clearly, you have no retort.
 
Really good yoga practioners who've worked their way up seem to have pretty good backs in my experience, but there are lots of folks trying to force flexibility who have crunched their spines pretty impressively doing too many upward facing dogs.

EDIT: my question about meta-analysis has always been, how can I get in on that action? Publishing High Quality Evidence without seeing any patients or dealing with IRBs sounds like the perfect gig.
 
SeniorWrangler said:
Really good yoga practioners who've worked their way up seem to have pretty good backs in my experience, but there are lots of folks trying to force flexibility who have crunched their spines pretty impressively doing too many upward facing dogs.

EDIT: my question about meta-analysis has always been, how can I get in on that action? Publishing High Quality Evidence without seeing any patients or dealing with IRBs sounds like the perfect gig.
Click to expand...

Just go do what I did. Get a MPH degree.
 
SeniorWrangler said:
Really good yoga practioners who've worked their way up seem to have pretty good backs in my experience, but there are lots of folks trying to force flexibility who have crunched their spines pretty impressively doing too many upward facing dogs.

EDIT: my question about meta-analysis has always been, how can I get in on that action? Publishing High Quality Evidence without seeing any patients or dealing with IRBs sounds like the perfect gig.
Click to expand...

Hard to get in on that racket like Chou.

The dude has zero clinical experience but can pontificate like an "expert" while eliminating all "low value" studies to obtain the conclusions he was paid to obtain.

Essentially, you already know the outcome of their "meta analyses" beforehand.

This is happening in all of procedural medicine, with the most recent in BMJ concerning knee replacement surgeries in the US.
 
You must log in or register to reply here.

Similar threads

drusso
Big Beautiful Bill: Implications for Pain...
6 7 8 9 10
Replies
497
Views
21K
Agast
Agast
drusso
Prescribing GLP-1 RA's in the Pain Clinic
Replies
21
Views
2K
epidural man
epidural man
P
Huge data theft. Check if your info was leaked
Replies
4
Views
960
clubdeac
clubdeac
drusso
NEJM: The Broken Promise of Profit-Driven Medicine
Replies
17
Views
1K
cowboydoc
cowboydoc
drusso
  • Poll Poll
Where's the smart money in Pain?
Replies
24
Views
3K
PMROralBoards
P
Top