Ok Facet, here's the rundown:
The statement you keep trying to make in this thread is... (I think anyways) that evidence based medicine isn't very evidence based at all. I try to see what point you are actually driving at within your worldview and the only one that makes sense is this as a counterpoint to challenges that your own profession is not based on evidence at all. So in addressing your links I am looking for obvious sources of bias.
Interestingly, in the first link (as I pointed out earlier) we have 3 men all with severe conflicts of interest associated with the topics at hand. One is even selling sham treatments. Now, I called you out last time we discussed this thread for posting editorials referencing published work. You've done this
constantly throughout these threads. What you need to understand is this: people editorializing on a publication are not necessarily inherently biased. It is possible for them to convey the understanding of a work that you or I would extract from it ourselves. But
if they are biased it is impossible to know the original author's meanings and conclusions by reading the secondary source.
There have been numerous times that you have posed a blog or source such as this that, when I open the original document, the publication doesn't agree in the slightest with what the blogger is saying. Ironically, it is selective interpretation of the work which is exactly what they are here claiming to oust. So let's look at what the JAMA paper actually says as compared to the blogger you cite....
Blogger:
suggests that the evidence base used to evaluate novel biomarkers may be seriously compromised by selective reporting bias.
JAMA:
Very large heterogeneity and small-study effects were seen in 9 and 13 meta-analyses, respectively. In 29 meta-analyses (52%), there was a significant excess of studies with statistically significant results. Selective reporting biases may be common in the evidence on emerging cardiovascular biomarkers. Most of the proposed associations of these biomarkers may be inflated.
These are vastly different statements. The purpose of the JAMA paper is this self-policing that I have mentioned many times (that same self-policing that is absent in every paper you cite from your side of the fence). The things at play here - it is true that negative results are not often published. This isn't some greater demon of western medicine witchcraft at work here. It is intuitive. See below:
Hypothesis: Shooting myself in the leg with a 12 gauge will cure sporadic headaches.
---experiment: n=2 (legs)
Conclusion: Correlation between leg shooting and reprieve from headaches was statistically insignificant.
When do you think Nature should pick this one up? You're right in that there is some potential bias when numerous studies on the same topic show mixed results and only the positive results are published. Personally, I am not aware of too many where this happens to a truly alarming degree. It is a form of systematic error that is difficult to purge at the meta level. But there are a number of watchdog labs who do nothing but take other people's studies and blend them together to see if a real difference exists, and IIRC the statistics are powerful enough to tease this out at times even in occasions when all such papers show correlation.
In the JAMA paper, they are holding 1 meta as the gold standard in terms of "unbiased" (the largest, which is kinda fair), and comparing each to it. They are looking to see if positive outcomes are likely to be confounded by group heterogeneity and meta size. They conclude that a large number of the metas out there suffer from "small study effects" and predict that positive reporting may somewhat skew the data upward.
Now, does this mean that the findings are simply wrong? Absolutely not. And this is where I am left wondering "what is your point?". You are, perhaps without knowing it, actually highlighting the mechanisms by which EBM works and
has worked for years. EBM is a conduction of studies until a base of knowledge is obtained, and then
also near constant review of that knowledge in the face of new knowledge that presents itself. It changes constantly and nearly always for the better. Studies like these (and all EBM studies) serve to shape the physicians approach to disease. They give "nudges" to clinical reasoning, they don't reform it with each new publication. You seem to think that I am as biased as you are here but I have absolutely decimated papers presented on the EBM side of things as well.
In a nutshell:
1) positive reporting is often not only unavoidable, but usually the best rational approach to research (see headache analogy if you are still confused)
2) In cases where data is mixed, the competitive (or even cutthroat) atmosphere of academic research means that negative findings will be published with increasing frequency.
3) I'm actually publishing a paper right now that calls out a number of prior publications for reporting genetic anomalies in hereditary disease that are actually common variants. (just throwing this in to demonstrate that I am not crusading for the holy sanctity of EBM. I am an equal opportunity critic. Some venues just facilitate the criticism more than others)
4) EBM stands as the best approach to a systematic way of employing knowledge that we have. The alternative by which you ascribe fails in a much larger way than EBM ever does as demonstrated in our discussion here which illuminated authors of significant bias both commercially and professionally as well as the complete lack of any system of self regulation.
5) and.... I simply cannot stress this one enough.... bloggers and editorial publications are not suitable works for discussion of facts. As shown above, the author of your link is dramatically exaggerating the findings of the JAMA paper, which... Ironically, is on the topic of exaggeration in publications. Come to think of it, I don't think you've posted a paper since I've joined that doesn't trigger my irono-meter.
Just.... just look at the original studies for what they are. All studies. There are good, there are bad, and there are those like the JAMA paper that are "ok" and aim not to undermine research but to improve the body of knowledge.
There is one last thing that I should mention about the JAMA paper. They are looking at non-standard biomarkers for cardiac health. That at least suggests that the metrics they analyzed are not necessarily even being used in the clinic and could potentially be the equivalent to "counter-culture" movements in medicine that are being critically analyzed.
This is an incredibly important distinction. Their importance states:
Importance Numerous cardiovascular biomarkers are proposed as potential predictors of cardiovascular risk.
further evidencing my last statement here. These are not mainstream clinical practice guides being analyzed.
So what the JAMA paper really says is: (in my words)
Non-standard biomarkers have been proposed as indicators of cardio risk. In determining their use in the clinic we considered 56 meta-analyses which investigated these markers and found 23% to be reasonably clear of bias, 52% had statistical significance above that which was predicted by our selected "gold standard", and another 23% suffered from hetergeneity (meta includes studies that are not necessarily/intuitively compatible, but not necessarily not either) and small study effects. We conclude that clinicians should exercise caution and sound clinical judgement when using the proposed non-standard biomarkers in the treatment of patients
Your article, on the other hand, states: (again, my own words, obviously)
JAMA ousts its own researchers for fraud! EBM can't be trusted! Go get accupuncture before it is too late!
Quite different..... Maybe I exaggerated a little in my interpretation of the latter, but hey.... turnabout is fair play
). The longest discussions occur when someone grabs a paper which was put out by a 3rd party or funded as such. Conflicts of interest are seen pretty readily and handled appropriately by the majority of clinicians.
