Articles

Posts Tagged Cochrane Collaboration

New evidence, same conclusion: Tamiflu only modestly useful for influenza

Tamiflu

Does Tamiflu have any meaningful effects on the prevention or treatment of influenza? Considering the drug’s been on the market for almost 15 years, and is widely used, you should expect this question has been answered after 15 flu seasons. Answering this question from a science-based perspective requires three steps: Consider prior probability, be systematic in the approach, and get all the data. It’s the third step that’s been (until now) impossible with Tamiflu: Some data was unpublished. In general, there’s good evidence to show that negative studies are less likely to be published than positive studies. Unless unpublished studies are included, systematic reviews are more likely to miss negative data, which means there’s the risk of bias in favor of an intervention.

The absence of a full data set on Tamiflu (oseltamivir) and the other neuraminidase inhibitor Relenza (zanamivir) became a rallying point for BMJ and the AllTrials campaign, which seeks to enhance the transparency and accessibility of clinical trials data by challenging trial investigators to make all evidence freely available. (Reforming and enhancing access to trial data was one of the most essential changes recommended by Ben Goldacre in his book, Bad Pharma.) In 2009, Tamiflu’s manufacturer, Hoffman-La Roche committed to making the Tamiflu data set available to investigators. Now after four years of back-and-forth between BMJ, investigators, and Roche, the full clinical trials data set has been made freely available. An updated systematic review was published today in BMJ (formerly The British Medical Journal), entitled “Oseltamivir for influenza in adults and children: systematic review of clinical study reports and summary of regulatory comments.” This will be a short post covering the highlights. As the entire study and accompanying data are freely available, I’ll await continued discussion in the comments. (more…)

Posted in: Clinical Trials, Pharmaceuticals, Public Health

Leave a Comment (30) →

Cochrane is Starting to ‘Get’ SBM!

This essay is the latest in the series indexed at the bottom.* It follows several (nos. 10-14) that responded to a critique by statistician Stephen Simon, who had taken issue with our asserting an important distinction between Science-Based Medicine (SBM) and Evidence-Based Medicine (EBM). (Dr. Gorski also posted a response to Dr. Simon’s critique). A quick-if-incomplete Review can be found here.

One of Dr. Simon’s points was this:

I am as harshly critical of the hierarchy of evidence as anyone. I see this as something that will self-correct over time, and I see people within EBM working both formally and informally to replace the rigid hierarchy with something that places each research study in context. I’m staying with EBM because I believe that people who practice EBM thoughtfully do consider mechanisms carefully. That includes the Cochrane Collaboration.

To which I responded:

We don’t see much evidence that people at the highest levels of EBM, eg, Sackett’s Center for EBM or Cochrane, are “working both formally and informally to replace the rigid hierarchy with something that places each research study in context.”

Hallafrickin’loo-ya

Well, perhaps I shouldn’t have been so quick to quip—or perhaps that was exactly what the doctor ordered, as will become clear—because on March 5th, nearly four months after writing those words, I received this email from Karianne Hammerstrøm, the Trials Search Coordinator and Managing Editor for The Campbell Collaboration, which lists Cochrane as one of its partners and which, together with the Norwegian Knowledge Centre for the Health Services, is a source of systematic reviews:

(more…)

Posted in: Clinical Trials, Energy Medicine, Medical Academia, Science and Medicine

Leave a Comment (20) →

Of SBM and EBM Redux. Part IV, Continued: More Cochrane and a little Bayes

OK, I admit that I pulled a fast one. I never finished the last post as promised, so here it is.

Cochrane Continued

In the last post I alluded to the 2006 Cochrane Laetrile review, the conclusion of which was:

This systematic review has clearly identified the need for randomised or controlled clinical trials assessing the effectiveness of Laetrile or amygdalin for cancer treatment.

I’d previously asserted that this conclusion “stand[s] the rationale for RCTs on its head,” because a rigorous, disconfirming case series had long ago put the matter to rest. Later I reported that Edzard Ernst, one of the Cochrane authors, had changed his mind, writing, “Would I argue for more Laetrile studies? NO.” That in itself is a reason for optimism, but Dr. Ernst is such an exception among “CAM” researchers that it almost seemed not to count.

Until recently, however, I’d only seen the abstract of the Cochrane Laetrile review. Now I’ve read the entire review, and there’s a very pleasant surprise in it (Professor Simon, take notice). In a section labeled “Feedback” is this letter from another Cochrane reviewer, which was apparently added in August of 2006, well before I voiced my own objections:

(more…)

Posted in: Clinical Trials, Homeopathy, Medical Academia, Science and Medicine

Leave a Comment (63) →

Of SBM and EBM Redux. Part IV: More Cochrane and a little Bayes

NB: This is a partial posting; I was up all night ‘on-call’ and too tired to continue. I’ll post the rest of the essay later…

Review

This is the fourth and final part of a series-within-a-series* inspired by statistician Steve Simon. Professor Simon had challenged the view, held by several bloggers here at SBM, that Evidence-Based Medicine (EBM) has been mostly inadequate to the task of reaching definitive conclusions about highly implausible medical claims. In Part I, I reiterated a fundamental problem with EBM, reflected in its Levels of Evidence scheme, that although it correctly recognizes basic science and other pre-clinical evidence as insufficient bases for introducing novel treatments into practice, it fails to acknowledge that they are necessary bases. I explained the difference between “plausibility” and “knowing the mechanism.”

I showed, with several examples, that in the EBM lexicon the word “evidence” refers almost exclusively to the results of clinical trials: thus, when faced with equivocal or no clinical trials of some highly implausible claim, EBM practitioners typically declare that there is “not enough evidence” to either accept or reject the claim, and call for more trials—although in many cases there is abundant evidence, other than clinical trials, that conclusively refutes the claim. I rejected Prof. Simon’s assertion that we at SBM want to “give (EBM) a new label,” making the point that we only want it to live up to its current label by considering all the evidence. I doubted Prof. Simon’s contention that “people within EBM (are) working both formally and informally to replace the rigid hierarchy with something that places each research study in context.”

In Part II I responded to the widely held assertion, also held by Prof. Simon, that there is “societal value in testing (highly implausible) therapies that are in wide use.” I made it clear that I don’t oppose simple tests of basic claims, such as the Emily Rosa experiment, but I noted that EBM reviewers, including those employed by the Cochrane Collaboration, typically ignore such tests. I wrote that I oppose large efficacy trials and public funding of such trials. I argued that the popularity gambit has resulted in human subjects being exposed to dangerous and unethical trials, and I quoted language from ethics treatises specifically contradicting the assertion that popularity justifies such trials. Finally, I showed that the alleged popularity of most “CAM” methods—as irrelevant as it may be to the question of human studies ethics—has been greatly exaggerated.

(more…)

Posted in: Clinical Trials, Energy Medicine, Faith Healing & Spirituality, Medical Academia, Medical Ethics, Science and Medicine

Leave a Comment (5) →

Of SBM and EBM Redux. Part III: Parapsychology is the Role Model for “CAM” Research

This is the third post in this series*; please see Part II for a review. Part II offered several arguments against the assertion that it is a good idea to perform efficacy trials of medical claims that have been refuted by basic science or by other, pre-trial evidence. This post will add to those arguments, continuing to identify the inadequacies of the tools of Evidence-Based Medicine (EBM) as applied to such claims.

Prof. Simon Replies

Prior to the posting of Part II, statistician Steve Simon, whose views had been the impetus for this series, posted another article on his blog, responding to Part I of this series. He agreed with some of what both Dr. Gorski and I had written:

The blog post by Dr. Atwood points out a critical distinction between “biologically implausible” and “no known mechanism of action” and I must concede this point. There are certain therapies in CAM that take the claim of biological plausibility to an extreme. It’s not as if those therapies are just implausible. It is that those therapies must posit a mechanism that “would necessarily violate scientific principles that rest on far more solid ground than any number of equivocal, bias-and-error-prone clinical trials could hope to overturn.” Examples of such therapies are homeopathy, energy medicine, chiropractic subluxations, craniosacral rhythms, and coffee enemas.

The Science Based Medicine site would argue that randomized trials for these therapies are never justified. And it bothers Dr. Atwood when a systematic review from the Cochrane Collaboration states that no conclusions can be drawn about homeopathy as a treatment for asthma because of a lack of evidence from well conducted clinical trials. There’s plenty of evidence from basic physics and chemistry that can allow you to draw strong conclusions about whether homeopathy is an effective treatment for asthma. So the Cochrane Collaboration is ignoring this evidence, and worse still, is implicitly (and sometimes explicitly) calling for more research in this area.

On the other hand:

There are a host of issues worth discussing here, but let me limit myself for now to one very basic issue. Is any research justified for a therapy like homeopathy when basic physics and chemistry will provide more than enough evidence by itself to suggest that such research is futile(?) Worse still, the randomized trial is subject to numerous biases that can lead to erroneous conclusions.

I disagree for a variety of reasons.

(more…)

Posted in: Acupuncture, Clinical Trials, Energy Medicine, Faith Healing & Spirituality, Herbs & Supplements, Homeopathy, Medical Academia, Medical Ethics, Science and Medicine

Leave a Comment (30) →

Of SBM and EBM Redux. Part II: Is it a Good Idea to test Highly Implausible Health Claims?

Review

This is the second post in a series* prompted by an essay by statistician Stephen Simon, who argued that Evidence-Based Medicine (EBM) is not lacking in the ways that we at Science-Based Medicine have argued. David Gorski responded here, and Prof. Simon responded to Dr. Gorski here. Between that response and the comments following Dr. Gorski’s post it became clear to me that a new round of discussion would be worth the effort.

Part I of this series provided ample evidence for EBM’s “scientific blind spot”: the EBM Levels of Evidence scheme and EBM’s most conspicuous exponents consistently fail to consider all of the evidence relevant to efficacy claims, choosing instead to rely almost exclusively on randomized, controlled trials (RCTs). The several quoted Cochrane abstracts, regarding homeopathy and Laetrile, suggest that in the EBM lexicon, “evidence” and “RCTs” are almost synonymous. Yet basic science or preliminary clinical studies provide evidence sufficient to refute some health claims (e.g., homeopathy and Laetrile), particularly those emanating from the social movement known by the euphemism “CAM.”

It’s remarkable to consider just how unremarkable that last sentence ought to be. EBM’s founders understood the proper role of the rigorous clinical trial: to be the final arbiter of any claim that had already demonstrated promise by all other criteria—basic science, animal studies, legitimate case series, small controlled trials, “expert opinion,” whatever (but not inexpert opinion). EBM’s founders knew that such pieces of evidence, promising though they may be, are insufficient because they “routinely lead to false positive conclusions about efficacy.” They must have assumed, even if they felt no need to articulate it, that claims lacking such promise were not part of the discussion. Nevertheless, the obvious point was somehow lost in the subsequent formalization of EBM methods, and seems to have been entirely forgotten just when it ought to have resurfaced: during the conception of the Center for Evidence-Based Medicine’s Introduction to Evidence-Based Complementary Medicine.

Thus, in 2000, the American Heart Journal (AHJ) could publish an unchallenged editorial arguing that Na2EDTA chelation “therapy” could not be ruled out as efficacious for atherosclerotic cardiovascular disease because it hadn’t yet been subjected to any large RCTs—never mind that there had been several small ones, and abundant additional evidence from basic science, case studies, and legal documents, all demonstrating that the treatment is both useless and dangerous. The well-powered RCT had somehow been transformed, for practical purposes, from the final arbiter of efficacy to the only arbiter. If preliminary evidence was no longer to have practical consequences, why bother with it at all? This was surely an example of what Prof. Simon calls “Poorly Implemented Evidence Based Medicine,” but one that was also implemented by the very EBM experts who ought to have recognized the fallacy.

There will be more evidence for these assertions as we proceed, but the main thrust of Part II is to begin to respond to this statement from Prof. Simon: “There is some societal value in testing therapies that are in wide use, even though there is no scientifically valid reason to believe that those therapies work.”

(more…)

Posted in: Chiropractic, Clinical Trials, Energy Medicine, Health Fraud, History, Homeopathy, Medical Academia, Medical Ethics, Naturopathy, Politics and Regulation, Science and Medicine

Leave a Comment (49) →