Posts Tagged evidence-based medicine

Of SBM and EBM Redux. Part III: Parapsychology is the Role Model for “CAM” Research

This is the third post in this series*; please see Part II for a review. Part II offered several arguments against the assertion that it is a good idea to perform efficacy trials of medical claims that have been refuted by basic science or by other, pre-trial evidence. This post will add to those arguments, continuing to identify the inadequacies of the tools of Evidence-Based Medicine (EBM) as applied to such claims.

Prof. Simon Replies

Prior to the posting of Part II, statistician Steve Simon, whose views had been the impetus for this series, posted another article on his blog, responding to Part I of this series. He agreed with some of what both Dr. Gorski and I had written:

The blog post by Dr. Atwood points out a critical distinction between “biologically implausible” and “no known mechanism of action” and I must concede this point. There are certain therapies in CAM that take the claim of biological plausibility to an extreme. It’s not as if those therapies are just implausible. It is that those therapies must posit a mechanism that “would necessarily violate scientific principles that rest on far more solid ground than any number of equivocal, bias-and-error-prone clinical trials could hope to overturn.” Examples of such therapies are homeopathy, energy medicine, chiropractic subluxations, craniosacral rhythms, and coffee enemas.

The Science Based Medicine site would argue that randomized trials for these therapies are never justified. And it bothers Dr. Atwood when a systematic review from the Cochrane Collaboration states that no conclusions can be drawn about homeopathy as a treatment for asthma because of a lack of evidence from well conducted clinical trials. There’s plenty of evidence from basic physics and chemistry that can allow you to draw strong conclusions about whether homeopathy is an effective treatment for asthma. So the Cochrane Collaboration is ignoring this evidence, and worse still, is implicitly (and sometimes explicitly) calling for more research in this area.

On the other hand:

There are a host of issues worth discussing here, but let me limit myself for now to one very basic issue. Is any research justified for a therapy like homeopathy when basic physics and chemistry will provide more than enough evidence by itself to suggest that such research is futile(?) Worse still, the randomized trial is subject to numerous biases that can lead to erroneous conclusions.

I disagree for a variety of reasons.


Posted in: Acupuncture, Clinical Trials, Energy Medicine, Faith Healing & Spirituality, Herbs & Supplements, Homeopathy, Medical Academia, Medical Ethics, Science and Medicine

Leave a Comment (30) →

Of SBM and EBM Redux. Part II: Is it a Good Idea to test Highly Implausible Health Claims?


This is the second post in a series* prompted by an essay by statistician Stephen Simon, who argued that Evidence-Based Medicine (EBM) is not lacking in the ways that we at Science-Based Medicine have argued. David Gorski responded here, and Prof. Simon responded to Dr. Gorski here. Between that response and the comments following Dr. Gorski’s post it became clear to me that a new round of discussion would be worth the effort.

Part I of this series provided ample evidence for EBM’s “scientific blind spot”: the EBM Levels of Evidence scheme and EBM’s most conspicuous exponents consistently fail to consider all of the evidence relevant to efficacy claims, choosing instead to rely almost exclusively on randomized, controlled trials (RCTs). The several quoted Cochrane abstracts, regarding homeopathy and Laetrile, suggest that in the EBM lexicon, “evidence” and “RCTs” are almost synonymous. Yet basic science or preliminary clinical studies provide evidence sufficient to refute some health claims (e.g., homeopathy and Laetrile), particularly those emanating from the social movement known by the euphemism “CAM.”

It’s remarkable to consider just how unremarkable that last sentence ought to be. EBM’s founders understood the proper role of the rigorous clinical trial: to be the final arbiter of any claim that had already demonstrated promise by all other criteria—basic science, animal studies, legitimate case series, small controlled trials, “expert opinion,” whatever (but not inexpert opinion). EBM’s founders knew that such pieces of evidence, promising though they may be, are insufficient because they “routinely lead to false positive conclusions about efficacy.” They must have assumed, even if they felt no need to articulate it, that claims lacking such promise were not part of the discussion. Nevertheless, the obvious point was somehow lost in the subsequent formalization of EBM methods, and seems to have been entirely forgotten just when it ought to have resurfaced: during the conception of the Center for Evidence-Based Medicine’s Introduction to Evidence-Based Complementary Medicine.

Thus, in 2000, the American Heart Journal (AHJ) could publish an unchallenged editorial arguing that Na2EDTA chelation “therapy” could not be ruled out as efficacious for atherosclerotic cardiovascular disease because it hadn’t yet been subjected to any large RCTs—never mind that there had been several small ones, and abundant additional evidence from basic science, case studies, and legal documents, all demonstrating that the treatment is both useless and dangerous. The well-powered RCT had somehow been transformed, for practical purposes, from the final arbiter of efficacy to the only arbiter. If preliminary evidence was no longer to have practical consequences, why bother with it at all? This was surely an example of what Prof. Simon calls “Poorly Implemented Evidence Based Medicine,” but one that was also implemented by the very EBM experts who ought to have recognized the fallacy.

There will be more evidence for these assertions as we proceed, but the main thrust of Part II is to begin to respond to this statement from Prof. Simon: “There is some societal value in testing therapies that are in wide use, even though there is no scientifically valid reason to believe that those therapies work.”


Posted in: Chiropractic, Clinical Trials, Energy Medicine, Health Fraud, History, Homeopathy, Medical Academia, Medical Ethics, Naturopathy, Politics and Regulation, Science and Medicine

Leave a Comment (49) →

Evidence-Based Medicine, Human Studies Ethics, and the ‘Gonzalez Regimen’: a Disappointing Editorial in the Journal of Clinical Oncology Part 1

Background: the distinction between EBM and SBM

An important theme on the Science-Based Medicine blog, and the very reason for its name, has been its emphasis on examining all the evidence—not merely the results of clinical trials—for various claims, particularly for those that are implausible. We’ve discussed the distinction between Science-Based Medicine (SBM) and the more limited Evidence-Based Medicine (EBM) several times, for example here (I began my own discussion here and added a bit of formality here, here, and here). Let me summarize by quoting John Ioannidis:

…the probability that a research finding is indeed true depends on the prior probability of it being true (before doing the study), the statistical power of the study, and the level of statistical significance.

EBM, in a nutshell, ignores prior probability† (unless there is no other available evidence) and falls for the “p-value fallacy”; SBM does not. Please don’t bicker about this if you haven’t read the links above and some of their own references, particularly the EBM Levels of Evidence scheme and two articles by Steven Goodman (here and here). Also, note that it is not necessary to agree with Ioannidis that “most published research findings are false” to agree with his assertion, quoted above, about what determines the probability that a research finding is true.

The distinction between SBM and EBM has important implications for medical practice ethics, research ethics, human subject protections, allocation of scarce resources, epistemology in health care, public perceptions of medical knowledge and of the health professions, and more. EBM, as practiced in the 20 years of its formal existence, is poorly equipped to evaluate implausible claims because it fails to acknowledge that even if scientific plausibility is not sufficient to establish the validity of a new treatment, it is necessary for doing so.

Thus, in their recent foray into applying the tools of EBM to implausible health claims, government and academic investigators have made at least two, serious mistakes: first, they have subjected unwary subjects to dangerous but unnecessary trials in a quest for “evidence,” failing to realize that definitive evidence already exists; second, they have been largely incapable of pronouncing ineffective methods ineffective. At best, even after conducting predictably disconfirming trials of vanishingly unlikely claims, they have declared such methods merely “unproven,” almost always urging “further research.” That may be the proper EBM response, but it is a far cry from the reality. As I opined a couple of years ago, the founders of the EBM movement apparently “never saw ‘CAM’ coming.”


Posted in: Cancer, Clinical Trials, Medical Academia, Medical Ethics, Politics and Regulation, Science and Medicine

Leave a Comment (59) →

New CMS Chief Donald Berwick: a Trojan Horse for Quackery?

NB: I posted this on Health Care Renewal a couple of days ago, figuring that Dr. Gorski’s post would suffice for the SBM readership (he and I had discussed the topic while at TAM8 last week). But Managing Editor Gorski has asked me to repost it here, which I’m happy to do. I am especially pleased to demonstrate that I am capable of writing a shorter post than is Dr. Gorski. 😉

On July 7, President Obama appointed Dr. Donald Berwick as Administrator of the Centers for Medicare and Medicaid Services (CMS). Dr. Berwick, a pediatrician, is well known as the CEO of the non-profit Institute for Healthcare Improvement (IHI), which “exists to close the enormous gap between the health care we have and the health care we should have — a gap so large in the US that the Institute of Medicine (IOM) in 2001 called it a ‘quality chasm’.” Dr. Berwick was one of the authors of that IOM report. His IHI has been a major player in the patient safety movement, most notably with its “100,000 Lives Campaign” and, more recently, its “5 Million Lives Campaign.”

Berwick’s CMS gig is a “recess appointment”: it was made during the Senate’s July 4th recess period, without a formal confirmation hearing—although such a hearing must take place before the end of this Senate term, if he is to remain in the position. A recent story suggested that Obama had made the recess appointment in order to avoid a reprise of “last year’s divisive health care debate.” The president had originally nominated Berwick for the position in April, and Republicans have opposed “Berwick’s views on rationing of care,” claiming that he “would deny needed care based on cost.”

A “Patient-Centered Extremist”

If there is a problem with the appointment, it is likely to be roughly the opposite of what Republicans might suppose: Dr. Berwick is a self-described “Patient-Centered Extremist.” He favors letting patients have the last word in decisions about their care even if that means, for example, choosing to have unnecessary and expensive hi-tech studies. In an article for Health Affairs published about a year ago, he explicitly argued against the “professionally dominant view of quality of health care”:


Posted in: Health Fraud, Homeopathy, Medical Ethics, Politics and Regulation, Science and Medicine, Science and the Media

Leave a Comment (19) →

Brief Note: The Chiropractic Subluxation is Dead

The General Chiropractic Council, a UK-wide statutory body with regulatory powers, has just published a new position statement on the chiropractic subluxation complex:

The chiropractic vertebral subluxation complex is an historical concept but it remains a theoretical model. It is not supported by any clinical research evidence that would allow claims to be made that it is the cause of disease or health concerns.

They remind chiropractors that they must make sure their own beliefs and values do not prejudice the patient’s care, and that they must provide evidence-based care. Unfortunately, they define evidence-based care as

clinical practice that incorporates the best available evidence from research, the preferences of the patient and the expertise of practitioners, including the individual chiropractor her/himself. [emphasis added]

This effectively allows “in my experience” and “the patient likes it” to be considered along with evidence, effectively negating the whole point of evidence-based medicine.

Posted in: Chiropractic

Leave a Comment (40) →

Placebo Therapies: Are They Ethical?

Is it ethical to overstate the efficacy of a treatment option, if it might lead to a patient’s enhanced experience of that treatment? Your response to this question may reveal the degree to which you favor Complementary and Alternative Medicine (CAM). Let me explain.

As far as I can tell, no CAM treatment has been proven effective beyond placebo. (If you’re not convinced of this, I suggest you take a look at Barker Bausell’s book on the subject.) That means that treatments like acupuncture, homeopathy, Reiki, energy healing, Traditional Chinese Medicine (such as cupping), and others (like “liver flushes”) perform about as well as placebos (inert alternatives) in head-to-head studies. Therefore, the effects of these treatments cannot be explained by inherent mechanisms of action, but rather the mind’s perception of their value. In essence, the majority of CAM treatments are likely to be placebo therapies, with different levels of associated ritual.

For the sake of argument, let’s assume that CAM therapies are in fact placebos – the question then becomes, is it ethical to prescribe placebos to patients?  It seems that many U.S. physicians believe that it is not appropriate to overstate potential therapeutic benefits to patients. In fact, the AMA strictly prohibits such a practice:

“Physicians may use [a] placebo for diagnosis or treatment only if the patient is informed of and agrees to its use.”

Moreover, a recent article in the New England Journal of Medicine concludes:

“Outside the setting of clinical trials, there is no justification for the use of placebos.”

However, there is some wavering on the absolute contraindication of placebos. A recent survey conducted by researchers at the Mayo Clinic asked physicians if it was permissible to give a dextrose tablet to a non-diabetic patient with fibromyalgia if that tablet was shown to be superior to no treatment in a clinical trial. In this case 62% of respondents said that it would be acceptable to give the pill.

The authors note:

“Before 1960, administration of inert substances to promote placebo effects or to satisfy patients’ expectations of receiving a prescribed treatment was commonplace in medical practice. With the development of effective pharmaceutical interventions and the increased emphasis on informed consent, the use of placebo treatments in clinical care has been widely criticized. Prescribing a placebo, it is claimed, involves deception and therefore violates patients’ autonomy and informed consent. Advocates of placebo treatments argue that promoting the placebo effect might be one of the most effective treatments available for many chronic conditions and can be accomplished without deception.”

How do you feel about placebos? Are they a legitimate option in some cases, or a violation of patient autonomy and informed consent?

Posted in: Medical Ethics, Science and Medicine

Leave a Comment (27) →

How To Get Physicians To Use The Same Science-Based Playbook

Pretty much everyone agrees that we need to improve the quality of healthcare delivered to patients in the US. We’ve all heard the frightening statistics from the Institute of Medicine about medical error rates – that as many as 98,000 patients die each year as a result of them – and we also know that the US spends about 33% more than most industrialized country on healthcare, without substantial improvements in outcomes.

However, a large number of quality improvement initiatives rely on additional rules, regulations, and penalties to inspire change (for example, decreasing Medicare payments to hospitals with higher readmission rates, and decreasing provider compensation based on quality indicators). Not only am I skeptical about this stick vs. carrot strategy, but I think it will further demoralize providers, pit key stakeholders against one another, and cause people to spend their energy figuring out how to game the system than do the right thing for patients.

There is a carrot approach that could theoretically result in a $757 billion savings/year that has not been fully explored – and I suggest that we take a look at it before we “release the hounds” on hospitals and providers in an attempt to improve healthcare quality.

I attended the Senate Finance Committee’s hearing on budget options for health care reform on February 25th. One of the potential areas of substantial cost savings identified by the Congressional Budget Office (CBO) is non evidence-based variations in practice patterns. In fact, at the recent Medicare Policy Summit, CBO staff identified this problem as one of the top three causes of rising healthcare costs. Just take a look at this map of variations of healthcare spending to get a feel for the local practice cultures that influence treatment choices and prices for those treatments. There seems to be no organizing principle at all.

Senator Baucus (Chairman of the Senate Finance Committee) appeared genuinely distressed about this situation and was unclear about the best way to incentivize (or penalize) doctors to make their care decisions more uniformly evidence-based. In my opinion, a “top down” approach will likely be received with mistrust and disgruntlement on the part of physicians. What the Senator needs to know is that there is a bottom up approach already in place that could provide a real win-win here.

Posted in: Politics and Regulation, Public Health, Science and Medicine

Leave a Comment (21) →

Knowledge Versus Expertise: The View From Consumer Land

“The internet, in democratizing knowledge, has led a lot of people to believe that it is also possible to democratize expertise.”

– SBM Commenter, yeahsurewhatever

I’ve spent the last few years of my life in Internet “Consumer Land,” doing what I can to bring accurate health information directly to patients. Of course, I have been surprised by the push-back, and the demand for misinformation. When I first left full time clinical work, it never occurred to me that people would prefer to read falsehoods when provided a clear choice between truth and error. I guess I was pretty naïve.

Journalist Lesley Stahl provided me with some helpful insights during a recent conference. She explained that the Internet has catalyzed a new method of information transfer – speed trumps accuracy, the line between pundits and journalists is blurred, and anyone who can get to a microphone can become an “expert.” Gone are the days of careful sourcing and fact-checking. And gone is the public trust in “mainstream media.”


Posted in: Health Fraud, Humor, Public Health, Science and Medicine, Science and the Media

Leave a Comment (17) →

Polypharmacy – Is It Evidence-Based?

Polypharmacy essentially means taking too many pills. It’s a real problem, especially in the elderly.

A family doctor gives an elderly patient one pill for diabetes, another for high blood pressure, and another to lower cholesterol. The patient sees a rheumatologist for his arthritis and gets arthritis pills. Then he sees a psychiatrist for depression and gets an antidepressant. He takes a sleeping pill. He takes a laxative. He buys some over-the-counter cold medicine and Tylenol. Then he goes to his local GNC store and buys a smorgasbord of vitamins, minerals, supplements and herbal products. It would be surprising if some of these didn’t interact with each other to cause some problems.

One doctor may not know what the other doctors have prescribed. The patient may not think to tell his doctors about the non-prescription products he’s taking. Or he may not want to admit it for fear the doctors will disapprove. (more…)

Posted in: Herbs & Supplements, Pharmaceuticals, Science and Medicine

Leave a Comment (12) →
Page 3 of 3 123