This is actually the second entry in this series;† the first was Part V of the Homeopathy and Evidence-Based Medicine series, which began the discussion of why Evidence-Based Medicine (EBM) is not up to the task of evaluating highly implausible claims. That discussion made the point that EBM favors equivocal clinical trial data over basic science, even if the latter is both firmly established and refutes the clinical claim. It suggested that this failure in calculus is not an indictment of EBM’s originators, but rather was an understandable lapse on their part: it never occurred to them, even as recently as 1990, that EBM would soon be asked to judge contests pitting low powered, bias-prone clinical investigations and reviews against facts of nature elucidated by voluminous and rigorous experimentation. Thus although EBM correctly recognizes that basic science is an insufficient basis for determining the safety and effectiveness of a new medical treatment, it overlooks its necessary place in that exercise.
This entry develops the argument in a more formal way. In so doing it advocates a solution to the problem that has been offered by several others, but so far without real success: the adoption of Bayesian inference for evaluating clinical trial data.
When I think back to my own ‘discovery’ of the skeptical movement, it grew out of my experience watching the James Randi Secrets of the Psychics NOVA special. After being enthralled with the special (and with several Randi books already in my library) I sought Mr. Randi out on the Internet. In chat rooms, blogs, forums and skeptical conferences such as TAM this is a tale I’ve heard repeated many times; folks heard about the JREF of CSICOP (now CSI) and then used the World Wide Web to learn more about these organizations.
Recently I began to wonder about my own personal pet peeve (unscientific medicine) and how it has benefited from the Web’s huge explosion and influence. Certainly there are plenty of great sites out there that help to show much of so-called Alternative Medicine for what it really is – blogs like this, Dr. Stephen Barrett’s Quackwatch.org site, the National Council Against Health Fraud, and many other important sites; still the number of sites extolling the virtues of science and critical thinking pale in comparison to those that forward notions embracing magical thinking and quack-related products and health claims. A quick examination of the web’s most popular search tool (Google) shows us the cold hard facts about who’s winning the war of medical woo:
The core principle of science-based medicine is that health care decisions should be based upon our best current scientific evidence and understanding. When applied to the regulation of health products this means that health claims should first be required to meet some reasonable threshold of scientific evidence before they are allowed. Admittedly this is not a purely scientific question but the application of scientific knowledge to an essentially political question – the balance of protection vs freedom.
Regardless of where one thinks this balance should be, I think most would agree that is it a problem if the public generally wants more protection than it is getting, or believes it is currently getting more protection than it is. A Harris poll from 2002 indicates that the majority of Americans believe that companies cannot make health claims about supplements unless they have been proven scientifically and approved by the FDA, when in fact this is not the case. The Dietary Supplement Health and Education Act of 1994 (DSHEA), largely through the efforts of Senator Orin Hatch from Utah, removed supplements from the control of the FDA and specifically allowed for so-called structure function claims to be made about products without any burden of proof. Most Americans are not aware of this fact.
Patients with heartburn are often diagnosed with GERD (gastroesophageal reflux disease) and treated with a drug called a proton pump inhibitor (PPI) to reduce stomach acid production. It is pretty effective, but it doesn’t always work. When it doesn’t, standard practice has been to double the dose of PPI. Doubling the dose only improves symptoms in 20-25%. Most patients who fail the single dose turn out to have normal esophageal acid exposure, or “functional” heartburn. In other words, the symptoms appear to be due to something other than excess acid – so it really may not make much sense to double the PPI dose. What else could doctors try?
How about acupuncture? A recent clinical trial compared acupuncture to doubling the proton pump inhibitor dose in refractory heartburn. (more…)
There it was on Friday greeting me on the ABC News website: “Study: Acupuncture May Boost Pregnancy” in bold blue letters, with the title of the webpage being “Needles Help You Become Pregnant.” The story began:
It sounds far-fetched sticking needles in women to help them become pregnant but a scientific review suggests that acupuncture might improve the odds of conceiving if done right before or after embryos are placed in the womb.
The surprising finding is far from proven, and there are only theories for how and why acupuncture might work. However, some fertility specialists say they are hopeful that this relatively inexpensive and simple treatment might ultimately prove to be a useful add-on to traditional methods.
By the end of the day, the story was all over the media, including radio, TV, news websites, the blogosphere, and various other outlets, all trumpeting the message that a scientific study says that acupuncture can help infertile couples conceive. Nary a skeptical word seemed to be found. Knowing very well just how far parents will go to conceive, I was curious: Did this study actually say what the media says it said? What was so new and radical about this study that it rated a press release and a lot of promotion? Do we here at SBM (particularly Steve) need to rethink our extreme skepticism about acupuncture, given the poor quality evidence and lack of even a glimmer of a convincing physiologic mechanism to explain its supposed activities?
News bulletin on BBC NEWS International version, 8 Feruary 2008:“Acupuncture ‘boosts IVF chances.’ Acupuncture may increase the success rates of fertility treatment, according to a study. “
(Manheimer E, Zhang G, Udoff L, Haramati A, Langenberg P, Berman BM, Bouter LM. Effects of acupuncture on rates of pregnancy and live birth among women undergoing in vitro fertilisation: systematic review and meta-analysis. BMJ. 2008 Feb 7)
First off, how plausible is the claim? The press release states that acupuncture had been used in China fior thousands of years for infertility. Has it? No medical historian writing I have seen made such an interpretation of ancient texts. Maybe I missed something…possible. But acupuncture was not used for specific disorders or purposes, but was used as a sort of panacea to cause balance of either the Yin and Yang or of the relationship of the individual with the 5 elements and the cosmos and the earth. There is nothing specific in claims of acupuncture in traditional Chinese Medicine history. Who gave the news people that misleading lead-in?
Second, what is the plausibility that acupuncture could possibly affect a laboratory procedure on tissue removed from the subject, regardless of timing? Negligible to none. There is no consistent and credible information that acupuncture is effective for anything, except as a conditiong agent for perception of symptoms.
So, does acupuncture increase the success of IVF?
Homeopathy and Science: Discussion, Summary and Conclusions
I was not surprised by a couple of the dissenting comments after Part IV of this blog. One writer worried that I had neglected, presumably for nefarious reasons, to cite replications of Benveniste’s results; another cited several examples of “positive” homeopathy studies that I had failed to mention. I answered some of those points here. I am fully aware of such “positive” reports, including those seeming to support Benveniste. I didn’t cite them, but not in some futile hope of concealing their existence from the watchful eyes of the readership. I also didn’t cite several “negative” reports, including an independent, disconfirming report of one of the claims of David Reilly, whose words began this series,* and the most recent of several reviews (referenced here) to conclude that “the clinical effects of homoeopathy are placebo effects.” I didn’t cite those reports for the same reasons that I didn’t cite the “positive” studies: they are mere footnotes to the overwhelming evidence against homeopathy.
To explain why, it will be necessary to discuss some of the strengths and weaknesses of the project known as “Evidence-Based Medicine.”
This is a story about a story and a story or two within that story. The first story is one of faulty epidemiology – data collection in a war zone. The first inside one is how medical news and journals affect not only national news, but are being used as political weaponry, to affect elections , and to change history.
Within that story is yet another – how editors contribute to fabrication, accepting or refusing to recognize fraud and misinformation. Yet another is that one cannot change some opinions, even after showing that the original information on which they were based was false. Sound familiar? We’ve been illustrating the point in classes for years.
The Iraq death studies. In 2004, weeks before the US presidential election, the journal, The Lancet published a study from a group at Johns Hopkins University, of Iraqi civilian deaths since the 2003 invasion (Lancet I). The results were unseemly high; a UN group estimated the deaths to be about one tenth of the Lancet’s report. The allied forces were still receiving approval for deposing Saddam Hussein, and the world press did not publicize them.
Then, 2-3 weeks before the 2006 US national congressional elections, with the Iraqi war wearing on and US and the world public tiring of stalemate and casualties, Lancet published a follow-up study (Lancet II) by the same group, concluding that in the years 2003-2006, Iraqi civilian war related deaths exceeded 600,000. It was shocking, made headline newspaper and television news. The study had such a significant impact partly because of where it appeared. The Lancet, despite its spotty record for off-beat articles, is revered by the public and the press. If the article’s publicity did not create a wave of political disapproval, it at least helped whip up the waves of discontent, washing in a major change in the Congress. Criticism of the study at the time seemed drowned out by its publicity. But a recent repeat study of civilian Iraqi deaths brings new light on the Lancet II study.
A new study by lead author Shelly Gray and published in the latest issue of the Journal for the American Geriatric Society, found no effect from taking Vitamin C or E, either alone or in combination, on the risk of dementia or Alzheimer’s disease after 5.5 years. Vitamins C and E were chosen because they both have significant antioxidant activity, and so this study was partly to test the hypothesis that oxidative stress causes or contributes to dementia.
The science behind the role of oxidative stress in aging and neurodegenerative disorders and the modulation of oxidative stress by nutritional antioxidants is complex and has not yielded many confident therapeutic recommendations. And yet, by contrast, antioxidants are sold to the public with dramatic health claims as if they were well established. It is common for marketing hype to out pace scientific reality, especially when the science is complex and preliminary so that there is as yet no firm scientific consensus.
For background, oxidative stress refers to the production in the cells of the body of certain oxygen-based compounds, collectively called reactive oxygen species (ROS), that are highly reactive. Some of these compounds are simply waste products of cell metabolism. Others serve a useful purpose, such as nitric oxide that is used in neurotransmission. What these ROS compounds have in common is that they react with proteins, DNA, and other cell components and cause damage.
There is an organization that calls itself The International Network of Cholesterol Skeptics (THINCS). Its members “thinc” they are smarter than the average doctor. They “thinc” that cholesterol has nothing to do with cardiovascular disease and that we have been deluded into waging a “cholesterol campaign” for which the scientific evidence is non-existent. They say, “What we all oppose is that animal fat and high cholesterol play a role.” I find even the wording of this statement problematical: one does not usually hear scientists “opposing” matters of fact or non-fact. They go on to say, “The aim with this website is to inform our colleagues and the public that this idea is not supported by scientific evidence; in fact, for many years a huge number of scientific studies have directly contradicted it.”
They tell us about those contradicting studies; but they don’t tell us about the flaws in those studies, they misrepresent some of the results, and they don’t tell us about the many good studies that support the cholesterol/heart link. The issue is a complex one, and it is easy to find studies to support any claim. Good science is about weighing all the evidence pro and con before reaching a conclusion. As far as I can see, these folks have cherry-picked the literature to support an agenda. They seem to have a vendetta against statin drugs in particular.
The website solicits complaints of adverse effects from statin drugs. It features a petition to the WHO that you can sign requesting an investigation of statin side effects. It alleges that lowering cholesterol endangers the elderly. It provides “what the medical journals and newspapers won’t let you hear” – letters and papers that have been rejected for publication. It lists books, published papers and talks by its members. It solicits financial contributions to the cause.
This movement seems to have started with Uffe Ravnskov’s book The Cholesterol Myths, published in Swedish in 1991 and in English in 2000. That book has been severely criticized, for instance in The Skeptic’s Dictionary , where Bob Carroll points out some of the distortions and deceptive techniques found in the cholesterol skeptics’ arguments. A typical claim: “Cholesterol is highly protective against cancer, infection and atherosclerosis” and “high TC [total cholesterol] and LDL levels are beneficial at all ages.” These statements are not only false, they are potentially dangerous to the health of those who believe them. (more…)