Last week I discussed a clinical trial comparing standardized acupuncture, individualized acupuncture, placebo-acupuncture, and usual care. In that discussion I emphasized the comparison between the three acupuncture groups, which did not show any difference in outcome. These results are consistent with the overall acupuncture literature, which shows in the better controlled trials that it does not matter where you stick the needles or even if you stick them through the skin. Therefore the scientific evidence fails to reject the null hypothesis (that acupuncture does not work). This did not stop the press from declaring, almost uniformly, that acupuncture works for back pain, contributing to the public misunderstanding of clinical science.
This week I am going to focus on the other aspect of the trial – the one the researchers and the press chose to focus on – the comparison of the two real and one placebo acupuncture arms to “usual care.” This too was misrepresented by the press, encouraged by the overinterpretation of the evidence by the researchers.
In the comments to Part I of this discussion David Gorski correctly pointed out that the study in fact did not even constitute a comparison of acupuncture to standard medical treatment. He is absolutely correct, and the many reasons for this are worth explaining in detail. Understanding the technology of clinical trials is central to science-based medicine, including all of their pitfalls and limitations. For practical and logistical reasons there is almost never a perfect clinical trial, but mischief only ensues when limitations are not understood, leading to a misinterpretation (and almost always an overinterpretation in the direction of the researcher’s bias) of the evidence.
A new study which randomized 638 adults to either standard acupuncture, individualized acupuncture, placebo acupuncture using tooth picks that did not penetrate the skin, and standard therapy found exactly what previous evidence has also suggested – it does not seem to matter where you stick the needles or even if you stick the needles through the skin. The only reasonable scientific conclusion to draw from this is that acupuncture does not work.
But let me back up a minute. Imagine if we were evaluating the efficacy of a new pain drug. This drug, when tested in open trials (no blinding or control) has an effect on reducing pain – it is superior to no treatment. When compared to a placebo, however, the drug is no more effective than the placebo, although both are more effective than no treatment.
Now imagine that the pharmaceutical company who manufactures this drug sends out a press release declaring that their drug is effective for pain, but that their research shows that a placebo of their drug is also effective (FDA applications are pending). Therefore more research is needed to determine how their drug works. Would you buy it?
That is the exact situation we are facing with acupuncture research.
In 1918 the Spanish Flu (named after the country of origin of the first reported case) swept the globe, killing 20-40 million people – more than the First World War (which killed 15 million) which was just ending. When an epidemic spreads to multiple regions, especially multiple countries or continents, it becomes a pandemic. Flu pandemics happen 2-3 times each century.
This is probably partly why there has been so much news attention, even some mild hysteria, surrounding recent outbreaks of swine flu, beginning in Mexico. It is hard to say how many cases and how many deaths there have been so far, because information from Mexico is spotty. Specifically it is difficult to say if people who have died with flu-like symptoms really had the swine flu or something else.
Estimates are that more than 800 people have contracted the swine flu in Mexico with 29 confirmed deaths, although none in the last week. The Centers for Disease Control (CDC) reports as of May 5th that there have been 403 confirmed cases in the United States, with the first death just reported. World-wide there have been 1,490 cases in 21 countries (according to the World Health Organization– WHO).
Some Background on Influenza
The influenza or flu virus is an RNA virus that comes in three genera – A, B, and C. Influenza A is the most common type. It can infect mammals and birds, with aquatic birds being its natural endemic host. Each year there is a seasonal epidemic of Influenza A, infecting millions of people world-wide and killing 100-200,000 – mostly the very old, the very young, and the sick.
It is unfortunately a common human reaction to respond to criticism by attacking those leveling the criticism, rather than addressing the points being made. This is especially true if the criticism is legitimate and one cannot reasonably counter it.
Substantive criticism is also a central part of the scientific endeavor, and so the culture of science has developed a tolerance for harsh criticism and a general understanding that the only proper response is with logic and evidence. Examples of exceptions are legion, human frailty being what it is, but you cannot live in the world of science for long without learning the rules of the game. Peers are expected to pull no punches when criticizing the errors or countering the arguments of their colleagues. Everyone is expected to be their own harshest critic (criticize your own data before someone has a chance to). And when criticized yourself, acknowledge what is legitimate and make appropriate corrections, but feel free to defend yourself against weak criticisms by pointing out additional data, interpretations, or errors in the arguments of your critics.
This meat grinder approach to scientific discourse works. Slowly, bad ideas and claims are beaten down, and only good ideas have the stamina to persist.
But here at science-based medicine we engage not only with the scientific community, but also with the public, and with those on the fringes of science. This means we often engage with those who do not play by the rules of science. A recent example is that of J.B. Handley from Age of Autism. David Gorski and I (and later Mark Crislip) wrote blog entries criticizing their 14 studies website with a detailed analysis. Handley responded with a full frontal personal assault sprinkled with irrelevant accusations. He ignored the vast majority of our actual criticisms, and those few he took on he completely botched.
As many have pointed out, we are in the midst of a transformation in the way news is created, distributed, and monetized – all brought on by the internet. Access to information has dramatically increased, while the traditional news outlets are fading away. The new internet-based outlets that are cropping up are often hybrids that do not fit into any existing definition. Science-based medicine itself is such an outlet – it’s primarily a group professional blog, but we have editors and take submissions. We also plan to expand the type of resources available on SBM. We’re experimenting.
Others, like Plos ONE, are experimenting with open-access peer-reviewed journals. And there are online newspapers that are part blog, part news feed, part something else.
While we are in this phase of experimentation it is important to monitor quality control, as the old institutions lose their grip on the flow of information. Health information in particular, now the most common type of information on the internet, suffers from poor quality control, leading the average consumer with too much information of too low quality.
Recently my co-blogger David Gorski wrote an excellent analysis of the latest propaganda effort from the anti-vaccine crowd – a website that attempts to deconstruct the fourteen studies most often cited to argue for a lack of association between vaccines and autism. As David pointed out, there are many more than 14 studies which demonstrate this, and no credible studies showing that there is any correlation. David covered some of the 14 discussed studies, and today I will discuss one more.
On that anti-vaccine propaganda site J.B. Handley begins his introduction with this logical fallacy:
Of all the remarkable frauds that will one day surround the autism epidemic, perhaps one of the most galling is the simple statement that the “science has spoken” and “vaccines don’t cause autism.” Anytime a public health official or other talking head states this, you can be assured that one of two things is true: they have never read the studies they are talking about, or they are lying through their teeth.
Of course this is a false dichotomy, or forced choice. I personally know of many people, including myself and David, who have both read all the studies and are telling the truth about our opinions that they do not support a link between autism and vaccines. It seems to be inconceivable to Mr. Handley that an informed professional could honestly disagree with his opinions – such is the nature of fanaticism.
Dr. Michael Dixon, the medical director of the Prince’s Foundation for Integrated Health, wrote an editorial for BBC news that is a densely packed rant of tiresome straw men often trotted out by the defenders of so-called “integrative” medicine. (The reason for the quotes in the headline, by the way, is because I stole that line from George Will who used in on This Week recently – it was too perfect not to co-opt.) Dixon was responding to an excellent commentary by Edzard Ernst, in which he characterized integrative medicine as a”shabby smokescreen for unproven treatments.”
Dixon was writing right out of the playbook of “integrative” propaganda, so it is worthwhile to expose his numerous logical fallacies and mischaracterizations of fact.
The Holism Gambit
Integrated health is not a new concept – the best doctors and their clinical colleagues have practised it for years.
It means treating patients as whole human beings – paying attention to body, mind and soul – instead of regarding them as nothing more than a set of symptoms to be got out the door as quickly as possible.
If Dixon wishes to be taken seriously by scientific practitioners he should make more of an effort to more fairly characterize mainstream medical practice. Of course, I must acknowledge up front, that there are mediocre and even bad doctors. There are also good doctors struggling within failing systems. And there are also many excellent doctors with effective practices. However, Dixon makes it seem as if the absolute worst of mainstream medicine is standard and typical. This is insulting, dismissive, and frankly ignorant of the facts on the ground. I find it interesting that defenders of integrative medicine are frequently whining about the dismissive attitudes of scientific practitioners of whom they are dismissive.
The LA Times recently published their analysis of data provided them by the state of California and found that there are pockets of high rates of exemption from vaccines among kindergarteners. In the US public schools require that all children receive the recommended vaccines. However, states can allow exemptions for the religious beliefs of the parents.
Over the years anti-vaccine activists have been successful in many states in expanding the rules for exemption. In California, for example, parents may seek excemption if they have “philosophical” objections to vaccines – which means there really isn’t any criteria beyond the parent’s wishes. The anti-vaccine movement has been active not only in pushing for the weakening of vaccine requirements but also in teaching parents how to use the laws to evade vaccination for their children.
The LA Times found that, while state wide the exemption rate was only 2%, exemptions were largely clustered in certain schools. They report:
In all, more than 10,000 kindergartners started school last fall with vaccine exemptions, up from about 8,300 the previous school year. In 1997, when enrollment was higher, the number of exempted kindergartners was 4,318.
At Ocean Charter School in Del Rey, near Marina del Rey, 40% of kindergartners entering school last fall and 58% entering the previous year were exempted from vaccines, the highest rates in the Los Angeles Unified School District.
When beginning a discussion of a controversial topic I like to establish the common ground upon which most or all people can agree. Everyone seems to agree that real conflicts of interest in medical research and practice is a bad thing and steps should be taken to minimize, eliminate, and illuminate any such conflicts. The controversy revolves around what constitutes a real conflict of interest.
There is broad agreement that researchers should not have a personal financial stake in the outcome of their own research – they should not make more money if their research is positive than if it’s negative. That creates a clear and powerful bias. There is also now broad agreement and adoption of standards that speakers, authors, and researchers should disclose any potential conflicts of interest – primarily the source of their funding. If someone is being paid by a drug company to say that their drug is effective for a particular disease, they should disclose that up front.
These same standard are now being applied to IRBs – institutional review boards, and that seems apprpriate. Every institution that does biomedical research must have an IRB, which is a committee of appropriate professionals (and there are rules as to the IRB’s constitution) that review all human research proposals to make sure they meet ethical guidelines and that subjects are adequately protected. This is a good system that generally works.
The primary goal of science-based medicine (SBM) is to connect the practice of medicine to the best currently available science. This is similar to evidence-based medicine (EBM), although we quibble about the relative roles of evidence vs prior plausibility. In a recent survey 86% of Americans said they thought that science education was “absolutely essential” or “very important” to the healthcare system. So there seems to be general agreement that science is a good way to determine which treatments are safe and work and which ones are not safe or don’t work.
The need for SBM also stems from an understanding of human frailty – there are a host of psychological effects and intellectual pitfalls that tend to lead us to wrong conclusions. Even the smartest and best-meaning among us can be lead astray by the failure to recognize a subtle error in logic or perception. In fact, coming to a reliable conclusion is hard work, and is always a work in progress.
There are also huge pressures at work that value things other than just the most effective healthcare. Industry, for example, is often motivated by profit. Institutions and health care providers may be motivated by the desire for prestige in addition to profits. Insurance companies are motivated by cost savings. Everyone is motivated by a desire to have the best health possible – we all want treatments that work safely, often more so than the desire to be logical or consistent. And often personal or institutional ideology comes into play – we want health care to validate our belief systems.