About a year ago the editors of Anesthesia & Analgesia solicited a written debate on whether or not acupuncture is effective or simply an elaborate placebo. Four experienced acupuncture researchers agreed to write the pro-acupuncture article, Wang, Harris, Lin and Gan. They asked David Colquhoun to write the con position, and David asked me to write it with him (which, of course, I enthusiastically agreed to do).
The article is fortunately published in open access, and so I can reprint it here (full article is below). What I think David and I convincingly demonstrated is that, according to the usual standards of medicine, acupuncture does not work.
Let me explain what I mean by that. Clinical research can never prove that an intervention has an effect size of zero. Rather, clinical research assumes the null hypothesis, that the treatment does not work, and the burden of proof lies with demonstrating adequate evidence to reject the null hypothesis. So, when being technical, researchers will conclude that a negative study “fails to reject the null hypothesis.”
We accept certain risks for the benefits of modern society. We pump explosive gas into homes, we run wires with potentially fatal electrical currents through our neighborhoods, and we ski at breakneck speeds down mountains for fun.
We also allow people to operate vehicles weighing thousands of pounds at speeds that are potentially deadly if a mishap occurs. In 2011 there were 32,367 motor vehicle deaths in the US (10.4 per 100,000 population). Interestingly, this is down quite a bit from previous years. As a percentage of population the highest motor vehicle death year was 1935, with 34,494 deaths, or 27.1 per 100,000. The highest absolute number of motor vehicle deaths was in 1970, at 52,627.
The number of deaths has been mostly trending down since 1996, which is interesting because over this same period of time cell phone use has risen tremendously. There are various reasons for the decreased in fatalities – helmet laws, seatbelt laws, cracking down on drunk driving, increased car safety, and intermediate drivers licenses for new drivers to name a few. These trends have probably obscured any increase in car accidents from using portable communication devices while driving.
It’s frustrating to read yet another story of the process of developing a potential new medical treatment derailed by the current infrastructure of quackery that we have in this and other countries. This is one of the unmeasurable harms that results when pseudoscience is given regulatory, academic, and professional legitimacy. The press then celebrates the nonsense that results.
The basic story is often the same, with a few variations. First, however, let me describe what should happen when someone comes up with an idea for a new medical treatment.
Background research – The first step, whether the innovator is within or without the medical community, is to familiarize oneself with existing research. Is the idea plausible, has it been investigated before, are there any similar treatments that can act as a guide to predicting how this new treatment will work?
“Patient-Centered” decision-making is a new buzz-word in medicine. It is a metaphor for a general approach to care that puts the patient’s experience and needs at the center, as opposed to the needs of the physician or the system.
While this is an effective marketing term, and a useful principle as far as it goes, as a guide to medical practice it is a bit simplistic. It needs to be viewed in the context of the overall medical infrastructure and the net effect specific practices have on the cost and effectiveness of medical care.
A 2012 NEJM editorial by Charles Bardes nicely summarizes the issues. He notes that patient-centered care represents the next step in a general trend (a good trend) in the medical profession over the last half-century:
The fifth edition of the Diagnostics and Statistical Manual (DSM-5) was recently released. This is the standard reference of mental disorders and psychiatric illnesses released by the American Psychiatric Association (APA).
As with previous editions there is a great deal of discussion and wringing of hands over the details – which disorders were created or eliminated. For example hoarding is now considered its own disorder, rather than part of obsessive compulsive disorder (it has its own reality TV show, why not its own DSM diagnosis?).
This time around, however, the debate over the DSM goes much deeper than the particulars of specific diagnoses. The real debate is about the very existence of the DSM – its validity and utility. While this discussion is nothing new, it has taken on an unprecedented dimension with the rejection of the DSM by the National Institutes of Mental Health (NIMH). Director Thomas Insel wrote:
The goal of this new manual, as with all previous editions, is to provide a common language for describing psychopathology. While DSM has been described as a “Bible” for the field, it is, at best, a dictionary, creating a set of labels and defining each. The strength of each of the editions of DSM has been “reliability” – each edition has ensured that clinicians use the same terms in the same ways. The weakness is its lack of validity. Unlike our definitions of ischemic heart disease, lymphoma, or AIDS, the DSM diagnoses are based on a consensus about clusters of clinical symptoms, not any objective laboratory measure. In the rest of medicine, this would be equivalent to creating diagnostic systems based on the nature of chest pain or the quality of fever. Indeed, symptom-based diagnosis, once common in other areas of medicine, has been largely replaced in the past half century as we have understood that symptoms alone rarely indicate the best choice of treatment.
The Star Trek universe is a fairly optimistic vision of the future. It’s what we would like it to be – an adventure fueled by advanced technology. In the world of Star Trek technology makes life better and causes few problems.
One of the most iconic examples of Star Trek technology is the medical tricorder. What doctor has not fantasized about walking up to a sick patient, waving a handheld device over them, and then having access to all the medical information you could possibly want. No needle sticks for blood tests, no invasive tests, scary MRI machines, and no wait. The information is available instantly.
It’s clear that we are heading in that direction as technology progresses, but how close are we?
The Smartphone in Medicine
Many people in developed nations today are walking around with supercomputers in their pocket – their smartphone. Technological advances are often strange – the ones we anticipate seem to never come, but then life-changing technology creeps up on us.
A recent article in the LA times tells of a husband’s quest to find a treatment for his wife’s Alzheimer’s disease. This is a narrative that journalists know and love—the brave patient or loved-one who won’t accept the nihilism of the medical establishment, who finds a maverick doctor willing to buck the system.
The article itself at least was not gushing, it tended toward a neutral tone, but such articles do tend to instill in the public a very counterproductive attitude toward science and medicine. I would have preferred an exposé of a dubious clinic exploiting desperate patients by peddling false hope. That is a narrative in which journalists rarely engage.
The story revolves around Dr. Edward Tobinick and his practice of perispinal etanercept (Enbrel) for a long and apparently growing list of conditions. Enbrel is an FDA-approved drug for the treatment of severe rheumatoid arthritis. It works by inhibiting tumor necrosis factor (TNF), which is a group of cytokines that are part of the immune system and cause cell death. Enbrel, therefore, can be a powerful anti-inflammatory drug. Tobinick is using Enbrel for many off-label indications, one of which is Alzheimer’s disease (the focus of the LA Times story).
A great deal of science is funded by the US government. The total research funding for 2009 was 54.8 billion dollars (much more if you include all R&D). A breakdown by agency of total R&D shows that the NIH (National Institutes for Health) funding is 28.5 billion while the NSF (National Science Foundation) is 4.1 billion.
There is general agreement that this expenditure is an investment on critical intellectual infrastructure for our nation and is vital to our competitiveness and standard of living. The government certainly has the right, and in fact the duty, to ensure that this money is well-invested. Government oversight is therefore understandable. Inevitably, however, politics is likely to intrude.
Representative Lamar Smith has been developing legislation that would in effect replace the peer-review process by which grants are currently given with a congressional process. Rather than having relevant scientists and experts decide on the merits of proposed research Smith would have politicians decide. It is difficult to imagine a more intrusive and disastrous process for public science funding.
Changing behavior is difficult. It is also an increasing priority for health care. We have entered a period of history when lifestyle choices have a dominant impact on health and longevity. People are no longer dying young of incurable infectious diseases in significant numbers. Rather they are surviving long enough to die from their bad habits.
Further, health behaviors are having a huge impact on the overall cost of health care. So the motivation is greater than ever to impact public health by influencing behavior. Yet, we are not very good at doing this.
It’s not that we’re not trying – it’s simply that having a large influence on people’s day-to-day behavior is remarkably difficult. There is ongoing research looking at how to effectively change behavior at the individual and public level, but it is complex, often conflicting, and new techniques at best yield only marginal gains.
Websites such as Luminosity.com make some bold promises about the effectiveness of computer-based brain-training programs. The site claims:
“Harness your brain’s neuroplasticity and train your way to a brighter life”
“Your brain’s abilities are unique. That’s why your Personalized Training Program adapts to fit your brain and your life goals.”
“Just 10 hours of Lumosity training can create drastic improvements. Track your own amazing progress with our sophisticated tools.”
Wow – in just 10 hours I can become smarter by playing fun video games personalized to my brain. I’m a huge fan of video games, and I would love to justify this hobby by saying that I’m training my brain while I play, but what does the scientific evidence have to say about such claims?
Not surprisingly, the published evidence is complex and mixed.