While we frequently on SBM target the worst abuses of science in medicine, it’s important to recognize that doing rigorous science is complex and mainstream scientists often fall short of the ideal. In fact, one of the advantages of exploring pseudoscience in medicine is developing a sensitive detector for errors in logic, method, and analysis. Many of the errors we point out in so-called “alternative” medicine also crop up elsewhere in medicine – although usually to a much less degree.
It is not uncommon, for example, for a paper to fail to adjust for multiple analysis – if you compare many variables you have to take that into consideration when doing the statistical analysis otherwise the probability of a chance correlation will be increased.
I discussed just yesterday on NeuroLogica the misapplication of meta-analysis – in this case to the question of whether or not CCSVI correlates with multiple sclerosis. I find this very common in the literature, essentially a failure to appreciate the limits of this particular analysis tool.
Another example comes recently from the journal Nature Neuroscience (an article I learned about from Ben Goldacre over at the Bad Science blog). Erroneous analyses of interactions in neuroscience: a problem of significance investigates the frequency of a subtle but important statistical error in high profile neuroscience journals.
Determining the net health effects of independent factors can be tricky, especially when those factors cannot be controlled for in experimental studies. For things like body mass index (BMI) we must rely on observational data and triangulate with multiple studies to isolate the contributions from BMI. But it can be done.
The data, however, are likely to be complex and noisy, and therefore there is plenty of opportunity for ideology to trump objectivity in interpreting the data. There are those who, for whatever reason, deny that we are having an obesity epidemic in the West, and those who deny the health implications of being overweight as an independent factor.
The terms overweight and obesity have had various definitions in the past, but in recent years the various health organizations have settled on consensus operational definitions (for obvious practical reasons). Their definition relates to body mass index, which is a person’s weight in kilograms (kg) divided by their height in meters (m) squared.
One of the themes of SBM is that modern health care should be based upon solid scientific ground. Interventions should be based on a risk vs benefit analysis using the best available scientific evidence (clinical and basic science).
As an extension of this, the standard of care needs to be a science-based standard. Science is (or at least should be) objective and transparent, and without such standards there is no way to have meaningful quality control. Without the filter of science there is no limit to the nonsense and magical thinking that can flow into the health care system. Increasingly we cannot afford the waste of fanciful and ineffective interventions, and even if limited resources were not an issue – individual patients deserve better.
It is for these reasons that we oppose the attempts by proponents of so-called complementary and alternative medicine (CAM) to erode or eliminate the science-based standard of care in medicine. Proponents differ mostly on how open they are about this goal, but there is no escaping the reality that at the heart of the very concept of CAM is at least a double standard – one in which the science-based bar for inclusion is lowered for some favored modalities.
It is an unfortunate truth that there is money in pseudoscience, particularly medical pseudoscience. Money both attracts charlatans and also funds their activities, which includes marketing pseudoscience and defending their claims from scientific scrutiny. In this way the game is rigged in favor of pseudoscience.
With0ut effective regulation, sites like ours are forced to play whack-a-mole with the medical pseudoscience du jour. The latest case in point is Titanium Ion Bands – which are just another version of the Power Balance bands that have been previously exposed as nonsense. The idea is that by wearing a small bracelet on one wrist you will experience improved athletic performance. This sounds impossible – because it is. But companies have successfully bamboozled enough of the public to rake in millions.
The marketing strategy is three-fold. First, get naive professional athletes to endorse the product. Second, give live demonstrations (deceptive parlor tricks) that convince the unsuspecting that something must be going on. And third, wow the scientifically illiterate with a confusing barrage of medical techno-babble. The combination is sadly effective.
Power Balance, for example, makes vague references to frequencies and energy as the explanation for how a little piece of rubber (with embedded holograms) can have any effect on human physiology. The company was eventually legally forced to admit: “”We admit that there is no credible scientific evidence that supports our claims.” The admission has not ended their sale, however.
The latest issue of the BMJ contains an editorial recommending that regulators (this is in the UK, but the argument applies in the US and elsewhere) should require pharmaceutical companies to provide research on direct comparison to existing therapies as part of the approval process. The authors, Sorenson, Naci, Cylus, and Mossialos, write:
When a drug comes to market, evidence on the comparative risks and benefits is needed to help regulatory authorities to safeguard public health from inferior and unsafe treatments, to ensure that health technology assessment agencies and payers make funding decisions based on the best available evidence of different treatments, and to aid clinicians’ and patients’ understanding of what therapies work best and their appropriate position in the treatment pathway.
They make a persuasive argument, but there are some interesting angles to this topic.
Here is yet another study claiming to show “how acupuncture works” when in fact it does nothing of the kind. It does, however, reveal the bias of the researchers – it is, in fact, surprising that it was published in a peer-reviewed journal. Unfortunately, the mainstream media is dutifully reporting the biased claims of the researchers without any independent verification or analysis.
There are numerous fatal problems with this study. The first, like in many physiological studies that purport to be about acupuncture, is that the connection to acupuncture is tenuous. The researchers claim that they are testing the effects of an acupuncture needle – but what makes a needle an acupuncture needle? Other such studies were ultimately just seeing the effects of local tissue trauma. The fact that this trauma was induced by an “acupuncture needle” is not necessarily relevant.
This study is far worse, because it is simply using the acupuncture needle as a mechanism for inducing an unrelated physiological stimulus. This is similar to “electroacupuncture” where electrical current is applied through an acupuncture needle – what you are actually studying is the effects of electricity, not “acupuncture.” Applying electrical stimulation, or some other physiological stimulus, is the equivalent of injecting morphine through a thin needle and then claiming this demonstrates how “acupuncture works.”
The fundamental concept of science-based medicine (SBM) is that medical practice should be based upon the best available science. This may seem obvious, but there are many important details to its application, such as the relationship between clinical and basic science. Clinical claims require clinical evidence, but clinical evidence can be tricky and is often preliminary. It is therefore helpful (I would say essential) to view the clinical evidence in light of all of the rest of science.
A thorough basic and clinical science analysis of a medical claim can be summarized by the term “plausibility,” or “prior probability” if you want to put it into statistical terms. When we say a certain belief is plausible we mean it is consistent with what we know from the rest of science. In other words, because of the many weaknesses of clinical evidence, in order for a therapy to be generally accepted as part of SBM it should have a certain minimal supporting clinical evidence and overall scientific plausibility.
These can exist in different proportions – for example one therapy may be highly plausible (it would be shocking if it were not true) and have modest supporting clinical evidence, while another may have unknown plausibility but with solid clinical evidence of efficacy. But no therapy should have clinical evidence that suggests lack of efficacy, nor extreme implausibility (not just an unknown mechanism, but no possible mechanism).
There have been many cases now of big companies or organizations, or wealthy individuals, threatening to sue or actually suing a blogger for libel. The most famous case is that of Simon Singh who was sued by the British Chiropractic Association over comments he made in an article. Simon braved through the expensive and exhaustive legal process (which is especially onerous in England), but he is not just a lone blogger. He is a successful author and was writing for the Guardian. Eventually the BCA was forced to drop the case – but only after the blogging community rallied behind Simon, magnifying his criticisms of the BCA by orders of magnitude. By all accounts it was a PR disaster.
The blogging community as a whole is rather passionate about this issue. We exist on the premise of free and open public discourse about important issues. At SBM we take on many controversial issues and we don’t pull our punches when criticizing what we see as pseudoscience in medicine. So of course we take notice when a large company tries to bully a blogger to silence their legitimate criticism.
According to the BMJ this has happened yet again – this time the international homeopathy producer, Boiron, is threatening a lone Italian blogger because he dared to criticize their product, Oscillococcinum. The blogger, Samuele Riva, wrote two articles on his blog, blogzero.it, criticizing what our own Mark Crislip has called “oh-so-silly-coccinum.” The blog is entirely in Italian, but he is maintaining a page in English with updates on the Boiron vs Blogzero affair.
In 1994 Congress (pushed by Senators Harkin and Hatch) passed DSHEA (the Dietary Supplement Health and Education Act). As regular readers of SBM know, we are not generally happy about this law, which essentially deregulated the supplement industry. Under DSHEA supplements, a category which specifically was defined to include herbals, are regulated more like food than like medicinals.
Since then the flood-gates opened, and there has been open competition in the marketplace for supplement products. This has not resulted, I would argue, in better products – only in slicker and more deceptive claims. What research we have into popular herbals and supplements shows that they are generally worthless (except for targeted vitamin supplementation, which was already part of science-based medicine, and remains so).
A company can essentially put a random combination of plants and vitamins into a pill or liquid and then make whatever health claims they wish for their product, as long as they stay within the “structure-function” guidelines. This means they cannot claim to cure or treat a specific disease, but this has proven to be an insignificant limitation on marketing supplements.
Part of the Complementary and Alternative Medicine (CAM) movement is an attempt to insert spirituality into the philosophy and practice of medicine. Most energy healing modalities, for example, have spiritual underpinnings. At the same time there are many attempts to use science to validate the healing power of faith. This is also an issue that is very attractive to the media, who love articles and headlines about the power of prayer. In our culture – faith sells.
A recent article in the Detroit Free Press is an excellent example of bad reporting and the sensationalizing of this issue. It does a good job of maximally confusing the issue.
To be clear, SBM is not anti-faith or anti-religion. But the issue of faith in medicine raises two main areas of concern. The first is the misrepresentation of the scientific evidence, both for intercessory prayer and the health effects of faith. The second are the ethical and professional implications of mixing faith with medical practice.
The Detroit Free Press article makes no attempt to distinguish the various issues with faith and medicine, and confuses them together in a misleading way. Intercessory prayer is, essentially, praying for the health of another person. There have been about a dozen such trials with reasonable design. In most the subjects know they may be prayed for. But of course, none of the trials can control for those who are not part of the study praying for a study subject.