There is an endless stream of supplement products on the market that are of questionable value. They tend to follow a similar pattern: put an essentially random assortment of vitamins, minerals, perhaps herbs and nutritional elements into a pill and then make whatever pseudo-health claims you want. Usually the claim is implied in the name of the product itself – sleepwell, or brainboost. The popular product Airborne fits this mold. It is essentially a multivitamin with the unfounded claim that it will prevent infection by boosting the immune system.
In the US, regulations (under DSHEA) specifically allow “structure/function” claims without any requirement for evidence to back up the claims. In other words, as long as you don’t mention a disease by name, you can make pretty much whatever claim you want. This was supposed to be good for the consumer, when in fact it is springtime for industry at the expense of the consumer. If your claims are outrageous enough the FTC can still go after you, but they are playing a game of whack-a-mole and losing.
Another pattern that is common is for a supplement product to contain specific components that are claimed to have specific benefits. Often these claims are based upon evidence – just the wrong kind of evidence. Basic science evidence is used inappropriately to support clinical claims. This strategy is more insidious, as it gives the public the sense that the product is science-based when it isn’t.
Reiki (pronounced raykey) is a form of “energy healing,” essentially the Asian version of faith healing or laying on of hands. Practitioners believe they are transferring life energy to the patient, increasing their well-being. The practice is popular among nurses, and in fact is practiced by nurses at my own institution (Yale).
From reiki.org, we get this description:
Reiki is a Japanese technique for stress reduction and relaxation that also promotes healing. It is administered by “laying on hands” and is based on the idea that an unseen “life force energy” flows through us and is what causes us to be alive. If one’s “life force energy” is low, then we are more likely to get sick or feel stress, and if it is high, we are more capable of being happy and healthy.
Reiki is therefore a form of vitalism – the pre-scientific belief that some spiritual energy animates the living, and is what separates living things from non-living things. The notion of vitalism was always an intellectual place-holder, responsible for whatever aspects of biology were not currently understood. But as science progressed, eventually we figured out all of the basic functions of life and there was simply nothing left for the vital force to do. It therefore faded from scientific thinking. We can add to that the fact that no one has been able to provide positive evidence for the existence of a vital force – it remains entirely unknown to science.
The discovery of various vitamins – essential micronutrients that cause disease when deficient – was one of the great advances of modern scientific medicine. This knowledge also led to several highly successful public health campaigns, such as vitamin-D supplementation to prevent rickets.
Today vitamins have a deserved reputation for being an important part of overall health. However, their reputation has gone beyond the science and taken on almost mythical proportions. Perhaps it is due to aggressive marketing from the supplement industry, perhaps recent generations have grown up being told by their parents thousands of times how important it is to take their vitamins, or eat vitamin-rich food. Culture also plays a role – Popeye eating spinach to make himself super strong is an example this pervasive message.
Regardless of the cause, the general feeling is that vitamins are all good – they are not only important for health, they promote health. Many people take vitamin supplements on the idea that more is better, or for nutritional “insurance” to make sure they are getting enough of every vitamin.
While we frequently on SBM target the worst abuses of science in medicine, it’s important to recognize that doing rigorous science is complex and mainstream scientists often fall short of the ideal. In fact, one of the advantages of exploring pseudoscience in medicine is developing a sensitive detector for errors in logic, method, and analysis. Many of the errors we point out in so-called “alternative” medicine also crop up elsewhere in medicine – although usually to a much less degree.
It is not uncommon, for example, for a paper to fail to adjust for multiple analysis – if you compare many variables you have to take that into consideration when doing the statistical analysis otherwise the probability of a chance correlation will be increased.
I discussed just yesterday on NeuroLogica the misapplication of meta-analysis – in this case to the question of whether or not CCSVI correlates with multiple sclerosis. I find this very common in the literature, essentially a failure to appreciate the limits of this particular analysis tool.
Another example comes recently from the journal Nature Neuroscience (an article I learned about from Ben Goldacre over at the Bad Science blog). Erroneous analyses of interactions in neuroscience: a problem of significance investigates the frequency of a subtle but important statistical error in high profile neuroscience journals.
Determining the net health effects of independent factors can be tricky, especially when those factors cannot be controlled for in experimental studies. For things like body mass index (BMI) we must rely on observational data and triangulate with multiple studies to isolate the contributions from BMI. But it can be done.
The data, however, are likely to be complex and noisy, and therefore there is plenty of opportunity for ideology to trump objectivity in interpreting the data. There are those who, for whatever reason, deny that we are having an obesity epidemic in the West, and those who deny the health implications of being overweight as an independent factor.
The terms overweight and obesity have had various definitions in the past, but in recent years the various health organizations have settled on consensus operational definitions (for obvious practical reasons). Their definition relates to body mass index, which is a person’s weight in kilograms (kg) divided by their height in meters (m) squared.
One of the themes of SBM is that modern health care should be based upon solid scientific ground. Interventions should be based on a risk vs benefit analysis using the best available scientific evidence (clinical and basic science).
As an extension of this, the standard of care needs to be a science-based standard. Science is (or at least should be) objective and transparent, and without such standards there is no way to have meaningful quality control. Without the filter of science there is no limit to the nonsense and magical thinking that can flow into the health care system. Increasingly we cannot afford the waste of fanciful and ineffective interventions, and even if limited resources were not an issue – individual patients deserve better.
It is for these reasons that we oppose the attempts by proponents of so-called complementary and alternative medicine (CAM) to erode or eliminate the science-based standard of care in medicine. Proponents differ mostly on how open they are about this goal, but there is no escaping the reality that at the heart of the very concept of CAM is at least a double standard – one in which the science-based bar for inclusion is lowered for some favored modalities.
It is an unfortunate truth that there is money in pseudoscience, particularly medical pseudoscience. Money both attracts charlatans and also funds their activities, which includes marketing pseudoscience and defending their claims from scientific scrutiny. In this way the game is rigged in favor of pseudoscience.
With0ut effective regulation, sites like ours are forced to play whack-a-mole with the medical pseudoscience du jour. The latest case in point is Titanium Ion Bands – which are just another version of the Power Balance bands that have been previously exposed as nonsense. The idea is that by wearing a small bracelet on one wrist you will experience improved athletic performance. This sounds impossible – because it is. But companies have successfully bamboozled enough of the public to rake in millions.
The marketing strategy is three-fold. First, get naive professional athletes to endorse the product. Second, give live demonstrations (deceptive parlor tricks) that convince the unsuspecting that something must be going on. And third, wow the scientifically illiterate with a confusing barrage of medical techno-babble. The combination is sadly effective.
Power Balance, for example, makes vague references to frequencies and energy as the explanation for how a little piece of rubber (with embedded holograms) can have any effect on human physiology. The company was eventually legally forced to admit: “”We admit that there is no credible scientific evidence that supports our claims.” The admission has not ended their sale, however.
The latest issue of the BMJ contains an editorial recommending that regulators (this is in the UK, but the argument applies in the US and elsewhere) should require pharmaceutical companies to provide research on direct comparison to existing therapies as part of the approval process. The authors, Sorenson, Naci, Cylus, and Mossialos, write:
When a drug comes to market, evidence on the comparative risks and benefits is needed to help regulatory authorities to safeguard public health from inferior and unsafe treatments, to ensure that health technology assessment agencies and payers make funding decisions based on the best available evidence of different treatments, and to aid clinicians’ and patients’ understanding of what therapies work best and their appropriate position in the treatment pathway.
They make a persuasive argument, but there are some interesting angles to this topic.
Here is yet another study claiming to show “how acupuncture works” when in fact it does nothing of the kind. It does, however, reveal the bias of the researchers – it is, in fact, surprising that it was published in a peer-reviewed journal. Unfortunately, the mainstream media is dutifully reporting the biased claims of the researchers without any independent verification or analysis.
There are numerous fatal problems with this study. The first, like in many physiological studies that purport to be about acupuncture, is that the connection to acupuncture is tenuous. The researchers claim that they are testing the effects of an acupuncture needle – but what makes a needle an acupuncture needle? Other such studies were ultimately just seeing the effects of local tissue trauma. The fact that this trauma was induced by an “acupuncture needle” is not necessarily relevant.
This study is far worse, because it is simply using the acupuncture needle as a mechanism for inducing an unrelated physiological stimulus. This is similar to “electroacupuncture” where electrical current is applied through an acupuncture needle – what you are actually studying is the effects of electricity, not “acupuncture.” Applying electrical stimulation, or some other physiological stimulus, is the equivalent of injecting morphine through a thin needle and then claiming this demonstrates how “acupuncture works.”
The fundamental concept of science-based medicine (SBM) is that medical practice should be based upon the best available science. This may seem obvious, but there are many important details to its application, such as the relationship between clinical and basic science. Clinical claims require clinical evidence, but clinical evidence can be tricky and is often preliminary. It is therefore helpful (I would say essential) to view the clinical evidence in light of all of the rest of science.
A thorough basic and clinical science analysis of a medical claim can be summarized by the term “plausibility,” or “prior probability” if you want to put it into statistical terms. When we say a certain belief is plausible we mean it is consistent with what we know from the rest of science. In other words, because of the many weaknesses of clinical evidence, in order for a therapy to be generally accepted as part of SBM it should have a certain minimal supporting clinical evidence and overall scientific plausibility.
These can exist in different proportions – for example one therapy may be highly plausible (it would be shocking if it were not true) and have modest supporting clinical evidence, while another may have unknown plausibility but with solid clinical evidence of efficacy. But no therapy should have clinical evidence that suggests lack of efficacy, nor extreme implausibility (not just an unknown mechanism, but no possible mechanism).