Apr 20 2009
It’s often puzzled me (and, I daresay, many other skeptics and boosters of science- and evidence-based medicine) why various forms of quackery and woo that have either about as close to zero prior probability as you can get and/or have failed to show evidence greater than placebo in clinical trials manage to retain so much traction among the public. Think homeopathy. Think reiki. The former is nothing more than sympathetic magic prettied up with science-y sounding terms, while the latter is nothing more than faith healing given a slant based on Eastern mysticism and religion instead of Christianity. Indeed, reiki was even inspired by stories of Jesus’ healing powers, complete with a trip into the wilderness for fasting and prayer, resulting in revelation. Or consider acupuncture, a modality that is seemingly more popular than ever, even invading the very sanctum sanctorum of the ivory towers of academic medicine, yet every study of which that is done under rigorous conditions with proper placebo controls shows it to be no more efficacious than a placebo. It’s easy enough to shake one’s head and chalk it up to irrationality, ignorance of science, or even religious faith, but I’ve always been dissatisfied with such glib explanations, even though admittedly I have myself used them on occasion.
That’s why a study released last week in PLoS One by Mark M. Tanaka, Jeremy R. Kendal, Kevin N. Laland out of the Evolution & Ecology Research Centre, School of Biotechnology & Biomolecular Sciences, University of New South Wales, the Department of Anthropology, University of Durham, and the School of Biology, University of St Andrews, Fife, respectively, entitled From Traditional Medicine to Witchcraft: Why Medical Treatments Are Not Always Efficacious. Besides loving the title, I also like the methodology, which in essence adapts the tools of modeling evolution and the spread of traits throughout a population and asks the question: Why do ineffective or even harmful (or, as the authors characterize them, “maladaptive”) treatments for various illnesses persist in populations? The results are surprising and counterintuitive, yet ring true. In essence, the authors conclude that the most efficacious self-treatments are not always the ones that spread and that even harmful treatments can spread. Both of these observations are entirely plausible based on the prevalence of usage of common woo and quackery, and what the authors have done, in essence, is to model mathematically why quackery persists.
Indeed, the authors set the stage:
In recent years, 60–80% of the world’s population, mainly from developing countries, depended primarily on traditional medicines, folk remedies and home cures, as well as treatment from witchdoctors and other ‘supernatural practices’, for their health-care needs . In western societies, complementary and alternative medicine is garnering increasing interest and acceptance. At current growth rates, two-thirds of Americans are projected to be using alternative medicine by 2010 . Asian governments are pouring billions of dollars into screening Traditional Chinese medicines in the hope that clinical trials will spawn lucrative drugs . Traditional medicine has become big business.
Indeed, it has, which is why I always laugh when advocates of “complementary and alternative medicine” (CAM) claim it’s all about the money for big pharma but not for CAM. But I digress:
While scientific studies have validated some traditional remedies, for instance, by confirming the biological activity of plant extracts , , the use of complementary and traditional medicines remains contentious, and doubts about the efficacy and safety of many treatments remain , , , . Reservations over safety and efficacy underpin controversy over USA and UK universities’ attempts to bring alternative medicines into medical school curricula . The active ingredients used in many traditional medicines are potentially toxic, often containing dangerous elements, including heavy metals , . Even the use of ineffective non-toxic remedies can be harmful if it delays effective treatment. For instance, fears have been expressed that, in Nigeria, witchcraft and traditional remedies of unknown efficacy are widely employed as treatments for malaria, instead of, or delaying access to, modern medicines of proven effectiveness . In sub-Sarahan Africa there is a concern that the use of traditional remedies for mastitis, a condition often attributed to sorcery, may inadvertently be contributing to the spread of HIV .
In 2002 the WHO  launched a global plan to make the use of traditional medicine safer by encouraging evidence-based research on the safety, efficacy and quality of traditional practices. Accordingly, traditional medicines are currently undergoing scrutiny to evaluate their effectiveness and monitor adverse events , . Such analyses have often failed to confirm the efficacy of traditional remedies: for instance, of nearly 25,000 applications for registration of traditional medicines received by Malaysian authorities, 37.3% were rejected, either on grounds of safety or ineffectiveness . However, there is currently no compelling explanation for the prevalence of low-efficacy treatments.
As I said before, I never found explanations of lack of scientific understanding or confusing correlation with causation entirely satisfactory. Don’t get me wrong; I definitely think these are major reasons why, for instance, so many parents believe vaccines cause autism or so many people think that reiki works. I just don’t think it’s enough.
Enter Tanaka et al. They basically construct a model in which they attempt to estimate the factors that could affect the spread of various treatments throughout the population. One aspect of this model to be kept in mind is that it primarily looks at self-medication, although parents medicating their children, as happens in too much autism quackery, could be included under this category, mainly because it is not the children who decide for themselves whether a treatment is working or not; it is the parents. The general assumptions of the model are summarized in Figure 1 below:
Table 1 lists the parameters inputted into the model:
I’m not going to go into the equations in much detail. In fact, I’m not going to go into the equations at all. The article is freely available to all; so you can read it yourself. What I will discuss are some of the assumptions going into the model. First among these is this:
We assume that individuals are either in a diseased state or in a healthy state. We model the spread of a behavioural trait expressed in treatment of disease. The behavioural trait in question is any innovation, practice or treatment that could potentially affect the outcome of this disease. To model the spread of a behavioural trait, we make the following assumptions. A new behavioural trait arises in (or is invented by) an ill individual who may then demonstrate this practice; others who are ill may adopt the practice upon being exposed to it, and then become demonstrators themselves. In other words, demonstrators convert observers. There is empirical support for the assumption that self-medicative treatments spread through social learning .
The most important assumption made for the model is that observers adopt the trait in an unbiased fashion. The authors don’t assume a prior that efficacy has anything to do with whether observers take up a treatment. They justify this by using the quite sensible reasoning that observers are not very good at judging the efficacy of a treatment. Any regular reader of this blog should be aware of why this is true; it’s basically because any observer trying to determine whether a treatment is efficacious on the basis of watching it being used, is using anecdotal evidence to come to a conclusion. If there’s one theme that’s run through this blog from the very beginning, it’s how easily we as humans are deceived by anecdotal evidence. Indeed, unless a remedy is rapidly fatal or produces a miraculous recovery, given the effects of regression to the mean, confirmation bias, treatment bias, and a variety of other biases, judging the efficacy of a therapy on the level of an individual can be very problematic even for physicians. For the untrained, it’s really tough.
The authors define the “cultural fitness” of a treatment as the mean number of converts produced by a demonstrator and then studied what the value of cultural fitness would be under various assumptions, both for instances of single illness and in the case of multiple relapses of illness. Interestingly, although highly efficacious treatments are predicted to spread through the population, even in the case of inefficacious treatments, multiple exposures of it and a remedy for it actually increase the chance that the treatment will spread, reagardless of whether the treatment is efficacious or not. The authors concluded that, although highly efficacious treatments have a high cultural fitness, treatments with efficacy near or at zero can also spread. The conditions under which such remedies could spread and achieve high cultural fitness are when treatments are primarily demonstrated in sickness and demonstrate low abandonment, particularly where the likelihood of relapse is small. The authors summarize:
This study offers a simple, novel and counter-intuitive hypothesis for the prevalence of ineffective medical treatments: unbiased copying of new treatments can frequently lead to the prevalence of ineffective practices because such treatments are demonstrated more persistently than efficacious alternatives, even when there is enhanced abandonment of ineffective cures. By unbiased copying, we mean copying in direct proportion to the rate at which the alternative variants are demonstrated. Here, in simple terms, treatment frequency dynamics are typically dominated by two processes, representing the rates of acquisition and loss of remedies. Maladaptive and superstitious treatments can become prevalent because their ineffectiveness prolongs illness, enhancing their rate of demonstration relative to efficacious cures, and leading to elevated rates of acquisition that may compensate for greater loss.
Our finding that superstitious treatments can easily spread is supported by reports of extraordinary treatments for conditions such as leprosy (treated with a drink made of rotting snakes) and syphilis (treated by eating a vulture), and by similar myths for poisonous snake bites (apply ‘guaco’ leaves, poisonous lizard skin or snake’s bile), dog bites (drink tea made from the dog’s tail) and scorpion stings (tie a scorpion against the stung finger) . The analysis also helps explain the persistence of medical treatments of animals, such as ‘firing’ (cautery) of working horses, employed for millennia as treatment for lameness, where recovery is rare, and still widely practiced in many countries in spite of trials establishing its ineffectiveness .
Another conclusion of this model is that even highly effective treatments can be lost due to stochasticity, in other words, due to random chance. Indeed, the authors point out that most highly efficacious innovations would be predicted to be lost due to stochasticity. Indeed, my reading of this study would suggest that one reason why highly effective treatments actually do persist in our society is because scientists, physicians, and science-based medicine validate what treatments are efficacious, retaining the treatments that are and trying (but not always succeeding) in discarding the ones that aren’t. For people self-medicating, such a mechanism is not operative.
One aspect of this model that I consider inadequate is that it doesn’t take into account the role of peer pressure and groupthink. It certainly doesn’t take into account the rapidity with which glowing stories of success for quackery can spread through both word of mouth and other methods. In this case, the Internet truly has changed everything. Never before, has it been possible for so many people to “listen to so many stories” in such a short period of time or to form online communities made up of people who mutually reinforce each other’s beliefs, even though most of them have never met face-to-face.
I return to one of my favorite examples, autism quackery such as “biomedical interventions” and chelation therapy. Because autism is a condition of developmental delay, not stasis, it’s very easy to confuse correlation with causation and believe that a given ineffective intervention “worked.” Prometheus points out this very phenomenon:
If you have the opportunity, check in to one of the many “biomedically oriented” Internet groups and see what happens when somebody questions the idea that “biomed” can “recover” autistic children. At the least, they will be admonished – “Don’t stand in the way of other parents getting their children the help they need!”. More likely, they will be told to “Shut up!” and banned from the group. In some cases, they will be harassed and even threatened.
I’m not sure how the authors could have modeled the “Emperor’s New Clothes” phenomenon, such as Prometheus describes, but I can’t help but think that such mechanisms are at work in addition to the ones described in this paper.
As much as I like this paper for its description of a plausible mechanism by which implausible and ineffective treatments can spread through a population and remain entrenched, I have to caution that, without some sort of experimental or observational validation, it remains just that: a model. Unfortunately, I’m having some difficulty figuring out just how one might test this model against reality in a real human population.
One thing that this model leads me to wonder, though, is whether, maybe–just maybe–there is a force that can prevent or halt the spread of implausible, unscientific, and ineffective treatments, such as homeopathy, throughout the population. Perhaps such a method both serves as a memory that recalls which therapies don’t work and a method of testing which methods do work. That method is science-based medicine.
I wonder how the effect of SBM can be incorporated into this model.
Tanaka, M., Kendal, J., & Laland, K. (2009). From Traditional Medicine to Witchcraft: Why Medical Treatments Are Not Always Efficacious PLoS ONE, 4 (4) DOI: 10.1371/journal.pone.0005192
30 Responses to “Mathematically modeling why quackery persists”