Articles

Mathematically modeling why quackery persists

ResearchBlogging.orgIt’s often puzzled me (and, I daresay, many other skeptics and boosters of science- and evidence-based medicine) why various forms of quackery and woo that have either about as close to zero prior probability as you can get and/or have failed to show evidence greater than placebo in clinical trials manage to retain so much traction among the public. Think homeopathy. Think reiki. The former is nothing more than sympathetic magic prettied up with science-y sounding terms, while the latter is nothing more than faith healing given a slant based on Eastern mysticism and religion instead of Christianity. Indeed, reiki was even inspired by stories of Jesus’ healing powers, complete with a trip into the wilderness for fasting and prayer, resulting in revelation. Or consider acupuncture, a modality that is seemingly more popular than ever, even invading the very sanctum sanctorum of the ivory towers of academic medicine, yet every study of which that is done under rigorous conditions with proper placebo controls shows it to be no more efficacious than a placebo. It’s easy enough to shake one’s head and chalk it up to irrationality, ignorance of science, or even religious faith, but I’ve always been dissatisfied with such glib explanations, even though admittedly I have myself used them on occasion.

That’s why a study released last week in PLoS One by Mark M. Tanaka, Jeremy R. Kendal, Kevin N. Laland out of the Evolution & Ecology Research Centre, School of Biotechnology & Biomolecular Sciences, University of New South Wales, the Department of Anthropology, University of Durham, and the School of Biology, University of St Andrews, Fife, respectively, entitled From Traditional Medicine to Witchcraft: Why Medical Treatments Are Not Always Efficacious. Besides loving the title, I also like the methodology, which in essence adapts the tools of modeling evolution and the spread of traits throughout a population and asks the question: Why do ineffective or even harmful (or, as the authors characterize them, “maladaptive”) treatments for various illnesses persist in populations? The results are surprising and counterintuitive, yet ring true. In essence, the authors conclude that the most efficacious self-treatments are not always the ones that spread and that even harmful treatments can spread. Both of these observations are entirely plausible based on the prevalence of usage of common woo and quackery, and what the authors have done, in essence, is to model mathematically why quackery persists.

Indeed, the authors set the stage:

In recent years, 60–80% of the world’s population, mainly from developing countries, depended primarily on traditional medicines, folk remedies and home cures, as well as treatment from witchdoctors and other ‘supernatural practices’, for their health-care needs [1]. In western societies, complementary and alternative medicine is garnering increasing interest and acceptance. At current growth rates, two-thirds of Americans are projected to be using alternative medicine by 2010 [2]. Asian governments are pouring billions of dollars into screening Traditional Chinese medicines in the hope that clinical trials will spawn lucrative drugs [3]. Traditional medicine has become big business.

Indeed, it has, which is why I always laugh when advocates of “complementary and alternative medicine” (CAM) claim it’s all about the money for big pharma but not for CAM. But I digress:

While scientific studies have validated some traditional remedies, for instance, by confirming the biological activity of plant extracts [4], [5], the use of complementary and traditional medicines remains contentious, and doubts about the efficacy and safety of many treatments remain [1], [6], [7], [8]. Reservations over safety and efficacy underpin controversy over USA and UK universities’ attempts to bring alternative medicines into medical school curricula [9]. The active ingredients used in many traditional medicines are potentially toxic, often containing dangerous elements, including heavy metals [5], [10]. Even the use of ineffective non-toxic remedies can be harmful if it delays effective treatment. For instance, fears have been expressed that, in Nigeria, witchcraft and traditional remedies of unknown efficacy are widely employed as treatments for malaria, instead of, or delaying access to, modern medicines of proven effectiveness [11]. In sub-Sarahan Africa there is a concern that the use of traditional remedies for mastitis, a condition often attributed to sorcery, may inadvertently be contributing to the spread of HIV [12].

In 2002 the WHO [1] launched a global plan to make the use of traditional medicine safer by encouraging evidence-based research on the safety, efficacy and quality of traditional practices. Accordingly, traditional medicines are currently undergoing scrutiny to evaluate their effectiveness and monitor adverse events [3], [13]. Such analyses have often failed to confirm the efficacy of traditional remedies: for instance, of nearly 25,000 applications for registration of traditional medicines received by Malaysian authorities, 37.3% were rejected, either on grounds of safety or ineffectiveness [14]. However, there is currently no compelling explanation for the prevalence of low-efficacy treatments.

As I said before, I never found explanations of lack of scientific understanding or confusing correlation with causation entirely satisfactory. Don’t get me wrong; I definitely think these are major reasons why, for instance, so many parents believe vaccines cause autism or so many people think that reiki works. I just don’t think it’s enough.

Enter Tanaka et al. They basically construct a model in which they attempt to estimate the factors that could affect the spread of various treatments throughout the population. One aspect of this model to be kept in mind is that it primarily looks at self-medication, although parents medicating their children, as happens in too much autism quackery, could be included under this category, mainly because it is not the children who decide for themselves whether a treatment is working or not; it is the parents. The general assumptions of the model are summarized in Figure 1 below:

Figure1

Table 1 lists the parameters inputted into the model:

Table1

I’m not going to go into the equations in much detail. In fact, I’m not going to go into the equations at all. The article is freely available to all; so you can read it yourself. What I will discuss are some of the assumptions going into the model. First among these is this:

We assume that individuals are either in a diseased state or in a healthy state. We model the spread of a behavioural trait expressed in treatment of disease. The behavioural trait in question is any innovation, practice or treatment that could potentially affect the outcome of this disease. To model the spread of a behavioural trait, we make the following assumptions. A new behavioural trait arises in (or is invented by) an ill individual who may then demonstrate this practice; others who are ill may adopt the practice upon being exposed to it, and then become demonstrators themselves. In other words, demonstrators convert observers. There is empirical support for the assumption that self-medicative treatments spread through social learning [22].

The most important assumption made for the model is that observers adopt the trait in an unbiased fashion. The authors don’t assume a prior that efficacy has anything to do with whether observers take up a treatment. They justify this by using the quite sensible reasoning that observers are not very good at judging the efficacy of a treatment. Any regular reader of this blog should be aware of why this is true; it’s basically because any observer trying to determine whether a treatment is efficacious on the basis of watching it being used, is using anecdotal evidence to come to a conclusion. If there’s one theme that’s run through this blog from the very beginning, it’s how easily we as humans are deceived by anecdotal evidence. Indeed, unless a remedy is rapidly fatal or produces a miraculous recovery, given the effects of regression to the mean, confirmation bias, treatment bias, and a variety of other biases, judging the efficacy of a therapy on the level of an individual can be very problematic even for physicians. For the untrained, it’s really tough.

The authors define the “cultural fitness” of a treatment as the mean number of converts produced by a demonstrator and then studied what the value of cultural fitness would be under various assumptions, both for instances of single illness and in the case of multiple relapses of illness. Interestingly, although highly efficacious treatments are predicted to spread through the population, even in the case of inefficacious treatments, multiple exposures of it and a remedy for it actually increase the chance that the treatment will spread, reagardless of whether the treatment is efficacious or not. The authors concluded that, although highly efficacious treatments have a high cultural fitness, treatments with efficacy near or at zero can also spread. The conditions under which such remedies could spread and achieve high cultural fitness are when treatments are primarily demonstrated in sickness and demonstrate low abandonment, particularly where the likelihood of relapse is small. The authors summarize:

This study offers a simple, novel and counter-intuitive hypothesis for the prevalence of ineffective medical treatments: unbiased copying of new treatments can frequently lead to the prevalence of ineffective practices because such treatments are demonstrated more persistently than efficacious alternatives, even when there is enhanced abandonment of ineffective cures. By unbiased copying, we mean copying in direct proportion to the rate at which the alternative variants are demonstrated. Here, in simple terms, treatment frequency dynamics are typically dominated by two processes, representing the rates of acquisition and loss of remedies. Maladaptive and superstitious treatments can become prevalent because their ineffectiveness prolongs illness, enhancing their rate of demonstration relative to efficacious cures, and leading to elevated rates of acquisition that may compensate for greater loss.

Our finding that superstitious treatments can easily spread is supported by reports of extraordinary treatments for conditions such as leprosy (treated with a drink made of rotting snakes) and syphilis (treated by eating a vulture), and by similar myths for poisonous snake bites (apply ‘guaco’ leaves, poisonous lizard skin or snake’s bile), dog bites (drink tea made from the dog’s tail) and scorpion stings (tie a scorpion against the stung finger) [23]. The analysis also helps explain the persistence of medical treatments of animals, such as ‘firing’ (cautery) of working horses, employed for millennia as treatment for lameness, where recovery is rare, and still widely practiced in many countries in spite of trials establishing its ineffectiveness [24].

Another conclusion of this model is that even highly effective treatments can be lost due to stochasticity, in other words, due to random chance. Indeed, the authors point out that most highly efficacious innovations would be predicted to be lost due to stochasticity. Indeed, my reading of this study would suggest that one reason why highly effective treatments actually do persist in our society is because scientists, physicians, and science-based medicine validate what treatments are efficacious, retaining the treatments that are and trying (but not always succeeding) in discarding the ones that aren’t. For people self-medicating, such a mechanism is not operative.

One aspect of this model that I consider inadequate is that it doesn’t take into account the role of peer pressure and groupthink. It certainly doesn’t take into account the rapidity with which glowing stories of success for quackery can spread through both word of mouth and other methods. In this case, the Internet truly has changed everything. Never before, has it been possible for so many people to “listen to so many stories” in such a short period of time or to form online communities made up of people who mutually reinforce each other’s beliefs, even though most of them have never met face-to-face.

I return to one of my favorite examples, autism quackery such as “biomedical interventions” and chelation therapy. Because autism is a condition of developmental delay, not stasis, it’s very easy to confuse correlation with causation and believe that a given ineffective intervention “worked.” Prometheus points out this very phenomenon:

If you have the opportunity, check in to one of the many “biomedically oriented” Internet groups and see what happens when somebody questions the idea that “biomed” can “recover” autistic children. At the least, they will be admonished – “Don’t stand in the way of other parents getting their children the help they need!”. More likely, they will be told to “Shut up!” and banned from the group. In some cases, they will be harassed and even threatened.

I’m not sure how the authors could have modeled the “Emperor’s New Clothes” phenomenon, such as Prometheus describes, but I can’t help but think that such mechanisms are at work in addition to the ones described in this paper.

As much as I like this paper for its description of a plausible mechanism by which implausible and ineffective treatments can spread through a population and remain entrenched, I have to caution that, without some sort of experimental or observational validation, it remains just that: a model. Unfortunately, I’m having some difficulty figuring out just how one might test this model against reality in a real human population.

One thing that this model leads me to wonder, though, is whether, maybe–just maybe–there is a force that can prevent or halt the spread of implausible, unscientific, and ineffective treatments, such as homeopathy, throughout the population. Perhaps such a method both serves as a memory that recalls which therapies don’t work and a method of testing which methods do work. That method is science-based medicine.

I wonder how the effect of SBM can be incorporated into this model.

REFERENCES:

Tanaka, M., Kendal, J., & Laland, K. (2009). From Traditional Medicine to Witchcraft: Why Medical Treatments Are Not Always Efficacious PLoS ONE, 4 (4) DOI: 10.1371/journal.pone.0005192

Posted in: Basic Science, Clinical Trials, Science and Medicine

Leave a Comment (30) ↓

30 thoughts on “Mathematically modeling why quackery persists

  1. Neuroskeptic says:

    It’s quite a good paper, although the model they present makes a number of dubious assumptions. #1 being that people start using treatments just because they see other people using them – rather than because they’re convinced that they work.

    However, the central argument – that bad treatments might end up being used for longer than good ones, just because they don’t work – is interesting…

  2. Beowulff says:

    Maybe this model also explains the lack of support of vaccination? Since vaccines are never demonstrated to cure a diseased person, but only prevents people from ever getting a particular disease, it won’t benefit from this mechanism of spreading. It does, however, suffer from being too effective.

  3. Gah! You’ve scooped me!
    Not that this will stop me my posting anyway.

    Interesting treatment.

    @Beowulf: In fact in the paper it explicitly says that prophylactic treatments often fare poorly in this model, for more or less the reasons you set out.

    It’d be interesting if they could incorporate prestige bias – how does real doctors advice, affect this model?

  4. DrBadger says:

    It’s quite a good paper, although the model they present makes a number of dubious assumptions. #1 being that people start using treatments just because they see other people using them – rather than because they’re convinced that they work.

    Yep. That’s exactly what I was thinking about last week. It may work under some special circumstances, but I doubt its why most americans are using quack medicine today.

  5. DVMKurmes says:

    I think they were trying to model the spread of treatments in traditional societies, and did not necessarily even consider things like the internet in that model. Still another excellent reason to be skeptical of the argument from antiquity or argument ad populi though.

  6. Harriet Hall says:

    I have a highly intelligent friend who is skeptical about most things but gullible about alternative medicine. Her rationale for trying a new treatment is that a friend told her it worked for him and it doesn’t appear to be dangerous. The testimonial, especially from a personal friend or relative, is the most effective persuasion even though it is the least scientifically meaningful.

  7. Karl Withakay says:

    whitecoattales:

    Prestige bias is probably trumped (and dwarfed) by the Cult of Celebrity/ Oprah factor.

    If Oprah promotes Airborne on her show, legions of Oprah disciples start using and swearing by Airborne. This produces a large jump start on the number of “demonstrators” in the population, which accelerates what is essentially a viral effect and self fueling firestorm.

  8. markmarijnissen says:

    It is a nice model, it would also be nice to datamine the data, or to use artifical intelligence (neural networks, genetic algorithms, etc) to come up with an explanation (or, how effective a treatment will be, which persons will use them, etc – depending on the data you have)

  9. thehatinthecat says:

    Very intresting. I’m going to have to take a deeper look at their study.

  10. David Gorski says:

    I think they were trying to model the spread of treatments in traditional societies, and did not necessarily even consider things like the internet in that model. Still another excellent reason to be skeptical of the argument from antiquity or argument ad populi though.

    Very true, but the authors explicitly mention CAM. They even start out pointing out how CAM is now big business. I repeat the first two paragraphs of the paper:

    Traditional remedies, utilising medicinal plant and animal products, have been used as treatments for human diseases and medical conditions for millennia [1]. In recent years, 60–80% of the world’s population, mainly from developing countries, depended primarily on traditional medicines, folk remedies and home cures, as well as treatment from witchdoctors and other ‘supernatural practices’, for their health-care needs [1]. In western societies, complementary and alternative medicine is garnering increasing interest and acceptance. At current growth rates, two-thirds of Americans are projected to be using alternative medicine by 2010 [2]. Asian governments are pouring billions of dollars into screening Traditional Chinese medicines in the hope that clinical trials will spawn lucrative drugs [3]. Traditional medicine has become big business.

    While scientific studies have validated some traditional remedies, for instance, by confirming the biological activity of plant extracts [4], [5], the use of complementary and traditional medicines remains contentious, and doubts about the efficacy and safety of many treatments remain [1], [6], [7], [8]. Reservations over safety and efficacy underpin controversy over USA and UK universities’ attempts to bring alternative medicines into medical school curricula [9]. The active ingredients used in many traditional medicines are potentially toxic, often containing dangerous elements, including heavy metals [5], [10]. Even the use of ineffective non-toxic remedies can be harmful if it delays effective treatment. For instance, fears have been expressed that, in Nigeria, witchcraft and traditional remedies of unknown efficacy are widely employed as treatments for malaria, instead of, or delaying access to, modern medicines of proven effectiveness [11]. In sub-Sarahan Africa there is a concern that the use of traditional remedies for mastitis, a condition often attributed to sorcery, may inadvertently be contributing to the spread of HIV [12].

    I don’t think the authors were limiting themselves to just traditional societies.

  11. David Gorski says:

    If Oprah promotes Airborne on her show, legions of Oprah disciples start using and swearing by Airborne. This produces a large jump start on the number of “demonstrators” in the population, which accelerates what is essentially a viral effect and self fueling firestorm.

    I wonder if there’s a way to account for “super-demonstrators” (like Oprah and, to a lesser extent, Jenny McCarthy, Mehmet Oz, Kevin Trudeau, etc.) in the model described in this paper; i.e., people who through their fame and recommendation produce a huge bump in the number of “demonstrators.” Such a phenomenon undoubtedly went on to a much lesser extent in traditional societies in the form of shamans, chieftains, and village elders, but with mass media and the Internet the ability of such “super-demonstrators” to pump up the numbers dwarfs what could ever happen in a traditional society living a hunter-gatherer or early agrarian kind of existence.

  12. Karl Withakay says:

    The super-demonstrators have at least two things that would need to be accounted for:

    1 Their large exposure to the rest of the population
    2 The weight of their influence with the population

    Theoretically, one should be able to take these factors into account in a model. In fact, one could modify the model to include this information for ALL demonstrators, with default baseline values for the “mundane-demonstrator”, making it easier to tweak the model to include various numbers and sizes of different “demonstrator strength” groups.

    For #1, exposure shouldn’t be too hard to model based on various forms of exposure such as TV, movie, magazine appearances, internet site hit counters, etc.

    One could give a decent estimation of #2 (for Oprah, at least) by looking at past “demonstration” performance by Oprah with book & movie recommendations, Airborne, etc.

  13. pec says:

    This theory explains why consumers generally buy the worst products at the highest prices — one person does it and then everyone else follows mindlessly. People don’t stop and think for themselves and decide that maybe there is a better product at a lower price. Look at cars, for example — obviously, the highest price pile of junk available is the one consumers go for.

    I am glad this has finally been explained.

  14. pec says:

    Oh and preferably the car doesn’t work at all. That’s a guaranteed best seller.

  15. Dacks says:

    I wonder if the use of “supplements” fits this model. These are usually taken by people who are not sick to “promote health.” Someone else would see that this person is healthy, and that they attribute their health to the supplements, and another convert is won.
    Musing a little further, perhaps this mechanism acts as a reinforcement when they do become ill. If supplements have kept them healthy so far, they might think supplements are the best treatment. In other words, this innocuous trend in self-help among the healthy becomes the dreaded “paradigm shift” that brings them to alternative medicine when they are sick.

  16. pec says:

    Yeah, never forget PEOPLE ARE STUPID. Except, of course, the ones who designed this model.

  17. Peter Lipson says:

    pec, you never fail to bring the Whiskey Tango Foxtrot.

  18. pec says:

    Well I try.

  19. Calli Arcale says:

    Fascinating, and it makes sense to me. I don’t find it counter-intuitive at all.

    Essentially, this model is looking at the propagation of memes, specifically, healthcare-related memes. I’m sure the same thing can be seen in other areas. pec jokingly referred to purchase of overpriced and ineffective products, but this does actually occur, although if a product is totally bogus it won’t sell for long. Consider how people become brand-loyal even when there is no real difference in the performance of different brands. Store-brand soap is much cheaper than the name-brand stuff and often functionally identical; why, then, are people so much more likely to buy brand-name that the store has to stock twice as much of that as of their in-house alternative?

    Really dramatic examples can be seen in drinks. Why in God’s name can a bottle of wine sell for upwards of $2,000 dollars? Yes, it’s probably better than the $2 bottle of plonk sold in the same store, but is $2,000 ever really justified? Some swear it is, that there is a difference beyond simply the prestige of a particularly rare vintage, and that there are times when a mere $100 bottle simply won’t do. I know I wouldn’t be able to tell the difference between a margaux and a Chateau Margeaux of the same year. (Difference? The first is a region controlee while the second is an actual vintner. Also, the second seriously can cost over ten times as much.)

    Obviously, there’s another factor going on, and that’s marketing, but there are definitely memes going through society, many without any conscious force driving them, which dramatically alter our perceptions of what is good and what is valuable. This is just as true in health care as it is in fashion accessories.

  20. DVMKurmes says:

    Sorry, I oversimplified and managed to misrepresent myself. I agree that the authors meant to apply the model to CAM, and not just the effectiveness of traditional treatments. I am not sure the model as it is and with the assumptions they used can account for modern mass media phenomena, but it is definitely applicable to CAM.
    As to the effect of SBM, it at least helps to understand what we are up against and can give a better understanding of why so many people adopt ineffective treatments. Probably the way people adopt treatments in traditional societies is not much different than in modern societies, there is just more to choose from in a modern situation.

  21. Karl Withakay:

    I guess I feel like celebrity factor is a special case of prestige bias
    In my highly uninformed opinion, it would seem doable to model them both: Prestige via doctors as limited exposure, but selective for efficacious treatments. Prestige via celebrities as larger exposure, nonselective, or selective for maladaptive behavior.

    Presumably modeling both would lean towards the celebrities yes, I’ll agree with you there!

  22. Chris says:

    Peter Lipson said

    pec, you never fail to bring the Whiskey Tango Foxtrot.

    pec said

    Well I try.

    Ack! Is pec a Poe dervied sockpuppet, or did she not understand the meaning behind “WTF”?

  23. yeahsurewhatever says:

    Ah, PLoS ONE. When you absolutely, positively have to publish your fringe science prestige article in what some people *might* call a peer-reviewed journal, but you don’t want to pay more than $1300 to do it, look no further.

  24. The Blind Watchmaker says:

    We didn’t evolve to think scientifically. We evolved to assign meaning to patterns and believe our first impressions. We evolved to carismatic leaders and reject what we don’t understand.

    We evolved to like CAM.

  25. Mark Crislip says:

    “Ah, PLoS ONE. When you absolutely, positively have to publish your fringe science prestige article in what some people *might* call a peer-reviewed journal, but you don’t want to pay more than $1300 to do it, look no further.”

    I don;t know. As a practicing physician who gets called to see all the weirdness, the lack of easily accessible information in a busy day is an annoyance. I can get anything through the library, but it costs to get copies and it can take a day, and I have often already paid for the research with my tax dollars.

    When it comes to medical information, I do not like barriers to accessing it for patient care.

    I would love it if all medical information were freely available and the world would be a better place if more people would publish in PLoS.

    It would also be a better place if I could make my am coffee with unicorn tears.

  26. David Gorski says:

    Ah, PLoS ONE. When you absolutely, positively have to publish your fringe science prestige article in what some people *might* call a peer-reviewed journal, but you don’t want to pay more than $1300 to do it, look no further.

    Quite frankly, you’re full of crap.

    PLoS ONE is a peer-reviewed journal of generally high quality. The publication charges are necessary because of the Open Access model of publishing.

  27. solanum says:

    It’s quite a good paper, although the model they present makes a number of dubious assumptions. #1 being that people start using treatments just because they see other people using them – rather than because they’re convinced that they work.

    It seems to me there would be a lot of factors making the ‘demonstration’ seem successful — placebo effect, regression to the mean in cases of chronic symptoms, and in the case of an acute disease, it simply running its course. The demonstrator is probably also favorably predisposed toward the treatment or they wouldn’t have tried it. It may be that the demonstrator’s belief and testimonial is really all that is needed to spread the treatment — as in religious beliefs.

  28. pec says:

    Everyone knows that people love to spend their money on things that don’t work at all. The evidence for this is everywhere you look — Americans buying cars that either won’t go, or will go but won’t stop. And all those computers we’re always buying that crash every six seconds — this ingenious mathematical model finally explains why we prefer garbage over quality every time.

    So this only confirms what we already know — given a choice between something that is utterly useless and broken and something of high quality and value, consumers will inevitably take the broken useless thing.

    Thank goodness someone has found a scientific explanation.

Comments are closed.