Articles

Andrew Weil Flirts with Evidence Based Medicine

Andrew Weil, MD, pops up quite frequently on SBM, most recently in this entry by Harriet Hall, so I will not spend much space introducing him. An excellent biography and critique of Dr. Weil was written by Arnold Relman, former Editor of the New England Journal of Medicine. It is over a decade old, but contemporary to some of the events described in this post, and still quite relevant.

Suffice it to say that Dr. Weil is one of the most successful and well recognized popularizers of alternative medicine. He has authored or coauthored dozens of books. His website sells everything from baby pacifiers to vitamins to breakfast sausages, packaged bearing his name and/or visage. He is an altmed rockstar. He has been a key player in the branding of alternative medicine. In particular, been an advocate of “integration” of traditional and alternative medicine. He has created and exported residency training programs, and more recently proposed board certification in integrative medicine.

I recently read a book entitled On Being Certain: Believing you are right, even when you’re not, by Robert Burton, nicely-reviewed and recommended by Harriet Hall. In his book Dr. Burton excerpted an interview with Dr. Andrew Weil, pointing out Dr. Weil’s profound certainty about the effectiveness of a particular alternative treatment in spite of contradictory evidence. Dr. Hall also discussed this section of the book in her review. I found the excerpts fascinating and decided to delve more deeply into the interview. I also found another interview with Dr. Weil relevant to his ideas about evidence.

An interview with Dr. Weil (among others) was incorporated as supplemental material to a PBS Frontline episode called: The Alternative Fix. The program was pretty well done, and prominently featured the skeptical views of emeritus SBM editor Wallace Samson, MD. This was the interview quoted by Robert Burton.

Early in the interview, while explaining the appeal of alternative medicine, he brandishes a common mantra of of the “alternative” community:

I also think that a lot of people seek out alternative practitioners in frustration. If their first choice were available, it would be to go to a medically trained person, an MD who was open-minded and had knowledge of things beyond conventional medicine and could advise them about how to use them.”

The implication, of course, is that those who accept the claims of alternative medicine possess the virtue of open-mindedness. This is demonstrably false, and Dr. Weil is about to prove it. Credulousness is not synonymous with open-mindedness. Later in the interview he relates a clinical observation:

I wrote up my experience with an old osteopath in Tucson, who was a master of method called cranial therapy. He would take a kid, one treatment of this very noninvasive, inexpensive method and they would never get another ear infection. I saw this again and again.

I have recommended in my writings on my website that kids with ear infections should go to osteopaths and get this method done.

Cranial manipulation fails by all science-based criteria. Think of it as chiropractic manipulation of the bones of the skull, only with less credibility. It is based on assumptions that have no basis in scientifically established anatomy or physiology. It also fails by evidence-based criteria. The British Columbia Office of Health Technology did a systematic review of craniosacral therapy, and found no evidence of efficacy.

Dr. Weil laments the fact that researchers and mainstream medicine ignored his observations:

After something like twenty years of trying to get the research community interested in this, we finally set up some tests of doing this with kids with recurrent ear infections.

Despite the lack of a plausibility and absence of supporting clinical data, the Office of Alternative Medicine, and it offspring, the National Center for Complementary and Alternative Medicine, came to his rescue and funded a randomized clinical trial of cranial manipulation (full text). The study was designed to investigate two therapies for the prevention of relapse in recurrent otitis media (middle ear infection) in children. In addition to cranial manipulation, they also studied the popular herbal supplement Echinacea pupurea. The methodology utilized a double randomized (factorial) study design. This is a legitimate study design that can answer 2 questions with one trial. 90 children with recurrent otitis media were enrolled. The study was blinded, to the extent that patients and their families were not informed of their randomized assignment. Half were randomized to cranial therapy. The control group received osteopathic examination without manipulation. The same 90 children were independently randomized to receive Echinacea or placebo.
Before getting to the results of the study, I will provide some excerpts from an article about Dr. Weil written at the time the grant was awarded.

Weil would not specify a success rate that he considers optimal for the study, but said that if the results were not positive, he “would be reluctant’” to continue using these methods in his clinic….

If he does not get the positive results he anticipates, however, Weil said the structure of the study would probably be more to blame than the treatments themselves.

“Either the testing was wrong or we had the wrong practitioners,” Weil said. “The first question would be whether we designed the experiment the right way.”

He is so certain in the effectiveness of cranial therapy that he is preemptively impugning the quality of a study that would dare fail to confirm his faith in cranial therapy. This theme will be replayed, shortly.

The results of the study:

Interestingly, patients who received Echinacea actually did worse than patients randomized to placebo. Unsurprisingly, cranial therapy did not improve the outcomes of children with recurrent otitis media.

There is a very curious aspect to this study. In the paper Dr. Weil is acknowledged as one of 2 principle investigators (PI) on the study, but neither he nor his co-PI is listed among the authors. In general, the principle investigator(s) is the driving force and team leader for a research program. He writes (or at least supervises the writing of) the proposal for the research grant. If the grant is found deserving, the funding is in awarded in the name of the principle investigator(s). The PI should be the prime mover in the design and execution of the study, as well as the analysis and presentation of the data. It is quite irregular for a PI not to be an author on a publication of the primary results of a research study. According to Dr Weil, he worked for 20 years to accomplish this study. We can only speculate why he chose not to appear as an author on a paper.

One has to give a great deal of credit to Dr. Weil for his successful navigation of the scientific method. He made an observation: Children who receive cranial manipulation seem to have fewer episodes of otitis media.

He formulated a hypothesis: Cranial manipulation may be effective in reducing recurrences in children with otitis media.

He designed a study to test his hypothesis. The study was funded and performed. The results were published. All kidding aside, this was not a trivial accomplishment. Dr. Weil comments on the study in the Frontline intervwiew.

He concludes:

We were unable in those tests to prove that this [cranial manipulation] had an effect.

Gathering evidence and reporting data are critical activities, but the real reward of research is seeing new results become integrated into patient care. In the light of the results of a randomized clinical trial, how have Dr. Weil’s opinions about the benefits of cranial therapy for prevention of recurrent otitis media changed?

The problem is, I’m sure there’s an effect there. We couldn’t capture it in the way we set up the experiment.

Dr. Weil may have some legitimate grounds to be critical of the study: Was it adequately powered? Were study treatments done properly? Were treatment group assignments maintained? Were endpoints captured properly? It is the responsibility of an investigator to consider and discuss all of these things in critiquing his own study.

However…

Considering that the only evidence for the benefit of cranial therapy is an unpublished, self-selected, anecdotal series; when faced with the results of a negative randomized clinical trial, an objective, open minded person must at least concede the possibility that the study failed to document an effect because… it doesn’t work! Dr. Weil, however, is undaunted: “I’m sure there is an effect there.” He leaves no room for any other conclusion. He anecdotal experience outweighs the data from a randomized trial. The science must be wrong.

Now, armed with the data he awaited for 20 years, he must have been relieved to have some evidence to inform his advice regarding treatment of otitis media in children. In response to his own study, he has concluded:

…I can see no harm in recommending to people that they try cranial therapy from a qualified osteopathic physician, even though we still have not yet been able to verify this in a randomized controlled trial.…

I checked his website, and indeed, found this entry (accessed 8/28/12).

I have described the use of an osteopathic technique called craniosacral manipulation to end recurrent ear infections in children. I recommend this treatment highly.

He also advises putting warm oils into the ear canal, and opines:

Sometimes this treatment, combined with oral doses of echinacea, will also end a middle-ear infection, if you get on it at the very start.

Yes, despite his own study suggesting that echinacea was harmful in otitis media, he is still promoting its efficacy on his website.

Proponents of science-based medicine are often accused of being closed minded for eschewing implausible and unsubstantiated treatments. It is implicitly or explicitly stated that altmed practioners and promoters are open minded. Neither statement is necessarily true, or false. The fact is, one cannot define whether an individual is open minded or close minded based on a snapshot of an opinion on any given subject at any particular time. One must analyze how they came to reach their opinion and how they adjust their opinions base on new information. Does their opinion change when presented with credible, contradictory evidence?

In this instance, Dr. Weil has demonstrated unwillingness to change his opinion or his behavior based on new information. Even when the evidence, his own evidence, betrayed his hypothesis, he was unwilling to acknowledge even the possibility that he might be wrong. He has continued to promote an implausible treatment which failed to demonstrate benefit in a randomized trial, and even continues to recommend a treatment that was associated with outcomes worse than placebo.

Posted in: Chiropractic

Leave a Comment (24) ↓

24 thoughts on “Andrew Weil Flirts with Evidence Based Medicine

  1. tmac57 says:

    Ironically,even if it were real,craniosacral therapy would not work on Dr.Weil,since he is clearly too hard headed.

  2. BKsea says:

    I have been readin Nate Silver’s book on signal versus noise. One of the interesting concepts it introduced to me was that of “hedgehogs” versus “foxes.” A hedgehog basically has very high confidence in ideological truths and is unwilling to shift in the light of evidence, whereas foxes have nuanced views and a willingness to change opinions in the light of new evidence.

    What is interesting is that while foxes tend to be “right” more often than hedgehogs, most people seek out hedgehogs for direction. In essence, the misplaced confidence of the hedgehogs gives them an air of authority.

    It struck me that this is a perfect description of medicine. SBM is run by foxes, whereas Weil is clearly a hedgehog.

  3. rwk says:

    I don’t know what David Weinberg’s credentials are but he’s certainly proved he hasn’t even begun to research
    his article before writing.
    First of all Craniosacral Therapy which is practiced by Massage Therapists,PTs,OTs and the dreaded DC is not the
    same thing as Cranial Osteopathy performed by Osteopaths. They are not the same technic not even close.
    The review Weinberg references is about Craniosacral Therapy is the Upledger,D.O technic for Non-DOs.
    MTs,PTsOTs and DCs would not be offered training in Cranial Osteopathy.
    And this is another authoritative article on SBM?

  4. WilliamLawrenceUtridge says:

    …completely missing the point that none of these techniques have been demonstrated to work.

    “Rearranging the deck chairs on the Titanic” should be retired, and replaced with “correcting the name of the pseudoscience”.

    The overall point of the article is that Weil places his beliefs before science, to the detriment of not just his patients, but anyone who listens to him. Have any comments about that?

  5. WilliamLawrenceUtridge says:

    Heh, also note quote:

    Cranial osteopathy (also called cranial therapy or craniosacral therapy) is one variety of osteopathic manipulative therapies. It stimulates healing by using gentle hand pressure to manipulate the skeleton and connective tissues, especially the skull and sacrum (the large, triangular bone at the base of the spinal column).

    From Weil’s website:
    http://www.drweil.com/drw/u/ART03223/Cranial-Osteopathy-Craniosacral-Therapy.html

  6. Harriet Hall says:

    Quackwatch reports:
    “In 2002, two basic science professors at the University of New England College of Osteopathic Medicine concluded:
    Our own and previously published findings suggest that the proposed mechanism for cranial osteopathy is invalid and that interexaminer (and, therefore, diagnostic) reliability is approximately zero. Since no properly randomized, blinded, and placebo-controlled outcome studies have been published, we conclude that cranial osteopathy should be removed from curricula of colleges of osteopathic medicine and from osteopathic licensing examinations”

  7. David Weinberg says:

    @rwk

    I confess to being sloppy with the terminology: Cranial Osteopathy vs Craniosacral Therapy.

    As WLU pointed out, Dr Weil also seems to use the terms interchangeably. Even in reference to treatment for otitis media, he states on his website: “I have described the use of an osteopathic technique called craniosacral manipulation to end recurrent ear infections in children. I recommend this treatment highly.”

    I stand behind my inclusion of the systematic review from The British Columbia Office of Health Technology. Although the title uses the term “craniosacral therapy”, the review is much more comprehensive. Look over the definitions, search strategies, and references. The authors clearly set out to evaluate the literature on claims about the physiology of the cranial bones, cerebrospinal fluid, and health claims about supposed manipulation of these structures. They include numerous references from the osteopathic literature, including those which predate Upledger’s contributions.

    rwk, you have pointed out a flaw in my article. When I get a few minutes, I will clean up the terminology (at least those parts that aren’t direct quotes from Dr. Weil). Please note that Dr. Weil’s website is still promoting treatments that were discredited in his study performed 10 years ago and published 4 years ago. That’s the point!

  8. rwk says:

    OK. Hall’s reference made it complete.

  9. Dr. Weinberg is not responsible for any sloppiness with the terminology, real or perceived — that would be me. As SBM proofreader, I changed his terminology based on my own good knowledge of this topic. I think my choice was defensible, but I’m not commenting to debate it, just to claim it as my choice, not Dr. Weinberg’s. My apologies to everyone for causing a distraction from the substance of the post.

    And now, back to the science of skull bone wiggling …

  10. WilliamLawrenceUtridge says:

    …and again – it’s not like the mistake was between X blatant quackery and Y justified, empirically validated option. It was a completely female condition.

    Because it’s XX.

    Because X is the blatant quackery option.

    What I’m saying is at no point was Dr. Weil endorsing an empirically justified option.

    Dr. Crislip would have done something great with that, far better than this punny mortal.

    Sic.

  11. tmac57 says:

    WLU-You seem to be suffering from paronomasia. You should get that checked out.

  12. BillyJoe says:

    tmac, I do believe you have made a misdiagnosis.
    He said he was sic, but that doesn’t mean he has a disease.

  13. tmac57 says:

    BJ-

    tmac, I do believe you have made a misdiagnosis.

    Perhaps,but just to be on the safe side I think he should get the punjab.

  14. Alia says:

    When I was a kid (about 4 or 5, I guess), I used to suffer from recurrent otitis media. I got antibiotics, of course, but apart from that my mom used to heat a swab of cotton wool using an iron and put it close to my ear. And I stopped getting otitis media. I’m sure it was antibiotics that helped but when I was a kid, this hot cotton wool made me feel better. And it has one distinct advantage over cranial osteopathy/craniosacral therapy – it is virtually free (if we forget the cost of cotton wool).

  15. lilady says:

    So Weil is still recommending pediatric cranio sacral therapy for recurrent ear infections, eh?

    A while back when I was “slumming” on the Ho-Po I came across a blogger who was touting cranio-sacral therapy for all newborn infants and was shilling for a quack cranio-sacral therapist that she had interviewed…

    “…..I recently chatted to Julie Rimmington, one of Italy’s top therapists. She graduated from the University of Wisconsin in radiotherapy and for the last 20 years has been dedicating herself to Biodynamic Craniosacral Therapy — with international certifications of the highest levels…..”

    http://www.huffingtonpost.com/juliet-linley/craniosacral-therapy-for-_b_827495.html

    Seizing on the opportunity to debunk this therapy and the *credentials* of the cranio-sacral therapist, I posted at the blogger…

    “What utter nonsense! I visited the University of Wisconsin website and they do offer courses in radiotherapy..for cancer radiation treatments technicians and for engineers who are designing machines and setting up radiation treatment facilities. Even if Rimmington graduated with a “degree” in radiotherapy, it has nothing to do with her “international certifications of the highest levels in biodynamic craniosacral therapy. BTW which universities here or abroad have undergrad or graduate programs that qualify a personal for certification in “biodynamic craniosacral therapy” and what are the names of the certifying “boards.”

    I googled “Biodynamic Craniosacral Therapy” and lo and behold, found a website with an “interesting” article by Michael Shea about “getting permission from the infant…before you provide therapy” and other pseudo-science.

    New parents, save your money and don’t let any quack therapists near your precious infant.”

    Michael Shea whom I referenced in my Ho-Po post and who claims to be an infant “cranio-sacral therapist” has his own website. Take a look at the sheer unadulterated bullsh!t on his website:

    http://www.craniosacraltherapy.org/Babies.htm

    “….The therapist must find out the story of the birth but without the infant present. Otherwise it can be retraumatizing to the infant. Remember, infants hear and understand everything but in their own way. Infants experience and process the world with their whole body…..”

    “….The infant needs to be fully informed of the purpose of his or her visit with the therapist. The craniosacral therapist must speak in the first person to the infant. “Do you know what I do?” The therapist explains to the infant what they do and who they are. Infants are quite capable of responding to this information except with certain kinds of shock. They often shake their head knowingly from side to side to indicate “no”….”

    “….Shock babies will often become distressed quite suddenly during play or contact with the therapist. If this activation occurs spontaneously the therapist acknowledges that to the infant, “Yes that is a very difficult place and you have every right to be upset”. Everyone’s feelings must be acknowledged in some way for deep healing to take place. This is especially true of the infants feelings and emotions. If the infant has an anger response then its important to acknowledge him or her by saying, “I see you’re angry” or “I see you’re happy”, etc. The therapist must accurately reflect the feeling states that are evoked in the infant during the treatment. The shock that results from medical interventions during the birth process not only must be acknowledged but also requires an apology. As a therapist I often find myself saying to an infant “I’m sorry that happened to you when you were born”…..”

    ” In part two I will continue to discuss the next themes for a therapist to explore with an infant in craniosacral therapy, which are the four stages of birth, conjunct sites and conjunct pathways. In part three I will continue to discuss the biodynamics of the ignition process in the fluid system of the infant during birth and evaluating specific biomechanical parts of the whole body and craniosacral system. Finally in part four I will discuss emotional development in infants. I am most grateful for my mentors, Ray Castellino and Franklyn Sills, for showing me a common sense approach to treating infant….”

    Don’t even bother reading Shea’s Part two, Part three and Part four….It’s more of the same bullsh!t.

  16. Sam Homola says:

    Differentiating nonsensical chiropractic sacro-occipital technique from nonsensical osteopathic cranio-sacral therapy:

    http://www.sotousa.com/wp/?page_id=14959
    http://www.quackwatch.org/01QuackerytRelatedTopics/cranial.html

  17. lilady says:

    @ Sam Homola:

    How about this quackery from The Netherlands, Sam?

    http://www.ebm-first.com/craniosacral-therapy.html

    ” Infant Dies After Craniosacral Therapy: Therapist Gets Off Scot–Free

    Update on the link below about a formerly healthy, three-month-old girl, who died in the Netherlands after manipulation of the neck and the vertebral column (called “holding”) by a craniosacral therapist. The case was investigated by the Netherlands Health Care Inspectorate and the Public Prosecutors Office. The Inspectorate decided not to press charges against the craniosacral therapist when he promised never to apply this treatment again. The Dutch Public Prosecutors Office is still investigating the case and may yet decide to prosecute. Anaximperator Blog (17th January 2010)”

    And,

    “Infant Dies after Craniosacral Therapy

    A formerly healthy, three-month-old girl died after manipulation of the neck and the vertebral column by a so called “craniosacral therapist.” During continued and deep bending of the neck, the patient developed incontinence of faeces, atonia and respiratory arrest followed by aystolia. Based on findings at the physical examination of the body, an additional MRI examination and the autopsy, it is likely that the cause of death was a local neurovascular or a mechanic respiratory-induced problem. This is the second report of infant death after forced manipulation of the neck. As long as there is no scientific evidence for the efficacy and safety of forced manipulation of the neck and the vertebral column, we advise against this treatment for newborns and infants.” English translation of a report in the Dutch Medical Journal (May 2009) [Micha Holla, Marloes M. IJland et al. Ned Tijdschr Geneeskd, 2009]“

  18. Sam Homola says:

    Thanks, lilady. The link to “Craniosacral Therapy” on the ebm-first.com site provides an important and valuable collection of science-based reviews on the subject of cranial manipulation.

  19. Jan Willem Nienhuys says:

    A formerly healthy, three-month-old girl died after manipulation of the neck and the vertebral column by a so called “craniosacral therapist.”

    Actually the therapist was sentenced to a six months of prison (suspended sentence with probation time of two years) and also had to pay the cossts of the funeral. The therapist has appealed. Among the reasons for suspending the sentence the court mentioned:
    - therapist didn’t have the intention to hurt the victim
    - therapist has immediately stopped his practice
    - therapist already has suffered a lot of negative criticism.

    See http://zoeken.rechtspraak.nl/detailpage.aspx?ljn=BU9133

    Craniosacral therapists universally have said that this manner of treatment (basically folding over babies until they become quiet – my lay opinion: because of lack of oxygen, heaven knows how many babies in his care suffered brain damages that showed up much later without the parents ever making the link) wasn’t craniosacral therapy at all. It was some addition he picked up somewhere.

    So the total sum of this case is that if you go to an alternative healer, you’ll never know whether what he does is mostly harmless or quite lethal, and when it goes wrong it’s the state’s burden to prove a causal link, which may be very hard.

    Compare that with the way safety in traffic is enhanced: if a camera catches you speeding you are fined irrespective of whether you have killed someone or not.

  20. DevoutCatalyst says:

    “Craniosacral therapists universally have said…some addition he picked up somewhere.”

    Craniosacral itself being an add-on merit badge for the eclectic healer, eclecticism being akin to godliness for some in the mumbo-jumbo healing professions. Maybe I’d be more sympathetic to a denial of affiliation with this particular schmuck if Big Craniosacral didn’t hold such a compelling desire to prostitute itself to all comers. For the end user, craniosacral is purchased as a pig in a poke, no telling how many craniosacral therapists are and have been folding babies.

  21. pmoran says:

    From a systematic review at http://www.ncbi.nlm.nih.gov/pubmed/14520089

    “Children entered recurrent AOM trials with a mean rate of 5.5 or more annual episodes but averaged only 2.8 annual episodes while on placebo (95% CI, 2.2-3.4). No AOM episodes occurred in 41%, and only 17% remained otitis prone (3 or more episodes).”

    The present study revealed the same. Over the six month period of the study about 40% of children had no further episodes of otitis media, regardless of what treatments they were using, even though they all had three or more episodes in the preceding six months. The outlying, low 20% figure for “no further episodes” in those given Echinacea is likely to be a s glitch rather than an adverse effect, I suspect.

    This rate of spontaneous resolution makes it easy to encounter a small cluster of favorable outcomes for any treatment at all. Then the usual confirmation biases come in, including looking for excuses whenever it doesn’t work — “did you miss out on any treatments, or not follow my other advice” Those odd cases with exceptionally compelling stories help trap the unwary mind (these are also encountered within the placebo arm of the controlled trials and are not always easy to explain away.)

    This is the stuff that CAM is made of, when not frankly fraudulent. Clinical studies were never intended to be a tool with which to persecute CAM supporters. They arose because we found we could not trust ourselves to make the right decisions as to which treatments “work”, or “work” best via intrinsic efficacy.

  22. lilady says:

    @ Sam Homola:

    I’m happy I was able to provide you with more information from the ebm-first.com website about this quackery visited upon newborns.

    Jan Willem Nienhuys, thanks for the additional information about the cases in The Netherlands. Paying for the infant’s funeral costs, is small comfort for the parents who are now left with their guilt and only memories of their dead baby.

    What if a medical examination found these horrendous injuries in an infant where it was determined that a parent/caregiver inflicted those injuries. Would that murderer be given a slap on the wrist is (s)he promised not to harm another infant?

Comments are closed.