Articles

The Hazards of “CAM”-Pandering

Steven Salzberg, a friend of this blog and Director of the Center for Bioinformatics and Computational Biology at the University of Maryland, is on the editorial boards of three of the many journals published by BioMed Central (BMC), an important source of open-access, peer-reviewed biomedical reports. He is disturbed by the presence of two other journals under the BMC umbrella: Chinese Medicine and BMC Complementary and Alternative Medicine. A couple of days ago, on his Forbes science blog, Dr. Salzberg explained why. Here are some excerpts:

The Chinese Medicine journal promotes, according to its own mission statement, studies of “acupuncture, Tui-na, Qi-qong, Tai Chi Quan, energy research,” and other nonsense. Tui na, for example, supposedly “affects the flow of energy by holding and pressing the body at acupressure points.”

Right. What is this doing in a scientific journal?… I support BMC…But their corporate leaders seem to care more about expanding their stable than about maintaining the integrity of science. Chinese Medicine simply does not belong in the company of respectable scientific journals.

Forming a scientific journal whose goal is to validate antiquated, unproven superstitions is simply not science, whatever the editors of Chinese Medicine claim.

BMC should be embarrassed to be publishing journals that promote anti-scientific theories and otherwise muddy the literature. By supporting these journals, they undermine the credibility of many excellent BMC journals. They should cut these journals loose.


Matthew Cockerill, a molecular biologist and self-described skeptic who is Managing Director of BMC, responded. Quoting Tim Minchin’s poem Storm (“You know what they call alternative medicine that’s been proved to work?…Medicine”), he wrote:

Well-known examples of “alternative medicines” that have made the transition to “medicine” include aspirin (explicitly cited in ‘Storm’) and artemisinin, a compound identified in a type of traditional Chinese herbal medicine, derivatives of which are now used to treat malaria.

Clearly for this transition to be possible, there must be a way for effective aspects of Complementary/Alternative Medicine (CAM) to prove themselves and to become “Medicine”. Journals focused on evaluating CAM are not the only approach to doing this (such research might also be published in a regular medical journal), but they are one valid approach…

Regular medical journals may tend to be dismissive of anything with CAM-associations…

In a second comment (which at the time of this writing is unavailable, but Dr. Salzberg sent me a copy) Dr. Cockerill continued:

Steven,

In your post you suggest that journals relating to complementary/alternative medicine (CAM) including Traditional Chinese Medicine have no place in BioMed Central’s portfolio, and your broader implication is that these fields represent nothing but pseudoscience and are unworthy of (or indeed are incompatible with) scientific investigation.

BioMed Central does not share this view. We accept that this is a challenging and controversial area, and that one of the problems to be addressed is that much CAM research is lacking in scientific rigor. However, to dismiss all aspects of CAM out-of-hand and to exclude research on CAM from scientific discourse would only give ammunition to those who try to characterize the scientific process as being close-minded and based on presumptions which are not open to question.

You refer to BMC Complementary and Alternative Medicine as a “pseudoscience journal” which “promote[s] anti-scientific theories”, but this is really not the case.

Dr. Cockerill argued that standards of peer-review and methodological rigor for that journal are not different from those for the rest of the BMC group. He named Edzard Ernst as a member of the editorial board. More:

Moving to the field of Traditional Chinese Medicine – while it is true that there are some aspects of TCM as traditionally practiced that are likely to hold only anthropological/psychological interest from a scientific point of view, it is also clear that there are many active ingredients in traditional Chinese Herbal Medicine (CHM) which are worthy of further investigation in terms of biological mechanism and clinical effectiveness. This is an area that has attracted significant funding from commercial and public sources – see for example this article from Science magazine in 2003. http://www.sciencemag.org/content/299/5604/188.long

Looking at articles published in the journal Chinese Medicine, the evaluation of active components within herbal remedies accounts for a large fraction of research articles published, and this cannot easily be dismissed as either irrelevant or inherently pseudoscientific.

In an earlier email to Steven Salzberg, Dr. Cockerill had written:

The major problem I have with such a blanket dismissal is that it lumps everything together as being equally valueless, whereas the goal of the scientific process is to winnow the wheat (however rare) from the chaff…

If you look at articles published in Chinese Medicine, you will find little if anything on “Energy fields” (not a fruitful area for scientific investigation, I quite agree, and not something which I believe should be in the scope of the journal), but a great deal on biochemical and functional genomic work in relation to traditional herbal medicine, which is quite a different matter.

Bait and Switch: the Yin and the Yang

Sigh. Where to begin? Let’s start by trying a slight revision of Dr. Cockerill’s own words: The major problem I have with such a blanket acceptance is that it lumps everything together as being equally valuable, whereas the goal of the scientific process is to winnow the wheat (however rare) from the chaff…

That’s more like it. I imagine that Dr. Cockerill will find that statement unfair, in that it fails to acknowledge that there may be reports in the two journals that disconfirm hypotheses that are not fruitful for scientific investigation (if you catch my drift). I concede that, but the point here is a different one: Dr. Salzberg objects to including anti-scientific topics in the journals precisely because doing so gives the appearance that real scientists believe that they must be intriguing enough to warrant space in real biomedical journals.

But they are not: it is clear, after a moment’s reflection, that natural products research is the ONLY worthwhile scientific endeavor within the editorial purview of Chinese Medicine, so why pretend that it justifies the rest? Matthew Cockerill may agree that “energy fields” should not be included; nevertheless they are—which was Dr. Salzberg’s point. Tim Minchin, bless his heart, probably doesn’t realize that his statement about “alternative medicine that’s been proved to work” is itself purely hypothetical: there has yet to be an example of such a medicine. Dr. Cockerill ought to know that, as I will explain a bit later in this piece.

An Article Brings Homeopathy to Mind

I have little time to investigate the non-[natural products pharmacology] articles in Chinese Medicine; Dr. Salzberg cited one, which he characterized as “a laughably bad study.” I’ll tell you a bit about another, just published in January:

Misdiagnosis and undiagnosis due to pattern similarity in Chinese medicine: a stochastic simulation study using pattern differentiation algorithm

The article is nearly incoherent. Passages such as this abound:

In Chinese medicine, diagnosis is also important. Practitioners recognise and label nosological conditions based on inspection (Ip, wang), auscultation and olfaction (AO, wen), inquiry (Iq, wen) and palpation (P, qie), also known as the Four Examinations (Sizhen). According to traditional literature, these methods should be applied in order to enhance recovery of the patients. Manifestations (ie signs and symptoms) collected from patients are interpreted using Chinese medicine theories (eg eight principles, five phases, vital substances, six channels, four levels, triple burner and Zangfu), which were developed on the basis of some observations of Nature. Similar to Western medicine, the collected manifestations are interpreted collectively; however, diagnosis is established through a pattern differentiation process whereby a unique, stable manifestation profile is obtained for the identification of a pattern among other diagnostic hypotheses.

The article appears to torture statistics to show that each additional Examination among the Four confers a slightly greater degree of certainty that the examiner will not confuse two different “diagnoses” (in the TCM sense of the term). At least I think that’s the point. But the investigator didn’t determine this by testing actual practitioners, as far as I can tell; he did it using “a stochastic computational simulation based on Monte Carlo method implemented for patient simulation from ZFSP in a dataset.” And a good thing, too, because if there’s anything useful to be gleaned from this paper, it’s found in references 19-24. They show, unsurprisingly, that there isn’t much agreement from one TCM practitioner to the next about either “diagnoses” or treatments.

Hmmm. No surprise when you consider what those diagnoses, i.e., patterns, involve. Here is the pattern of “Deficient Cold of Small Intestine…often discussed as Deficient Spleen Qi,” according to Ted Kaptchuk, an American who seems to have invented the “OMD” degree that won him a professorship at the Harvard Medical School, and who is on the editorial board of Chinese Medicine:

Signs: slight, persistent discomfort in lower abdomen; gurgling noises in abdomen; watery stools.

Tongue: pale material; thin white moss

Pulse: empty

Here is the pattern of “Deficient Cold in Stomach…often called Deficient Spleen Yang”:

Signs: chronic diarrhea; slight persistent pain in epigastrium; discomfort relieved by warmth, eating, and touching; shy; easily influenced by others

Tongue: pale material; most white moss

Pulse: deep or moderate without strength

And so on, for hundreds of patterns. According to Kaptchuk, “pulse examination can be the most important of the Four Examinations and is crucial to pattern discernment in general.” Paul Unschuld, a prolific historian of Chinese medicine, explains how this is done:

The physician has the choice of either feeling the (radial) pulse with three fingers or with one finger. Using three fingers, light pressure with the index finger above the imaginary line level with the styloid process allows him to feel the state of the lung and heart, greater pressure with the middle finger on the imaginary line enables him to feel the state of the spleen, and greater pressure still with the ring finger allows him to feel the liver and kidney. According to a third (and certainly not the last) variant, he can, with one finger, apply a pressure equal to the weight of three beans to feel the lung, a pressure of six beans to feel the heart, a pressure of nine beans to feel the spleen, a pressure of twelve beans to feel the liver, and finally a pressure that brings the fingertip almost to the bone, to feel the kidney.

What does “correct” pattern recognition mean, in the practical sense? Well, it means that the correct therapeutic intervention will occur. Kaptchuk again:

The goal of all treatment methods in Chinese medicine is to rebalance those aspects of the body’s Yin and Yang whose harmonious proportion and movement have become disordered…inappropriate anger such as that characterized by excessive Liver activity must be calmed…insufficient activity, say of the Kidney Yang, must be tonified to avoid lack of sexual energy…If there is not enough Qi in the Lungs, it must be replenished…If the Qi in the Spleen descends, causing chronic diarrhea, it must be lifted; if the Qi of the stomach ascends, it causing nausea, it must be sent down. Stagnant Qi must be moved…Too much Cold in the Kidney must be warmed; extra Fire in the Lungs must be cooled. Whatever is out of balance must be rebalanced. The complementary aspects of Yin and Yang must be harmonious.

Here’s how it’s done:

…the insertion of very fine needles into points along the Meridians can rebalance bodily disharmonies. A related technique [is] moxibustion…The action of the needles or of moxibustion affects the Qi and Blood in the Meridians, thus affecting all the fundamental textures and Organs. The needles can reduce what is excessive, increase what is deficient, warm what is cold, cool what is hot, circulate what is stagnant, move what is congealed, stabilize what is reckless, raise what is falling, and lower what is rising.

OK, enough already. Is it not obvious that this is the classic assortment of metaphors, myths, traditions, and appeals to authority that is common to the medicine of pre-scientific cultures? Were Chinese medicine theories developed on the basis of some observations of Nature? I don’t think so.

The claims regarding pulse examination are quite fanciful, to say the least: that changing the pressure of the examining finger would allow the practitioner to feel the state of the various, named organs is hard to conceive, given what is now known about anatomy and physiology; it’s also clear that pre-modern Chinese physicians would have had no way test this claim. There is a smattering of empirically gathered symptoms and signs that are based on history taking and external examinations, some of which are repeatable and might have eventually become useful, if pre-modern Chinese medicine had pursued anatomy, physiology, and pathology before those fields were developed elsewhere—thus making that aspect of “TCM” moot.

Funny: the “patterns” remind me of another conspicuous pre-scientific “school” of medicine: homeopathy, with its elaborate ‘symptom’-gathering scheme, its voluminous repertories, rubrics, and simillima. And, of course, its lack of agreement from one homeopath to the next regarding the ‘correct’ constellation of ‘symptoms’ and therefore the correct ‘remedy.’

Pharmacognosy is neither “CAM” nor “Chinese Medicine”

Dr. Cockerill’s other points are a mixture of reason and misunderstanding. Legitimate natural products research is not “CAM,” and should neither be billed as such (or as some other misnomer such as “Chinese Medicine”) nor should it be purported to justify “CAM,” which, if anything, hinders such research. Aspirin is not an example of an “alternative medicine that made the transition to medicine” any more than quinine or atropine or digoxin are such examples. Nor is artemisinin. Everyone with a modicum of education knows that many useful drugs have been, and will continue to be, derived from natural sources, exactly as biology would predict. This is completely unsurprising. The term “alternative medicine,” along with its various synonyms, refers to a recent political and quasi-religious movement, not to a longstanding branch of drug development that is scientific and that has not required the help of pseudoscientific zealots to pursue its investigations.

If Chinese Medicine were limited to pharmacognosy and had a more appropriate title—Asian Pharmacognosy, for example, referring to the geographic origin of the products studied, not to some pre-scientific medical ‘system’—I’d have little problem with it, assuming that the natural products reports are as rigorous as Cockerill claims them to be.

“Chinese Medicine” is a misnomer in several ways. First, the medicine overwhelmingly used in China today is modern, scientific medicine. Second, pre-scientific Chinese medicine was not one or even a few schools of medicine but many disparate schools, occurring at various times over millennia and in various places over a huge geographical region. Some schools were completely unrelated or even contradictory. There was much influence from abroad, including India and Greece. Third, the term “Traditional Chinese Medicine” was coined only about 55 years ago in the PRC, mainly for pitching to (gullible) Westerners. It heralded something that, paradoxically, had never before been true in China: a forced standardization, such as to make ‘Chinese Medicine’ appear to be something that it was not: a conceptual whole.

Last Thoughts

I’m running out of gas, but another straw man used by the “CAM” movement, repeated by Matthew Cockerill, is that “Regular medical journals may tend to be dismissive of anything with CAM-associations.” There was a time in the 1990s when editors of the major journals had to answer this charge on a regular basis, and each time it went something like this:

Dr. Siegel’s charge that medical journals will not publish studies of alternative medicine comes out of thin air. Journals compete avidly to publish important new clinical research, if it is rigorously done. As we indicated in our editorial, the problem with the studies funded by the Office of Alternative Medicine in 1993 was not their size, as suggested by Dr. Cherkin and Ms. Street, but their quality.

In fact, major medical journals have, to their own embarrassment and to the detriment of uninformed readers, bent over backward to accommodate substandard “CAM” treatises.

“CAM” advocacy journals, moreover, have become far more evident in the past few years, thanks in part to political pressure applied by anti-intellectual demagogues such as Dan Burton—even as the only journal committed to an appropriate, skeptical view of “CAM” was excluded from listing by the US National Library of Medicine after similar political pressure.

Perhaps someone else will look at some of the articles in BMC Complementary and Alternative Medicine. I have neither the time nor the inclination, but I would like to reiterate a point that Steven Salzberg and we have all made at one time or another: even the subsequent publication of a reasonable scientific report doesn’t justify creating a journal for the purpose of “helping aspects of CAM prove themselves.” If there’s scientific promise for some new way to solve a problem, and if a legitimate study is done, there are plenty of opportunities to report it in the medical literature.

Posted in: Acupuncture, Clinical Trials, Energy Medicine, Herbs & Supplements, History, Homeopathy, Medical Academia, Pharmaceuticals, Science and Medicine, Science and the Media

Leave a Comment (50) ↓

50 thoughts on “The Hazards of “CAM”-Pandering

  1. GLaDOS says:

    Dr. Salzberg is wrong when he says, “BioMed Central does not share this view,” as “views” are beside the point.

    The onus rule means anyone claiming a therapy works must prove it before we promote the therapy to the public.

    CAM *is* the promotion of unproven therapies to the public and so it violates the onus rule, among others.

    Dr. Salzberg does not appear brain injured generally, so his obvious failure to see the obvious is puzzling.

  2. pmoran says:

    Oddly for anyone deeply committed to CAM, Ted Kaptchuk has published a lot of quite sophisticated work on placebos. I have a lot of respect for his work; he asks the right questions.

    Does he actually practice TCM diagnosis and methods, or is the book you quote merely descriptive of TCM theory?

  3. @GLaDOS:

    It was Dr. Cockerill who wrote “BioMed Central does not share this view”—not Dr. Salzberg, who brought the complaint in the first place.

    KA

  4. windriven says:

    “Legitimate natural products research is not “CAM,” ”

    Amen. Physics does not indulge a “Journal of Contemporary Astrology” nor Chemistry a “Journal of Investigational Alchemy.” That there are good and sufficient reasons for this should be clear to anyone with a pulse. Why then medicine?

    The tacit endorsement of quackery is, to steal a phrase, a cancer on the noblest of professions. Medice, cura te ipsum!

  5. Ed Whitney says:

    I believe that money in the bank is a form of blocked energy. However, if you send me your bank account data, I will channel it from your account into mine, and will thereby release it into the universe in the form of abundant prosperity.

  6. Anthro says:

    Phew! This comes just in time for me to show to a New Age acquaintance I am seeing tomorrow. I can simply hand this to her rather than engaging in a game of back and forth every time she starts in with her woo. It’s very difficult to maintain these “friendships” and I’ve already dumped a bunch of them, but alas, the woo is widespread and I’m running out of “friends”.

  7. David Gorski says:

    Legitimate natural products research is not “CAM,”

    Hallelujah! I’ve said this time and time and time again. Legitimate natural products research is pharmacognosy, a longstanding and respected branch of pharmacology. Unfortunately, it’s been hijacked by the herbalists and CAMsters.

  8. I have the impression that

    a) most of the tradition-based uses of herbal products that have been tested have not been validated and

    b) most of the effective drugs that have been scientifically validated did not come from ivestigating traditional uses of plant products.

    Anybody have any solid information related to those assumptions?

  9. GLaDOS says:

    Thanks for fixing my name confusion, Dr. Atwood.

    “CAM” is just another way of saying, “the argument from ignorance.” Prove me wrong.

  10. Scott says:

    @ Ed:

    Gotta say, that’s a beautiful analogy! Definitely gave me a good laugh.

  11. JPZ says:

    I have the impression that the author is largely upset by the semantics of CAM, TCM, and other terms. Looking at the Chinese Medicine journal online, there are good papers with good quality science (and bad ones). It seems a call for revising the mission statement and editorial standards of this journal would be more productive than Dr. Salzberg questioning its existance – throwing out good papers in the process. That sounds like an over-reaction to me.

    Another semantic issue, willow bark was used as a pain reliever before science was applied to learning why. I tend to call that traditional medicine, but the author may prefer another name for it that doesn’t involve saying “medicine” (traditional treatments?). Based on traditional uses, pharmacognosy led us to aspirin. So, I see the value of examining “traditional treatments” for medical efficacy even if they come from CAM or TCM. If there is some political word war over how the term “medicine” can be used and by whom, then I think you are doing it wrong.

    (www.nutrition-industry.blogspot.com)

  12. Costner says:

    “[...] he can, with one finger, apply a pressure equal to the weight of three beans to feel the lung, a pressure of six beans to feel the heart, a pressure of nine beans to feel the spleen, a pressure of twelve beans to feel the liver [...]“

    I’d love to see a control test to see if these “practitioners” could consistently apply pressure to that level of accuracy – or better yet come up with the same results after reading the pulse. Then again it might matter what kinds of beans they are using… there is a difference between a soybean, coffee bean, kidney bean, and Mr. Bean.

    Are we really to believe with all the various tools and methodologies in practice today that we still need to rely upon the weight of a bean to be a guideline? Dont bother with all of those new fangled CT scans, MRI machines, or even blood pressure cuffs… we can determine everything via measuring a pulse with bean-like precision.

  13. daedalus2u says:

    Costner, you know it is the bean counters that really do run everything and decide which machines are really needed, and they do that by counting beans.

  14. JPZ says:

    @GlaDOS

    says “CAM” is just another way of saying, “the argument from ignorance.” Prove me wrong.

    To me, CAM is just a way of saying “here is something that many/few people think works.” As a scientist I say, prove it right or wrong. Don’t assume people who don’t agree with you are ignorant.

    (www.nutrition-industry.blogspot.com)

  15. JPZ says:

    @Brennen

    “Most” is a hard term to answer since it implies limits of time or effort. It would help if you set those, just to help me a bit.

    There are “many” natural products that have proved efficacious based on scientific evidence. From willow bark and the pacific yew tree to cranberries and saw palmetto.

    Pharmaceutical companies prefer to patent their drugs, so they look for unique molecules that are patentable. Even if there is a natural molecule that is efficacious, these companies will look for an un-natural molecule that is more efficacious.

    The hard line of distinction is difficult to find. Regulations and profit drive many of these decisions.

    (www.nutrition-industry.blogspot.com)

  16. @JPZ

    I don’t think “most” is that difficult a term here. If we looked at, say, 500 herbal remedies in the TCM tradition, for example, and searched for scientific evidence validating their claims of safety and efficacy, would we find 5, 10, 200 of them had strong scientific evidence to support their use? I suspect that if we required high-quality scientific data with proper controls, as we do to license pharmaceuticals, the number that met that standard would be negligible. So the argument that traditional use is a promising indicator of real efficacy that will be born out in scientific testing would fail on that point.

    Similarly, if we looked at the top 500 conventional pharmaceuticals prescribed today and traced their origins, how many would turn out to have been investigated because of traditional use? How many would have been used in folk traditions for the indication that they are now used for? Again, if the number is a tiny fraction, as I suspect, then that makes the argument that traditional use should be viewed as a reliable source of ideas to be tested scientifically pretty untenable.

    But I suspect we are starting with perspectives so different we may not even understand the issues int he same way. For example, cranberry strikes me as a terrible example, because it’s not an especially effective product (http://www.sciencebasedmedicine.org/?p=9805), and pacific yew was not a traditional cancer therapy but was part of a broad program of screening plant compounds for potential therapeutic value (http://en.wikipedia.org/wiki/Paclitaxel) and thus an example of scientific pharmacognosy rather than the value of traditional use of herbals.

    Of course, sicne herbals are traditionally used for a wide variety of unrelated complaints, almost any plant you can isolate a drug from for a specific indication was probably used for that indication in some tradition somewhere. But that’s just counting accidental hits and ignoring all the misses.

    The patent issue is a straw man since compounds derived from plants and purified/tested as drugs can be patented and money made from them, which does happen. You are assuming that use of unstandardized, unregulated, whole-plant products is as safe or effective, which is not true. And herbal medicines make plenty of money for the companies that produce them and the practitioners who sell them, so they are not an example of an altruistic, non-profit approach to medicine. If the cure for cancer is out there in a plant, someone will find a way to bring it to the public, and probably to make money from it.

    The failure of these remedies to be scientifically validated cannot simply be passed off as due to a lack of Big Pharma interest in researching them. NIH spends billions on CAM research, and the herb and supplement industry, as well as academia, spends plenty more, so if there is convincing data to be generated, there are ways it can be done. I will be happy to accept and make use of any traditional medicine that passes as rigorous a validation process as the typical pharmaceutical, and I bet mainstream journals would be happy to publish the trial.

  17. pmoran says:

    Many here will be interested in a recent editorial on the Cochrane site acknowledging SBM’s criticisms of it’s recent interpretation of EBM.

    http://www.thecochranelibrary.com/details/editorial/1034087/If-there-are-no-randomised-controlled-trials-do-we-always-need-more-research.html#fb1

  18. GLaDOS says:

    To me, CAM is just a way of saying “here is something that many/few people think works.” As a scientist I say, prove it right or wrong. Don’t assume people who don’t agree with you are ignorant.

    When you say, “some think treatment X works, but this has not been proven,” you are admitting ignorance.

    It is an error to justify a treatment on the basis of ignorance.

    Ignorance = doesn’t work.

    You can’t say something works until you have evidence that it does.

    The onus is always on the person claiming something works to provide the evidence.

    CAM is ignorance of effectiveness.

  19. Mark P says:

    As a scientist I say, prove it right or wrong.

    No dude, a scientist gives a hypothesis and proves it could be right. You don’t suggest a hypothesis and then tell the other side to prove it wrong. Or rather, you have to prove it right before you start to treat people.

    There would be little issue with CAM if the hypotheses were proposed based on alternative concepts, but solidly tested before they were used. But we all know the reality is the other way round: the “research” is there solely to justify treatments already in use.

    The duty of care is on the CAM community to prove treatments before use.

    Imagine if the rest of the world worked the way you propose. How much would you trust a car built by an “alternative” engineer? Would you prefer the local bridges to be built with “traditional” methods, or modern ones?

    The “prove it wrong” argument would appear ridiculous when applied to engineering or mechanics. That is because it is ridiculous. And is just as ridiculous in medicine.

  20. JPZ says:

    A few clarifications in response to your thoughtful comments:

    @Brennen – I changed your use of “traditional” to “natural” in my reply, creating confusion. My bad. And, maybe a patent attorney can correct me here, but I have been led to believe composition of matter patents on new chemical entities are much more clear cut to defend than use patents in the crowded natural products patent space. No straw man intent here. Cranberry was found partially effective in a 2008 Cochrane review and is recommended by ACOG.

    @ GLaDOS and Mark – Let me see if I can clarify my overly terse assertion. Let’s say someone tells me, “Rubbing a cat on your head cures headaches.” The fact that they believe this does not make it true or false. I can believe them or not (or even laugh), but I need objective evidence to know if they are right or wrong. If they are rubbing a cat against their head with no objective evidence it works or doesn’t, you can endlessly argue placebo effect, belief system, silliness or a heretofor unknown property of cats. In the absence of evidence, I can form a hypothesis that they are right or wrong and test it in a RCT. Then I could say the belief is supported or unsupported.

    If someone has a belief where there is no objective evidence for or against it, it is unscientific to automatically classify it as false (If they are clearly harming themselves, they need to stop regardless). If someone holds on to a belief after it has been tested and found false, they are not making decisions using reason and should not be engaged in debate. This blog does a great job of saying what beliefs are or are not supported. I’m just addressing assumptions here.

  21. S.C. former shruggie says:

    Legitimate natural products research is not “CAM,”

    Windriven

    Amen. Physics does not indulge a “Journal of Contemporary Astrology” nor Chemistry a “Journal of Investigational Alchemy.”

    Ever done a literature search at a University that subscribes to Elsevier? The signal to noise ratio is not good. Journals of Zodiac-based Alchemy abound.

    Dr. Gorski

    Legitimate natural products research is pharmacognosy, a longstanding and respected branch of pharmacology. Unfortunately, it’s been hijacked by the herbalists and CAMsters.

    I got that impression too, just from writing undergrad assignments. Noise outnumbered signal like spam to sausage in the famous Monty Python sketch.

    Cultural relativism is big these days, and framing science as “Western” gives fraud a sort of false-equivalence to anything called “Eastern.” Add to that connotations of “Western” as racist, wicked, mean and nasty, and you’ve got solid propaganda to spread woo word-of-mouth among my past arts and philosophy classmates. Including me.

    We need more publicity recognizing science as an international effort. When TCM is rightly called the culturally relative alternative to Western bloodletting and German New Medicine, one historically and geographically limited lie among many, arraigned against a global co-operative effort, it won’t sound so appealing to arts students.

  22. GLaDOS says:

    If someone has a belief where there is no objective evidence for or against it, it is unscientific to automatically classify it as false.

    Although unproven claims are not the same as disproven claims, for practical purposes they are the same thing.

    The set of unproven claims is infinitely large and so by default any single unproven claim must be ignored until proven.

  23. aaronupnorth says:

    I feel the editors at BMC should be evaluated for the condition deficient warmth of the head; often discussed as deficient brain QI. The practitioner of Chinese medicine can detect this state through olfaction, the smell of bovine offal, and treat this state through palpation of the forehead pulse (located between the eyes) using the pressure of ten thousand beans.

    Dr. J

  24. pmoran says:

    GlaDOS:”Although unproven claims are not the same as disproven claims, for practical purposes they are the same thing.

    I agree that holds, more or less, for medicine as a body of knowledge, but not for medicine in practice. When there are no other options it is quite rational to try out unproven or even quite unlikely methods.

    This is one of the ways in which we skeptics can lose the confidence of our rightful constituency. Patients cannot understand why we would not be on their side, prepared to try anything that might help, when they have a problem we cannot solve for them.

  25. GLaDOS says:

    When there are no other options it is quite rational to try out unproven or even quite unlikely methods.

    Still you should be ready to defend your decision to your peers as *reasonable* given current practice and understanding. And when the patient is a young child unable to consent you must take that into account.

    The alt med tribe develop their interventions on their own, without including people like me in the process. Over time their cultishness invites crank magnetism, resulting in a mental state indistinguishable from brain damage.

    I was largely a shruggie until I had to care for a child who was also seeing a DAN! doctor. I wish I could inflict that experience upon you somehow.

    The mystification of medicine is not good, nor is the mystification of “leadership” within the business world. The next stop for this crazy train will be the mystification of politics.

  26. JPZ says:

    @GLaDOS

    Thanks for the impetus to look this up! I enjoy learning new things.

    “Appeal to Ignorance” logical fallacy…

    “Because you have not proven me wrong, I am right.”
    or
    “Because you have not proven yourself right, you are wrong.”

    There are some exceptions. One is if a claim is so outrageously impossible, you can shift the burden of proof on to the claimant. I am guessing that this is what you meant.

    I am sure we could find plenty of claims that we both would agree are outrageously impossible. And, if we disagree on other claims, the world is a richer place from the discussion.

  27. pmoran says:

    Glados:I was largely a shruggie until I had to care for a child who was also seeing a DAN! doctor. I wish I could inflict that experience upon you somehow.

    As I said, I would be by no means neutral in such a case. If the parents insisted on trying it, and there was no way of legally intervening, I would try and get them to agree to a finite trial period so that the child’s exposure to steroids was minimal.

    Point out that adrenal insufficiency should respond rapidly to quite small doses of hydrocortisone.

    I am sure you will have done all that.

  28. Ed Whitney says:

    A tip of the hat to pmoran for pointing us to a discussion of some pertinent issues concerning some of the tensions between SBM and EBM. Sometimes EBM comes up with results that are counterintuitive to SBM, especially when specific mechanisms of action of some interventions are obscure or appear not to exist.

    Perhaps SBM, judging from earlier posts and threads, generally requires that a known biochemical, physiological, or anatomical mode of action of an intervention be articulated before that intervention can be taken seriously and results of clinical trials can be considered plausible. EBM, by contrast, is less particular about disease mechanisms and specific actions of interventions in the evaluation of clinical trials, and does not fuss when methodologically satisfactory trials produce results that have no apparent explanations in terms of physiology, pathology, and biochemistry. None of the Cochrane considerations for risk of bias deal with the need for a plausible biomedical explanation of the specific effects of the intervention under study.

    Laetrile is a funny example to look at. Some of us can remember the outrage that erupted when Franz Ingelfinger, stipulating that he thought laetrile was quackery, and that he would never take it himself for his cancer, approved the publication in the New England Journal of Medicine of a clinical trial of same. In the late 1970s, this was just as much a political as a scientific hot potato. For some reason, it was embraced mostly on the far right by John Birchers and other types, who also tended to oppose fluoridation of water and strove to prevent the sapping and impurification of all of our precious bodily fluids. The NEJM trial was clear-cut, but perhaps laetrile faded away as much because of its association with the far right political fringe (and the death of a celebrity like Steve McQueen) as because of the clinical trial.

    As GLaDOS is no doubt aware, the “mystification of politics” is already at an advanced stage. The staggeringly stupid positions of the Tea Party and its panderers make one worry about the future of reason. I wonder if laetrile will have a comeback soon.

  29. Harriet Hall says:

    @ Ed Whitney,

    “Perhaps SBM, judging from earlier posts and threads, generally requires that a known biochemical, physiological, or anatomical mode of action of an intervention be articulated before that intervention can be taken seriously and results of clinical trials can be considered plausible.”

    Not true at all. When there is solid evidence that something works, we are quite willing to accept that evidence and use the treatment. If the mechanism is unknown, we are content to leave its elucidation to later studies. But if accepting that it works (homeopathy?) would require rejecting a huge body of other established knowledge, we could only accept it if the evidence were overwhelming. Carl Sagan: extraordinary claims require extraordinary proof. An RCT or two simply won’t do. Do you see the difference?

    Prior plausibility need not come from understanding the mechanism of action; it can come from other lines of evidence.

  30. nybgrus says:

    Perhaps SBM, judging from earlier posts and threads, generally requires that a known biochemical, physiological, or anatomical mode of action of an intervention be articulated before that intervention can be taken seriously and results of clinical trials can be considered plausible. EBM, by contrast, is less particular about disease mechanisms and specific actions of interventions in the evaluation of clinical trials, and does not fuss when methodologically satisfactory trials produce results that have no apparent explanations in terms of physiology, pathology, and biochemistry.

    Dr. Hall has already answered this, but I will do so again because it is a very important point that Ed has clearly missed.

    SBM does not necessitate a need for mechanism in order to deem something efficacious. The brief version of the difference between SBM and EBM is that when a mechanism is known not to be possible, it can inform the results of equivocal RCTs. SBM would take the basic sciences and postulate that since the mechanism by which [treatment X] purports to work does not conform to known laws of physics, chemistry, and biology then an RCT is not sufficient evidence to make a positive claim. EBM would ignore the basic sciences and state that the results are equivocal (and potentially slightly positive) and recommend further study.

  31. Ed Whitney says:

    Harriet: I think I do get it. Carl Sagan says, essentially, that the likelihood ratio of a test needs to be very high when the prior probability is very low. The laws of chemistry and physics developed in the past 300 years have a very high prior probability, and their nullification has, so to speak, a homeopathically small probability.

    Putting MMR vaccine phobia to one side (and the low prior probability that a Playboy centerfold will be an authority on autism), it may be instructive to look at a Cochrane review of something with a fairly high prior probability: flu shots prevent flu and its complications.

    Interestingly, Cochrane reviews of influenza vaccine for older adults have been quite lukewarm in relation to the views of most of the public health community. “Influenza vaccines have a modest effect in reducing influenza symptoms and working days lost. There is no evidence that they affect complications, such as pneumonia, or transmission… Studies funded from public sources were significantly less likely to report conclusions favorable to the vaccines. The review showed that reliable evidence on influenza vaccines is thin but there is evidence of widespread manipulation of conclusions and spurious notoriety of the studies. The content and conclusions of this review should be interpreted in light of this finding.”

    Those differences between the benefits of the vaccine in industry-sponsored studies compared to publicly funded studies are in line with such differences in studies of many pharmacological products. Cochrane is reporting something counterintuitive, and Thomas Jefferson has not persuaded very many geriatricians or public health physicians to abandon recommendations for flu shots.

  32. daedalus2u says:

    I would put it more simply, SBM looks at all the evidence. EBM ignores everything except clinical trials, CAM ignores everything.

    If clinical trials are equivocal, EBM says we need more clinical trials.

    If there are no clinical trials and there is no other evidence, SBM says lets get some other evidence before we do any clinical trials.

    If there are no clinical trials, and there is other evidence, SBM says lets do some clinical trials to see if this potential treatment works the way we think it will.

    If clinical trials are great and there is no other evidence, SBM says we better understand the physiology behind this treatment ASAP!

    If clinical trials are equivocal and there is other evidence, SBM says lets do more research and figure out why these clinical trials are not working

    If clinical trials are equivocal and there is no other evidence, SBM says why was this nonsense with no prior plausibility tested in these unethical clinical trials?

    CAM says by my Chakra I feel this will do something for someone while making me rich, that is all the evidence I need to tell people it works and start charging them for it.

  33. Ed Whitney says:

    Wait a minute, daedalus2u! “CAM says by my Chakra I feel this will do something for someone while making me rich, that is all the evidence I need to tell people it works and start charging them for it.”

    I thought the C in CAM stood for Complementary! I thought it was free!

  34. Ed Whitney says:

    Of course, I meant “complimentary.”

  35. JPZ says:

    @Ed

    I am really enjoying your posts! Keep it up.

    @Harriet

    Can I try a different tack on Ed’s first post? If a RCT agrees with the body of evidence including mechanism, we are more likely to accept the results without in-depth questions. If a RCT is not supported by a body of evidence, but empirically works – we demand more support before we “buy in.” Once that support is there, we accept the results even if the body of supporting evidence is lacking. If a CAM assertion is neither supported by a RCT nor a body of evidence, then it has no external validity.

    BTW, “The plural of anecdote is not data” – wonderful quote!

  36. Ed Whitney says:

    Thanks, JPZ!

    Most EBM folks would deny that they only look at the RCTs, since they also insist that the individual patient’s situation must be taken into account. If the RCT enrolled patients who differ (e.g., in comorbidities) from the patient under consideration, then the RCT is placed into context as one factor informing the decision-making process. So EBM will also claim to look at all the evidence.

    We all remember the claptrap about the new “biopsychosocial” model of medicine that arose in the late 70s, in contrast to an alleged “biomedical” model that considered only what the patient’s transaminases were and failed to consider whether he lived in a house full of heavy drinkers. Practitioners of the latter proved difficult to find in the real world, though they abounded in the world of the imagination.

    Jerome Groopman wrote an essay in the New York Review of Books (available online at http://www.nybooks.com/articles/archives/2010/feb/11/health-care-who-knows-best/?pagination=false ) talking about a topic directly related to EBM, namely, comparative effectiveness research and “best practices” and “behavioral economics.” Dr. Groopman describes a number of “best practices” which have proven illusory, including tight glucose control for diabetes, quality metrics for hip and knee replacement, statins for dialysis patients, and from his own experience, erythropoetin for the anemia of cancer. The firestorm over changes in screening mammography in 2009 also gets a mention, with the American College of Physicians accepting the new guidelines and the American Society of Clinical Oncology opposing them.

    It seems that SBM and EBM are equally aware of the short half-life of “best practices,” and of the need for vigilant revision of current knowledge as it becomes obsolete. The NYRB article should concern us all, since comparative effectiveness research is likely to be with us for a long time. President Obama thinks that we can study and figure out “what works and what doesn’t”. Dr. Groopman shows that he is at risk of being disillusioned.

    Does SBM have anything to contribute to the problems in the Groopman article that EBM lacks? Are they in fact very different? Could SBM have done anything to calm the waters when some of the PSA screening recommendations came out for prostate cancer, and the EBM task force which prepared it was accused of being a bunch of murderers? If there are differences between them, let us hope that this is where that difference could make a difference!

  37. GLaDOS says:

    There are some exceptions. One is if a claim is so outrageously impossible, you can shift the burden of proof on to the claimant. I am guessing that this is what you meant.

    I think my point about CAM being an argument from ignorance and my point about onus have become conflated.

    “You can’t prove that acupuncture is nothing more than a placebo, therefore doctors are justified in prescribing it,” is an argument from ignorance.

    When someone says, “acupuncture is more than just a placebo,” the onus is upon them to prove that claim. The default position for unproven claims is rejection.

  38. GLaDOS says:

    pmoran, the bigger problem for me is, how do I talk to the MD prescribing the hydrocortisone?

    The standard of care where I work requires communication between treating MDs.

  39. Jan Willem Nienhuys says:

    @ daedalus2u on 02 Apr 2011 at 9:59 pm

    CAM ignores everything

    That’s too simplistic. Look at pro-homeopathy sites. They quote RCTs and meta-analyses and lost of fundamental research too. But their quotes are selective. When a meta-analysis concludes that the effects of homeopathy cannot be explained as only placebo (which is of course a euphemism for the effects of widespread custom of publication bias, data selection, fraud, fidgeting with multiple endpoints and other calculation tricks after data collection) then this phrase is quoted, and the next part of the sentence is omitted when it says that for no combination of disease and treatment scheme homeopathy has consistently shown to work.

    So the homeopaths really scrutinise those articles for phrases that out of context seem to support the homeopathic view. That is subtly different from ‘ignoring’. In their own magazines they usually present anecdotes, which they themselves find very convincing. No homeopath bases his or beliefs on those selected phrases from reviews. But when they present themselves to the general public, they pretend that homeopathy is scientifically proved and that they have a huge stack of papers to prove it. The WHO is or has been working on a report entitled ‘Homeopathy: overview and analysis of clinical research’ of 111 pages, written by (judging by the contents) a bunch of homeopaths, and containing at the end about 600 or more references.

    Metaphorically it is like

    Fit 6 “Fit the Sixth: THE BARRISTER’S DREAM”
    in the Hunting of the Snark.
    (the Snark the lawyer of the accused pig)

    … he Snark undertook it instead,
    And summed it so well that it came to far more
    Than the Witnesses ever had said!

    … their wild exultation was suddenly checked
    When the jailer informed them, with tears,
    Such a sentence would have not the slightest effect,
    As the pig had been dead for some years.

  40. GLaDOS says:

    But when they present themselves to the general public, they pretend that homeopathy is scientifically proved and that they have a huge stack of papers to prove it.

    The skeptical movement has caught the eye of marketing firms. In my Facebook travels I’ve encountered a fair number of Scientologists and other New Agers with “likes” that include: “Sorry I’m allergic to bullsh_t,” and “Science,” “Richard Dawkins.” But they seem to confine their skepticism to attacks upon BigPharma, genetically modified food, psychiatry, the FDA, etc.

    Another trick: a self-described skeptic offers a snarky rejection of homeopathy or energy healing, followed by a credulous promotion of some other unproven intervention.

  41. daedalus2u says:

    No Ed, you are confusing cause and effect. You are neglecting the timing of the beliefs of homeopaths and when they created “data” to support those beliefs. They don’t believe in homeopathy because of their cherry-picked “evidence”, they cherry-picked the “evidence” because of their beliefs.

    It is the cherry-picking of evidence that leads acupuncturists to declare that “toothpicks work too!”. They cherry-pick the evidence because the believe acupuncture works and are unable to accept that their belief in acupuncture might be mistaken.

    When your world-view allows you to ignore evidence, then you can’t have an accurate world-view. You have prioritized your world-view over the actual world.

    This is the problem with the idea that GLaDOS expressed that the “onus” is on the person making a hypothesis to “prove” it and one is free to “ignore” all that has not been “proven”. This is a very tricky standard to actually make work. There is never actually “proof” of anything. Applying the standard of “proof” allows cherry-picking unless it is done very carefully. Virtually no one who asks for “proof” applies the standard correctly. EBM frequentist methodolitry allows people to trick themselves into accepting as “proof”anything with a p less than 0.05.

    To apply standards of evidence and proof carefully requires dissociation of evidence from conclusions and also from what you want the conclusions to be. The ability to dissociate conclusions from what you want those conclusions to be is extremely important, but many people simply can’t do it and can’t accept that other people can do it. They prioritize what they want to believe over what there is evidence for. The ability to dissociate is one of the side effects of trauma. What that ability to dissociate is applied to is very important to control and regulate. Acquiring that ability is costly and has adverse effects in other areas, particularly in relationships.

    A problem that people have in evaluating other people’s ideas is that how the idea is expressed has a great affect on the evaluation of the idea. On the one hand, a proponent needs to be enthusiastically supportive of the idea, but if they are perceived to be too enthusiastic, then their ability to be objective is discounted and the idea is dismissed without being evaluated.

    Most of what we think we know does not derive from a “hypothesis” which has been “proven” by “data”. Most of what we think we know, we arrived at by copying someone else’s world-view. This is what is going on in childhood, adolescence and early adulthood during the process of learning. People find a “role model” and then try to emulate them. It is extremely important to recognize that all models are wrong and that all role models have feet of clay.

    The adoption of the frequentist methodolitry is a way of formally dissociating acceptance of the data as “proof” from everything else by making it a purely mechanical decision. This does protect the decision making process from wishful thinking, but it also “protects” the decision making process from everything else that is known. The “toothpicks work too!” result is compatible with the frequentist methodolitry method.

    The problem is with the frequentist methodolitry method. We want to use a method that produces personal satisfaction that the result is correct so we can place that brick of truth in the world-view edifice that we are constructing. This is the problem. The frequentist methodolitry method can’t produce a brick of truth. No method can.

    What SBM allows us to do is construct a structure with each brick supporting its own weight plus some more. The “weight” that each brick can support depends on the Bayes Factor associated with that piece of data. We may not know what the Bayes Factor is exactly, but we can estimate it and when our experiments produce certain outcomes we know our Bayes Factor is approximately correct.

    The “bricks” at the foundation of homeopathy and acupuncture are not even clay, they are shadows, phantasms with no substance, with no ability to carry any weight.

  42. @GLaDOS:

    http://readersandrootworkers.org/index.php?title=Fake_Psychics#Don.27t_Be_Fooled_By_Fake_Psychic_Sites.21

    Heh.

    @Jan Willem Nienhuys:

    Absolutely correct, and it is not limited to the homeopaths (or whatever) themselves, but to the authors of the ‘widespread customs’ to which you refer. They even admit it:

    http://www.ncbi.nlm.nih.gov/pubmed/19891621?itool=EntrezSystem2.PEntrez.Pubmed.Pubmed_ResultsPanel.Pubmed_RVDocSum&ordinalpos=4

    @Ed Whitney:

    You are asking too much, or perhaps just changing the subject. We’ve identified an important problem with EBM. We’ve shown that this problem has important consequences for patient care, for public policy, for research ethics, and for research funding, among other things (if you follow the links to and from the post cited by the Cochrane editorial, you’ll find examples of those and more). Those are “differences that make a difference,” but this is quite different from claiming that solving this problem will solve all of the health care problems named or implied in Groopman’s article.

  43. Ed Whitney says:

    Kimball–very interesting links; much obliged to you for pointing to them. The EBM/SBM tension, at a first glance at the links, appears to be in part a frequentist vs. Bayes contrast. I have certain pet peeves, some of which involve frequentist presentation of study results. The presentation of p values for correlation coefficients, and nothing else, drives me to start throwing things very quickly. Fortunately, there is a paper recycling program where I work, and I can place these studies into the bin where they will be shipped to Mexico and processed into toilet paper, so that they do not go entirely to waste. I will pursue the links further.

  44. GLaDOS says:

    From your link above, Dr. A:

    As CAM researchers navigate a broadening of biomedicine’s boundaries, while still needing to maintain the identity and research methods of a biomedical scientist, this article explores the constant pressure on CAM researchers to appear and act a little more ‘scientific’.

    These politicians are too clever by half.

    While we’ve been at medicine’s front door debatin a motley crew of cargo-cultists dressed in white coats, the other team has sent its larger divisions –including a corps of engineers aka wealthy IT corporations– around the back. They’ve been building blocks of condos to house their own graduate schools, standardized national exams, laboratories, accreditation boards, conferences and e-learning CME courses, malpractice insurers, journals, textbooks, high-tech equipment (Google “Brain State Technologies”), etc. They call this bizarro world “science based medicine” or “evidence based medicine.”

    Naturopaths lobby congress for moar credibility than they deserve in April 2011

    “A committee in the New Hampshire House of Representatives has voted unanimously to retain HB 351, a bill to guarantee health insurance parity for NDs and their patients.” 26 people like this.

    Popularity contests are irrelevant to figuring out what’s true, so people like us pretty much ignore politics. We are slow on the draw and light on teh sexy.

    Yet often ordinary Americans can spot the difference between someone trying very hard to be honest verses an indoctrinated fool. So maybe a few guys from our team should go to DC for equal time or something?

    I’ll volunteer to cover airfare for one dude, but not first class.

  45. Ed Whitney says:

    @ Kimball—those were pretty good links. For what it is worth, the concept of statistical significance came before the Supreme Court, and it appears that its status as a legal standard may be changing. http://online.wsj.com/article/SB10001424052748703712504576235683249040812.html for the legal story and http://blogs.wsj.com/numbersguy/a-statistical-test-gets-its-closeup-1050/ for a story about Bayes as a better way of assessing data.

  46. Artour says:

    I teach breathing retraining techniques developed by Russian medical doctors, and our goal is simple: to normalize automatic breathing patterns of our patients (or students) or to achieve normal breathing parameters (10-12 breaths/min; about 40 mm Hg for the arterial and alveolar CO2, 40 seconds for stress-free breath holding time test and so forth). These are international medical norms, but mainstream medicine ignore horrible breathing habits in modern people since overwhelming majority of us are chest breathers, mouth breathers, and hyperventilators:
    http://www.normalbreathing.com/i-hyperventilation.php

    Furthermore, over 90% of modern people believe that breathing more air at rest increases body oxygen levels, while thousands of medical studies tell the opposite story.

    Are these breathing retraining methods (which have the purpose to achieve the medical norm) alternative or complementary?

  47. nybgrus says:

    First of it isn’t “complimentary or alternative” it is CAM which encompasses everything that isn’t medicine. So if your question is “are the methods medicine that isn’t accepted to the mainstream or CAM?” then my answer is in 5 seconds of perusing the link you offered, I would say it is CAM and it is garbage. I could be wrong, and I certainly have put no effort into dissecting why. But here are my reasons for such a quick judgement:

    1. You posted a link about something that was only tangentially related to the topic at hand
    2. You asked for an evaluation that was poorly worded and when interpreted sounds like a CAM pandering for medical validation in a classic bait and switch
    3. The website itself is rather poorly designed – it lacks any sort of professional finish to it and appears childishly done.
    4. The key logo in the top left has the letters “NB” as the teeth – seems very chintzy and like a CAM gimmick logo
    5. The claim is that 90% of all people are hyperventilators (such a large number is a claim that is inherently hard to prove, especially to say so conclusively).
    6. (The big one!) The claim is that “normal breathing” can “defeat” chronic disease.

    And done… may have some grain of truth, may have none, but the claims are too grandiose, the website too crappy, and the notion of normal breathing defeating chronic disease too overarching and nonsensical.

  48. Scott says:

    I seem to recall that this sort of thing has been discussed here before and shown to be pure quackery, but I can’t seem to find the link.

Comments are closed.