Articles

Acupuncture and the Hazard of Nonsense

A recent study published in the Archives of Opthalmology compare patching of one eye vs acupuncture in the treatment of amblyopia in older children, and finds positive results from acupuncture. The study, and its press, are a good example of the hazards of studying highly implausible modalities.

First let’s dissect the study itself – from the abstract:

In a single-center randomized controlled trial, 88 eligible children with an amblyopic eye who had a best spectacle-corrected visual acuity (BSCVA) of 0.3 to 0.8 logMAR at baseline were randomly assigned to receive 2 hours of patching of the sound eye daily or 5 sessions of acupuncture weekly. All participants in our study received constant optical correction, plus 1 hour of near-vision activities daily, and were followed up at weeks 5, 10, 15, and 25. The main outcome measure was BSCVA in the amblyopic eye at 15 weeks.

For background, amblyopia occurs when the brain tends to ignore visual information from one eye. This results from a variety of causes, but commonly from the two eyes having different refractive errors (anisometropic) – one eye may be more near-sighted or far-sighted than the other. The brain cannot combine information from both eyes, so it ignores one. This can be corrected in younger children, up to age 7, by correcting the vision for the refractive problems. If visual correction alone is not enough, then patching one eye (the strong eye) to force the brain to use the weak eye can be effective. This is usually done for only 2 hours a day, otherwise amblyopia of the patched eye can occur.

In children over 7 years old amblyopia is harder to treat. As the brain matures it becomes less plastic and less responsive to interventions used to treat amblyopia. In this study the researchers investigated standard treatment – constant optical correction plus near-vision activities – plus either patching or acupuncture. Not surprisingly, both groups improved:

The mean BSCVA of the amblyopic eye at 15 weeks improved from baseline by 1.83 and 2.27 lines in the patching and acupuncture groups, respectively. After baseline adjustment, the mean difference of BSCVA between the 2 groups was 0.049 logMAR (95% confidence interval, 0.005-0.092; P = .03), meeting the definition of equivalence (difference within 1 line). The BSCVA had improved by 2 lines or more in 28 (66.7%) and 31 (75.6%) eyes in the patching and acupuncture groups, respectively. Amblyopia was resolved in 7 (16.7%) and 17 (41.5%) eyes in the patching and acupuncture groups, respectively.

The weaknesses of this study make interpretation of the results difficult. It is a small, single-center study. Follow up was relatively short for this outcome. But most importantly – both groups received some standard therapy, and the variables of interest were not blinded at all. Further, there were no acupuncture controls – no sham or placebo acupuncture.

At best this is a pilot study. In the abstract the authors conclude: “Further studies are warranted to investigate its value in the treatment of amblyopia.” That is the only conclusion warranted by pilot studies. However, in the discussion the authors go further:

“Because of the good results obtained in our study, the acupoints that we used could be considered for use in clinical practice.”

Changes to clinical practice are not warranted based upon an unblinded pilot study such as this. The history of acupuncture specifically is one in which unblinded pilot studies tend to be positive, but then follow up well-controlled blinded acupuncture studies have tended to be negative. If history is any judge, these results will not hold up under further study, and therefore changes to clinical practice are premature.

This episode is just one example of the hazard of studying was is essentially a nonsensical system – the notion that acupuncture needles placed in specific (and non-existent) acupoints can cause specific physiological effects. The authors write:

“Although the treatment effect of acupuncture appears promising, the mechanism underlying its success as a treatment for amblyopia remains unclear,” the authors write. Targeting vision-related acupoints may change the activity of the visual cortex, the part of the brain that receives data from the eyes. It may also increase blood flow to the eye and surrounding structures as well as stimulate the generation of compounds that support the growth of retinal nerves, they note.

There is no evidence for such wild speculation about possible mechanisms of acupuncture, and speculating about mechanisms is premature when the best acupuncture studies all find no effect.

This is the merry-go-round of such highly implausible but culturally supported therapies. Studies that carefully control for the specific elements of acupuncture (sticking needles through the skin at specific points) show no effect from those specific elements. As I have written many times before – it doesn’t matter where you stick the needles or even if you stick the needles. What we get from acupuncture, at best, are very non-specific effects from the therapeutic ritual that surrounds acupuncture. There also appears to be non-specific effects from the local trauma of piercing the skin with needles, essentially mechanisms that dampen down pain and inflammation following local trauma.

Proponents of acupuncture typically will justify the claim that “acupuncture works” with these non-specific effects, missing the point (either naively or disingenuously) that non-specific effects do not justify specific claims or mechanisms. Often the claim is made that it does not matter how acupuncture works (again, missing the point) if it shows some clinical utility – non-specific effects (the argument goes) are worthwhile.

But such arguments are ultimately a bait-and-switch, a desperate attempt at misinterpreting the literature to justify the specific interventions of acupuncture. And then – with the next acupuncture study that does not control for needle location or insertion (those elements that define acupuncture) the authors happily credit a positive result to the specific elements of acupuncture and start speculating wildly about possible specific physiological mechanisms.

And around we go again on the merry-go-round of acupuncture – a poorly-controlled and unblinded study with (not surprisingly) positive results. The authors are superficially circumspect, but ultimately promote the findings as sufficient to justify clinical practice and continued speculation about the magical mechanisms of acupuncture. Follow up studies (when they occur) tend to be negative, meaning that any positive effects have nothing to do with the acupuncture itself, in which case proponents trumpet the non-specific and placebo effects as also supporting acupuncture (head I win, tails I win).

The ultimate problem is that the underlying notions of acupuncture itself – that there are specific acupuncture points on the body, and that there are mysterious energies that can be manipulated by sticking needles in these points that then have specific physiological effects – are highly implausible. They are, in fact, nothing but pre-scientific superstition. The energy and acupuncture points of acupuncture, according to decades of research and multiple independent lines of evidence, simply do not exist. Acupuncture is ultimately a shell game of preliminary unreliable results and misinterpreted non-specific/placebo effects.

Posted in: Acupuncture

Leave a Comment (38) ↓

38 thoughts on “Acupuncture and the Hazard of Nonsense

  1. Tantalus Prime says:

    “At best this is a pilot study.”

    Funny, those were almost the exact same words I used to describe this study.

    Really, without sham accupuncture, vision exercise alone, and wait-list controls you really can’t tell anything from this study.

  2. windriven says:

    And the beat goes on: the increasingly common infiltration of quack practices into mainstream medical journals, educational institutions and medical centers. As a non-physician I find myself wondering just what the foundations of medical education are. Is it really possible to become a medical doctor, perhaps board certified in a specialty, without understanding the scientific method and the necessary structure of experiments and studies by which hypotheses are tested?

    How long before we see glowing reports on the pioneering work of John of God in the pages of Surgery?

    To paraphrase John Dean, there is a cancer growing on the practice of medicine. When sCAM artists pull these stunts and they fail, patients – and their survivors – may rue the day that they abandoned medicine for quackery. But what do they do when their medical doctor is the one quacking?

    The intrusion of sCAM into mainstream medical practice denigrates the profession and will ultimately erode the trust of patients. The difficulty of the profession policing itself has been discussed in these pages before. But the difficulty only underscores the importance of getting it done. If not, in a few generations what will be the difference between an MD and an ND?

  3. David Gorski says:

    There’s also the large contingent of investigators from China. True, there was one investigator from the Einhorn Clinical Research Center, New York Eye and Ear Infirmary and the Department of Ophthalmology at New York Medical College, but the vast majority of other investigators were from China and India. As was pointed out in Jonah Lehrer’s article about the “decline effect,” which I blogged about on Monday, acupuncture studies from China are almost always positive. At least, they are far more often positive than studies done in the “West.” This has been known for quite some time, so much so that there is an ongoing debate over whether Chinese studies should be included in acupuncture meta-analyses.

  4. Dacks says:

    My sister has amblyopia which was treated unsuccessfully with an eye patch when she was a child. I thought I heard recently that the eye patch approach has been seen as ineffective, and is not in widespread use. Am I remembering correctly?

    Also -
    “This can be corrected in younger children, up to age 7, but correcting the vision.”

    ?

  5. marilynmann says:

    Sorry to be dense, but where exactly were they sticking the needles? Hopefully not in the kids’ eyes!? What, exactly, are “vision-related acupoints”?

    @Dacks I assume “correcting the vision” means having the kids wear corrective lenses (i.e., glasses).

  6. WeiterGen says:

    German weekly “Der Spiegel” covered the story with a disturbingly uncritical article.
    http://www.spiegel.de/wissenschaft/medizin/0,1518,734398,00.html

  7. I corrected the statement to: “by correcting the vision for the refractive problems.”

  8. mikerattlesnake says:

    I don’t have access to the full article, but someone elsewhere pointed out that one of the primary authors of the study had applied for a patent for this treatment prior to the study. Can you confirm or deny this? Seems like it would be a huge deal in an unblinded study.

  9. windriven says:

    Dr. Gorski wrote: “… but the vast majority of other investigators were from China and India.”

    Still, Archives of Opthalmology, its editors and reviewers saw fit to accept it for publication. Professional journals in mathematics and physics don’t, in my experience, have a lower bar for Chinese or Indian scientists.

    “… acupuncture studies from China are almost always positive.”

    Which one would hope would increase skepticism among Opthalmology’s reviewers and editors.

  10. Robin says:

    Were the children amendable to acupucture? Anyone who has tried to extract blood from a small child knows how traumatic and frightening needles are for them.

    I have this condition though I never knew what it was called medically (thank you.) As a small child I was treated with a patch and visual exercises designed to build my eye muscles to moderate success (I still get wonk eye when I get tired). I think if someone had tried to stick me with needles I would have been terrified of my appointments.

  11. iDoc says:

    A few thoughts from someone who has treated a number of amblyopic patients…

    The first step is always optimal optical correction. I don’t prescribe patching until they have worn the new glasses/contacts for around 2 months. Often when I see them back after the 2 months, the patient’s amblyopic eye has improved at least 2 lines of acuity as a result of the brain having a clear image projected on that retina.

    Getting the first couple lines of improvement is the easy part. Patching Therapy with near activity then begins to try and improve the acuity further.

    Relating that to this study: These kids were all mild to moderate amblyopes (.3-.8 logMAR is equivalent to 20/40 – 20/120). All kids were given optimal optical correction and prescribed near activity for an hour per day which, as Dr. Novella points out, is part of standard therapy. That alone should improve 2 lines of acuity.

    The big surprise to me from this study is that the kids who received patching did not improve more than they did. I don’t have full access to the article, but what I would be curious to see is if the kids who received acupuncture “therapy” were more on the 20/40 (mild, easily treatable) side of the scale to begin with and the ones who received patching were more on the 20/120 (moderate, more difficult to get a “resolved” outcome) end. That would account for the large disparity of “resolved” amblyopia in the acupuncture group over the patching group, as well as the slightly better average acuity improvement of the acupuncture group over the patching group.

  12. squirrelelite says:

    Perhaps it would help if some of the major medical journals would set up an auxiliary journal specifically for publishing pilot studies. To be published in the main journal, a study would have to meet certain minimum methodological standards, like a sufficient size to have some statistical power and some plausible control group, not just an apples and oranges comparison like this study.

  13. squirrelelite says:

    Also, publishing cost would be a major factor, but with electronic only publication it might be feasible.

  14. Regarding baseline characteristics, the authors write:
    “There was no significant interaction between treatment group and baseline VA (P = .82), baseline refractive error (P = .34), age at enrollment (P = .17), or sex (P = .12) with regard to the 15-week outcome VA in the amblyopic eye.”

    Regarding financial disclosures:
    “Drs Zhao, Lam, Chen, Zheng, Fan, and Zhang have filed with the US Patent and Trademark Office a provisional patent application for the stimulation of specific acupuncture points for the improvement of vision.”

  15. windriven says:

    Squirrelelite said: “Perhaps it would help if some of the major medical journals would set up an auxiliary journal specifically for publishing pilot studies.”

    Perhaps it would help if some of the major medical journals would refrain from publishing junk.

    The journals incur no greater costs by rejecting studies that do not meet minimum methodological criteria. Whose interests are served by publishing seriously flawed studies, even in an auxiliary journal?

  16. crs says:

    Applying for a provisional patent isn’t a huge deal at all. It’s relatively cheap and not too difficult. So that alone shouldn’t be taken to imply a major financial interest in outcome! More is needed…

  17. windriven says:

    @crs

    Using – and sometimes abusing – publications as marketing tools has been around for a long time. If you’re doing a road show for investors in Dr. Zhao’s Miracle Acupuncture Clinics, Inc., supporting papers in major journals is a big plus.

  18. crs says:

    Yes indeed, depending on the majority of the journal and the sophistication or lack thereof of the investor.

    But don’t consider a patent application to be simply a type of publication, the standards are quite different. That said, mentioning “we have filed applications (and would you like white or red?)” may initially impress your investors, at least until they have a patent lawyer actually read them.

  19. windriven says:

    crs, I was referring to the journal article rather than the patent. Journal articles carry weight with investors and with other physicians. The fact that they’ve filed a patent simply suggests that they intend to commercialize this sCAM.

  20. daedalus2u says:

    there are only business reasons for filing a patent application. There are no non-business reasons (other than ego self-gratification) for filing a patent application.

    They are trying to commercialize this. If they were not, they would not file a patent application. The patent might even issue, so long as no one has previously mentioned using acupuncture for the improvement of vision.

    The patent office does not test to see if the treatment works, only if it is new.

  21. Joe says:

    I wondered why the main outcome measure was taken at 15 weeks since they had data going to 25 weeks. According to the article (penultimate paragraph on p.5 of the PDF). “By 25 weeks, the mean BSCVA was 0.28 logMAR in the patching group and 0.22 logMAR in the acupuncture group (P=.06). The resolved rate was similar between groups (30% in the patching group vs 42.1% in the acupuncture group, P=.27).”

    Also: (“Comment” penultimate paragraph on p. 6) “By 15 weeks, the proportion of responders in both groups was similar (75.6% in the acupuncture group vs 66.7% in the patching group; P=.37), whereas the resolution rate was significantly higher in the acupuncture group (41.5%) than in the patching group (16.7%) (P=.01). However, the resolution rates were similar by 25 weeks.”

    Isn’t the longest-term result the most significant? Doesn’t it show no significant difference in the long run?

  22. “… acupuncture studies from China are almost always positive.”

    From 1966 to 1995 they were all positive:

    http://www.ncbi.nlm.nih.gov/pubmed/9551280

    (We don’t know what the incidence has been since 1995).

    Here’s another example of how this information ought to contribute, but hardly ever does, to reviews of acupuncture claims:

    http://www.sram.org/0802/acupuncture.html

    My view is that even reviewers who don’t understand the concept of nonsense, but who fancy themselves committed to responsible reviewing, ought to exclude all positive acupuncture trials from China, Japan, Hong Kong, and Taiwan until someone repeats the “Do Certain Countries” study and shows that things have changed.

  23. Kyle says:

    I wonder how well the kids wore the glasses they were prescribed. As an optician I have seen more than a few kids over the years who just wouldn’t wear their glasses and had their amblyopia get worse. The parents didn’t or wouldn’t make their child wear them either.

    Have to dig in my bosses journals to see if I can find the article.

  24. pmoran says:

    Kimball: “Here’s another example of how this information ought to contribute, but hardly ever does, to reviews of acupuncture claims:

    http://www.sram.org/0802/acupuncture.html

    A most telling critique. Tell me, who is minding the shop at Cochrane, deciding what gets published in its name?

  25. daijiyobu says:

    Not to be a d@#k,

    but,

    not “opthalmology”,

    it’s ophthalmology.

    As a teacher of medical terminology, that detail catches my attention.

    -r.c.

  26. daedalus2u says:

    pmoran, my guess would be the accountants.

  27. windriven says:

    @daijiyobu

    You are, of course, correct. No dickishness involved. Thanks for the correction.

  28. Werdna says:

    (95% confidence interval, 0.005-0.092; P = .03)

    Perhaps this is a hole in my statistical knowledge but doesn’t that CI look a little wide. Also I can never remember if the probability distribution inside a CI is assumed to be linear or normal. If the former then the width of the CI makes me wonder at the clinical significance even if it’s statistically significant.

  29. Dr. N.: “This episode is just one example of the hazard of studying was is essentially a nonsensical system” should be “hazard of studying what is essentially…”

  30. Always Curious says:

    They did indeed write the abstract from the perspective of week 15 because that was the only point where the treatment group comparison had a significant p-value. The study is divided into 5 week intervals, and it shows that 2 unimproved patients dropped from the acupuncture group between week 10 & week 15. Compare this to the patching group which lost 1 patient who improved 2 lines.

    @ iDoc: I’m far from being an eye expert, but it appears that your guess is correct. Both groups had similar average levels of severity. However, the distribution indicates that the acupuncture group had a larger number of mild cases (with just enough severe cases to make up the difference). So it might be reasonable to assume the acupuncture group would have improved faster under any conditions.

  31. Joe says:

    @Always Curious on 16 Dec 2010 at 12:27 pm

    So you agree that they probably chose their endpoint post hoc.

    As for your response to iDoc, it seems the reported ‘good’ result was an artifact of the small sample size.

    The bottom line seems to be that the publication was simply a failure of peer-review. That is why publication in a ‘good’ journal is only the first step in scientific acceptance. It (peer-review) streamlines our appraisal of new ideas- we don’t have to look at every claim by a quack.

  32. “However, the distribution indicates that the acupuncture group had a larger number of mild cases (with just enough severe cases to make up the difference). So it might be reasonable to assume the acupuncture group would have improved faster under any conditions.”

    I think this is a large part of the magic trick.
    There are six levels of severity. Here is how the two groups were distributed across those six groups, from 20/40 (least impaired vision at randomization) to 20/125 (most impared vision):
    acupu 13, 13, 5, 3, 8, 1;
    patch 13, 9, 6, 6, 9, 2.

    Yes, as noted: 15 wk was the only week to have a statistically sig diff favoring acupuncture, but the trend across the 4 follow-up periods was the same: acupuncture wins; so, it is not a matter of results bouncing around, favoring one then the other.

    It may have been a matter those most likely to experience easy, or power-of-suggeston/try harder improvement to be in the acup group vs. patch group.

    Randomization was old-fashioned, and I have read crituques of this method, but I cannto recall the exact weakness or criticism:
    as people were consented, they were given sequential study participant numbers. These were, sequentially, matched to a number in a “computer-generated” random number generator (versus a published table of random numbers); odds went into one group, evens into another.

  33. Joe says:

    @MedsVsTherapy on 16 Dec 2010 at 2:56 pm wrote “… Randomization was old-fashioned …”

    Yes, I did not understand this (quoting the article) “… participants were randomly assigned to either patching or acupuncture by use of a simple randomization method [23] that consisted of a list of computer generated random numbers. The participants were assigned by matching their serial numbers of trial enrollment to the sequence of the random numbers, with odd numbers being assigned to the patching group and even ones to the acupuncture group.”

    What was the randomization procedure- random number generation or by serial numbers (odd or even)? This sounds like they were ordered according alphabet, by height. Is this another failure of my understanding, or of peer-review?

  34. Calli Arcale says:

    In children over 7 years old amblyopia is harder to treat. As the brain matures it becomes less plastic and less responsive to interventions used to treat amblyopia.

    There is an additional problem — the muscles that control the eyeball tend to atrophy from disuse, and there is a point of no return after which they have shrunk so much that no matter how successful the brain re-training is, the eye simply cannot be pointed in the desired direction anymore. Surgery is sometimes effective, depending on how much the muscles have contracted, but otherwise prismatic eyeglasses are required to correct for the one eye not pointing *quite* the right way. My littlest brother had this happen, and has the fancy glasses to correct that last little bit that therapy couldn’t get. Ironically, he is the only one in the family who has 20/20 acuity in both eyes. ;-)

  35. fealgu says:

    Maybe someone could help me to answer a question. Has anyone heard about William Bengston? Apparently, he has conducted 10 ‘experiments’ on how touch healing cured mice with cancer. I have not being able to find any reliable information on this guy, except for his own website and being mention by Dr. OZ. Please post if you know anything related to this person.

  36. verquer says:

    I was in brief contact with the author of the Spiegel article that WeiterGen mentioned. I argued that eye patches could be useless in this context, which would then lead to the conclusion that one treatment (acupuncture) is as useless as another (eye patches). She quoted the article with

    “Participants were also instructed to perform near-vision activities for 1 hour during patching. The activities included writing homework assignments,
    reading, computer work, and other eye-hand coordination activities.”

    which seems to be in contradiction to Novellas sentence:

    “In this study the researchers investigated standard treatment – constant optical correction plus near-vision activities – plus either patching or acupuncture.”

    Was the acupuncture group now asked to do the near-vision activities or not? That info would really help my discussion.

  37. Werdna says:

    I admit I’m not sure how that randomization was supposed to work – what I don’t understand is the phrase”The participants were assigned by matching their serial numbers of trial enrollment to the sequence of the random numbers”. Perhaps some big string of numbers and the person takes the xth digit (or maybe all of the digits depending on how long the string is) of their enrollment number and goes down the list of digits until they find their number? Whether their number is an odd or even number of digits in decides what group they are in. If so, and I’m guessing this seems random – if a bit elaborate.

    http://scientopia.org/blogs/whitecoatunderground/2010/12/15/stick-a-needle-in-your-eye/

    Mentions another problem with subject bias: “Also, subjects (or victims?) of this experiment were recruited from eye clinics. They were not, for example, chosen serially (at least as reported). This can introduce bias into the experiment. For example, parents responding to recruitment announcements might be more predisposed to “believe in” acupuncture, and therefore be more responsive to placebo effects.”

    You’re correct that there’s also a slight bias in the variety of cases with harder cases going to the patching group.

    Based on the information given (i.e. a difficulty group of 1 to 6) the Acupuncture group would have a median of 2 and the patch group a median of 3.

  38. red rabbit says:

    @Windriven: the foundations of medical education vary widely. Some schools are more prone to sCAM than others. In particular, those which reject the more traditional teaching methods and hard science prerequisites in favour of so-called problem-based learning seem to attract the woo-susceptible.

    At McGill, a traditional “old school” medical school, there was very little tolerance for bullshit, and the “EBM” lectures we had there were based on critical thinking, statistics and their failings, and developing a jaundiced eye for the literature. We needed the hard science prereqs, and we used them.

    At McMaster, there are no hard-and-fast prerequisites. Woo-ey thinking abounds. The EBM lectures were much more the stuff you hear about here. My fellow residents included a guy who went on a fast-and-sauna ersatz Indian camp in Arizona as an elective, and a guy whose brother was a chiropractor and who saw nothing odd about the methods they used.

    Blergh. I was, suffice it to say, unpopular in that venue.

Comments are closed.