May 20 2009
Last week I discussed a clinical trial comparing standardized acupuncture, individualized acupuncture, placebo-acupuncture, and usual care. In that discussion I emphasized the comparison between the three acupuncture groups, which did not show any difference in outcome. These results are consistent with the overall acupuncture literature, which shows in the better controlled trials that it does not matter where you stick the needles or even if you stick them through the skin. Therefore the scientific evidence fails to reject the null hypothesis (that acupuncture does not work). This did not stop the press from declaring, almost uniformly, that acupuncture works for back pain, contributing to the public misunderstanding of clinical science.
This week I am going to focus on the other aspect of the trial – the one the researchers and the press chose to focus on – the comparison of the two real and one placebo acupuncture arms to “usual care.” This too was misrepresented by the press, encouraged by the overinterpretation of the evidence by the researchers.
In the comments to Part I of this discussion David Gorski correctly pointed out that the study in fact did not even constitute a comparison of acupuncture to standard medical treatment. He is absolutely correct, and the many reasons for this are worth explaining in detail. Understanding the technology of clinical trials is central to science-based medicine, including all of their pitfalls and limitations. For practical and logistical reasons there is almost never a perfect clinical trial, but mischief only ensues when limitations are not understood, leading to a misinterpretation (and almost always an overinterpretation in the direction of the researcher’s bias) of the evidence.