I’ll begin with the possibly shocking admission that I’m a strong supporter of the collection of ideas and techniques known as evidence-based medicine (EBM). I’m even the current President of the Evidence-Based Veterinary Medicine Association (EBVMA). This may seem a bit heretical in this context, since EBM takes a lot of heat in this blog. But as Dr. Atwood has said, “we at SBM are in total agreement…that EBM “should not be without consideration of prior probability, laws of physics, or plain common sense,” and that SBM and EBM should not only be mutually inclusive, they should be synonymous.” So I have hope that by emphasizing the distinction between SBM and EBM and the limitations of EBM, we can engender the kind of changes in approach needed to address those limitations and eliminate the need for the distinction. One way of doing this is to critically evaluate the misuses of EBM in support of alternative therapies.
One of the highest levels of evidence in the hierarchy of evidence-based medicine is the systematic review. Unlike narrative reviews, in which an author selects those studies they consider relevant and then summarizes what they think the studies mean, which is a process subject to a high risk of bias, a systematic review identifies randomized controlled clinical trials according to an explicit and objective set of criteria established ahead of time. Predetermined criteria are also used to grade the studies evaluated by quality so any relationship between how well studies are conducted and the results can be identified. Done well, a systematic review gives a good sense of the balance of the evidence for a specific medical question.
Unfortunately, poorly done systematic reviews can create an strong but inaccurate impression that there is high-level, high-quality evidence in favor of a hypothesis when there really isn’t. Reviews of acupuncture research illustrate this quite well.