Explanatory studies are done under controlled conditions to determine whether a treatment has any efficacy compared to a placebo. Pragmatic studies are designed to assess how the intervention performs in everyday real world practice. Pragmatic studies measure practical success but don’t determine actual efficacy: that requires a proper randomized controlled trial (RCT) with an appropriate control. Pragmatic studies have their place, but they can sometimes make an ineffective treatment look good: a phenomenon I have christened Cinderella Medicine.
In my recent review of Peter Palmieri’s book Suffer the Children I said I would later try to cover some of the many other important issues he brings up. One of the themes in the book is the process of critical thinking and the various cognitive traps doctors fall into. I will address some of them here. This is not meant to be systematic or comprehensive, but rather a miscellany of things to think about. Some of these overlap.
Everything is attributed to a pet diagnosis. Palmieri gives the example of a colleague of his who thinks everything from septic shock to behavior disorders are due to low levels of HDL, which he treats with high doses of niacin. There is a tendency to widen the criteria so that any collection of symptoms can be seen as evidence of the condition. If the hole is big enough, pegs of any shape will fit through. Some doctors attribute everything to food allergies, depression, environmental sensitivities, hormone imbalances, and other favorite diagnoses. CAM is notorious for claiming to have found the one true cause of all disease (subluxations, an imbalance of qi, etc.).
A friend asked me to look at the evidence for hash oil as a treatment for glioma. His teenage daughter was recently diagnosed with brain cancer: a grade 3 anaplastic ependymoma. It recurred very rapidly after surgery and radiotherapy and the latest tissue diagnosis shows an aggressive grade IV glioma. Her prognosis is not good. No further attempts at curative therapy are indicated; the oncologist prescribed only palliative therapy with temozolomide. Her father, who had recently lost his wife to cancer (breast cancer metastatic to lungs and brain), was understandably devastated. As he puts it, he remains “focused on the belief that just maybe a cure can be found.” He stumbled on what he calls “earth-shattering news” regarding hash oil. He and his friends established a private wiki website which they are constantly updating with information about THC (tetrahydrocannabinol, the active ingredient in marijuana and hash) and other possible cancer cures: everything from curcumin to diet. He asked me to look at the information he has accumulated. He said
I hope to convince you in the same way I have done with my daughter’s GPs and her neuro-oncologist at BC Children’s Hospital.
The oncologist was not exactly convinced. He didn’t say he thought hash oil was likely to work; he only said it would be reasonable to try it as a complementary therapy. He said
the data published so far appears very preliminary, most of its potential effectiveness in vivo so far appears in colonic disease, having said that there doesn’t appear to be any obvious down side as a complementary therapy and may have synergistic effect, so may be reasonable as add on to temodal if she tolerates it
I wasn’t convinced either.
I will discuss two issues here:
- What does the evidence say about gliomas and hash oil?
- When is it reasonable to try an unproven treatment as a last resort?
When an article is published in a medical journal, the authors must disclose any conflicts of interest. This is important, because even if they think owning stock in the drug company won’t influence their scientific judgment, we know that subtle biases can creep in to somehow affect the findings of studies. It has been shown that studies funded by drug companies are more likely to get positive results for their drug than studies funded by independent sources. Andrew Wakefield, author of the infamous retracted Lancet study suggesting a relationship between MMR vaccine and autism, was severely chastised for not disclosing that he received money from autism litigators and expected to earn a fortune from his own patented products if the MMR vaccine could be discredited.
I was recently contacted by an acupuncturist who plans to critique an article I wrote. It was a commentary in the journal Pain that accompanied a systematic review of systematic reviews of acupuncture by Ernst et al. For details of Ernst’s and my articles, see my previous post. He challenged my statement that I had no conflicts of interest to report. He apparently thinks I should have said I have a conflict of interest in that I am anti-CAM and anti-acupuncture. When he writes about my article, he plans to attack me for not declaring this alleged conflict of interest and he plans to set a good example with a conflict of interest statement of his own, divulging that he makes his living practicing acupuncture, has financial investments in it and many personal relationships, that his self-identity and prestige are dependent on his belief in acupuncture’s efficacy, and that he is biased towards constructivism and away from positivism. (I think this is a fancy way of saying he favors experience over the scientific method.) I agree that he has conflicts of interest, but was I wrong to say I had no conflicts of interest? I don’t think so. (more…)
Some of our readers have complained that we pick on alternative medicine while ignoring the problems in conventional medicine. That criticism is unjustified: we oppose non-science-based medicine wherever we find it. We find it regularly in alternative medicine; we find it less frequently in conventional medicine, but when we do, we speak out. A new book by Dr. Peter Palmieri is aimed squarely at failure to use science-based medicine in conventional practice.
Dr. Palmieri is a pediatrician who strives to provide the best compassionate, cost-effective, science-based care to all his patients. Over 15 years of practice in various settings, he observed that many of his colleagues were practicing substandard medicine. He tried to understand what led to that situation and how it might be remedied. The result is a gem of a book: Suffer the Children: Flaws, Foibles, Fallacies and the Grave Shortcomings of Pediatric Care. Its lessons are important and are not limited to pediatrics: every health care provider and every patient could benefit from reading this book.
The chapters cover these subjects:
- How doctors mishandle the most common childhood illnesses
- How doctors succumb to parental demands
- How they embrace superstition and magical beliefs
- How they fall prey to cognitive errors
- How they order the wrong test at the wrong time on the wrong patient
- How financial conflicts of interest defile the medical profession
- How doctors undermine parents’ confidence by labeling their children as ill
- A prescription for change
Before we had EBM (evidence-based medicine) we had another kind of EBM: experience-based medicine. Mark Crislip has said that the three most dangerous words in medicine are “In my experience.” I agree wholeheartedly. On the other hand, it would be a mistake to discount experience entirely. Dynamite is dangerous too, but when handled with proper safety precautions it can be very useful in mining, road-building, and other endeavors.
When I was in med school, the professor would say “In my experience, drug A works better than drug B.” and we would take careful notes, follow his lead, and prescribe drug A unquestioningly. That is no longer acceptable. Today we ask for controlled studies that objectively compare drug A to drug B. That doesn’t mean the professor’s observations were entirely useless: experience, like anecdotes, can draw attention to things that are worth evaluating with the scientific method.
We don’t always have the pertinent scientific studies needed to make a clinical decision. When there is no hard evidence, a clinician’s experience may be all we have to go on. Knowing that a patient with disease X got better following treatment Y is a step above having no knowledge at all about X or Y. A small step, but arguably better than no step at all. (more…)
The word “frequency” ranks right up there with “quantum” and “energy” as a pseudoscientific buzzword. It is increasingly prevalent in product advertisements and in CAM claims about human biofields and energy medicine. It doesn’t mean what they think it means.
I have written about Power Balance products, the wristbands and cards that allegedly improve sports performance through frequencies embedded in a hologram. They amount to nothing but a new version of the old rabbit’s foot carried for superstition and their sales demonstrations fool people with simple musculoskeletal tricks. I addressed their ridiculous claims (including “We are a frequency”). I pointed out that
The definition of frequency is “the number of repetitions of a periodic process in a unit of time.” A frequency can’t exist in isolation. There has to be a periodic process, like a sound wave, a radio wave, a clock pendulum, or a train passing by at the rate of x boxcars per minute. The phrase “33⅓ per minute” is meaningless: you can’t have an rpm without an r. A periodic process can have a frequency, but an armadillo and a tomato can’t. Neither a periodic process nor a person can “be” a frequency.
Believers in acupuncture claim it is supported by plenty of published scientific evidence. Critics disagree. Thousands of acupuncture studies have been done over the last several decades, with conflicting results. Even systematic reviews have disagreed with each other. The time had come to re-visit the entire body of acupuncture research and try to make sense out of it all. The indefatigable CAM researcher Edzard Ernst stepped up to the plate. He and his colleagues in Korea and Exeter did an exhaustive study that was published in the April 2011 issue of the medical journal Pain: “Acupuncture: Does it alleviate pain and are there serious risks? A review of reviews.” It is accompanied by an editorial commentary written by yours truly: “Acupuncture’s claims punctured: Not proven effective for pain, not harmless.” (The editorial is reproduced in full below.)
Ernst et al. systematically reviewed all the systematic reviews of acupuncture published in the last 10 years: 57 systematic reviews met the criteria they set for inclusion in their analysis. They found a mix of negative, positive, and inconclusive results. There were only four conditions for which more than one systematic review reached the same conclusions, and only one of the four was positive (neck pain). They explain how inconsistencies, biases, conflicting conclusions, and recent high quality studies throw doubt on even the most positive reviews.
They also demolished the “acupuncture is harmless” myth by reporting 95 published cases of serious adverse effects including infection, pneumothorax, and 5 deaths. Some but not all of these might have been avoided by better training in anatomy and infection control. (more…)
David Kroll’s recent article on thunder god vine is a great example of what can be learned by using science to study plants identified by herbalists as therapeutic. The herbalists’ arsenal can be a rich source of potential knowledge. But Kroll’s article is also a reminder that blindly trusting herbalists’ recommendations for treatment can be risky.
Herbal medicine has always fascinated me. How did early humans determine which plants worked? They had no record-keeping, no scientific methods, only trial and error and word of mouth. How many intrepid investigators poisoned themselves and died in the quest? Imagine yourself in the jungle: which plants would you be willing to try? How would you decide whether to use the leaf or the root? How would you decide whether to chew the raw leaf or brew an infusion? It is truly remarkable that our forbears were able to identify useful natural medicines and pass the knowledge down to us.
It is equally remarkable that modern humans with all the advantages of science are willing to put useless and potentially dangerous plant products into their bodies based on nothing better than prescientific hearsay. (more…)