The scientific approach to understanding the world includes the process of carefully separating out variables and effects. Experiments, in fact, are designed specifically to control for variables. This can be especially challenging in medicine, since the body is a complex and variable system and there are always numerous factors at play. We often characterize the many variables that can influence the outcome in a clinical study as “placebo effects” or “non-specific effect” – things other than a specific response to the treatment in question.
A common error to make when interpreting clinical studies is to confuse non-specific effects – those that result from the therapeutic interaction or the process of observation – with a specific effect from the treatment being studied. While this is broadly understood within the scientific medical community, it seems that within certain fields proponents are going out of their way to sell non-specific effects as if they were specific effects of the favored treatment.
This is perhaps most true for acupuncture. As has been discussed numerous times on SBM, the consensus of the best clinical studies on acupuncture show that there is no specific effect of sticking needles into acupuncture points. Choosing random points works just as well, as does poking the skin with toothpicks rather than penetrating the skin with a needle to elicit the alleged “de qi”.
Mark Tonelli, MD has problems with evidence-based medicine (EBM). He has published a few articles detailing his issues, and he makes some legitimate points. We at science-based medicine (SBM) have a few issues with the execution of EBM as well, so I am sympathetic to constructive criticism.
In an article titled: Integrating evidence into clinical practice: an alternative to evidence-based approaches. The abstract states:
Evidence-based medicine (EBM) has thus far failed to adequately account for the appropriate incorporation of other potential warrants for medical decision making into clinical practice. In particular, EBM has struggled with the value and integration of other kinds of medical knowledge, such as those derived from clinical experience or based on pathophysiologic rationale. The general priority given to empirical evidence derived from clinical research in all EBM approaches is not epistemically tenable. A casuistic alternative to EBM approaches recognizes that five distinct topics, 1) empirical evidence, 2) experiential evidence, 3) pathophysiologic rationale, 4) patient goals and values, and 5) system features are potentially relevant to any clinical decision. No single topic has a general priority over any other and the relative importance of a topic will depend upon the circumstances of the particular case. The skilled clinician must weigh these potentially conflicting evidentiary and non-evidentiary warrants for action, employing both practical and theoretical reasoning, in order to arrive at the best choice for an individual patient.
I often receive e-mail from SBM readers (or SGU listeners) who have had the experience of their doctor, nurse, dentist, physical therapist, or other health care provider recommending to them a treatment option that seems dubious, if not outright pseudoscientific. They want advice on what to do. There are common themes to the e-mails – the writer often feels very uncomfortable in the situation. They do not feel comfortable confronting their provider directly, yet they do not want to acquiesce to the advice either. They are also often asking my opinion about the advice – is it really as wacky as it seems. This uncertainty saps them of their resolve, leaving them feeling a bit helpless.
Here is one such e-mail:
Ten days ago, my wife and I welcomed our first child into the world. She was born a couple weeks early, which left her mouth a bit too small and week to breastfeed effectively. To prevent her from losing too much weight, we were referred to a lactation consultant (who works out of the pediatrics department at the hospital where our daughter was born). This consultant (who is also an RN) suggested a regimen of supplementing nursing with pumped breast milk.
This was working great until my wife’s milk production dropped the day before our follow-up appointment. When we asked what to do about this, the nurse recommended that my wife take fenugreek, an herbal supplement. I was a bit skeptical of this advice, so I asked what it was about fenugreek that helped with milk production. The lactation nurse’s answer was vague — she said things like, Herbs can be helpful for lots of health issues, and, a lot of women I see seem to think it helps (oh, the logical fallacies). When we pushed her on this a little more, she handed us a flyer, printed by the hospital about fenugreek. The flyer seemed to support the use of the supplement, but mentioned that there was no scientific research demonstrating that fenugreek increases milk supply. When we asked why it hadn’t been researched, the nurse responded that there wasn’t a lot of money in lactation and that scientists generally aren’t interested in the kind of things she does (basically, that she was doing the good work that cold-hearted scientists refused to do).
“You are not going to change what we do, you’re not going to change our determination to make these patients better. I see these patients, I know these patients, I value these patients, I’ve looked after them for years. I’ve seen them after the procedure, the vast majority are improved.”
The above quote could be a reference to just about any fringe medical treatment. It is partly an expression of faith in anecdotal experience over scientific evidence. It is partly the fallacy of justifying a treatment because it is needed – whereas the real question is whether or not the treatment works. It is an attempt to justify specific claims with compassion, as if the person quoted cares more for the health of their patients than those who might be skeptical of their claims. And it is an expression of stubbornness – I know the truth, so don’t confuse me with evidence and logic.
Is this person talking about acupuncture? Perhaps they run a stem cell clinic in China, India or somewhere outside the reach of regulation. Or maybe they are defending hyperbaric oxygen therapy for unproven indications, like autism. It could be anything, because this sentiment is the standard mantra of the dubious practitioner, practicing outside the bounds of science-based medicine.
It has long been recognized that there are substantial multifactorial placebo effects that create real and illusory improvements in response to even an inactive treatment. There is a tendency, however (especially in popular discussion), to oversimplify placebo effects – to treat them as one mind-over-matter effect for all outcomes. Meanwhile researchers are elucidating the many mechanisms that go into measured placebo effects, and the differing magnitude of placebo effects for different outcomes.
For example, placebo effects for pain appear to be maximal, while placebo effects for outcomes like cancer survival appear to be minimal.
A recent study sheds additional light on the expectation placebo effect for pain. The effect is, not surprisingly, substantial. However it does not extrapolate to placebo effects for outcomes other than pain, and the results of this very study give some indication why. From the abstract:
The effect of a fixed concentration of the μ-opioid agonist remifentanil on constant heat pain was assessed under three experimental conditions using a within-subject design: with no expectation of analgesia, with expectancy of a positive analgesic effect, and with negative expectancy of analgesia (that is, expectation of hyperalgesia or exacerbation of pain).
What they found was that the positive expectation group reported twice the analgesic effect as the no expectation group, and the negative expectation group reported no analgesic effect. This is a dramatic effect, but not surprising.
Last week David Gorski wrote a excellent post about why we have not yet cured cancer. It turns out, cancer is a category of many individual diseases that are very challenging to treat. We have made steady progress, and many people with cancer can now be cured – but we have not discovered the one cure for all cancer. I personally am not convinced that we will discover a single cure for all cancer, at least not with any extrapolation of current technology. But if we continue to make progress as we are cancer will become an increasingly treatable and even curable type of disease.
This topic also brings up a meme that has been around for a long time – the notion that scientists have already cured cancer but the cure is being suppressed by the powers that be, to protect cancer as a source of income. In the comments to David’s article, Zuvrick writes:
So we can find a cure. It has probably happened multiple times. But nobody wants to cure cancer. Too many researchers earn a living seeking a cure by remaining inside a narrow, restricted channel of dogma. Their institutions get grant money and survive from the funding. Big Pharma makes big bucks selling chemotherapy drugs, surgeons remove tumors and various radiation devices employ radiologists and firms making these machines. MRI and CT scans would not be needed for cancer if Rife technology were available today.
I have heard or read some version of this claim since before I entered medical school. Superficially it may sound like profound wisdom (cynicism is a cheap way to sound wise) – but the idea collapses under the slightest bit of logical scrutiny.
For the last week I have had a cold. I usually get one each winter. I have two kids in school and they bring home a lot of viruses. I also work in a hospital, which tends (for some reason) to have lots of sick people. Although this year I think I caught my cold while traveling. I’m almost over it now, but it’s certainly a miserable interlude to my normal routine.
One thing we can say for certain about the common cold – it’s common. It is therefore no surprise that there are lots of cold remedies, folk remedies, pharmaceuticals, and “alternative” treatments. Finding a “cure for the common cold” has also become a journalistic cliche – reporters will jump on any chance to claim that some new research may one day lead to a cure for the common cold. Just about any research into viruses, no matter how basic or preliminary, seems to get tagged with this headline. (It’s right up there with every fossil being a “missing link.”)
But despite the commonality of the cold, the overall success of modern medicine, and the many attempts to treat or prevent the cold – there are very few treatments that are actually of any benefit. The only certain treatment is tincture of time. Most colds will get better on their own in about a week. This also creates the impression that any treatment works – no matter what you do, your symptoms are likely to improve. It is also very common to get a mild cold that lasts just a day or so. Many people my feel a cold “coming on” but then it never manifests. This is likely because there was already some partial immunity, so the infection was wiped out quickly by the immune system. But this can also create the impression that whatever treatment was taken at the onset of symptoms worked really well, and even prevented the cold altogether.
Last year it was reported that there was a possible increase in narcolepsy, a sleep disorder characterized by excessive sleepiness, in children who had received the Pandemrix brand of H1N1 flu vaccine in Sweden, Finland, and Iceland. However a review of the data did not find a convincing connection, although concluded there was insufficient data at present and recommended further surveillance. A narcolepsy task force was formed in Finland, and now we have their preliminary report.
They conclude that the evidence suggests there is a connection:
Based on the preliminary analyses, the risk of falling ill with narcolepsy among those vaccinated in the 4-19 years age group was 9-fold in comparison to those unvaccinated in the same age group. This increase was most pronounced among those 5–15 years of age. No cases were observed among those under 4 years of age. Also, no increase in cases of narcolepsy or signs of vaccination impacting risk of falling ill with narcolepsy was observed among those above 19 years of age.
The World Health Organization (WHO) has reviewed these results and concluded:
WHO’s Global Advisory Committee on Vaccine Safety (GACVS) reviewed this data by telephone conference on 4 February 2011. GACVS agrees that further investigation is warranted concerning narcolepsy and vaccination against influenza (H1N1) 2009 with Pandemrix and other pandemic H1N1 vaccines. An increased risk of narcolepsy has not been observed in association with the use of any vaccines whether against influenza or other diseases in the past. Even at this stage, it does not appear that narcolepsy following vaccination against pandemic influenza is a general worldwide phenomenon and this complicates interpretation of the findings in Finland.
During my recent stint covering the Neuro ICU I noticed for the first time a checklist posted above each patient bed. The checklist covered the steps to undergo whenever performing an invasive procedure on the patient. I was glad to see that the checklist phenomenon had penetrated my hospital, although the implementation of safety checklists is far from complete.
A recent study published in the BMJ offers support for the efficacy of using checklists to reduce complications and improve patient outcomes. This is a retrospective study looking at mortality and length of stay in Michigan area ICUs, comparing those that had implemented the Michigan Keystone ICU project (including a safety checklist for the placement of central lines) with local ICUs that had not implemented the project. They found a 10% decrease in overall mortality, but the results were not significant for length of stay. Because this was a retrospective study it was not designed to prove cause and effect, but it is highly suggestive of the efficacy of implementing such checklists.
The checklist trend represents a culture change within medicine – and a good one. This change received its greatest boost with the publication of The Checklist Manifesto by Dr. Atul Gawande. He presents a compelling case for the need and efficacy of using checklists in order to minimize error.
A recent Cochrane review of the use of cholesterol-lowering statin drugs in primary prevention has sparked some controversy. The controversy is not so much over what the data says, but in what conclusions to draw from the data.
Statin drugs have been surrounded by controversy for a number of reasons. On the one hand they demonstrably lower cholesterol, and the evidence has shown that they also reduce the incidence of heart attacks and strokes. The data on whether or not they reduce mortality has been less clear, although this latest data actually supports that claim. However, statins have also been blockbuster drugs for pharmaceutical companies and this has spawned concerns (some might say paranoia) that drug companies are pushing billions of dollars worth of marginally effective drugs onto the public.
So are statins a savior or a scam? Life does not always provide nice clean answers to such simple dichotomies. The evidence clearly shows that statins work and are safe. However, pharmaceutical companies do like to present their data in the best light possible, and they need to be watched closely for this. The recent review does call them on some practices that might tend to exaggerate the utility of statins. Finally, the real question comes down to – where should we draw the line in terms of cost-benefit of a preventive measure like statins.
Let’s look as this recent review of the data to see what it actually shows.