In part 2 of the Science-Based Medicine 101 series we take a look at the second pillar of good science: plausibility. This blog post was written for a lay audience so more advanced readers will need to indulge me here…
I really enjoy sci-fi action movies. I love the convincing special effects and the fact that heroes can accomplish the physically impossible without skipping a beat. Implausible events unfurl with convincing reality, and you never know what might happen with the plot.
I also enjoy the TV show, America’s Funniest Home Videos, for different reasons. The mundane nature of actual reality, and the often predictable, but hilarious mistakes made by those I relate to result in some pretty hearty laughs.
But there is a big difference between these two forms of entertainment: science-fiction requires the suspension of belief in plausibility, while home videos are based on plausible outcomes. When it comes to medical research, though, plausibility can mean the difference between science fiction and reality.
The more things change, the more they stay the same.
Not every post will be an in-depth, authoritative review of a topic like yesterday’s on Dr. Sears. A change of pace can be nice, and I have always liked history.
JAMA likes to run articles called “JAMA 100 YEARS AGO” and the reprint from the July 24, 1909 issue is interesting. It is called BUTTERMILK THERAPY. They liked all caps at the turn of the century.
1909 was at the very beginning of the biologic sciences and the understanding of disease pathophysiology. Physicians had almost no useful, or more importantly, rigorously tested therapeutic interventions for diseases. So they relied on traditional method of determining what worked: expert opinion and anecdote. And that lead to buttermilk.
“Metchnikoff, Massol and several other authors have recommended fermented sour milk as prepared in Bulgaria, or a similar product, prepared according to Metchnikoff’s method from pure cultures of bacteria, as a panacea for many ills.”
Metchnikoff was a Russian microbiologist who won a Nobel prize in 1908 for discovering phagocytosis and was responsible for many early discoveries in the immune system and in host-bacterial interactions. As a preeminent scientist of the time, his word was respected and carried weight. As a side note, when he tried to commit suicide he did so in a manner that would benefit science: he injected himself with the relapsing fever organisms and proved it could be blood borne, and while he evidently became very ill in the process, it failed to kill him.
In searching for just what FM is, one has to in a way read between lines. Claiming to treat the “underlying cause” of a condition raises the usual straw man argument that modern medicine does not, which of course is untrue. It also implies that there are underlying causes known to them and not to straights. FM claims to treat chronic disease which FM claims is inadequately treated by medicine. FM claims to be a more advanced approach both in conceptual thinking and in practical management. Such claims are on the face doubtful, but hard to disprove. The way to find out would be to analyze cases they manage and critique them.
I tried to see specific examples of treatments but the web page text book links were not working at the time. I understand others have seen the contents and perhaps can add some information. I sense a difference between “CAM” and FM – at least among the MDs and DOs - is that FMers tend to use methods and substances with some degree of scientific or biochemical rationale, even if not proved, moreso than many of the CAMers. Many seem to practice both systems or do not distinguish between the two systems. In order to get a sense of the degree to which FM is known, I requested from the web page the names of practitioners in a 50 mile radius of my home (near Palo Alto, Calif.). The names ranged from Santa Cruz (40 miles) to Berkeley (50) and San Francosco (40) and Marin County (Sausalito – 50 miles) The population of that area is about 5 million. They sent 46 names: MD/DO 31 – (including a nephrologist formerly on the staff of my teaching hospital) PhD 1 DC 8 Lac 3 ND 2 RN 1 Because I had become aware of FM only 1-2 years ago, I thought 46 was a relatively large number. The Web page lists four text books published in the past few years. A manuscript of the first one is available on line for downloading (not functioning when I tried.) . 21st Century Medicine: A New Model for Medical Education and PracticeMonograph Set – Functional Medicine Clinical Monograph Set – CME Available Textbook of Functional Medicine Clinical Nutrition: A Functional ApproachAs mentioned, I could not activate the links to those books, and did not have time to get to them individually. No authors were listed.
Revised 7/23/9 to correct an error.
While there are many taxonomies of alternative medicines, one thing almost all alternative therapies have in common is they are originally the de novo discovery of one lone individual. Working outside of the mainstream, they are the gadflies who see farther because those around them are midgets.
Hanneman conceives of homeopathy, the treatment of all disease.
Palmer conceives the cause of all disease and its treatment in chiropractic
Mikao Usui, while having a mid-life crisis, conceives Reiki.
Virgin births all. These pioneers boldly go where no man has gone before.
Others have been less acclaimed after seeking out new life. An example is Virginia Livingston, MD, the discoverer of the cause of all cancer (1). She discovered a bacterium, the cause of cancer, she called Progenitor cryptocides, which, unfortunately only she could grow. Her therapies include an autogenous ‘vaccine” made from your own urine, which will probably preclude widespread use even in alternative therapies circles. I wonder if Jenny would object to vaccines if there were naturally derived from the patients urine?
Discovering a new form of pathogenic microbiology that no one else can see or grow is not uncommon, since people seem to be unable to recognise artifact on slides, be it Oscillococcinum being seen by Joseph Roy 200 years ago or Virginia Livingston in the 1960’s. Sometimes I regret the discovery of H. pylori as a cause of gastritis as it gives the alternative microbiologists a medical Galileo to point at. H. pylori is used as an example, erroneously, of a bacteria causing disease that was laughed at by the medical establishment (Parenthetically, as my flawed memory has it, while I was an Infectious Disease Fellow the data for H. pylori came trickling in. I remember discussing the papers with one of my attendings who was an expert in GI infections. We all thought is was an interesting hypothesis and waited further data with interest. I cannot remember anyone dismissing the idea out of hand with derisive laughter. But then, I remain convinced that infections are the cause of all disease, at least the diseases that matter).
On July 9th we held our first Science Based Medicine conference in Las Vegas. The event was definitely a success – we filled our room to capacity (150 attendees) and almost everyone stayed until the end. It also appeared that most attendees were actually awake, a rarity for a full-day medical conference. The Q&A session at the end was lively and interesting.
Kimball Atwood and I covered the history of science-based medicine and explored the differences between EBM and SBM. David Gorski discussed cancer quackery, including specific cases to illustrate the potential harm of pursuing worthless therapies for serious diseases. Harriet Hall gave us an overview of the the pseudoscience endemic in chiropractic. Mark Crislip discussed the chronic Lyme disease controversy. And Val Jones discussed health information online.
Based upon the feedback from those attending the conference, as well as the general enthusiasm, it seems that there is a hunger for this type of information. The audience was split about even between health care professionals and interested lay public. Many people asked if we plan on giving the conference again, and the answer is definitely yes. We have no plans set as of yet, and will certainly announce any future conferences here.
In the three prior posts of this series I tried to analyze some of the defects in the randomized clinical rials (RCTs) of homeopathic remedies for childhood diarrhea. The first entry showed that the first two RCTs’ (done in Nicaragua) methods could not produce a meaningful result because of the way the RCTs were set up (methods.) The second entry showed that the results obtained in the first two trials were meaningless clinically even if assumed to have resulted from more legitimate methods. The same applied to the third trial in Nepal, analyzed in the third entry.
This entry will suggest that the authors’ fourth paper (Jacobs J, Jonas WB, Jimenez-Perez M, Crothers D. Homeopathy for childhood diarrhea: combined results and metaanalysis from three randomized, controlled clinical trials. Pediat Inf Dis J, 2005;22:229-234.)- a meta-analysis (MA) of the data from the three RCTs resulted in conclusions equally as meaningless as those of the three trials.
The MA authors – several of the same workers from the three RCTs – begin by agreeing that the data from the RCTs, taken individually, were of borderline significance:
In our previous three studies, we evaluated the use of individualized homeopathic treatment of childhood diarrhea … The results of the two larger studies (n = 81, n = 116) were just at or near level of statistical significance. Because all three studies followed the same basic study design , […] we analyzed the combined data from these three studies to obtain greater statistical power. In addition we conducted a meta-analysis of effect-size difference […] to look for consistency of effects.
MAs and systematic reviews (SRs) are the two consensus methods for summarizing data from multiple individual studies. The inclusion and search methods of RCTs for SRs and MAs are similar, but the objectives of the two are a bit different, as are the forms of the reports. In SRs, the results are summarized in more in narrative form, whereas in MAs the data are treated mathematically and the results are defined in statistical terms. Thus authors of SRs are freer to speculate on the degree of confidence that a method is effective based on what is shown by the numbers of positive and negative RCTs collected. Authors of MAs usually limit their comments to what the mathematical formulation of the summarized data show.
An audience member at a recent NYC Skeptics meeting asked me how I handled conflict surrounding strongly held beliefs that are not supported by conclusive evidence. As a dentist, he argued, he often witnessed professionals touting procedure A over procedure B as the “best way” to do X, when in reality there are no controlled clinical trials comparing A and B. “How am I to know what’s right in these circumstances?” He asked.
And this is more-or-less what I said:
The truth is, you probably can’t know which procedure is better. At least, not at this point in history. The beauty of science is that it’s evolving. We are constantly learning more about our bodies and our environment, so that we are getting an ever-clearer degree of resolution on what we see and experience.
It’s like having a blurry camera lens at a farm. At first we can only perceive that there are living things moving around on the other side of the lens – but as we begin to focus the camera, we begin to make out that the animals are in the horse or cattle family. With further focus we might be able to differentiate a horse from a cow… and eventually we’ll be able to tell if the horse has a saddle on it, and maybe one day we’ll be able to see what brand of saddle it is. Each scientific conundrum that we approach is often quite blurry at the onset. People get very invested in their theories of the presence or absence of cows, and whether or not the moving objects could in fact be horses. Others say that those looking through the camera contradict one another too much to be trusted – that they must be offering false ideas or willfully misleading people about the picture they’re describing.
In fact, we just have different degrees of clarity on issues at any given point in time. This is not cause for alarm, nor is it a reason to abandon our cameras. No, it just gives us more reason to continue to review, analyze, and revise our understanding of the picture at hand. We should try not to make more out of photo than we can at a given resolution – and understand that contradicting opinions are more likely to be evidence of insufficient information than a fundamental flaw of the scientific method.
On May 9th I had the pleasure of lecturing to an audience of critical thinkers at the NYC Skeptics meeting. The topic of discussion was pseudoscience on the Internet – and I spent about 50 minutes talking about all the misleading health information and websites available to (and frequented by) patients. The common denominator for most of these well-intentioned but misguided efforts is a fundamental lack of understanding of the scientific method, and the myriad ways that humans can fool ourselves into perceiving a cause and effect relationship between unrelated phenomena.
But most importantly, we had the chance to touch upon a theme that has been troubling me greatly over the past couple of years: the rise in influence of those untrained in science on matters of medicine. I have been astonished by the ability of “thought leaders” like Jenny McCarthy to gain a broad platform of influence (i.e. Oprah Winfrey’s TV network) despite her obviously flawed beliefs about the pathophysiology of autism. Why is it so hard to find a medical voice of reason in mainstream media?
The answer is probably related to two issues: first, good science makes bad television, and second, physicians are going about PR and communications in the wrong way. We are taught to put emotions aside as we carefully weigh evidence to get to the bottom of things. But we are not taught to reinfuse the subject with emotion once we’ve come to an impartial consensus. Instead, we tend to bicker about statistical analyses, and alienate John Q. Public with what appears to him as academic minutiae and hair-splitting.
I’m not sure what we can or should offer in place of our “business as usual” behavior – but I’ve noticed that being right isn’t the same as being influential. I wonder how we can better advance the cause of science (for the sake of public health at a minimum) to an audience drawn more to passion than to substance?
I would really enjoy your input, dear readers of Science Based Medicine, because I’m at a loss as to what we should do next to reach people in our current culture, and with new communications platforms. What would you recommend?
One of the more frustrating things about practitioners who promote unsafe and scientifically discredited medical practices is their tendency to change their message for different audiences. One day they’ll tell you that they espouse only evidence-based practices and the next they’ll be promoting snake oil. This double talk is hard to combat, since to disprove them one would essentially have to provide a video of their contradictory remarks.
One day I participated in a series of business meetings with a CAM practitioner in attendance (he was an MD who graduated from UCSF). During one meeting he boldly proclaimed his support of scientifically rigorous research, and praised the Cochrane Collaborative’s efforts to provide systematic reviews of the evidence (or lack thereof) for various practices.
Several hours later we were sitting together in another meeting in which I objected to the publication of a consumer article that would assist parents of children with autism in finding a DAN! practitioner who could provide chelation therapy to their children. I explained that there was no evidence for the efficacy of such treatments, and plenty of evidence for their harm (including the death of at least one child that I’d read about in the news). I suggested that an article describing these dangers might be in order, but that an article encouraging chelation use for autism was simply unethical and I would not allow it to be published.
Instead of agreeing with me, the CAM MD suggested that I was being “narrow” and that I should allow consumers to “explore all their options.” I was stunned. This was the same person who had just said that he fully supported scientific inquiry. So I asked him how he could say that he supported evidence-based medicine, and then turn around and ignore evidence at will – even at the peril of human life.
His response dumbfounded me:
“I am just as comfortable practicing within an evidence-based framework as I am outside it.”