As I mentioned in a not-so-recent blog post, I’ve often marveled at the marketing successes of “integrative” medicine – a practice which generally refers to the act of blending effective treatments with ineffective or untested/equivocal ones. Only the marketing elite know how integrative medicine became associated with enlightenment. Perhaps they’re the very same people who came up with the idea that women should be excessively concerned about cellulite? Thanks a lot, guys.
But I do find a lot of integrative medicine vexing because it often starts with a grain of truth, and then usually proceeds to make wildly exaggerated claims about its efficacy.
So far I have explained why most research (if not carefully designed) will lead to a false positive result. This inherent bias is responsible for many of the illusionary treatment benefits that we hear about so commonly through the media (whether they’re reporting about CAM or Western medicine), because it is their job to relay information in an entertaining way more so than an accurate manner (i.e. good science makes bad television).Then I explained a three step process for determining the trustworthiness of health news and research. We can remember these steps with a simple mnemonic: C-P-R.
The C stands for credibility- in other words, “consider the source” – is the research published in a top tier medical journal with a scientifically rigorous review process?
The P stands for plausibility- is the proposed finding consistent with known principles of physics, chemistry, and physiology or would accepting the result require us to suspend belief in everything we’ve learned about science to date?
And finally we arrive at R – reproducibility. If the research study were repeated, would similar results be obtained? (more…)
In part 2 of the Science-Based Medicine 101 series we take a look at the second pillar of good science: plausibility. This blog post was written for a lay audience so more advanced readers will need to indulge me here…
I really enjoy sci-fi action movies. I love the convincing special effects and the fact that heroes can accomplish the physically impossible without skipping a beat. Implausible events unfurl with convincing reality, and you never know what might happen with the plot.
I also enjoy the TV show, America’s Funniest Home Videos, for different reasons. The mundane nature of actual reality, and the often predictable, but hilarious mistakes made by those I relate to result in some pretty hearty laughs.
But there is a big difference between these two forms of entertainment: science-fiction requires the suspension of belief in plausibility, while home videos are based on plausible outcomes. When it comes to medical research, though, plausibility can mean the difference between science fiction and reality.
The more things change, the more they stay the same.
Not every post will be an in-depth, authoritative review of a topic like yesterday’s on Dr. Sears. A change of pace can be nice, and I have always liked history.
JAMA likes to run articles called “JAMA 100 YEARS AGO” and the reprint from the July 24, 1909 issue is interesting. It is called BUTTERMILK THERAPY. They liked all caps at the turn of the century.
1909 was at the very beginning of the biologic sciences and the understanding of disease pathophysiology. Physicians had almost no useful, or more importantly, rigorously tested therapeutic interventions for diseases. So they relied on traditional method of determining what worked: expert opinion and anecdote. And that lead to buttermilk.
“Metchnikoff, Massol and several other authors have recommended fermented sour milk as prepared in Bulgaria, or a similar product, prepared according to Metchnikoff’s method from pure cultures of bacteria, as a panacea for many ills.”
Metchnikoff was a Russian microbiologist who won a Nobel prize in 1908 for discovering phagocytosis and was responsible for many early discoveries in the immune system and in host-bacterial interactions. As a preeminent scientist of the time, his word was respected and carried weight. As a side note, when he tried to commit suicide he did so in a manner that would benefit science: he injected himself with the relapsing fever organisms and proved it could be blood borne, and while he evidently became very ill in the process, it failed to kill him.
In searching for just what FM is, one has to in a way read between lines. Claiming to treat the “underlying cause” of a condition raises the usual straw man argument that modern medicine does not, which of course is untrue. It also implies that there are underlying causes known to them and not to straights. FM claims to treat chronic disease which FM claims is inadequately treated by medicine. FM claims to be a more advanced approach both in conceptual thinking and in practical management. Such claims are on the face doubtful, but hard to disprove. The way to find out would be to analyze cases they manage and critique them.
I tried to see specific examples of treatments but the web page text book links were not working at the time. I understand others have seen the contents and perhaps can add some information. I sense a difference between “CAM” and FM – at least among the MDs and DOs - is that FMers tend to use methods and substances with some degree of scientific or biochemical rationale, even if not proved, moreso than many of the CAMers. Many seem to practice both systems or do not distinguish between the two systems. In order to get a sense of the degree to which FM is known, I requested from the web page the names of practitioners in a 50 mile radius of my home (near Palo Alto, Calif.). The names ranged from Santa Cruz (40 miles) to Berkeley (50) and San Francosco (40) and Marin County (Sausalito – 50 miles) The population of that area is about 5 million. They sent 46 names: MD/DO 31 – (including a nephrologist formerly on the staff of my teaching hospital) PhD 1 DC 8 Lac 3 ND 2 RN 1 Because I had become aware of FM only 1-2 years ago, I thought 46 was a relatively large number. The Web page lists four text books published in the past few years. A manuscript of the first one is available on line for downloading (not functioning when I tried.) . 21st Century Medicine: A New Model for Medical Education and PracticeMonograph Set – Functional Medicine Clinical Monograph Set – CME Available Textbook of Functional Medicine Clinical Nutrition: A Functional ApproachAs mentioned, I could not activate the links to those books, and did not have time to get to them individually. No authors were listed.
Revised 7/23/9 to correct an error.
While there are many taxonomies of alternative medicines, one thing almost all alternative therapies have in common is they are originally the de novo discovery of one lone individual. Working outside of the mainstream, they are the gadflies who see farther because those around them are midgets.
- Hanneman conceives of homeopathy, the treatment of all disease.
- Palmer conceives the cause of all disease and its treatment in chiropractic
- Mikao Usui, while having a mid-life crisis, conceives Reiki.
Virgin births all. These pioneers boldly go where no man has gone before.
Others have been less acclaimed after seeking out new life. An example is Virginia Livingston, MD, the discoverer of the cause of all cancer (1). She discovered a bacterium, the cause of cancer, she called Progenitor cryptocides, which, unfortunately only she could grow. Her therapies include an autogenous ‘vaccine” made from your own urine, which will probably preclude widespread use even in alternative therapies circles. I wonder if Jenny would object to vaccines if there were naturally derived from the patients urine?
Discovering a new form of pathogenic microbiology that no one else can see or grow is not uncommon, since people seem to be unable to recognise artifact on slides, be it Oscillococcinum being seen by Joseph Roy 200 years ago or Virginia Livingston in the 1960s. Sometimes I regret the discovery of H. pylori as a cause of gastritis as it gives the alternative microbiologists a medical Galileo to point at. H. pylori is used as an example, erroneously, of a bacteria causing disease that was laughed at by the medical establishment (Parenthetically, as my flawed memory has it, while I was an Infectious Disease Fellow the data for H. pylori came trickling in. I remember discussing the papers with one of my attendings who was an expert in GI infections. We all thought it was an interesting hypothesis and waited further data with interest. I cannot remember anyone dismissing the idea out of hand with derisive laughter. But then, I remain convinced that infections are the cause of all disease, at least the diseases that matter).
On July 9th we held our first Science Based Medicine conference in Las Vegas. The event was definitely a success – we filled our room to capacity (150 attendees) and almost everyone stayed until the end. It also appeared that most attendees were actually awake, a rarity for a full-day medical conference. The Q&A session at the end was lively and interesting.
Kimball Atwood and I covered the history of science-based medicine and explored the differences between EBM and SBM. David Gorski discussed cancer quackery, including specific cases to illustrate the potential harm of pursuing worthless therapies for serious diseases. Harriet Hall gave us an overview of the the pseudoscience endemic in chiropractic. Mark Crislip discussed the chronic Lyme disease controversy. And Val Jones discussed health information online.
Based upon the feedback from those attending the conference, as well as the general enthusiasm, it seems that there is a hunger for this type of information. The audience was split about even between health care professionals and interested lay public. Many people asked if we plan on giving the conference again, and the answer is definitely yes. We have no plans set as of yet, and will certainly announce any future conferences here.
In the three prior posts of this series I tried to analyze some of the defects in the randomized clinical rials (RCTs) of homeopathic remedies for childhood diarrhea. The first entry showed that the first two RCTs’ (done in Nicaragua) methods could not produce a meaningful result because of the way the RCTs were set up (methods.) The second entry showed that the results obtained in the first two trials were meaningless clinically even if assumed to have resulted from more legitimate methods. The same applied to the third trial in Nepal, analyzed in the third entry.
This entry will suggest that the authors’ fourth paper (Jacobs J, Jonas WB, Jimenez-Perez M, Crothers D. Homeopathy for childhood diarrhea: combined results and metaanalysis from three randomized, controlled clinical trials. Pediat Inf Dis J, 2005;22:229-234.)- a meta-analysis (MA) of the data from the three RCTs resulted in conclusions equally as meaningless as those of the three trials.
The MA authors – several of the same workers from the three RCTs – begin by agreeing that the data from the RCTs, taken individually, were of borderline significance:
In our previous three studies, we evaluated the use of individualized homeopathic treatment of childhood diarrhea … The results of the two larger studies (n = 81, n = 116) were just at or near level of statistical significance. Because all three studies followed the same basic study design , […] we analyzed the combined data from these three studies to obtain greater statistical power. In addition we conducted a meta-analysis of effect-size difference […] to look for consistency of effects.
MAs and systematic reviews (SRs) are the two consensus methods for summarizing data from multiple individual studies. The inclusion and search methods of RCTs for SRs and MAs are similar, but the objectives of the two are a bit different, as are the forms of the reports. In SRs, the results are summarized in more in narrative form, whereas in MAs the data are treated mathematically and the results are defined in statistical terms. Thus authors of SRs are freer to speculate on the degree of confidence that a method is effective based on what is shown by the numbers of positive and negative RCTs collected. Authors of MAs usually limit their comments to what the mathematical formulation of the summarized data show.
An audience member at a recent NYC Skeptics meeting asked me how I handled conflict surrounding strongly held beliefs that are not supported by conclusive evidence. As a dentist, he argued, he often witnessed professionals touting procedure A over procedure B as the “best way” to do X, when in reality there are no controlled clinical trials comparing A and B. “How am I to know what’s right in these circumstances?” He asked.
And this is more-or-less what I said:
The truth is, you probably can’t know which procedure is better. At least, not at this point in history. The beauty of science is that it’s evolving. We are constantly learning more about our bodies and our environment, so that we are getting an ever-clearer degree of resolution on what we see and experience.
It’s like having a blurry camera lens at a farm. At first we can only perceive that there are living things moving around on the other side of the lens – but as we begin to focus the camera, we begin to make out that the animals are in the horse or cattle family. With further focus we might be able to differentiate a horse from a cow… and eventually we’ll be able to tell if the horse has a saddle on it, and maybe one day we’ll be able to see what brand of saddle it is. Each scientific conundrum that we approach is often quite blurry at the onset. People get very invested in their theories of the presence or absence of cows, and whether or not the moving objects could in fact be horses. Others say that those looking through the camera contradict one another too much to be trusted – that they must be offering false ideas or willfully misleading people about the picture they’re describing.
In fact, we just have different degrees of clarity on issues at any given point in time. This is not cause for alarm, nor is it a reason to abandon our cameras. No, it just gives us more reason to continue to review, analyze, and revise our understanding of the picture at hand. We should try not to make more out of photo than we can at a given resolution – and understand that contradicting opinions are more likely to be evidence of insufficient information than a fundamental flaw of the scientific method.