So far I have explained why most research (if not carefully designed) will lead to a false positive result. This inherent bias is responsible for many of the illusionary treatment benefits that we hear about so commonly through the media (whether they’re reporting about CAM or Western medicine), because it is their job to relay information in an entertaining way more so than an accurate manner (i.e. good science makes bad television).Then I explained a three step process for determining the trustworthiness of health news and research. We can remember these steps with a simple mnemonic: C-P-R.
The C stands for credibility– in other words, “consider the source” – is the research published in a top tier medical journal with a scientifically rigorous review process?
The P stands for plausibility– is the proposed finding consistent with known principles of physics, chemistry, and physiology or would accepting the result require us to suspend belief in everything we’ve learned about science to date?
And finally we arrive at R – reproducibility. If the research study were repeated, would similar results be obtained? (more…)
I recently wrote an article for a community newspaper attempting to explain to scientifically naive readers why testimonial “evidence” is unreliable; unfortunately, they decided not to print it. I considered using it here, but I thought it was too elementary for this audience. I have changed my mind and I am offering it below (with apologies to the majority of our readers), because it seems a few of our readers still don’t “get” why we have to use rigorous science to evaluate claims. People can be fooled, folks. All people. That includes me and it includes you. Richard Feynman said
The first principle is that you must not fool yourself–and you are the easiest person to fool.
Science is the only way to correct for our errors of perception and of attribution. It is the only way to make sure we are not fooling ourselves. Either Science-Based Medicine has not done a good job of explaining these vital facts, or some of our readers are unable or unwilling to understand our explanations.
Our commenters still frequently offer testimonials about how some CAM method “really worked for me.” They fail to understand that they have no basis for claiming that it “worked.” All they can really claim is that they observed an improvement following the treatment. That could indicate a real effect or it could indicate an inaccurate observation or it could indicate a post hoc ergo propter hoc error, a false assumption that temporal correlation meant causation. Such observations are only a starting point: we need to do science to find out what the observations mean. (more…)
First some background. I was first directed to the Marshall protocol by a reader who wondered about the information the found on the web. So I went to the web and looked at the available information, much as any patient would, and discussed what I found there.
I have subsequently been lead to believe that none of the information on the website http://www.marshallprotocol.com can be considered up to date or accurate. As as result of, I have told that my post is chockablock with errors, although, outside of writing doxycycline where I should have put minocycline, I am left in the dark as to exactly what my errors are. I am told that it is my responsibility to locate the errors in the last post, yet I can find none when compared to the website.
However, to remedy the deficiency of having reviewed inaccurate and out of date material, I have been sent 6 articles that I am informed represent the state of the art in understanding the science behind the Marshall protocol. Ah, the peer reviewed medical literature. An opportunity to carefully read and critique new ideas. It is one of the reasons people publish: to see if their ideas can withstand the scrutiny of others.
Several of these papers concern Vitamin D, the Vitamin D receptor, and olmesartan which I will review, perhaps, another time. I don’t find them a compelling read, but it not an area about which I have more than a standard medical knowledge. The other papers concern the role of infection in autoimmune diseases, which I will discuss here. It is easier as an infectious disease doctor to read this literature as I am, as least as far as the American Board on Internal Medicine is concerned, a specialist in the field. Alternatively, I am a closed minded tool of the medical industrial complex who only seeks to push his own twisted, narrow agenda at the expense of suffering patients (1). We can’t all be perfect.
It is my contention that terms such as “complementary and alternative medicine” and “integrative medicine” exist for two primary purposes. The first is marketing – they are an attempt at rebranding methods that do not meet the usual standards of unqualified “medicine”. The second is a very deliberate and often calculating attempt at creating a double standard.
We already have a standard of care within medicine, and although its application is imperfect its principles are clear – the best available scientific evidence should be used to determine that medical interventions meet a minimum standard of safety and effectiveness. Regulations have largely (although also imperfectly) reflected that principle, as have academia, publishing standards, professional organizations, licensing boards, and product regulation.
With the creation of the new brand of medicine (CAM and integrative) came the opportunity to change the rules of science and medicine to create an alternative standard, one tailor made for those modalities that do not meet existing scientific and even ethical standards for medicine. This manifests in many ways – the NCCAM was created so that these modalities would have an alternate standard for garnering federal dollars for research. Many states now have “health care freedom laws” which create a separate standard of care (actually an elimination of the standard of care) for self-proclaimed “alternative” practices.
One thing I always encourage my residents and students to do is to go to primary sources. If someone tells you that thiazide diruetics should be the first line treatment for hypertension, get on MedLine and see if that assertion is congruent with the evidence. It’s important to see how we arrive at broad treatment recommendations, how strong and consistent the evidence is, and the best way to do this is go back to the beginning.
This is not an explicitly political blog, and for that reason, I don’t feel it’s appropriate for me to advocate for one or another proposed health care reform plan. But I do want to encourage everyone to follow health care reform closely, and to go to the primary sources. Certain aspects of the proposed bill will be hard for any of us to understand, especially cost. There are all sorts of wild claims about how much reform will save us or cost us, and I’m betting that none of these claims is completely congruent with the truth. But some of what we’re hearing on the news is so far from the truth that to call them lies would be generous. (more…)
If there’s one thing we emphasize here on the Science-Based Medicine blog, it’s that the best medical care is based on science. In other words, we are far more for science-based medicine, than we are against against so-called “complementary and alternative medicine” (CAM). My perspective on the issue is that treatments not based on science need to be either subjected to scientific scrutiny if they have sufficient prior plausibility or strong clinical data suggesting efficacy or abandoned if they do not.
Unfortunately, even though the proportion of medical therapies not based on science is far lower than CAM advocates would like you to believe, there are still more treatments in “conventional” medicine that are insufficiently based on science or that have never been validated by proper randomized clinical trials than we as practitioners of science-based medicine would like. This is true for some because there are simply too few patients with a given disease; i.e., the disease is rare. Indeed, for some diseases, there will never be a definitive trial because they are just too uncommon. For others, it’s because of what I like to call medical fads, whereby a treatment appears effective anecdotally or in small uncontrolled trials and, due to the bandwagon effect, becomes widely adopted. Sometimes there is a financial incentive for such treatments to persist; sometimes it’s habit. Indeed, there’s an old saying that, for a treatment truly to disappear, the older generation of physicians has to retire or die off.
That is why I consider it worthwhile to write about a treatment that appears to be on the way to disappearing. At least, I hope that’s what’s going on. It’s also a cautionary tale about how the very same sorts of factors, such as placebo effects, reliance on anecdotal evidence, and regression to the mean, can bedevil those of us dedicated to SBM just as much as it does the investigation of CAM. It should serve as a warning to those of us who might feel a bit too smug about just how dedicated to SBM modern medicine is. Given that the technique in question is an invasive (although not a surgical technique), I also feel that it is my duty as the resident surgeon on SBM to tackle this topic. On the other hand, this case also demonstrates how SBM is, like the science upon which it is based, self-correcting. The question is: What will physicians do with the most recent information from very recently reported clinical trials that clearly show a very favored and lucrative treatment does not work better than a placebo?
Here’s the story that illustrates these issues, fresh from the New York Times this week:
Let’s look at one example.
A unknown number of Functional Medicine adherents broadcast call-in programs on radio stations. One FM physician, a Dr. “D” in Northern California graduated from UC Davis School of Medicine (Central California’s Sacramento Valley.) I find her program fascinating, requiring some attentive listening.
Dr. D’s recommendations for people’s complaints and conditions are often complex, a chimera of standard explanations and therapies, but painted with a variety of views that are anything but standard. The problem I found was that some of each answer was rational – especially the logic of her differential diagnosis – but suddenly spun out into space with unfamiliar methods or some recognizable as one component or another of sectarianism. Some answers had no relationship to the problem at hand, but seemed to be plucked out of a firmament of independent ideas, theories, ideologies, and personal anecdotes – a medical Separate Reality.
One can be carried along by an answer that sounds on surface reasonable because of the confidence and the delivery’s vocal tone. Her voice is medium-low, sort of a mezzo or contralto. It’s a voice ideal for advice; confidence oozes. Some of her separate reality recommendations she precedes with a biochemical or physiological explanation, so the shifting from standard to “separate reality” grids goes so smoothly, the usual recognizable red flags may not spring up.
Reuters recently reported on the raid of a stem-cell clinic in Hungary. This is welcome news, if the allegations are correct, but really is only scratching the surface of this problem – clinics offering dubious stem cell therapies to desperate patients. And in fact this is only one manifestation of a far greater problem – the quack clinic. They represent a serious problem for patients, doctors, and health care regulation.
Stem Cell Clinics
There is a very disturbing trend in the last few years – the proliferation of clinics offering stem cell therapy for a variety of serious, often incurable, diseases such as spinal cord injury, ALS, Parkinson’s disease, and other neurological disorders. These clinics claim to improve and even cure these diseases by injecting stem cells into the spinal cord or other parts of the body. Treatments typically cost 20-25,000 dollars, plus travel expenses, for a single treatment.
The problem is that these clinics do not have any published evidence that their treatments are valid. There is good reason to think that they are not – stem cell technology is simply not at the point yet where we can use them to cure such diseases. There are many technical hurdles to be overcome first – knowing how to control the stem cells, to get them to survive and become the types of cells necessary to have the desired therapeutic effect, and also figuring out how to keep them from growing into tumors. Basic issues of safety have not yet been sorted out.
The more things change, the more they stay the same.
Not every post will be an in-depth, authoritative review of a topic like yesterday’s on Dr. Sears. A change of pace can be nice, and I have always liked history.
JAMA likes to run articles called “JAMA 100 YEARS AGO” and the reprint from the July 24, 1909 issue is interesting. It is called BUTTERMILK THERAPY. They liked all caps at the turn of the century.
1909 was at the very beginning of the biologic sciences and the understanding of disease pathophysiology. Physicians had almost no useful, or more importantly, rigorously tested therapeutic interventions for diseases. So they relied on traditional method of determining what worked: expert opinion and anecdote. And that lead to buttermilk.
“Metchnikoff, Massol and several other authors have recommended fermented sour milk as prepared in Bulgaria, or a similar product, prepared according to Metchnikoff’s method from pure cultures of bacteria, as a panacea for many ills.”
Metchnikoff was a Russian microbiologist who won a Nobel prize in 1908 for discovering phagocytosis and was responsible for many early discoveries in the immune system and in host-bacterial interactions. As a preeminent scientist of the time, his word was respected and carried weight. As a side note, when he tried to commit suicide he did so in a manner that would benefit science: he injected himself with the relapsing fever organisms and proved it could be blood borne, and while he evidently became very ill in the process, it failed to kill him.