So far I have explained why most research (if not carefully designed) will lead to a false positive result. This inherent bias is responsible for many of the illusionary treatment benefits that we hear about so commonly through the media (whether they’re reporting about CAM or Western medicine), because it is their job to relay information in an entertaining way more so than an accurate manner (i.e. good science makes bad television).Then I explained a three step process for determining the trustworthiness of health news and research. We can remember these steps with a simple mnemonic: C-P-R.
The C stands for credibility– in other words, “consider the source” – is the research published in a top tier medical journal with a scientifically rigorous review process?
The P stands for plausibility– is the proposed finding consistent with known principles of physics, chemistry, and physiology or would accepting the result require us to suspend belief in everything we’ve learned about science to date?
And finally we arrive at R – reproducibility. If the research study were repeated, would similar results be obtained? (more…)
We frequently receive requests from readers, our colleagues in medicine or fellow science bloggers for the best reference site that has all the information they need on a specific topic. There are many excellent resources on the net, but nothing I know of that quite puts it all together in that way – one-stop shopping for up-to-date information on the topics we are most concerned with.
So we decided to create just such a resource.
You will now see at the top of this page a new link for SBM Topic-Based Reference which leads to our new section by that name. There you will see the list of topics we are currently working on, and once they are complete more will be added. As of today only one topic is reasonably complete, Vaccines and Autism.
The format (which is subject to change as we build and use the resource) is as follows: We start with a brief topic overview. This is not meant to be a thorough discussion of the topic, but a quick summary to get people started. This is followed by an index of all SBM posts on that topic and then links to outside resources that we recommend.
I recently wrote an article for a community newspaper attempting to explain to scientifically naive readers why testimonial “evidence” is unreliable; unfortunately, they decided not to print it. I considered using it here, but I thought it was too elementary for this audience. I have changed my mind and I am offering it below (with apologies to the majority of our readers), because it seems a few of our readers still don’t “get” why we have to use rigorous science to evaluate claims. People can be fooled, folks. All people. That includes me and it includes you. Richard Feynman said
The first principle is that you must not fool yourself–and you are the easiest person to fool.
Science is the only way to correct for our errors of perception and of attribution. It is the only way to make sure we are not fooling ourselves. Either Science-Based Medicine has not done a good job of explaining these vital facts, or some of our readers are unable or unwilling to understand our explanations.
Our commenters still frequently offer testimonials about how some CAM method “really worked for me.” They fail to understand that they have no basis for claiming that it “worked.” All they can really claim is that they observed an improvement following the treatment. That could indicate a real effect or it could indicate an inaccurate observation or it could indicate a post hoc ergo propter hoc error, a false assumption that temporal correlation meant causation. Such observations are only a starting point: we need to do science to find out what the observations mean. (more…)
I don’t recall if I’ve mentioned it on SBM before, but I went to the University of Michigan. In fact, I didn’t go there just for undergraduate studies or medical school, but rather for both, graduating with a B.S. in Chemistry with Honors in 1984 and from medical school in 1988. In my eight years in Ann Arbor, I came to love the place, and I still have an affinity for it, even though it’s been over 20 years since I last walked about the campus as a student, although I have been back from time to time for various functions, most recently to see Brian Deer speak last winter. True, I’m not fanatical about it, as some of my contemporaries and friends who attened U. of M. with me back in the 1980s (and, sadly, the string of losses to Ohio State and the definitively mediocre last season Michigan had last year make it very hard to be a Michigan football fan these days). However, I do have considerable affection for the place. It molded me, trained me in science, taught me medicine, and provided me the basis for everything I do professionally today.
First some background. I was first directed to the Marshall protocol by a reader who wondered about the information the found on the web. So I went to the web and looked at the available information, much as any patient would, and discussed what I found there.
I have subsequently been lead to believe that none of the information on the website http://www.marshallprotocol.com can be considered up to date or accurate. As as result of, I have told that my post is chockablock with errors, although, outside of writing doxycycline where I should have put minocycline, I am left in the dark as to exactly what my errors are. I am told that it is my responsibility to locate the errors in the last post, yet I can find none when compared to the website.
However, to remedy the deficiency of having reviewed inaccurate and out of date material, I have been sent 6 articles that I am informed represent the state of the art in understanding the science behind the Marshall protocol. Ah, the peer reviewed medical literature. An opportunity to carefully read and critique new ideas. It is one of the reasons people publish: to see if their ideas can withstand the scrutiny of others.
Several of these papers concern Vitamin D, the Vitamin D receptor, and olmesartan which I will review, perhaps, another time. I don’t find them a compelling read, but it not an area about which I have more than a standard medical knowledge. The other papers concern the role of infection in autoimmune diseases, which I will discuss here. It is easier as an infectious disease doctor to read this literature as I am, as least as far as the American Board on Internal Medicine is concerned, a specialist in the field. Alternatively, I am a closed minded tool of the medical industrial complex who only seeks to push his own twisted, narrow agenda at the expense of suffering patients (1). We can’t all be perfect.
In part 2 of the Science-Based Medicine 101 series we take a look at the second pillar of good science: plausibility. This blog post was written for a lay audience so more advanced readers will need to indulge me here…
I really enjoy sci-fi action movies. I love the convincing special effects and the fact that heroes can accomplish the physically impossible without skipping a beat. Implausible events unfurl with convincing reality, and you never know what might happen with the plot.
I also enjoy the TV show, America’s Funniest Home Videos, for different reasons. The mundane nature of actual reality, and the often predictable, but hilarious mistakes made by those I relate to result in some pretty hearty laughs.
But there is a big difference between these two forms of entertainment: science-fiction requires the suspension of belief in plausibility, while home videos are based on plausible outcomes. When it comes to medical research, though, plausibility can mean the difference between science fiction and reality.
It is my contention that terms such as “complementary and alternative medicine” and “integrative medicine” exist for two primary purposes. The first is marketing – they are an attempt at rebranding methods that do not meet the usual standards of unqualified “medicine”. The second is a very deliberate and often calculating attempt at creating a double standard.
We already have a standard of care within medicine, and although its application is imperfect its principles are clear – the best available scientific evidence should be used to determine that medical interventions meet a minimum standard of safety and effectiveness. Regulations have largely (although also imperfectly) reflected that principle, as have academia, publishing standards, professional organizations, licensing boards, and product regulation.
With the creation of the new brand of medicine (CAM and integrative) came the opportunity to change the rules of science and medicine to create an alternative standard, one tailor made for those modalities that do not meet existing scientific and even ethical standards for medicine. This manifests in many ways – the NCCAM was created so that these modalities would have an alternate standard for garnering federal dollars for research. Many states now have “health care freedom laws” which create a separate standard of care (actually an elimination of the standard of care) for self-proclaimed “alternative” practices.
Chiropractic originated in 1895 when D.D. Palmer claimed to have restored deaf janitor Harvey Lillard’s hearing by manipulating his spine. This makes no anatomical sense, and few if any chiropractors claim to be able to reverse deafness today. But now a chiropractic website is attempting to vindicate D. D. Palmer. They list deafness among a long (wrong) list of “Conditions That Respond Well to Chiropractic”
They list 6 supporting studies. Three are case reports, two are case series, and one is a review of the literature that is labeled in its title as “A Search to Validate” D.D. Palmer. (more…)
One thing I always encourage my residents and students to do is to go to primary sources. If someone tells you that thiazide diruetics should be the first line treatment for hypertension, get on MedLine and see if that assertion is congruent with the evidence. It’s important to see how we arrive at broad treatment recommendations, how strong and consistent the evidence is, and the best way to do this is go back to the beginning.
This is not an explicitly political blog, and for that reason, I don’t feel it’s appropriate for me to advocate for one or another proposed health care reform plan. But I do want to encourage everyone to follow health care reform closely, and to go to the primary sources. Certain aspects of the proposed bill will be hard for any of us to understand, especially cost. There are all sorts of wild claims about how much reform will save us or cost us, and I’m betting that none of these claims is completely congruent with the truth. But some of what we’re hearing on the news is so far from the truth that to call them lies would be generous. (more…)
If there’s one thing we emphasize here on the Science-Based Medicine blog, it’s that the best medical care is based on science. In other words, we are far more for science-based medicine, than we are against against so-called “complementary and alternative medicine” (CAM). My perspective on the issue is that treatments not based on science need to be either subjected to scientific scrutiny if they have sufficient prior plausibility or strong clinical data suggesting efficacy or abandoned if they do not.
Unfortunately, even though the proportion of medical therapies not based on science is far lower than CAM advocates would like you to believe, there are still more treatments in “conventional” medicine that are insufficiently based on science or that have never been validated by proper randomized clinical trials than we as practitioners of science-based medicine would like. This is true for some because there are simply too few patients with a given disease; i.e., the disease is rare. Indeed, for some diseases, there will never be a definitive trial because they are just too uncommon. For others, it’s because of what I like to call medical fads, whereby a treatment appears effective anecdotally or in small uncontrolled trials and, due to the bandwagon effect, becomes widely adopted. Sometimes there is a financial incentive for such treatments to persist; sometimes it’s habit. Indeed, there’s an old saying that, for a treatment truly to disappear, the older generation of physicians has to retire or die off.
That is why I consider it worthwhile to write about a treatment that appears to be on the way to disappearing. At least, I hope that’s what’s going on. It’s also a cautionary tale about how the very same sorts of factors, such as placebo effects, reliance on anecdotal evidence, and regression to the mean, can bedevil those of us dedicated to SBM just as much as it does the investigation of CAM. It should serve as a warning to those of us who might feel a bit too smug about just how dedicated to SBM modern medicine is. Given that the technique in question is an invasive (although not a surgical technique), I also feel that it is my duty as the resident surgeon on SBM to tackle this topic. On the other hand, this case also demonstrates how SBM is, like the science upon which it is based, self-correcting. The question is: What will physicians do with the most recent information from very recently reported clinical trials that clearly show a very favored and lucrative treatment does not work better than a placebo?
Here’s the story that illustrates these issues, fresh from the New York Times this week: