Science-Based Medicine

On Friday, you might have noticed that Mark Crislip hinted at a foreshadowing of a blog post to come. This is that blog post. He knew it was coming because when I saw the article that inspired it, I sent an e-mail to my fellow bloggers marking out my territory like a dog peeing on every tree or protecting my newfound topic like a mother bear protecting her cubs. In other words, I was telling them all to back off. This article is mine.

Mine! Mine! Mine! I tell you!

My extreme territorial tendencies (even towards my friends and colleagues) notwithstanding on this issue aside, if you read Mark’s post (and if you didn’t go back and read it now—seriously, go now), you might also remember that he was discussing a “reality bias” in science-based medicine (SBM), a bias that we like to call prior plausibility. In brief, positive randomized clinical trials (RCTs) testing highly implausible treatments are far more likely to be false positives than RCTs testing more plausible treatments. That is the lesson that John Ioannidis has taught us and that I’ve written about multiple times before, as have other SBM bloggers, most prominently Kimball Atwood, although nearly all of us have chimed in at one time or another about this issue.

Apparently a homeopath disagrees and expressed his disagreement in an article published last week online in Medicine, Health Care, and Philosophy entitled Plausibility and evidence: the case of homeopathy. You’ll get an idea of what it is that affected us at SBM like the proverbial matador waving his cape in front of a bull by reading this brief passage from the abstract:

Prior disbelief in homeopathy is rooted in the perceived implausibility of any conceivable mechanism of action. Using the ‘crossword analogy’, we demonstrate that plausibility bias impedes assessment of the clinical evidence. Sweeping statements about the scientific impossibility of homeopathy are themselves unscientific: scientific statements must be precise and testable.

Scientific. You keep using that word. I do not think it means what you think it means. Of course, his being a homeopath is about as close to a guarantee as I can think of that a person doesn’t have the first clue what is and is not scientific. If he did, he wouldn’t be a homeopath. Still, this particular line of attack is often effective, whether yielded by a homeopath or other CAM apologist. After all, why not test these therapies in human beings and see if they work? What’s wrong with that? Isn’t it “close-minded” to claim that scientific considerations of prior plausibility consign homeopathy to the eternal dustbin of pseudoscience?

Not at all. There’s a difference between being open-minded and being so “open-minded” that your brains threaten to fall out. Guess which category homeopaths like Rutten fall into. But to hear them tell it, homeopathy is rejected because because we scientists have a “negative plausibility bias” towards it. At least, that’s what Rutten and some other homeopaths have been trying to convince us. This article seems to be an attempt to put some meat on the bones of their initial trial balloon of this argument published last summer, which Steve Novella duly deconstructed.

Before I dig in, however, I think it’s necessary for me to “confess” my bias and why I think it should be your bias too.
Continue Reading »

There have not been a lot of topics of late that warrant extensive analysis and discussion.  But there are a number of little topics of interest, each worthy of a few paragraphs of discussion, archetypes of issues in medicine, science based and otherwise.

Xigirs. No, it is not whale vomit, but close.

Last month Xigris  was pulled from the market by Lilly.  Yes, I understand the shock. Xigris, we hardly knew ye.   Xigris is the brand name for drotrecogin alfa, or activated protein C.  It is an enzyme in the clotting cascade that is/was given for the treatment of sepsis. Continue Reading »

There is a tradition in medical training called Journal Club. The first rule of Journal Club is you do not talk about Journal Club. In Journal Club, at least in the iterations in which I have participated, one article is selected by an attending, everyone reads it, then the strengths, weakness and applicability are discussed by the group. Usually a top notch, ground breaking article was the focus, one that had high potential clinical impact. But since they were good articles in good journals, there was not a lot to learn about in reguards to critical thinking. While the attending would put the article in context and maybe discuss some rudimentary statistics, there was little that was discussed about the quality of the study. The main take home from every study was to question the applicability of the results to populations that were not old, white males, since it seemed all the ground breaking studies back in the day were a VA Cooperative study of one sort or another.

As I remember it, there was not really a conceptual frame work with which to evaluate studies. Bayes theorem, and its application to clinical medicine was never explicitly discussed outside of testing, where you have to consider the prior plausibility of the patient having a disease before you can decide if the test results is a true positive or not. In Portland, Oregon, the chance that a Lyme serology is a false positive is much greater than a test done in Portland, Maine. Generally speaking in the information overload state that is the practice of medicine, clinical trials are generally taken at face value and tests are considered infallible. Which is a shame, as I wonder how much suboptimal medicine is inflicted on patients by not considering prior plausibility and how accurate a given test is in either ruling in or out a disease. There seems to be a whole industry built around treating patients with no risks for Lyme but have positive tests of doubtful provenance. We never discussed the prior plausibility and its effect on the outcomes of a studied treatment. Continue Reading »

Some of my fellow Science-Based Medicine (SBM) bloggers and I have been wondering lately what’s up with The Atlantic. It used to be one of my favorite magazines, so much so that I subscribed to it for roughly 25 years (and before that I used to read my mother’s copy). In general I enjoyed its mix of politics, culture, science, and other topics. Unfortunately, my opinion changed back in the fall of 2009, when, on the rising crest of the H1N1 pandemic, The Atlantic published what can only be described as an terrible bit of journalism lionizing the “brave maverick doctor” Tom Jefferson of the Cochrane Collaboration. The article, written by Shannon Brownlee and Jeanne Lenzer, argued, in essence, that vaccinating against H1N1 at the time was a horrendous waste of time and effort because the vaccine didn’t work. So bad was the cherry picking of data and framing of the issue as a narrative that consisted primarily of the classic lazy journalistic device of a “lone maverick” against the entire medical establishment that it earned the lovely sarcasm of our very own Mark Crislip, who wrote a complete annotated rebuttal, while I referred to the methodology presented in the article as “methodolatry.” Even public health epidemiologist Revere (who is, alas, no longer blogging but in his day provided a very balanced, science-based perspective on vaccination for influenza, complete with its shortcomings) was most definitely not pleased.

I let my subscription to The Atlantic lapse and have not to this day renewed it.

Be that as it may, last year The Atlantic published an article that wasn’t nearly as bad as the H1N1 piece but was nonetheless pretty darned annoying to us at SBM. Entitled Lies, Damned Lies, and Medical Science, by David Freedman, it was an article lionizing John Ioannidis (whom I, too, greatly admire) while largely missing the point of his work, turning it into an argument for why we shouldn’t believe most medical science. Now, Freedman’s back again, this time with a much, much, much worse story in The Atlantic in the July/August 2011 issue under the heading “Ideas” and entitled The Triumph of New Age Medicine, complete with a picture of a doctor in a lab coat in the lotus position. It appears to be the logical follow up to Freedman’s article about Ioannidis in that Freedman apparently seems to think that, if we can’t trust medical science, then there’s no reason why we shouldn’t embrace medical pseudoscience.

Basically, the whole idea behind the article appears to be that, even if most of alternative medicine is quackery (which it is, by the way, as we’ve documented ad nauseam on this very blog), it’s making patients better because of placebo effects and because its practitioners take the time to talk to patients and doctors do not. In other words, Freedman’s thesis appears to be a massive “What’s the harm?” argument coupled with a false dichotomy; that is, if real doctors don’t have the time to listen to patients and provide the human touch, then let’s let the quacks do it. Tacked on to that bad idea is a massive argumentum ad populum portraying alternative medicine as the wave of the future, in contrast to what Freedman calls the “failure” of conventional medicine.

Let’s dig in, shall we? I’ll start with the article itself, after which I’ll examine a few of the responses. I’ll also note that our very own Steve Novella, who was interviewed for Freedman’s article, has written a response to Freedman’s article that is very much worth reading as well.

Continue Reading »

From the very outset, the founders of Science Based Medicine have have emphasized the importance of plausibility in the critical evaluation of scientific claims in medicine. What exactly does “plausibility” mean, and how should we apply it in science? My simple definition of plausibility would be “the likelihood that a premise is true.” The application in science is a little more complicated.

Consciously or unconsciously, we all consider plausibility in interpreting events in our lives. For example, if one of your coworkers showed up late for work and grumbled about a traffic jam, you would likely accept his story without question. If, instead, the same coworker attributed his tardiness to an alien abduction, you would not be so charitable. In each case, he has provided the same level of evidence: his anecdotal account. You are likely to accept one story and reject the other because of a perceived difference in the plausibility. The skeptic’s mantra “Extraordinary Claims Require Extraordinary Evidence” expresses this concept in a qualitative way.

Evidence-based medicine has traditionally ignored plausibility when interpreting the evidence for a medical intervention. Science-based medicine, as envisioned by the creators of this blog, includes plausibility when making these judgements.

Since experiment research employs rigorous controls, and statistical criteria, you might assume that plausibility is not an issue, however, this is not entirely true. An article written by John Ioannidis entitled “Why Most Published Research Findings Are False” is cited frequently as a reference for the impact of plausibility on the interpretation of research results. This article enumerates numerous factor leading to erroneous research conclusions. Most of them have been dealt with on this blog at one time or another. To me, the most eye-opening aspect of the paper was a quantitive approach to the influence of plausibility in interpreting positive research findings. I was never taught this approach in medical school, or in any other venue. When it comes to implausible hypotheses, the traditional P-value can be very misleading.

As good as Ioannidis’ article is, it is not easy reading for the statistically or mathematically challenged. What I attempt to do in this post is to demonstrate the importance of plausibility in graphic format, without a lot of complex math. If you can grasp the concepts in this post, you will have an understanding that many researchers, and consumers of research, lack.
Continue Reading »

Believers in acupuncture claim it is supported by plenty of published scientific evidence. Critics disagree. Thousands of acupuncture studies have been done over the last several decades, with conflicting results. Even systematic reviews have disagreed with each other. The time had come to re-visit the entire body of acupuncture research and try to make sense out of it all. The indefatigable CAM researcher Edzard Ernst stepped up to the plate. He and his colleagues in Korea and Exeter did an exhaustive study that was published in the April 2011 issue of the medical journal Pain:   “Acupuncture: Does it alleviate pain and are there serious risks? A review of reviews.” It is accompanied by an editorial commentary written by yours truly: “Acupuncture’s claims punctured: Not proven effective for pain, not harmless.” (The editorial is reproduced in full below.)

Ernst et al. systematically reviewed all the systematic reviews of acupuncture published in the last 10 years: 57 systematic reviews met the criteria they set for inclusion in their analysis. They found a mix of negative, positive, and inconclusive results. There were only four conditions for which more than one systematic review reached the same conclusions, and only one of the four was positive (neck pain). They explain how inconsistencies, biases, conflicting conclusions, and recent high quality studies throw doubt on even the most positive reviews.

They also demolished the “acupuncture is harmless” myth by reporting 95 published cases of serious adverse effects including infection, pneumothorax, and 5 deaths. Some but not all of these might have been avoided by better training in anatomy and infection control. Continue Reading »

Homeopathy amuses me.

Well, actually it both amuses me and appalls me. The amusement comes from just how utterly ridiculous the concepts behind homeopathy are. Think about it. It is nothing but pure magical thinking. Indeed, at the very core of homeopathy is a concept that can only be considered to be magic. In homeopathy, the main principles are that “like heals like” and that dilution increases potency. Thus, in homeopathy, to cure an illness, you pick something that causes symptoms similar to those of that illness and then dilute it from 20C to 30C, where each “C” represents a 1:100 dilution. Given that such levels of dilution exceed Avagaddro’s number by many orders of magnitude, even if any sort of active medicine was used, there is no active ingredient left after a series of homeopathic dilutions. Indeed, this was known as far back as the mid-1800′s. Of course, this doesn’t stop homeopaths, who argue that water somehow retains the “essence” of whatever homeopathic remedy it has been in contact with, and that’s how homeopathy “works.” Add to that the mystical need to “succuss” (vigorously shake) the homeopathic remedy at each dilution (I’ve been told by homeopaths, with all seriousness, that if each dilution isn’t properly succussed then the homeopathic remedy will not attain its potency), and it’s magic all the way down, just as creationism has been described as “turtles all the way down.” Even more amusing are the contortions of science and logic that are used by otherwise intelligent people to make arguments for homeopathy. For example, just read some of Lionel Milgrom‘s inappropriate invocations of quantum theory at the macroscopic level for some of the most amazing woo you’ve ever seen, or Rustum Roy‘s claims for the “memory of water.” Indeed, if you want to find out just how scientifically bankrupt everything about homepathy is, my co-blogger Dr. Kimball Atwood started his tenure on Science-Based Medicine with a five part series on homeopathy.

At the same time, homeopathy appalls me. There are many reasons for this, not the least of which is how anyone claiming to have a rational or scientific viewpoint can fall so far as to twist science brutally to justify magic. Worse, homepaths and physicians sucked into belief into the sorcery that his homeopathy are driven by their belief to carry out unethical clinical trials in Third World countries, even on children. Meanwhile, time, resources, and precious cash are wasted chasing after pixie dust by our own government through the National Center for Complementary and Alternative Medicine (NCCAM). So while I laugh at the antics of homeopaths going on and on about the “memory of water” or quantum gyroscopic models” in order to justify homeopathy as anything more than an elaborate placebo, I’m crying a little inside as I watch.

The Lancet, meta-analysis, and homeopathy

If there’s one thing that homepaths hate–I mean really, really, really hate–it’s a meta-analysis of high quality homeopathy trials published by Professor Matthias Egger in the Department of Social and Preventative Medicine at the University of Berne in Switzerland, entitled Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy.
Continue Reading »

This is an addendum to my previous entry on Bayesian statistics for clinical research.† After that posting, a few comments made it clear that I needed to add some words about estimating prior probabilities of therapeutic hypotheses. This is a huge topic that I will discuss briefly. In that, happily, I am abetted by my own ignorance. Thus I apologize in advance for simplistic or incomplete explanations. Also, when I mention misconceptions about either Bayesian or “frequentist” statistics, I am not doing so with particular readers in mind, even if certain comments may have triggered my thinking. I am quite willing to give readers credit for more insight into these issues than might be apparent from my own comments, which reflect common, initial difficulties in digesting the differences between the two inferential approaches. Those include my own difficulties, after years of assuming that the “frequentist” approach was both comprehensive and rational—while I had only a cursory understanding of it. That, I imagine, placed me well within two standard deviations of the mean level of statistical knowledge held by physicians in general.

Continue Reading »

A reader recently sent in a link to a New York Times article that discussed an alternative breathing technique developed in Russia for the treatment of asthma called the Buteyko Method, or the Buteyko Breathing Technique (BBT), and asked for an evaluation of the claims on SBM.  This post will attempt to be a reasonably comprehensive evaluation of Buteyko and his therapy so that subsequent discussions, should they be necessary, may be more terse.

The NYT article is primarily an anecdote of a friend of the author who suffered from severe asthma, but who had improved since he began using the BBT.  The author briefly discusses asthma, the history and theory behind Buteyko and hyperventilation before wrapping up with an attempt to provide evidence to support the legitimacy of the story.  The friend’s pulmonologist is quoted to confirm that “based on objective data, his breathing has improved…”  She cites controlled clinical trials “in Australia and elsewhere” where patients have reduced their use of medications, including a purported British study of 384 patients where patients had a 90% reduction in rescue inhaler use and 50% reduction in steroids.  She ends by pointing out that the British Thoracic Society has given BBT a “B” rating, and an admonition to “the pharmaceutically supported American medical community to explore this nondrug technique.”

Never having heard of BBT before, the NYT article left me with several questions.  Who was Buteyko?  How did he develop the BBT?  What is BBT, what does it claim to do, and how does it claim to work?  Is the evidence as presented in the NYT article accurate? And finally, what evidence exists within the literature that BBT is an effective treatment for asthma?

Continue Reading »

RCT Plausibility Scale

After a few intro paragraphs, I want to present a scale of probability to estimate a value of a “prior” to plug into the formula for obtaining a Bayes Factor. The scale can help to estimate a value, but will still rely on an estimate, the non-quantitative element in Bayesian simulations. However, the checklist may at least provide some objective bases on which to hang a value, and that value would actually make a semi-quantitative statement of its own. Although that value would retain some subjective quality, it would at least be backed by known quantities and laws of nature.

Begging your patience again, I became aware of this problem in 1999 when asked to moderate an online (BioMednet.com) debate on “CAM” among 4 physicians. My role soon morphed into participant-debater when I could not get all to agree on what I thought was obvious common ground to proceed with the discussion – that 1) concepts that violate scientific laws do not have to be subjected to clinical trial (RCT) and that trial results had to be interpreted in light of previous knowledge; and 2) clinical trials could not constitute adequate evidence in the absence of plausibility because their results were too varied and inconsistent. The matter was p-recipitated by systematic reviews (SRs) showing efficacy of acupuncture in back pain. I was truly surprised when one of the participants (Dr. Edzard Ernst) assured me that indeed, RCTs were now the gold standard for efficacy. The debate went downhill from there.

Continue Reading »