The Science-Based Medicine Conference has begun! Val is Twittering it at:
Check it out at the tags #TAM7 and #SBM.
The Science-Based Medicine Conference has begun! Val is Twittering it at:
Check it out at the tags #TAM7 and #SBM.
One of the major themes here on the Science-Based Medicine (SBM) blog has been about one major shortcoming of the more commonly used evidence-based medicine paradigm (EBM) that has been in ascendance as the preferred method of evaluating clinical evidence. Specifically, as Kim Atwood (1, 2, 3, 4, 5, 6, 7, 8) has pointed out before, EBM values clinical studies above all else and devalues plausibility based on well-established basic science as one of the “lower” forms of evidence. While this sounds quite reasonable on the surface (after all, what we as physicians really want to know is whether a treatment works better than a placebo or not), it ignores one very important problem with clinical trials, namely that prior scientific probability matters. Indeed, four years ago, John Ioannidis made a bit of a splash with a paper published in JAMA entitled Contradicted and Initially Stronger Effects in Highly Cited Clinical Research and, more provocatively in PLoS Medicine, Why Most Published Research Findings Are Wrong. In his study, he examined a panel of highly cited clinical trials and determined that the results of many of them were not replicated and validated in subsequent studies. His conclusion was that a significant proportion, perhaps most, of the results of clinical trials turn out not to be true after further replication and that the likelihood of such incorrect results increases with increasing improbability of the hypothesis being tested.
Not surprisingly, CAM advocates piled onto these studies as “evidence” that clinical research is hopelessly flawed and biased, but that is not the correct interpretation. Basically, as Steve Novella and, especially, Alex Tabarrok pointed out, prior probability is critical. What Ioannidis’ research shows is that clinical trials examining highly improbable hypotheses are far more likely to produce false positive results than clinical trials examining hypotheses with a stronger basis in science. Of course, estimating prior probability can be tricky based on science. After all, if we could tell beforehand which modalities would work and which didn’t we wouldn’t need to do clinical trials, but there are modalities for which we can estimate the prior probability as being very close to zero. Not surprisingly (at least to readers of this blog), these modalities tend to be “alternative medicine” modalities. Indeed, the purest test of this phenomenon is homeopathy, which is nothing more than pure placebo, mainly because it is water. Of course, another principle that applies to clinical trials is that smaller, more preliminary studies often yield seemingly positive results that fail to hold up with repetition in larger, more rigorously designed randomized, double-blind clinical trials.
Last week, a paper was published in PLoS ONE Thomas by Thomas Pfeiffer at Harvard University and Robert Hoffmann at MIT that brings up another factor that may affect the reliability of research. Oddly enough, it is somewhat counterintuitive. Specifically, Pfeiffer and Hoffmann’s study was entitled Large-Scale Assessment of the Effect of Popularity on the Reliability of Research. In other words, the hypothesis being tested is whether the reliability of findings published in the scientific literature decreases with the popularity of a research field. Although this phenomenon is hypothesized based on theoretical reasoning, Pfeiffer and Hoffmann claim to present the first empirical evidence to support this hypothesis.
I love the British comedy duo Mitchell and Webb, and this is just one reason why. They totally get homeopathy, as this video e-mailed to me by a reader demonstrates:
Pay close attention to the signs in the A & E.
No doubt Dana Ullman will show up to cry foul over how Mitchell and Webb are totally “misrepresenting” homeopathy…
A couple of weeks ago, our resident skeptical medical student Tim Kreider wrote an excellent article about an op-ed in NEWSWEEK by science correspondent Sharon Begley, in which he pointed out many misconceptions she had regarding basic science versus translational research, journal impact factors, and how journals actually determine what they will publish. Basically, her thesis rested on little more than a few anecdotes by scientists who didn’t get funded or published in journals with as high an impact factor as they thought they deserved, with no data, science, or statistics to tell us whether the scientists featured in her article were in fact representative of the general situation. Begley’s article caught flak from others, including Mike the Mad Biologist and our very own Steve Novella. Naturally, as the resident cancer surgeon and researcher, I had thought of weighing in, but other issues interested me more at the time.
In retrospect, I rather regret it, given that this issue crops up time and time again. In essence, it’s a variant of the lament that pops up in the press periodically, when science journalists look at survival rates for various cancers and ask why, after nearly 40 years, we haven’t yet won the war on cancer. Because of his youth, Tim probably hasn’t seen this issue crop up before, but, trust me, every couple of years or so it does. Begley’s article and the NYT article strike me as simply “Why are we losing the war on cancer?” 2009 edition.
Now the New York Times has given me an excuse both to revisit Begley’s article and discuss yesterday’s front page article in the NYT Grant System Leads Cancer Researchers to Play It Safe. Basically, they are variants of the same complaints I’ve heard time and time again. Now, don’t get me wrong. By no means am I saying that the current system that the NIH uses to determine which scientists get funded. Those who complain that the system is often too conservative have a point. The problem, all too often, however, is that the proposals for how to fix the problem are usually either never spelled out or rest on dubious assumptions about the nature of cancer research themselves.
Last week, I wrote one of my characteristically logorrheic meandering posts about what turns a scientist into a crank or a doctor into a quack. In a sort of continuation of this line of thinking, this week I’ll turn my attention to one of the other most common characteristics of a crank, be he scientific crank (i.e., a creationist), a quack, or historical crank (i.e., Holocaust deniers), specifically how he views the peer review system.
Not suprisingly, one of the favorite targets of pseudoscientists is, in fact, the peer review system. Indeed, it’s a very safe thing to say that, almost without exception, cranks really, really, really don’t like the peer review system for scientific journals and grant review. After all, it’s the system through which scientists submit their manuscripts describing their scientific findings or their grant proposals to their peers, and their peers make a judgment whether manuscripts are scientifically meritorious enough to be published and grant applications scientifically compelling enough to be funded. Creationists hate peer review. HIV/AIDS denialists hate it. Anti-vaccine cranks like those at Age of Autism hate it. Indeed, as a friend of mine, Mark Hoofnagle pointed out a couple of years ago, pseudoscientists and cranks of all stripes hate it. There’s a reason for that, of course, namely that vigorous peer review is a major part of science that keeps pseudoscientists from attaining the respectability that science possesses and that they crave so.
As a physician and scientists who’s dedicated his life to the application of science to the development of better medical treatments, I’ve often wondered how formerly admired scientists and physicians fall into pseudoscience or even generate into out-and-out cranks. Examples are numerous and depressing to contemplate. For example, there’s Linus Pauling, a highly respected chemist and Nobel Laureate, who in his later years became convinced that high dose vitamin C could cure cancer. Indeed, Pauling’s belief that high dose vitamin C could cure the common cold and cancer fueled the development of a whole new form of quackery known as “orthomolecular medicine,” whose entire philosophy seems to be based on the concept that if some vitamins are good more must be better. In essence, “orthomolecular medicine” is a parody of nutritional science; indeed, its advocates take credit for how some strains of “complementary and alternative medicine” (CAM) so frequently advocate the ingestion of huge amounts of dietary “supplements.” I could even go farther and say that orthomolecular medicine is clearly a major part of the “intellectual” (and I do use that term loosely) underpinning of the various biomedical treatments for autism that Jenny McCarthy and Generation Rescue advcoate.
There are other examples as well, all just as depressing to contemplate. For example, consider Peter Duesberg, a brilliant virologist who in the 1980s was widely believed to be on track for a Nobel Prize; that is, until he became fixated on the idea that HIV does not cause AIDS. True, lately he’s been trying to resurrect his scientific reputation with his interesting and possibly even promising chromosomal aneuploidy hypothesis of cancer, but, alas, true to form he’s been doing it by acting like a crank. Specifically, he sees his hypothesis as The One True Cause of Cancer and disparages conventional thinking as having been so very, very wrong all these years (with his being, of course, so very, very brilliant that he saw what no one else could see). Then there are people like Dr. Lorraine Day, who was a respected academic orthopedic surgeon in the 1980s. In the late 1980s, she started to flirt with AIDS pseudoscience through a scare campaign about catching AIDS from aerosolized blood. Of course, given the mystery and fear over HIV in the early years of the epidemic, such a fear, although overblown, was not so far out of the mainstream as to be worthy of the appellation crank. However, after being diagnosed with breast cancer, unfortunately Dr. Day rapidly degenerated into a purveyor of rank pseudoscience, as well as a New World Order conspiracy theorist, religious loon, and Holocaust denier. And let’s not forget Mark Geier, who, although not a distinguished scientist, did, before his conversion to antivaccinationism, apparently do a real fellowship at the NIH and appeared to be on track to a respectable, maybe even impressive, career as an academic physician. Now he’s doing “research” in his basement, injecting autistic children with a powerful anti-sex hormone drug and abusing epidemiology. There are innumerable other examples.
EDITOR’S NOTE: Unfortunately, this weekend, I was forced to get my slides together for the upcoming SBM Conference, plus editing a manuscript for resubmission, plus working on a manuscript that I should have submitted six months ago, plus reading over some grants, plus…well, you get the idea. What this means is that, alas, I didn’t have any time to prepare one of the new, long posts that you’ve come to love (or hate). Fortunately, there are a lot of other things I’ve written out there that can be rapidly adapted to SBM. For instance, what I am about to present now. Since I wrote this, I’ve thought of a couple of things that I should have said the first time (and was kicking myself for not having done so); so publishing an updated version here allows me to rectify those omissions.
A couple of weeks ago, there was a lot of hype about a study that hadn’t been released yet. Indeed, there was a story in Wired entitled To Survive Cancer, Live With It and an editorial by the study’s lead author in Nature entitled A change in strategy in the war on cancer. Not bad for a study that hadn’t been released yet. Intrepid medical and science blogger that I am, I waited until the actual study was published a week ago the June 1 episode of Cancer Research. It’s a clever study, but the hype over it was a bit overblown. For example:
For all the weapons deployed in the war on cancer, from chemicals to radiation to nanotechnology, the underlying strategy has remained the same: Detect and destroy, with no compromise given to the killer. But Robert Gatenby wants to strike a peace.
A mathematical oncologist at the Moffitt Cancer Center, Gatenby is part of a new generation of researchers who conceive of cancer as a dynamic, evolutionary system. According to his models, trying to wipe cancer out altogether actually makes it stronger by helping drug-resistant cells flourish. Rather than fighting cancer by trying to eradicate its every last cell, he suggests doctors might fare better by intentionally keeping tumors in a long-term stalemate.
Maybe I’m being a bit picky, but what annoys me about the news reports on this study is that the concept of turning cancer into a manageable chronic disease like diabetes or hypertension is not by any means a new idea. Remember, one of my major research interests is the inhibition of tumor angiogenesis. Consequently, I know that the late, great Judah Folkman first proposed the concept of using antiangiogenic therapy to turn cancer into a chronic disease at least as early as the mid-1990′s. The only difference is the strategy that he proposed. The idea had also been floating around for quite a while before that, although I honestly do not know who first came up with it.
But let’s see what Dr. Gatenby proposes. What makes it interesting is that his study actually looks at how scientists have applied evolutionary principles to cancer until recently, argues that we’ve been doing it wrong. He then proposes a way to use the evolutionary dynamics of applied ecology. He may well be on to something. First, here’s the problem:
I’d just like to take a moment to engage in a little bit of shameless self-promotion and point out that an SBM post has actually seen print. Specifically, my post about the malign influence Oprah Winfrey’s promotion of dubious medical practices on her show (The Oprah-fication of Medicine) has been adapted (with heavy cutting and editing) into an op-ed piece in The Toronto Star, entitled Is Oprah Winfrey Giving Us Bad Medicine?
No one was more shocked than I was when the editor of Sunday Insight section of The Toronto Star contacted me earlier this week to ask if he could adapt my post to a newspaper editorial. Actually, he and his editors did the vast majority of the work in whittling my usual logorrheic prose down to a manageable size and paraphrasing the sections of the NEWSWEEK article on Oprah that I had quoted liberally from. (After all, I didn’t want to be accused of plagiarism.) It was a rather educational experience, actually. Unfortunately, reading the finished version again, I don’t think it quite makes the link between Oprah and the infiltration of pseudoscientific CAM practices into modern medicine as clear as the original post, perhaps because the context of all the other blog posts on the topic by SBM bloggers is missing, which is why I hope that some Star readers will find their way here and be able to read the full length version.
In any case, compare:
The Oprah-fication of medicine (the original, full-length blog post)
Is Oprah Winfrey Giving Us Bad Medicine? (the heavily edited op-ed piece)
And see what you think.
Unfortunately, a frequent topic on SBM has been the anti-vaccine movement, personified these days by celebrity spokesmodel for Generation Rescue Jenny McCarthy and her dimmer than dim boyfriend comedian and actor Jim Carrey. Unfortunately, it is a topic that is unlikely to go away. We’ve all speculated why the anti-scientific emotion-based notion that vaccines somehow must cause autism persists in spite of mountains of evidence to the contrary, but I think the question goes much deeper than that because it’s not just about vaccines. The anti-vaccine movement is but one of the most visible components of a much deeper problem in our public discourse, a problem that values feelings and personal experience over evidence, compelling stories and anecdotes over science.
I’m referring to the Oprah-fication of medicine in America.
We’ve written a lot about anti-vaccine zealotry on this blog, as Steve and I take a particular interest in this particular form of dangerous pseudoscience for a number of reasons. One reason, of course, is that the activities of antivaccine groups like Generation Rescue and its spokesmodel since 2007 (Jenny McCarthy, a frequent topic on this blog) have started to frighten parents about vaccines enough that vaccination rates are falling well below that required for herd immunity in some parts of the country. Indeed, McCarthy, at the behest of her handlers in Generation Rescue, serves up a regular “toxic” brew of misinformation and nonsense about vaccines, most recently in a video that was the subject of a post by Val Jones about her unbelievably pseudoscience-laden blather. Truly, it has to be seen to be believed. Meanwhile, Generation Rescue has sent McCarthy on a media propaganda tour for her latest antivaccine pro-quackery book and set up a misinformation-laden propaganda site called Fourteen Studies (blogged about by Steve Novella, Mark Crislip, and, of course, yours truly) in which they attack well-designed studies that have failed to confirm their pet idea that somehow, some way, vaccines must be the cause of autism. And, when their pseudoscience is criticized, the antivaccine movement has a tendency to launch vicious ad hominem attacks, as they recently did against Steve Novella and have done multiple times in the past against me.
However, there is one other consequence of the antivaccine movement, however, and it is at least as important as the public health implications of the potential dimunition of herd immunity caused by the fear mongering of groups like Generation Rescue. That consequence is the cottage industry of “biomedical” treatments to which desperate parents subject their children. Gluten-free diets, chelation therapy (which has caused deaths), hyperbaric oxygen chambers (a recent story described a child getting severely burned when one of these caught fire), autistic children have been subjected to it all. But of all the biomedical woo to which autistic children have been subjected, one form of woo stands out as being particularly heinous. Indeed, I agree with our fearless leader Steve in characterizing it as an “atrocity.”
I’m referring to Mark and David Geier’s favored “treatment” for autistic children, namely a drug called Lupron.