It infuriates me when someone misappropriates the word “science” to promote treatments that are not actually based on science. I have just read a book entitled The PTSD Breakthrough: The Revolutionary Science-Based Compass Reset Program by Dr. Frank Lawlis, a psychologist who is the chief content advisor for Dr Phil and The Doctors. There is very little science in the book and references are not provided. It amounts to an indiscriminate catalog of everything Dr. Lawlis can imagine that might help post-traumatic stress disorder (PTSD) patients. (more…)
A critical aspect of both evidence-based medicine (EBM) and science-based medicine (SBM) is the randomized clinical trial. Ideally, particularly for conditions with a large subjective component in symptomatology, the trial should be randomized, double-blind, and placebo-controlled. As Kimball Atwood pointed out just last week, in EBM, scientific prior probability tends to be discounted while in SBM it is not, particularly for therapies that are wildly improbable strictly on the basis of basic science, but for both the randomized clinical trial remains, in essence, where the “rubber hits the road,” so to speak. Indeed, when the prior probability of a therapy working based on preclinical basic science investigations appears high, EBM and SBM should be (and are, for the most part) more or less indistinguishable.
The ethics of clinical trials, however, demand a characteristic known as clinical equipoise. Stated briefly, for purposes of clinical trials, clinical equipoise demands that at the time a clinical trial is being carried out there be a state of genuine scientific uncertainty in the medical community over which of the drugs or treatments being tested is more efficacious and safer. One reason (among many) why the Gonzalez trial was completely unethical was a lack of clinical equipose. (Lack of adequate informed consent was another.) Lack of clinical equipoise is also the reason why a prospective randomized, double-blind, placebo-controlled clinical trial testing an unvaccinated group versus a vaccinated control group to determine whether vaccines cause autism would be completely unethical. Such a trial would egregiously violate the principle of clinical equipoise because the unvaccinated group would be left unprotected against potentially life-threatening vaccine-preventable diseases, and that is completely unacceptable from an ethical perspective. Consequently, we have had to rely on on the accumulation of data from less rigorous trial designs to demonstrate that there is no correlation between vaccines and autism. Even so, the accumulated weight of such evidence is enough, and for some questions that is the best we can do because scientific rigor sometimes conflicts with human subjects research ethics. This is an extreme example of lack of clinical equipoise, but it illustrates the point. If we know (or have good scientific reason to suspect) that one treatment is better than another, it is unethical to randomize patients to the arm that receives what is, based on what is known at the time of the trial, likely to be an inferior treatment.
Sometimes, however, the question of whether clinical equipoise exists in a clinical trial is not so obvious as it is for trials proposed by cranks. This situation sometimes crops up in clinical trials for cancer. I was reminded of this issue by a front page story in the New York Times yesterday, New Drugs Stir Debate on Basic Rules of Clinical Trials. In it, reporter Amy Harmon uses a classic human interest story to highlight the issue of clinical equipoise in a clinical trial for a new drug for melanoma that shows great promise. In brief, it is the story of two cousins, one of whom is receiving the new “wonder drug” (whether it is truly a wonder drug or not remains to be seen) in a clinical trial and one of whom is receiving the current standard of care for stage IV melanoma, which, to put it bluntly, sucks in that it has very little effect in prolonging life:
Evidence-Based Medicine, Human Studies Ethics, and the ‘Gonzalez Regimen’: a Disappointing Editorial in the Journal of Clinical Oncology Part 2
NB: If you haven’t yet read Part 1 of this blog, please do so now; Part 2 will not summarize it.
At the end of Part 1, I wrote:
We do not need formal statistics or a new, randomized trial with a larger sample size to justify dismissing the Gonzalez regimen.
In his editorial for the JCO, Mark Levine made a different argument:
Can it be concluded that [the] study proves that enzyme therapy is markedly inferior? On the basis of the study design, my answer is no. It is not possible to make a silk purse out of a sow’s ear.
That conclusion may be correct in the EBM sense, but it misses the crucial point of why the trial was (ostensibly) done: to determine, once and for all, whether there was anything to the near-miraculous claims that proponents had made for a highly implausible “detoxification” regimen for cancer of the pancreas. Gonzalez himself had admitted at the trial’s inception that nothing short of an outcome matching the hype would do:
DR. GONZALEZ: It’s set up as a survival study. We’re looking at survival.
SPEAKER: Do you have an idea of what you’re looking for?
DR. GONZALEZ: Well, Jeff [Jeffrey White, the director of the Office of Cancer Complementary and Alternative Medicine at the NCI—KA] and I were just talking a couple weeks ago. You know, to get any kind of data that would be beyond criticism is—-always be criticism, but at least three times.
You would want in the successful group to be three times — the median to be three times out from the lesser successful groups.
So, for example, if the average survival with chemo, which we suspect will be 5 months, you would want my therapy to be at least — the median survival to be at least 15, 16, 17 months, as it was in the pilot study.
We’re looking for a median survival three times out from the chemo group to be significant.
Recall that the median survival in the Gonzalez arm eventually turned out to be 4.3 months.
Last week, I wrote one of my usual ridiculously detailed posts analyzing a recent study (Price et al) that, if science and reason ruled, would be the last nail in the coffin of the hypothesis connecting autism with the mercury-containing preservative, thimerosal, which used to be in many childhood vaccines but was phased out beginning in 1999 and disappearing in infant vaccines except for the flu vaccine by early 2002. Of course, for at least the last five years, the thimerosal-autism hypothesis has been a notion whose coffin already had so many nails pounded into it that Price et al probably had a hard time finding even a tiny area of virgin wood into which to pound even a tiny nail of a study published in an impact factor one journal, much less the spike that their study in Pediatrics represented.
Unfortunately, as we know, in the anti-vaccine movement unreason rules, and, not unexpectedly, as a result this study has changed little in the debate, the fortuitously ironic happenstance of its being released the day before Mark Blaxill and Dan Olmsted’s anti-mercury screed Age of Autism not withstanding. To physicians and scientists, it is another strong piece of data being added to the confluence of evidence that has shown no link between mercury in vaccines and autism (or vaccines themselves and autism, for that matter). It is yet another confirmation that vaccines are safe. In contrast, to the anti-vaccine movement, it is simply yet another confirmation that the CDC is hopelessly biased, that scientists are in on a conspiracy to suppress The Truth, and that they are the poor persecuted minority, the only ones who know What Is Really Going On.
When I wrote my post last week, I didn’t know whether or not it would be worth my while to comment on the response of anti-vaccine activists to the study. The reason is that, as fun as it is to reveal their responses to be as vacuous as they are, I wasn’t sure that it would be educational. Granted, sometimes educational value takes a back seat to criticism, but sometimes it’s just too easy. In any case, by mid-week, there had been virtually no criticism of the study yet from the usual sources; so I figured it to be a moot point whether or not I would end up writing about this study one last time. Then, on Thursday morning I noted an e-mail in my in box. In order to keep my finger on the pulse of various pseudoscience movements, I subscribe to e-mail lists of various crank organizations, one of which is Generation Rescue and another of which is SafeMinds. SafeMinds, as you may recall, is the organization headed up by Sallie Bernard. As you may also recall, Bernard was originally on the external consulting committee that participated in the design of Price et al, and, before it, Thompson et al, the two of which ultimately made up a one-two punch against the mercury-autism hypothesis. When she saw that the results of Thompson et al were going against her idea and that no link between thimerosal-containing vaccines and neurodevelopmental disorders was showing up in the preliminary analyses, she resigned from the committee and started attacking Thompson et al. What surprised me was that she wasn’t ready with a criticism of Price et al when it was released.
Evidence-Based Medicine, Human Studies Ethics, and the ‘Gonzalez Regimen’: a Disappointing Editorial in the Journal of Clinical Oncology Part 1
Background: the distinction between EBM and SBM
An important theme on the Science-Based Medicine blog, and the very reason for its name, has been its emphasis on examining all the evidence—not merely the results of clinical trials—for various claims, particularly for those that are implausible. We’ve discussed the distinction between Science-Based Medicine (SBM) and the more limited Evidence-Based Medicine (EBM) several times, for example here (I began my own discussion here and added a bit of formality here, here, and here). Let me summarize by quoting John Ioannidis:
…the probability that a research finding is indeed true depends on the prior probability of it being true (before doing the study), the statistical power of the study, and the level of statistical significance.
EBM, in a nutshell, ignores prior probability† (unless there is no other available evidence) and falls for the “p-value fallacy”; SBM does not. Please don’t bicker about this if you haven’t read the links above and some of their own references, particularly the EBM Levels of Evidence scheme and two articles by Steven Goodman (here and here). Also, note that it is not necessary to agree with Ioannidis that “most published research findings are false” to agree with his assertion, quoted above, about what determines the probability that a research finding is true.
The distinction between SBM and EBM has important implications for medical practice ethics, research ethics, human subject protections, allocation of scarce resources, epistemology in health care, public perceptions of medical knowledge and of the health professions, and more. EBM, as practiced in the 20 years of its formal existence, is poorly equipped to evaluate implausible claims because it fails to acknowledge that even if scientific plausibility is not sufficient to establish the validity of a new treatment, it is necessary for doing so.
Thus, in their recent foray into applying the tools of EBM to implausible health claims, government and academic investigators have made at least two, serious mistakes: first, they have subjected unwary subjects to dangerous but unnecessary trials in a quest for “evidence,” failing to realize that definitive evidence already exists; second, they have been largely incapable of pronouncing ineffective methods ineffective. At best, even after conducting predictably disconfirming trials of vanishingly unlikely claims, they have declared such methods merely “unproven,” almost always urging “further research.” That may be the proper EBM response, but it is a far cry from the reality. As I opined a couple of years ago, the founders of the EBM movement apparently “never saw ‘CAM’ coming.”
If you believe everything you read on the internet, then is seems that a chemical found in thousands of products is causing an epidemic of severe neurological and systemic diseases, like multiple sclerosis and lupus. The FDA, the companies that make the product, and the “medical industrial complex” all know about the dangers of this chemical but are hiding the truth from the public in order to protect corporate profits and avoid the pesky paper work that would accompany the truth being revealed. The only glimmer of hope is a dedicated band of bloggers and anonymous e-mail chain letter authors who aren’t afraid to speak the truth. Armed with the latest anecdotal evidence, unverified speculation, and scientifically implausible claims, they have been tirelessly ranting about the evils of this chemical for years. Undeterred by the countless published studies manufactured by the food cartel that show this chemical is safe, they continue to protect the public by spreading baseless fear and hysteria.
Hopefully, you don’t believe everything you read on the internet, and you don’t get your science news from e-mail SPAM, where the above scenario is a common theme. While there are many manifestations of this type of urban legend, I am speaking specifically about aspartame – an artificial sweetener used since the early 1980s. The notion that aspartame is unsafe has been circulating almost since it first appeared, and like rumors and misinformation have a tendency to do, fears surrounding aspartame have taken on a life of their own.
I am frequently asked my opinion about the safety of aspartame. Nutritionists often council to avoid the sweetener, citing unverified claims that it is unsafe. I was recently sent a chain letter warning that aspartame causes MS (which of course can be cured by simply avoiding aspartame), and Snopes informs me that this particular letter first appeared in 1998.
A member of Quackwatch’s Healthfraud discussion list recently reported from a health fair:
One booth was a bit of a mystery for me: Brain Balance. “Is your child struggling with ADHD, dyslexia, autism, Asperger’s, Tourette’s, or other related disorders?” A quick glance at their website makes it seem that they may be legitimate.
PROLOGUE: BAD LUCK AND BAD TIMING
Two and a half years ago, very early in the history of this blog, I wrote one of my usual logorrheic (although I prefer the word “comprehensive”) posts entitled Mercury in vaccines as a cause of autism and autism spectrum disorders (ASDs): A failed hypothesis. In that post, I characterized the scientifically discredited notion that the mercury in the thimerosal preservative that used to be in several childhood vaccines was the cause of the “autism epidemic” as “one of the most pernicious medical myths of recent years.” And so it is. I like to characterize the notion that thimerosal-containing vaccines (TCVs) cause autism as the American version of the British myth, popularized by Andrew Wakefield and a sensationalistic British press, that the measles-mumps-rubella (MMR) vaccine causes autism and “autistic enterocolitis.”
Both notions were based on confusing correlation with causation, aided and abetted by some truly bad science, and both notions have been painfully difficult to dislodge. Indeed, in the case of Wakefield, only now that Wakefield was stripped of his license to practice in the U.K. by its General Medical Council, leading to The Lancet finally doing what it should have done six years ago and retracting Wakefield’s 1998 study that sparked the MMR frenzy in the U.K. and arguably kickstarted the modern anti-vaccine movement, do I sense that journalists are finally “getting” that science does not support the idea that the MMR vaccine causes autism. Andrew Wakefield may be trying to fight back with his book Callous Disregard after his disgrace was complete, basking in the glow of admiration of die-hard anti-vaccine groups, but, for now, at least, Wakefield and his MMR fear mongering are yesterday’s news, and that’s a very good thing indeed–at least for as long as it lasts.
Perhaps it is the fall of Andy Wakefield that has led to an apparent resurgence of the concept that mercury in TCVs somehow causes autism, after having faded into the background after the CDC and AAP recommended that thimerosal be removed from all childhood vaccines in 1999 and the last TCV having expired towards the end of 2001. After all, if the hypothesis that TCVs cause autism had been correct, we should have expected to see a marked decrease in the incidence of autism and autism spectrum disorders (ASDs) within about 5 years of 2002, given that the vast majority of cases of ASDs are diagnosed between the ages of 2 and 5. We have not, and, even though its adherents have kept moving the goalposts back regarding the date that we should start to see a leveling off and drop in the incidence of ASDs, starting with 2005, then 2007, and now, apparently, 2011 (which is only less than four months away, by the way), even Jenny McCarthy’s anti-vaccine organization originally founded by J.B. Handley and his wife, namely Generation Rescue, began demphasizing mercury in 2007, after having stated flatly on its website that autism is a “misdiagnosis for mercury poisoning” for so long. Since then, “too many, too soon” has been the favored propaganda talking point.
Of course, not every crank is ready to abandon the myth that TCVs cause autism. Indeed, tomorrow two mercury militia “heavy hitters” and bloggers for the anti-vaccine propaganda blog Age of Autism, Mark Blaxill and Dan Olmsted, will be releasing a book entitled Age of Autism: Mercury, Medicine, and a Manmade Epidemic. In anticipation, four weeks ago I actually e-mailed the publicist to send me a review copy of Age of Autism. I have yet to receive the book. I wonder why. Be that as it may, it amuses me that the official release of the release of the not-so-dynamic duo of the mercury militia’s book actually will one day after a study that is arguably the last nail in the coffin of the very dead hypothesis that TCVs cause autism was released. Either the great pharma conspiracy is far more conniving and effective than even J.B. Handley thinks, or Blaxill and Olmsted’s luck is just that bad. As I anticipate the conspiracy mongering posts about this bad timing aside, let’s just take a look at this last coffin nail, which is a study by Price et al that was released today in the journal Pediatrics entitled Prenatal and Infant Exposure to Thimerosal From Vaccines and Immunoglobulins and Risk of Autism.
Humans love to find patterns in the world. Sometimes patterns exist, sometimes they are imaginary. Sometimes you can see a pattern that may be interesting and ignore its significance. As a resident I used to say that anyone who smokes three packs of cigarettes a day has to be schizophrenic, it was meant more as a joke, when, in fact, it was later discovered that tobacco helps ameliorate the symptoms of schizophrenia. I need to pay more attention.
Part of my job is to look for patterns as a key to the patients diagnosis. Diseases and pathogens tend to (more or less) cause reproducible signs and symptoms and looking for that pattern is often the most helpful clue towards finding the diagnosis. Of course things are never as easy as one would like, as you have to consider whether you are seeing common manifestations of a common disease, uncommon manifestations of a common disease, common manifestations of a uncommon disease and, the hardest, uncommon manifestations of an uncommon disease. When I have a complex or uncertain cause, I explicitly run through that, and other, litanies so I do not miss a unusual diagnosis.
Chronic Fatigue Syndrome (CFS) has, at least to my way of thinking, two patterns. I see the occasional CFS patient in clinic and, I hope, pay attention to their disease patterns. I keep in mind I may be seeing a pattern that does not exist, but looking for disease patterns is what doctors are trained to do.
Earlier this week, my colleague Dr. Gorski explored a common theme in alternative medicine: the idea that all disease is preventable. This implies that all disease has a discrete cause and that individual behavior can mitigate this cause.
If biology worked this way, my job as an internist would be very different. Many people would love to believe that life is this predictable, and that they have that much control over their health, but they don’t. Most disease represents the interaction of environment and genetics, and you can’t change your genes (with a few exceptions, of course).
It’s natural to want to be able to exert an impossible level of control over your health, but when unscrupulous charlatans (redundant redundancy alert!) play on these beliefs and fears, they can cause, rather than prevent problems.