The Science-Based Medicine Conference has begun! Val is Twittering it at:
Check it out at the tags #TAM7 and #SBM.
The Science-Based Medicine Conference has begun! Val is Twittering it at:
Check it out at the tags #TAM7 and #SBM.
In case you’re coming late to this discussion (or have ADD), I’ve summarized Dr. Kimball Atwood’s terrific analysis of the ongoing clinical trial (TACT trial) in which convicted felons were awarded $30 million by the NIH.
In one of the most unethical clinical trial debacles of our time, the NIH approved a research study (called the TACT Trial – Trial to Assess Chelation Therapy – a supposed treatment for arteriosclerosis) in which the treatment had no evidence for potential benefit, and clear evidence of potential harm – and even the risk of death. Amazingly, the researchers neglected to mention this risk in their informed consent document. The NIH awarded $30 million of our tax dollars to ~100 researchers to enroll 2000 patients in this risky study (ongoing from 2003-present). Even more astounding is the fact that several of the researchers have been disciplined for substandard practices by state medical boards; several have been involved in insurance fraud; at least 3 are convicted felons.
But wait, there’s more.
The treatment under investigation, IV injection of Na2EDTA, is specifically contraindicated for “generalized arteriosclerosis” by the FDA. There have been over 30 reported cases of accidental death caused by the administration of this drug – and prior to the TACT, 4 RCTs and several substudies of chelation for either CAD or PVD, involving 285 subjects, had been reported. None found chelation superior to placebo.
So, Why Was This Study Approved?
The NIH and the TACT principal investigator (PI) argued that there was a substantial demand for chelation, creating a “public health imperative” to perform a large trial as soon as possible. In reality, the number of people using the therapy was only a small fraction of what the PI reported.
It’s hard to know exactly what happened “behind the scenes” to pressure NIH to go forward with the study – however a few things are clear: 1) the National Heart, Lung, and Blood Institute (NHLBI) initially declined to approve the study based on lack of scientific merit 2) congressman Dan Burton and at least one of his staffers (Beth Clay) and a lobbyist (Bill Chatfield) worked tirelessly to get the study approved through a different institute – NCCAM 3) some of the evidence used to support the trial was falsified (The RFA cited several articles by Edward McDonagh, the chelationist who had previously admitted in a court of law to having falsified his data.) 4) The NIH Special Emphasis Panel that approved the TACT protocol included L. Terry Chappell, whom the protocol had named as a participant in the TACT.
All evidence seems to suggest that political meddling managed to trump science in this case – putting the lives of 2000 study subjects at risk, without any likely benefit to them or medicine.
A formal analysis of the sordid history and ethical violations of the TACT trial was published by the Medscape Journal of Medicine on May 13, 2008. Atwood et al. provide a rigorous, 9-part commentary with 326 references in review of the case. Congressman Burton’s staffer, Beth Clay, published what is essentially a character assassination of Dr. Atwood in response.
The NIH Writes TACT Investigators a Strongly Worded Letter
On May 27, 2009 the Office for Human Research Protections Committee sent a letter to the investigators of TACT, stating that they found, “multiple instances of substandard practices, insurance fraud, and felony activity on the part of the investigators.” The letter describes a list of irregularities and recommends various changes to the research protocol.
It is almost unheard of for a letter from the NIH to state that research study investigators are guilty of fraud and felony activity – but what I don’t understand is why they haven’t shut down the study. Perhaps this is their first step towards that goal? Let’s hope so.
The TACT trial has subjected 2000 unwary subjects and $30 million of public money to an unethical trial of a dubious treatment that, had it been accurately represented and judged by the usual criteria, would certainly have been disqualified. Political meddling in health and medical affairs is dangerous business, and must be opposed as strongly as possible. Congressmen like Tom Harkin and Dan Burton should not be allowed to push their political agendas and requests for publicly funded pseudoscience on the NIH. I can only hope that the new NIH director will have the courage to fend off demands for unethical trials from political appointees.
Simon Singh is a science journalist who last year wrote an article in the Guardian critical of the British Chiropractic Association (BCA) for promoting chiropractic treatment for certain childhood ailments. Singh characterized these treatments as “bogus” because they lack evidence to back up claims for clinical efficacy. The BCA responded by suing Singh for libel. In the English court system the person being sued for libel is essentially guilty until proven innocent, and even successfully defending oneself can be ruinously expensive. Therefore suing for libel in English court is a very successful strategy for silencing critics.
This case resulted in a bit of a backlash against the BCA, who were accused of silencing legitimate and very necessary public scientific debate regarding the safety and efficacy of medical interventions. The BCA could have simply responded by providing evidence to back up their claims, and the Guardian even offered them space to do so, but instead they sued.
Part of this backlash is a movement, supported by many scientific organizations, to keep libel laws out of science.
A study published in Alternative Therapies in Health and Medicine is being cited as evidence for the efficacy of healing touch (HT). It enrolled 237 subjects who were scheduled for coronary bypass, randomized them to receive HT, a visitor, or no treatment; and found that HT was associated with a greater decrease in anxiety and shorter hospital stays.
This study is a good example of what I have called “Tooth Fairy Science.” You can study how much money the Tooth Fairy leaves in different situations (first vs. last tooth, age of child, tooth in baggie vs. tooth wrapped in Kleenex, etc.), and your results can be replicable and statistically significant, and you can think you have learned something about the Tooth Fairy; but your results don’t mean what you think they do because you didn’t stop to find out whether the Tooth Fairy was real or whether some more mundane explanation (parents) might account for the phenomenon. (more…)
One of the major themes here on the Science-Based Medicine (SBM) blog has been about one major shortcoming of the more commonly used evidence-based medicine paradigm (EBM) that has been in ascendance as the preferred method of evaluating clinical evidence. Specifically, as Kim Atwood (1, 2, 3, 4, 5, 6, 7, 8) has pointed out before, EBM values clinical studies above all else and devalues plausibility based on well-established basic science as one of the “lower” forms of evidence. While this sounds quite reasonable on the surface (after all, what we as physicians really want to know is whether a treatment works better than a placebo or not), it ignores one very important problem with clinical trials, namely that prior scientific probability matters. Indeed, four years ago, John Ioannidis made a bit of a splash with a paper published in JAMA entitled Contradicted and Initially Stronger Effects in Highly Cited Clinical Research and, more provocatively in PLoS Medicine, Why Most Published Research Findings Are Wrong. In his study, he examined a panel of highly cited clinical trials and determined that the results of many of them were not replicated and validated in subsequent studies. His conclusion was that a significant proportion, perhaps most, of the results of clinical trials turn out not to be true after further replication and that the likelihood of such incorrect results increases with increasing improbability of the hypothesis being tested.
Not surprisingly, CAM advocates piled onto these studies as “evidence” that clinical research is hopelessly flawed and biased, but that is not the correct interpretation. Basically, as Steve Novella and, especially, Alex Tabarrok pointed out, prior probability is critical. What Ioannidis’ research shows is that clinical trials examining highly improbable hypotheses are far more likely to produce false positive results than clinical trials examining hypotheses with a stronger basis in science. Of course, estimating prior probability can be tricky based on science. After all, if we could tell beforehand which modalities would work and which didn’t we wouldn’t need to do clinical trials, but there are modalities for which we can estimate the prior probability as being very close to zero. Not surprisingly (at least to readers of this blog), these modalities tend to be “alternative medicine” modalities. Indeed, the purest test of this phenomenon is homeopathy, which is nothing more than pure placebo, mainly because it is water. Of course, another principle that applies to clinical trials is that smaller, more preliminary studies often yield seemingly positive results that fail to hold up with repetition in larger, more rigorously designed randomized, double-blind clinical trials.
Last week, a paper was published in PLoS ONE Thomas by Thomas Pfeiffer at Harvard University and Robert Hoffmann at MIT that brings up another factor that may affect the reliability of research. Oddly enough, it is somewhat counterintuitive. Specifically, Pfeiffer and Hoffmann’s study was entitled Large-Scale Assessment of the Effect of Popularity on the Reliability of Research. In other words, the hypothesis being tested is whether the reliability of findings published in the scientific literature decreases with the popularity of a research field. Although this phenomenon is hypothesized based on theoretical reasoning, Pfeiffer and Hoffmann claim to present the first empirical evidence to support this hypothesis.
This month I will begin my third year of medical school, after a three-year break for laboratory research. Living alternately in the worlds of med school and grad school has prompted me to reflect on differences between these training programs.
[Obvious disclaimer: I have studied at a single institution, and only for five years.]
I am enrolled in a dual-degree MD/PhD program. About 120 US medical schools have such programs, and the National Institutes of Health funds a third of them (MSTP). The schedule of such programs is generally: 2 years of medical school (culminating in USMLE Step 1), 3+ years of graduate school (culminating in dissertation and PhD), and then the last 2 years of medical school (which I begin this month). The most popular residency choices for MD/PhD graduates are internal medicine, pediatrics, and pathology (match data). Other residencies that attract these graduates include dermatology, neurology, ophthamology, and radiology (survey data). The hopes of those funding the MD/PhD training programs and of those accepting the graduates is that these individuals will become physician-scientists, bridging the divide between lab bench and patient bedside with insights from both. (more…)
I love the British comedy duo Mitchell and Webb, and this is just one reason why. They totally get homeopathy, as this video e-mailed to me by a reader demonstrates:
Pay close attention to the signs in the A & E.
No doubt Dana Ullman will show up to cry foul over how Mitchell and Webb are totally “misrepresenting” homeopathy…
Transcribed minutes of a meeting, provenance unknown.
Chairman: “I would like to call this emergency meeting to order and thank you all for coming under short notice. If there are no objections, I will dispense with the usual formalities and get right to the business at hand.”
Murmur of approval.
Chairman: Ladies, Gentleman and Demons of all kinds, we have been, to use the vernacular of the day, outed.”
Murmur of consternation
Voice from the crowd: “I am most certainly not gay.”
Chairman: “Not gay. We have been outed as a society. The Evil Society of the Evil Medical Industrial Complex, our beloved ESEMIC, is no longer secret. Our works have been discovered. The true meaning of the Caduceus has been published for all to see. And by your fruits you shall be known. Sorry demons”
Louder murmur of consternation.
Loud voice from the crowd: “I demand to know how this occurred. This is impossible.”
Shouts of agreement.
Chairman: “If you will please all remain calm, I will explain.”
Francis Collins, M.D., Ph.D., is probably best known for his leadership of the Human Genome Project, though his discoveries of the Cystic Fibrosis, Huntington’s, and Neurofibromatosis genes are also extraordinary accomplishments. Dr. Collins is a world-renowned scientist and geneticist, and also a committed Christian. In his recent best-selling book, The Language Of God, Dr. Collins attempts to harmonize his commitment to both science and religion.
Some critics (such as Richard Dawkins) have expressed reservations about Dr. Collins’ faith, wondering if it might cloud his scientific judgment. Since Collins is rumored to be the most likely candidate for directorship of the NIH, and because I wanted to know if Dawkins et al. had any reason for concern, I decided to read The Language Of God.
First of all, Christians are a rather heterogeneous group – with a range of viewpoints on evolution, science, and the interpretation of Biblical texts. On one extreme there are Christians (often referred to as “young earth creationists” or simply “creationists”) who believe in an absolutely literal interpretation of the Genesis story, and see evolution as antithetical to true faith. Dr. Collins suggests that as many as 45% of Christians may actually be in this camp.
When it comes to “alternative medicine” trials, it seems that the NIH is willing to experiment on people in ways that would be unthinkable for real biomedical research. The federal Office for Human Research Protections (OHRP) has posted a preliminary determination letter, dated May 27, 2009, addressing some of the charges we had made against the politics-driven NIH Trial to Assess Chelation Therapy (TACT).
It is a remarkably damning statement, particularly regarding an NIH study. That is, it found—or the recipients admitted—that each of several charges was valid. Among these are misleading statements and unstated risks in the consent form, and the embarrassing backgrounds of TACT investigators. According to the determination letter,
…investigations revealed multiple instances of substandard practices, insurance fraud, and felony activity on the part of investigators.”