An experiment is ethical or not at its inception; it does not become ethical post hoc—ends do not justify means.
~ Henry K. Beecher
A couple of weeks ago, Dr. Josephine Briggs, the Director of the National Center for Complementary and Alternative Medicine (NCCAM), posted a short essay on the NCCAM Research Blog touting the results of the Trial to Assess Chelation Therapy (TACT) (italics added):
The authors found that those receiving the active treatment clearly fared better than those receiving placebo. The accompanying editorial in the AHJ reminds readers about the value of equipoise and the need to “test our beliefs against evidence.”
Most physicians did not expect benefit from chelation treatment for cardiovascular disease. I readily admit, initially, I also did not expect we would find evidence that these treatments reduce heart attack, strokes, or death. So, the evidence of benefit coming from analyses of the TACT trial has been a surprise to many of us. The subgroup analyses are suggesting sizable benefit for diabetic patients—and also, importantly, no benefit for the non-diabetic patient. Clearly subgroup analyses, even if prespecified, do not give us the final answer. But it is also clear that more research is needed to test these important findings.
And TACT findings are indeed a reminder of the importance of retaining equipoise [sic], seeking further research aimed at replicating the findings, and neither accepting nor rejecting findings based on personal biases. The scientific process is designed to weed out our preconceived notions and replace them with evidence.
Dr. Briggs concluded:
So, TACT is a reminder—an open mind is at the center of the scientific method.
Dr. Briggs’s title was “Bayes’ Rule and Being Ready To Change Our Minds”, a reference to a recent editorial that had accompanied one of the TACT papers. That editorial, by Dr. Sanjay Kaul, a physician and statistician from UCLA, begins with this quotation:
Preconceived notions are the locks on the door to wisdom.
~ Merry Browne
Here is the relevant passage from Dr. Kaul’s editorial (italics added):
Sixth, it has been argued that the trial was unethical because there was no compelling clinical or preclinical evidence that chelation therapy has significant efficacy against atherosclerotic cardiovascular disease, and given that chelation therapy can cause harm, the risk was not minimal. A Bayesian analysis would not look kindly on the results because of the low prior probability of treatment effect (the so-called implausibility argument).6 This is an uncharitable (and unwarranted) interpretation of the data because previous systematic reviews concluded, “insufficient evidence to decide on the effectiveness or ineffectiveness of chelation therapy in improving clinical outcomes among people with atherosclerotic cardiovascular disease.” It is axiomatic that absence of evidence of efficacy is not the same as evidence of the absence of efficacy.
From a Bayesian perspective, the strength of evidence is often summarized using a Bayes factor, which is a measure of how well 2 competing hypotheses (the null and the alternate) predict the data. The Bayes factor and the corresponding strength of evidence for the primary end point result in TACT overall, and diabetic cohorts are shown in Table 1. The p-value of 0.035 for TACT overall cohort translates into a Bayes factor of 0.108, which means the evidence supports the null hypothesis ≈1/9th as strongly as it does the alternative. This reduces the null probability from 50% pretrial (justified by suspension of one’s belief in treatment effect) to 10% post-trial. Although this does not represent strong evidence against the null, it does reduce the level of skepticism surrounding chelation therapy. In the diabetic cohort, the nominal p-value of 0.0002 translates into a Bayes factor of 0.002 (1/500), which reduces the extremely skeptical prior null probability of 95% to 4% post- trial, indicating very strong evidence against the null.
In concluding, Dr. Kaul states:
Finally, TACT highlights the double standard when it comes to accepting inconvenient results not aligned with our preconceived notions on so-called dubious quack cures such as chelation…
Dr. Kaul’s reference “6” above is to a lengthy article that we published in 2008 titled “Why the NIH Trial to Assess Chelation Therapy Should Be Abandoned”. So, it seems, both Drs. Briggs and Kaul were chastising us for our biased, preconceived beliefs about so-called dubious quack cures. Our minds were, apparently, not open. Let’s examine this contention.
Here is the relevant passage from our 2008 article (references in the original, italics added now):
The Likely Outcome
Even if the TACT is completed, which it should not be, it is unlikely to reduce the promotion of reckless uses of Na2EDTA. Whatever the outcome, chelationists have already positioned themselves to continue the practice: By virtue of ridding the body of toxic heavy metals, they claim, chelation is useful for more than 70 conditions. Without exception, chelationists have refused to accept previous results that contradict their beliefs. Their response to the TACT, should it yield definitive, negative results, is unlikely to be different.
The trial, moreover, is unlikely to yield “an informative negative result” even though chelation is almost certainly ineffective for CAD (coronary artery disease). It is more likely to yield ambiguous results. There are multiple endpoints, including subjective quality-of-life measures, and several subgroup analyses. The variety of trial settings increases the likelihood of heterogeneity of procedures and reporting. Promotions of chelation by TACT co-investigators have already introduced unacceptable bias into the trial. There is ample, additional opportunity for mischief, and ample reason to think that several co-investigators are inclined to make it. The statistical analyses will not be Bayesian.
Thus, merely on the basis of chance and bias, it is likely that some outcome data in some subgroups will differ sufficiently, between those receiving Na2EDTA and those receiving placebo, to reassure chelationists that chelation “works” and to sustain “lingering questions of efficacy” in the minds of apologists. Dr. Lamas himself has made much of 2 or 3 “tantalizing positive secondary outcomes” of a previous trial in which only 15 subjects received Na2EDTA, and in which the remaining 30 secondary outcomes and all 7 primary outcomes were unequivocally negative. The all-but-inevitable “tantalizing positive secondary outcomes” of the TACT would likely lead to years of additional, unnecessary trials or, at the very least, unremitting peddling of chelation by practitioners armed with fresh fodder in their perpetual battle against rational standards of care.
Preconceived? Or perhaps just prescient? Dr. Kaul correctly cites our paper as having “argued that the [TACT] was unethical (etc.).” He then opines that our argument was “uncharitable and unwarranted,” because systematic reviews (published after the TACT grant had been approved!) concluded that there was insufficient evidence either for or against chelation for atherosclerotic cardiovascular disease.
He misses the point, and also missed the pre-existing evidence of the absence of efficacy. The point was made by the TACT investigators themselves, in their pitch for the grant. While acknowledging that every RCT of Na2EDTA for atherosclerotic disease had failed to show an effect, they argued:
The very large number of published case reports and case series support a hypothesis that EDTA chelation therapy provides clinical benefits in atherosclerotic vascular disease.
A very small number of trials of very small sample size have randomized 275 patients in aggregate, a number far too small to reliably detect or exclude the most plausible benefits.
Thus the case for clinical equipoise was not based on previous trials having been too small to exclude a small effect, but on “thousands” of confirming case reports and case series contradicting the disconfirming data from those clinical trials.
Even this, however, was wrong. In our paper we reviewed the purportedly confirming case reports and case series (ours is the only such review that we are aware of), and showed that they were neither credible nor accurately represented by the TACT protocol (the most charitable interpretation is that its authors had never read those case reports). That left only the RCT data and a couple of credible but disconfirming case series, which, even if inadequate to exclude a small effect, were more than adequate to exclude the huge effect claimed by chelationists, including those involved with the TACT—which, after all, was the only reason the trial was being done. They were also more than adequate to exclude, in the absence of favorable phase I and II trials, a $30 million, phase III trial, particularly according to the NCCAM’s own language. These and other considerations also argued strongly against a state of clinical equipoise, as we discussed at some length.
Both Drs. Briggs and Kaul seem innocent of the meaning of clinical equipoise. Dr. Briggs, in particular, appears to believe that “retaining equipoise” is akin to “maintaining an open mind” or “withholding judgment” and, furthermore, that there is value or virtue in doing so. Such are not the case. The long-accepted understanding is as follows:
According to this concept of “clinical equipoise,” the requirement is satisfied if there is genuine uncertainty within the expert medical community—not necessarily on the part of the individual investigator—about the preferred treatment.
Thus one cannot simply choose to “retain equipoise” or “suspend one’s (dis)belief in the treatment effect” in order to be less “uncharitable” than the alternative. Clinical equipoise—necessary to render a clinical trial of an unproven treatment ethical—either does or doesn’t exist for a particular treatment, even if the expert medical community is not unanimous in its opinion, and even if it isn’t always possible to know what that opinion is. In the case of EDTA chelation for CAD, the expert medical community was overwhelmingly skeptical, as we documented (that a small effect of EDTA had not been utterly disproved—what Dr. Kaul calls “the uncertainty associated with the possible benefit of chelation therapy”—is beside the point).
Henry Beecher understood, even if Drs. Briggs and Kaul do not, that this fact cannot be changed retroactively.
Bayes and human studies ethics
Regarding Bayes, our implication that the prior probability of a favorable treatment effect for Na2EDTA ought to be very low was based on a warranted interpretation of the existing data, which were entirely disconfirming, and on theoretical considerations discussed in our paper. It was also based on a lack of trust in the judgments of the only practitioners who believed that chelation works: those practitioners also peddle homeopathy, laetrile, Nambudripad’s Allergy Elimination Technique, and myriad other pseudoscientific nonsense, and have a rich history of corrupt, even criminal, behavior (many of them were also TACT investigators).
All of this is meticulously documented and referenced in our paper. We suggest that of all those who have published papers in the medical literature about the TACT, we are the only ones sufficiently open-minded and unbiased to have comprehensively reviewed the prior evidence and the history of the trial—and that includes the TACT investigators themselves.
We don’t see how Dr. Kaul can justify a “suspension of one’s belief in treatment effect” and thus arrive at a prior probability of 0.5 (or even 0.05). Systematic reviews notwithstanding, there were substantial prior trial data, even discounting our other reasons for judging the treatment implausible. If pushed to do so, we would have chosen a prior considerably less than 0.01. Na2EDTA still has a very low probability of being therapeutic for CAD, even if our opinion is now slightly less skeptical (for diabetics only), as Bayes Theorem dictates.
Laetrile and internal mammary artery ligation
Regarding NIH policy and human studies ethics, the existing evidence against chelation for atherosclerotic disease, as we explained in our paper, was considerably stronger than the evidence against many once-popular claims—we cited laetrile for cancer and internal mammary artery ligation for coronary disease—that no responsible scientist or ethicist would now consider subjecting to human trials. Drs. Kaul and Briggs might consider that if their post hoc reasoning about the TACT were to become the norm at the NIH, numerous dubious claims would be fair game for large human trials, each awaiting only the political arm-twisting that was the real impetus for a chelation trial. Then again, that’s why Congress created the NCCAM. Silly me.
Ironically, the TACT also violated the NCCAM’s own, stated intention to “to elucidate mechanisms of action and conduct small, well-developed phase I and II trials” prior to phase III trials. In 2002, prior to the enrollment of TACT subjects, that plan included “studies of the biology of EDTA chelation therapy in animal models,” which, although dubious, would have been more in accordance not only with sound science and public policy but with accepted ethical preconditions for a large human trial. Those studies, apparently, never happened.
The small effect that the TACT authors have now reported, limited to diabetics, may or may not be real, as Drs. Kaul and Briggs (sort of) acknowledge. Here’s hoping that the NIH doesn’t waste more public money pursuing it, unless some legitimate scientific clue suggests doing so, perhaps stemming from the sort of work described in the previous paragraph.
TACT: the real winners
Dr. Kaul is reassured because he thinks, based on formal reports, that “the presumption that the TACT trial [sic] provides actionable evidence for clinical practice is not endorsed even by the TACT investigators…” Alas, he and most academics have little idea whom they’re dealing with. Here is TACT investigator L. Terry Chappell on his own website (italics added):
The Trial to Assess Chelation Therapy (TACT) showed statistically significant benefits for patients with heart disease…This is the first major randomized clinical trial on chelation for vascular disease, and it is a positive study…
I cannot wait to share with our patients and friends what TACT and previous studies mean to current medical science and to those who could benefit from them…
Our experience is that chelation can reduce your chances for blindness, kidney failure, amputation and other vascular complications of diabetes. I believe that ALL diabetics should take chelation, starting about age 30…
I believe that chelation is the most powerful prevention intervention that we have. Everyone should take it to improve his or her chances for living a disease-free life. Scientific evidence is accumulating to back me up…
You do not have to be a doctor, nurse or paramedic to save lives. Just share the vital information contained in this message with others. Bring them to our clinic. Or ask them to set up a free phone consultation with me (Dr. Chappell) to discuss whether we can be of help. See to it that they know what they should know so that they can save their lives too!
Chelation is not just a treatment that is slowly emerging into the spotlight. It is a movement against the forces that are resisting change, especially from powerful economic interests…
TACT is the breakthrough study we needed. More studies will follow, but that will take years…but we want to help people now. Join the movement. Spread the word.
Isn’t it clear what has happened here? It was Dr. Chappell who—as President of the American College for Advancement in Medicine (ACAM), a group of several hundred chelationists—made the original pitch to congressman Dan Burton, who subsequently bullied NHLBI Director Claude Lenfant into accepting a chelation trial. Chappell was also a member, contrary to the NIH conflict of interest policy, of the scientific review committee that approved Dr. Lamas’s application—the same application that had named the ACAM the “the world’s largest and most respected organization of physicians who employ chelation therapy” and Dr. Chappell a “prominent expert.”
Chelationists seem to have got exactly what they’d hoped for from the TACT: a free pass to peddle their favorite, lucrative ‘remedy’—which they tout not only for CAD but for COPD, dementia, schizophrenia, skin wrinkles, and 70 other disparate indications—without being inconvenienced by pesky regulators. If a double standard is highlighted here, it’s a double standard of care, unwittingly encouraged by naive medical academics and a cowardly NIH.
There are numerous additional reasons that the TACT was unethical, including an almost universal lack of informed consent and incompetent medical care given by practitioner/investigators. These will be further explained in future postings.