An experiment is ethical or not at its inception; it does not become ethical post hoc—ends do not justify means.
~ Henry K. Beecher
A couple of weeks ago, Dr. Josephine Briggs, the Director of the National Center for Complementary and Alternative Medicine (NCCAM), posted a short essay on the NCCAM Research Blog touting the results of the Trial to Assess Chelation Therapy (TACT) (italics added):
The authors found that those receiving the active treatment clearly fared better than those receiving placebo. The accompanying editorial in the AHJ reminds readers about the value of equipoise and the need to “test our beliefs against evidence.”
Most physicians did not expect benefit from chelation treatment for cardiovascular disease. I readily admit, initially, I also did not expect we would find evidence that these treatments reduce heart attack, strokes, or death. So, the evidence of benefit coming from analyses of the TACT trial has been a surprise to many of us. The subgroup analyses are suggesting sizable benefit for diabetic patients—and also, importantly, no benefit for the non-diabetic patient. Clearly subgroup analyses, even if prespecified, do not give us the final answer. But it is also clear that more research is needed to test these important findings.
And TACT findings are indeed a reminder of the importance of retaining equipoise [sic], seeking further research aimed at replicating the findings, and neither accepting nor rejecting findings based on personal biases. The scientific process is designed to weed out our preconceived notions and replace them with evidence.
Dr. Briggs concluded:
So, TACT is a reminder—an open mind is at the center of the scientific method.
Dr. Briggs’s title was “Bayes’ Rule and Being Ready To Change Our Minds”, a reference to a recent editorial that had accompanied one of the TACT papers. That editorial, by Dr. Sanjay Kaul, a physician and statistician from UCLA, begins with this quotation:
Preconceived notions are the locks on the door to wisdom.
~ Merry Browne
Here is the relevant passage from Dr. Kaul’s editorial (italics added):
Sixth, it has been argued that the trial was unethical because there was no compelling clinical or preclinical evidence that chelation therapy has significant efficacy against atherosclerotic cardiovascular disease, and given that chelation therapy can cause harm, the risk was not minimal. A Bayesian analysis would not look kindly on the results because of the low prior probability of treatment effect (the so-called implausibility argument).6 This is an uncharitable (and unwarranted) interpretation of the data because previous systematic reviews concluded, “insufficient evidence to decide on the effectiveness or ineffectiveness of chelation therapy in improving clinical outcomes among people with atherosclerotic cardiovascular disease.” It is axiomatic that absence of evidence of efficacy is not the same as evidence of the absence of efficacy.
From a Bayesian perspective, the strength of evidence is often summarized using a Bayes factor, which is a measure of how well 2 competing hypotheses (the null and the alternate) predict the data. The Bayes factor and the corresponding strength of evidence for the primary end point result in TACT overall, and diabetic cohorts are shown in Table 1. The p-value of 0.035 for TACT overall cohort translates into a Bayes factor of 0.108, which means the evidence supports the null hypothesis ≈1/9th as strongly as it does the alternative. This reduces the null probability from 50% pretrial (justified by suspension of one’s belief in treatment effect) to 10% post-trial. Although this does not represent strong evidence against the null, it does reduce the level of skepticism surrounding chelation therapy. In the diabetic cohort, the nominal p-value of 0.0002 translates into a Bayes factor of 0.002 (1/500), which reduces the extremely skeptical prior null probability of 95% to 4% post- trial, indicating very strong evidence against the null.
In concluding, Dr. Kaul states:
Finally, TACT highlights the double standard when it comes to accepting inconvenient results not aligned with our preconceived notions on so-called dubious quack cures such as chelation…
Dr. Kaul’s reference “6” above is to a lengthy article that we published in 2008 titled “Why the NIH Trial to Assess Chelation Therapy Should Be Abandoned”. So, it seems, both Drs. Briggs and Kaul were chastising us for our biased, preconceived beliefs about so-called dubious quack cures. Our minds were, apparently, not open. Let’s examine this contention. (more…)