Flag of the World Health Organization (WHO)
The World Health Organization (WHO) has recently released a new position statement on mandatory reporting of all interventional clinical trials. This is a positive step in the trend towards higher quality and greater transparency in clinical trials.
The underlying ethical concept here is that the public has a right to data that results from experimentation on humans. The researchers do not ethically own that data. They have been granted the privilege of performing research on humans as part of a social contract that includes the timely public reporting of the data.
The WHO statement calls for all registered clinical trial results to be made public in an open-access public forum in a searchable format within 12 months, and publication in the peer-reviewed literature within 24 months. They urge more rapid dissemination, but state that the 12/24 month time frame is the upper limit of when results should be reported.
The WHO statement is part of a larger trend toward greater transparency but also quality in scientific research. It is part of a recognition that we cannot understand how to best practice medicine and allocate resources based upon individual studies. We need to look at the entire scientific literature as a whole. Even when individual studies may be of high quality, there are many potential factors that can distort the scientific literature and therefore misinform doctors, scientists, and regulatory agencies. (more…)
Wikipedia’s front page
Wikipedia, an online open-source encyclopedia, can boast 470 million visitors each month, making it one of the most popular websites on the internet. It is an incredibly useful resource – I think it’s fair to say it is the online reference of record. For that reason people care how topics important to them are represented in Wikipedia.
Wikipedia, in fact, has become no less than a battleground over certain controversial topics. In essence people generally want Wikipedia to reflect their opinions on controversial topics, and if it doesn’t then there must be something wrong with Wikipedia (rather than there being something wrong with their opinions). I don’t mean to imply that Wikipedia always gets it right – it is a crowdsourced reference and the content is only as good as the editors. But at least they make honest efforts to be neutral and to have standards.
Those standards are the real conflict here, and it is part of a broader conflict over standards. In medicine there is a standard of care, which in turn is based on an underlying system of professional and scientific standards. Medical education is standardized, students have to pass standardized exams, post-graduate clinical training is standardized, there are standardized exams for specialty certification, there are ethical standards enforced by institutions, hospitals, professional organizations, and the state boards of health, and peer-reviewed journals have standards.
This is all meant to ensure that individual patients receive the highest quality of care.
Last year Edward Tobinick sued the Society for Science-Based Medicine, SGU Productions (the producers of the Skeptics’ Guide to the Universe podcast), Yale University, and me personally for libel and (of all things) false advertising. I am frequently asked how the suit is going so here is an update.
The lawsuit involved an article I wrote on Science-Based Medicine on May 8, 2013. Dr. Tobinick’s practice involves giving perispinal etanercept to treat sciatica, Alzheimer’s disease, traumatic brain injury (TBI), and chronic deficits following stroke. He claims that he can reverse the symptoms of these various conditions within minutes. He uses highly emotional videos and anecdotal evidence to promote his treatments.
Further, he has medical use patents on these and other treatments and charges other physicians a substantial fee for training and a royalty simply for treating patients with his methods. Medical use patents are considered by the AMA and other medical organizations to be unethical and are banned in 80 countries (but not the US).
My original article was highly critical of Tobinick’s practice. I emphasized the fact that he is making dramatic clinical claims, which he himself characterizes as revolutionary, without ever having conducted a single double-blind placebo controlled trial. In my opinion none of these uses of etanercept are supported by adequate clinical evidence. In fact, there are no published double-blind placebo-controlled trials of etanercept for post-stroke symptoms or TBI. There are some studies for sciatica, but a 2013 systematic review concluded:
There was insufficient evidence to recommend these agents when treating sciatica, but sufficient evidence to suggest that larger RCTs are needed.
There is a single pilot study of etanercept for Alzheimer’s – a phase II trial with 41 subjects total. That’s it – this is hardly sufficient evidence on which to revolutionize the treatment of multiple complex neurological disorders. In fact, during the hearing, his own expert testified that we are “not there yet.”
Of course, any story illustrating the issues surrounding brain death is going to be a sad and tragic tale. In December of 2013, Jahi McMath suffered bleeding complications following a tonsillectomy and tissue removal for sleep apnea. This resulted in a cardiac arrest with an apparent prolonged period of lack of blood flow to the brain. While her heart function was brought back, Jahi suffered severe brain anoxia (damage due to lack of oxygen) and was declared brain dead on December 12, 2013.
Jahi’s tragic story is not over, however, because her family refused to accept the diagnosis of brain death. They took legal action to keep the hospital from pulling life support, and eventually worked out a compromise where the family was able to remove Jahi to their own care. At present Jahi is apparently being cared for in an apartment in New Jersey, on a ventilator and fed through a feeding tube.
There is often some confusion as to what brain death actually is. The term is unfortunately often used to refer to a persistent vegetative state or other severe impairment of consciousness, but this is not accurate. Brain death refers to a complete lack of function of the brain, including basic reflexes in the brain stem. There is a specific protocol for declaring a person brain dead, requiring detailed examination by at least two attending physicians to document the complete absence of any brain function. If the slightest pupillary reflex is present, then the patient cannot be declared brain dead. The criteria also include provisions that there are no medications in the person’s system that can suppress neurological function and their core body temperature is sufficiently high (being too cold can also suppress neurological function).
Vani Hari, the “Food Babe”.
The default mode of human activity is to construct our own internal model of reality based upon our desires, biases, flawed perceptions, memories, and reasoning, and received narratives from the culture in which we live. That model of reality is then reinforced by confirmation bias and jealously defended.
But we also have the capacity to transcend this pathway of least resistance. Philosophy is the discipline of thinking carefully and systematically about ideas to see if they at least are internally consistent. Science is the discipline of systematically and carefully comparing our internal models of reality against objective reality, and then changing those models to suit the evidence.
Everyone engages in a combination of bias, superstition, logic, and evidence-based reason to varying degrees – the question is, to what degree? The goal of science-based medicine is to increase the proportion of science and reason in the mix with respect to the practice of medicine and public health.
There are many forces at work in society, however, that explicitly oppose the role of science because, in my opinion, they find it inconvenient to their internal model of reality or whatever narrative they are selling.
It is long past time to close the door on homeopathy. After thousands of studies, homeopaths are still unable to produce convincing evidence that homeopathy works for any indication. Multiple reviews of the evidence have come to this conclusion, and now we have one more to add to the pile – the Australian National Health and Medical Research Council (NHMRC) just published their report after reviewing the clinical evidence and have concluded that homeopathy doesn’t work for anything.
Homeopathy is a prescientific medical philosophy based upon the fanciful notions that like cures like (which is really an expression of sympathetic magic) and that extreme dilutions of a substance can retain the magical essence of the substance. These ideas were silly two centuries ago when they were invented. The scientific advances we have made since them have only deepened this conclusion. Homeopathy should have been tossed onto the scrap heap of history along with phrenology, humoral theory, mesmerism, and other quaint ideas. Its persistence is testimony to the power of cultural inertia.
Despite the fact that homeopathic potions have essentially zero scientific plausibility (as close to zero as we can get in science), a great deal of resources have been wasted testing homeopathy clinically. The recent NHMRC review identified more than 1,800 studies, of which 225 were of sufficient size and rigor to include in the review. They report:
The review found no good quality, well-designed studies with enough participants to support the idea that homeopathy works better than a placebo, or causes health improvements equal to those of another treatment.
Although some studies did report that homeopathy was effective, the quality of those studies was assessed as being small and/or of poor quality. These studies had either too few participants, poor design, poor conduct and or [sic] reporting to allow reliable conclusions to be drawn on the effectiveness of homeopathy.
According to CEO Professor Warwick Anderson, “All medical treatments and interventions should be underpinned by reliable evidence. NHMRC’s review shows that there is no good quality evidence to support the claim that homeopathy works better than a placebo.”
In 2013 the NHMRC published a review of the clinical evidence for homeopathy, and they broke this down by medical condition. Of the 68 medical conditions they examined, for 7 of them there was no quality evidence from which to draw any conclusions. For 61 of the conditions there was evidence for lack of efficacy – not just a lack of evidence showing that homeopathy works, but evidence showing that homeopathy does not work. (more…)
Click to enlarge
Positive change not only requires a valid argument, it requires political will. My colleagues and I have been pointing out for years that vaccines are safe and effective, and the anti-vaccine movement, which is built largely on misinformation, threatens the public health by eroding herd immunity. These arguments are no more valid today than they were five or ten years ago (except that new scientific evidence continues to support our conclusion).
We also predicted that it will likely take the significant return of vaccine-preventable diseases to muster the political will to effectively push back against the anti-vaccine movement. Parents need to be more afraid of infectious disease than the false fearmongering surrounding vaccines. We, of course, did not want this to happen, we just thought this was a likely scenario.
I did not think, however, that it would be so sudden and dramatic. The Disneyland measles outbreak created an undeniable media and popular backlash against the anti-vaccine movement. Recent evidence for this is the Jimmy Kimmel segment in which he blasted anti-vaxxers and showed a fake PSA in which real doctors express their frustration over vaccine refusers. Anti-vaxxers replied with their usual shrill nonsense, comparing Kimmel’s statements to hate speech and falsely accusing him of attacking autistic children. Kimmel responded with still more ridicule, making a mockery of anti-vaxxer tweets attacking him. Being the butt of late night comedian jokes is a reasonable sign of popular backlash.
Popular opinion, which is turning against vaccine refusers for threatening the public health, translates into political will. In the case of vaccines there is a specific focus for this political will – state laws allowing exemptions from the requirement for children to be up to date on their vaccines in order to attend public school.
This is perhaps the first real crack in the wall for the almost-universal use of the null hypothesis significance testing procedure (NHSTP). The journal, Basic and Applied Social Psychology (BASP), has banned the use of NHSTP and related statistical procedures from their journal. They previously had stated that use of these statistical methods was no longer required but can be optional included. Now they have proceeded to a full ban.
The type of analysis being banned is often called a frequentist analysis, and we have been highly critical in the pages of SBM of overreliance on such methods. This is the iconic p-value where <0.05 is generally considered to be statistically significant.
The process of hypothesis testing and rigorous statistical methods for doing so were worked out in the 1920s. Ronald Fisher developed the statistical methods, while Jerzy Neyman and Egon Pearson developed the process of hypothesis testing. They certainly deserve a great deal of credit for their role in crafting modern scientific procedures and making them far more quantitative and rigorous.
However, the p-value was never meant to be the sole measure of whether or not a particular hypothesis is true. Rather it was meant only as a measure of whether or not the data should be taken seriously. Further, the p-value is widely misunderstood. The precise definition is:
The p value is the probability to obtain an effect equal to or more extreme than the one observed presuming the null hypothesis of no effect is true.
Perhaps one of the greatest threats to the enterprise of Science-Based Medicine is research fraud and misconduct. Rigorous research methods can be used to minimize the effects of bias, but when those methods themselves are the problem there is no easy fix. Related to this is the need for transparency. When fraud or misconduct is uncovered it erodes confidence in the system because it provokes speculation about how much fraud and misconduct has not been uncovered.
A recent study published in JAMA looks at one aspect of this issue – reporting of misconduct uncovered by the FDA. The good news here is that FDA trials, those that will be used to apply to the FDA for approval of a drug, are carefully monitored and inspected by the FDA. This is an important quality control measure. When the FDA uncovers misconduct it takes steps to correct it. If the misconduct is severe enough then any data that is associated with the poor research practices will be excluded from the trial so as not to taint the results. Even an entire study can be disqualified if necessary.
The problem highlighted by the study is that there is no systematic way for the FDA to communicate its findings through the peer-reviewed literature. Tainted studies, or ones that require a correction or retraction (because the violations were discovered after publication) may therefore persist in the peer-reviewed literature without any indication of the uncovered misconduct.
The standard features of quackery are all there. Proponents of this particular therapy claim that a normal condition is a disease. They make false claims about the cause of this disease. They then charge thousands of dollars for their fake treatment to cure the fake disease, and claim success rates that are not backed by any statistics.
In this case the fake disease is homosexuality, for which there is now a solid consensus that it is a normal variation of human sexuality. The fake treatment is conversion therapy. Recently a New Jersey judge ruled that conversion therapists cannot claim that homosexuality is a disease or disorder. The Southern Poverty Law Center reports:
Superior Court Judge Peter F. Barsio Jr. found that it “is a misrepresentation in violation of [New Jersey’s Consumer Fraud Act], in advertising or selling conversion therapy services, to describe homosexuality, not as being a normal variation of human sexuality, but as being a mental illness, disease, disorder, or equivalent thereof.”
The judge also ruled that [New Jersey conversion therapy provider Jews Offering New Alternatives for Healing (JONAH)] is in violation of the Consumer Fraud Act if it offers specific success statistics for its services when “client outcomes are not tracked and no records of client outcomes are maintained” because “there is no factual basis for calculating such statistics.”