Call me naive, but I did not expect the volume or the emotional depth of the responses to the Iraqi civilian death post. I thought many would respond to the new NEJMed survey as I did; wondering about the validity of the previous surveys and recognizing that they have a validity problem. And, that there is a question about what is printed in major journals, from unexpected sources. I did not mean that studies such as Lancet II not be printed. I stated that it should not have been printed in a first line journal for the general medical public. It could have been printed in a 2nd or 3rd line specialty journal where its methods and conclusions could have been debated and reforms shaped by colleagues. I find that hints and clues to errors in pseudoscientific reports mostly lie in the methods section. But questioning a study’s validity can involve more than just a knowledge of the methods and recalculation of the data. Because the “CAM” movement has redefined the borders of the playing field as well as the rules of the game, the entire environment of the scientific system surrounding implausible or unusual reports has to be examined – this goes beyond limits of methods, and includes motivations, funding, characters, and subtexts.
In developing criteria for estimating plausibility (prior probability) the most important criterion of course is consistency and consilience with established knowledge. But there are more. One can increase the effectiveness of investigation by using indicators not presently included in “Evidence Based Medicine” or in science, but that are used in criminology (previous arrests, convictions,) business (trustworthiness, profit vs loss,) and ideology and politics (elevation of the trivial, manipulation of the system; example: sectarian medicine.)
Recently, I’ve had a number of people bring to my attention a news story that has apparently been sweeping the wire services and showing up in all sorts of venues. It is, on its surface, a story of hope, hope for the millions of elderly (and even the not-so-elderly) who are or will be afflicted by that scourge of the mind, memory, and personality, Alzheimer’s disease. This disease is one of the most feared of diseases. A progressive and fatal disease of the brain, it robs a person of his memory and personality, until he no longer recognizes loved ones and becomes too demented to care for himself. The pathophysiology involves the accumulation in the brain of a protein known as β-amyloid, which forms plaques outside of cells, while neurofibrillary tangles believed to be due to the hyperphosphorylation of a protein known as tau develop in dying cells. The exact mechanism by which neuron death occurs is not fully understood, but over time this process leads to a decrease in the amount of gray matter in the cortex. There is no known cure, and the current treatments that we have result in at best a modest delay of the inevitable dementia that accompanies progression of the disease.
Given this grim backdrop and the general aging of the population in developed nations, it is expected that there will be a large increase in the number of people developing Alzheimer’s disease over the next few decades. Naturally, this provides a great deal of incentive to develop more effective treatments. Not surprisingly, sometimes the treatments proposed may sound somewhat outlandish and may even be somewhat outlandish. The treatment about which people were e-mailing me falls into this category, and I haven’t decided yet whether it’s science or pseudoscience. It could be legitimate. What I do know, however, is that I don’t like the way its inventors are promoting it by press conference before any evidence of its clinical efficacy in humans has been accepted by a peer-reviewed publication, leading to a flurry of stories about a new possible “miracle cure” for Alzheimer’s disease grounded in not a lot of science. I’m referring, of course, to the “Alzheimer’s helmet” developed by Dr. Gordon Dougal and his colleagues Dr. Paul Chazot and Abdel Ennaceur at Durham University. Dr. Dougal is a director of Virulite, a medical company based in County Durham in the U.K. Here’s a widely cited article from the Daily Mail that describes the device:
Part IV of the ongoing Homeopathy series will have to wait a day or two, because it is superceded by a recent, comment-worthy publication. Nevertheless, “H series” fans will find here a bit of grist for that mill, too.
An important role for this blog is to discuss problems of interpreting data from clinical studies. Academic medicine has committed itself, on the whole, to scientific rigor—to the extent that this is possible in messy, clinical (especially human) trials. Several tools have been proposed, and to a varying extent used, to enhance the rigor of clinical research and the reporting of clinical research. One of those tools is the registering of clinical trials prior to recruiting subjects. Registration would stipulate a trial’s a priori hypothesis(es), design, planned endpoints, and planned statistical methods, among other things. This would guard against several problems: publication bias—the tendency for some trials, usually “negative” ones, to go unreported; selective reporting of the results of a trial, if some are pleasing but others are not; and post hoc data analysis—finding data after the fact to suggest a novel hypothesis that will falsely be portrayed as an a priori hypothesis. Publication bias is also known as “selective publication” or the “file drawer problem”; post hoc analysis is also known as “data dredging” or “HARKing” (Hypothesizing After the Results are Known).
An article in the Jan. 17 issue of the New England Journal of Medicine demonstrates the usefulness of a trial registry:
Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy
Erick H. Turner, M.D., Annette M. Matthews, M.D., Eftihia Linardatos, B.S., Robert A. Tell, L.C.S.W., and Robert Rosenthal, Ph.D.
A recent study published in the Journal of General Internal Medicine and featured in a Time Magazine article, indicated that of 466 academic physicians in the Chicago area, 45% indicated that they have prescribed a placebo for a patient. This has sparked a discussion of the ethics of prescribing placebos in particular and deception in general in medicine.
A placebo is a biologically inactive treatment, such as a sugar pill. Any perceived benefit from taking a placebo is due to a combination of factors, mostly biased observation and non-specific effects, collectively referred to as the placebo effect. I discussed the placebo effect at length last week, and now will delve deeper into the question of deception in medicine more generally.
Prior to about 30 years ago the relationship between a physician and their patient was functionally paternalistic. This means that the physician did what they thought was best for their patients as a parent would for their child. It also meant that “benign deception” was often used, including prescribing treatments that were known to be inactive or ineffective. Sometimes the deception was one of omission – for example, not telling a patient that their disease was terminal and incurable so as not to upset them needlessly.