There is a new industry offering preventive health screening services direct to the public. A few years ago it was common to see ads for whole body CT scan screening at free-standing CT centers. That fad sort of faded away after numerous organizations pointed out that there was considerable radiation involved and the dangers outweighed any potential benefits.
Now what I most commonly see are ads for ultrasound screening. In fact, I am sick and tired of finding them in my mailbox and between the pages of my local newspaper. Ultrasound is certainly safe, with no radiation exposure. It sounds like it might be a good idea, but it isn’t.
Life Line Screening advertises itself as “America’s leading provider of quality health screenings.” They offer “4 tests in less than 1 hour – tests that can save your life.” They travel around the country, setting up their equipment in community centers, churches, and YMCAs. For $129 you get ultrasounds of your carotid arteries, your abdominal aorta, your legs, and your heel bone. They mail you your results 21 days later. (more…)
After the previous posting on the Bayesian approach to clinical trial data, several new comments made it clear to me that more needed to be said. This posting addresses those comments and adds a few more observations regarding the unfortunate consequences of EBM’s neglect of prior probability as it applies to “complementary and alternative medicine” (“CAM”).†
The “Galileo Gambit” and the Statistics Gambit
Reader durvit wrote:
A very interesting example, for a number of people, might be estimating the prior probability for Marshall and Warren’s early work on Helicobacter pylori and its impact on gastroduodenal management. I frequently have Marshall quoted to me as a variation on the Galileo gambit, so establishing whether he and Warren would have been helped or hindered by Bayesian techniques would be useful.
This suggestion raises a couple of issues. First, the “Galileo gambit” regarding Marshall and Warren’s discovery is a straw man (as durvit seems to have surmised). (more…)
In science- and evidence-based medicine, the evaluation of surgical procedures represents a unique challenge that is truly qualitatively different from the challenges in medical specialties. Perhaps the most daunting of these challenges is that it is often either ethically unacceptable or logistically impossible to do the gold-standard clinical trial, a double-blind, randomized placebo trial for an operation. After all, the “placebo” in a surgical trial involves patients to anaesthesia, making an incision or incisions like the ones used for the operation under study, and then not doing the operation. Clearly, even leaving the ethics aside, it’s impossible to blind the surgeons and operative team involved to which treatment, real surgery or placebo, the patient is receiving without having a different surgeon do the surgery from the one overseeing the postoperative care of the patient, with the operative surgeon barred from communicating to the postoperative surgeon what happened in the operating room and from participating in the postoperative care of the patient upon whom he operated. This sort of restriction, besides being also highly dubious ethically speaking, goes against the grain of surgical culture, in which a surgeon is expected to provide the postoperative care for his patients almost as a matter of surgical honor. A final problem that complicates any surgical trial is that surgeons of differing technical operating skill will necessarily be involved, and surgical skill is indeed very important in determining outcome. Although there have been examples of double-blinded trials with sham surgery as placebo, for example, in injecting dopamine-producing cells into the brain to treat Parkinson’s disease, difficulties doing such studies tend to force us as surgeons in many cases either to rely on retrospective data, prospective non-randomized data, or, when we’re lucky, a prospective randomized (but not double-blinded) trial of one surgical procedure versus another.
Call me naive, but I did not expect the volume or the emotional depth of the responses to the Iraqi civilian death post. I thought many would respond to the new NEJMed survey as I did; wondering about the validity of the previous surveys and recognizing that they have a validity problem. And, that there is a question about what is printed in major journals, from unexpected sources. I did not mean that studies such as Lancet II not be printed. I stated that it should not have been printed in a first line journal for the general medical public. It could have been printed in a 2nd or 3rd line specialty journal where its methods and conclusions could have been debated and reforms shaped by colleagues. I find that hints and clues to errors in pseudoscientific reports mostly lie in the methods section. But questioning a study’s validity can involve more than just a knowledge of the methods and recalculation of the data. Because the “CAM” movement has redefined the borders of the playing field as well as the rules of the game, the entire environment of the scientific system surrounding implausible or unusual reports has to be examined – this goes beyond limits of methods, and includes motivations, funding, characters, and subtexts.
In developing criteria for estimating plausibility (prior probability) the most important criterion of course is consistency and consilience with established knowledge. But there are more. One can increase the effectiveness of investigation by using indicators not presently included in “Evidence Based Medicine” or in science, but that are used in criminology (previous arrests, convictions,) business (trustworthiness, profit vs loss,) and ideology and politics (elevation of the trivial, manipulation of the system; example: sectarian medicine.)
Recently, I’ve had a number of people bring to my attention a news story that has apparently been sweeping the wire services and showing up in all sorts of venues. It is, on its surface, a story of hope, hope for the millions of elderly (and even the not-so-elderly) who are or will be afflicted by that scourge of the mind, memory, and personality, Alzheimer’s disease. This disease is one of the most feared of diseases. A progressive and fatal disease of the brain, it robs a person of his memory and personality, until he no longer recognizes loved ones and becomes too demented to care for himself. The pathophysiology involves the accumulation in the brain of a protein known as β-amyloid, which forms plaques outside of cells, while neurofibrillary tangles believed to be due to the hyperphosphorylation of a protein known as tau develop in dying cells. The exact mechanism by which neuron death occurs is not fully understood, but over time this process leads to a decrease in the amount of gray matter in the cortex. There is no known cure, and the current treatments that we have result in at best a modest delay of the inevitable dementia that accompanies progression of the disease.
Given this grim backdrop and the general aging of the population in developed nations, it is expected that there will be a large increase in the number of people developing Alzheimer’s disease over the next few decades. Naturally, this provides a great deal of incentive to develop more effective treatments. Not surprisingly, sometimes the treatments proposed may sound somewhat outlandish and may even be somewhat outlandish. The treatment about which people were e-mailing me falls into this category, and I haven’t decided yet whether it’s science or pseudoscience. It could be legitimate. What I do know, however, is that I don’t like the way its inventors are promoting it by press conference before any evidence of its clinical efficacy in humans has been accepted by a peer-reviewed publication, leading to a flurry of stories about a new possible “miracle cure” for Alzheimer’s disease grounded in not a lot of science. I’m referring, of course, to the “Alzheimer’s helmet” developed by Dr. Gordon Dougal and his colleagues Dr. Paul Chazot and Abdel Ennaceur at Durham University. Dr. Dougal is a director of Virulite, a medical company based in County Durham in the U.K. Here’s a widely cited article from the Daily Mail that describes the device:
Part IV of the ongoing Homeopathy series will have to wait a day or two, because it is superceded by a recent, comment-worthy publication. Nevertheless, “H series” fans will find here a bit of grist for that mill, too.
An important role for this blog is to discuss problems of interpreting data from clinical studies. Academic medicine has committed itself, on the whole, to scientific rigor—to the extent that this is possible in messy, clinical (especially human) trials. Several tools have been proposed, and to a varying extent used, to enhance the rigor of clinical research and the reporting of clinical research. One of those tools is the registering of clinical trials prior to recruiting subjects. Registration would stipulate a trial’s a priori hypothesis(es), design, planned endpoints, and planned statistical methods, among other things. This would guard against several problems: publication bias—the tendency for some trials, usually “negative” ones, to go unreported; selective reporting of the results of a trial, if some are pleasing but others are not; and post hoc data analysis—finding data after the fact to suggest a novel hypothesis that will falsely be portrayed as an a priori hypothesis. Publication bias is also known as “selective publication” or the “file drawer problem”; post hoc analysis is also known as “data dredging” or “HARKing” (Hypothesizing After the Results are Known).
An article in the Jan. 17 issue of the New England Journal of Medicine demonstrates the usefulness of a trial registry:
Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy
Erick H. Turner, M.D., Annette M. Matthews, M.D., Eftihia Linardatos, B.S., Robert A. Tell, L.C.S.W., and Robert Rosenthal, Ph.D.
A recent study published in the Journal of General Internal Medicine and featured in a Time Magazine article, indicated that of 466 academic physicians in the Chicago area, 45% indicated that they have prescribed a placebo for a patient. This has sparked a discussion of the ethics of prescribing placebos in particular and deception in general in medicine.
A placebo is a biologically inactive treatment, such as a sugar pill. Any perceived benefit from taking a placebo is due to a combination of factors, mostly biased observation and non-specific effects, collectively referred to as the placebo effect. I discussed the placebo effect at length last week, and now will delve deeper into the question of deception in medicine more generally.
Prior to about 30 years ago the relationship between a physician and their patient was functionally paternalistic. This means that the physician did what they thought was best for their patients as a parent would for their child. It also meant that “benign deception” was often used, including prescribing treatments that were known to be inactive or ineffective. Sometimes the deception was one of omission – for example, not telling a patient that their disease was terminal and incurable so as not to upset them needlessly.