Everything you know may be wrong. Well, not really, but reading the research of John Ioannidis does make you wonder. His work, concentrated on research about research, is a popular topic here at SBM. And that’s because he’s focused on improving the way evidence is brought to bear on decision-making. His most famous papers get to the core of questioning how we know what we know (or what we assume) to be evidence. (more…)
Archive for Clinical Trials
Well, it’s been a tough month for herbs since my last monthly soiree here at SBM.
Just last week at the American Society for Clinical Oncology (ASCO) meeting, a group out of the Mayo Clinic presented data from a study showing that a well-characterized flaxseed extract was ineffective against hot flashes in postmenopausal women. But as Steve Novella noted here earlier this week, negative clinical trials data on supplements rarely influence the behavior of those who continue to advocate for their herbal use.
Flaxseed, known to contain phytoestrogen compounds such as secoisolariciresinol diglucoside (SDG) and enterolactone, has been purported to relieve hot flashes.
But I think the hypothesis was flawed in the first place: while these compounds bind the estrogen receptor, they have largely been shown to be estrogen receptor modulators that act in a negative manner. Work from the group of Dr. Lillian Thompson at the University of Toronto has repeatedly shown in an estrogen-dependent animal model of human breast cancer that flaxseed components act in a predominantly anti-estrogenic manner. One might suspect that hot flashes would be made worse by flaxseed, although this was not the case in the study presented as ASCO.
Some people have made the mistake of seeing Shunt’s work as a load of rubbish about railway timetables, but clever people like me, who talk loudly in restaurants, see this as a deliberate ambiguity, a plea for understanding in a mechanized world. The points are frozen, the beast is dead. What is the difference? What indeed is the point? The point is frozen, the beast is late out of Paddington. The point is taken. If La Fontaine’s elk would spurn Tom Jones the engine must be our head, the dining car our esophagus, the guard’s van our left lung, the cattle truck our shins, the first-class compartment the piece of skin at the nape of the neck and the level crossing an electric elk called Simon. The clarity is devastating. But where is the ambiguity? It’s over there in a box. Shunt is saying the 8:15 from Gillingham when in reality he means the 8:13 from Gillingham. The train is the same only the time is altered. Ecce homo, ergo elk. La Fontaine knew his sister and knew her bloody well. The point is taken, the beast is moulting, the fluff gets up your nose. The illusion is complete; it is reality, the reality is illusion and the ambiguity is the only truth. But is the truth, as Hitchcock observes, in the box? No there isn’t room, the ambiguity has put on weight. The point is taken, the elk is dead, the beast stops at Swindon, Chabrol stops at nothing, I’m having treatment and La Fontaine can get knotted.
— Art Critic
Ambiguity. Medicine, like art, is filled with ambiguity, at least the way I practice it. Most of my practice is in the hospital. I am sometimes called to see patients that other physicians cannot figure out. And that puts me at a disadvantage, because the doctors who were referring patients to me are all bright, excellent doctors. Often the question is ‘Why does the patient have a fever?’ or ‘Why is the patient ill?’ Sometimes I have an answer. Most of the time I do not.
I am happy, however, to be able to tell the patient what they don’t have. I can often inform the patient and their family that whatever they have is probably not life-threatening or life-damaging, just life-inconveniencing, and most acute illnesses go away with no diagnosis. I always put the ‘just’ in air quotes, because illnesses that require hospitalization are rarely ‘just.’ Just without quotes is reserved for the antivaccine crowd and applied to the small number of deaths from vaccine preventable diseases in unvaccinated children. John Donne they ain’t.
We are excellent, I tell them, at diagnosing life-threatening problems that we can treat, and terrible at diagnosing processes that are self-limited. Of course diagnostic testing is always variable. No test is 100% in making a diagnosis, and often with infections I cannot grow the organism that I suspect is causing the patient’s disease. So for hospitalized patients, ambiguity and uncertainty are the rule of the day. (more…)
The course of research into so-called alternative medicine (CAM) over the last 20 years has largely followed the same pattern. There was little research into many of the popular CAM modalities, but proponents supported them anyway. We don’t need science, they argued, because we have anecdotes, history, and intuition.
When media attention, which drove public attention, was increasingly paid to CAM then serious scientific research increased. A specific manifestation of this was the National Center for Complementary and Alternative Medicine (NCCAM). CAM proponents then argued that their modalities were legitimate because they were being studied (as if that’s enough). Just you wait until all the positive evidence comes rolling in showing how right we were all along.
But then the evidence started coming in negative. A review of the research funded by NCCAM, for example, found that 10 years and 2.5 billion dollars of research had found no proof for any CAM modality. They must be doing something wrong, Senator Harkin (the NCCAM’s major backer) complained. They engaged in a bit of the kettle defense – they argue that the evidence is positive (by cherry picking, usually preliminary evidence), but when it is pointed out to them that evidence is actually negative they argue that the studies were not done fairly. But then when they are allowed to have studies done their way, but still well-controlled, and they are still negative, they argue that “Western science cannot test my CAM modalities.”
Explanatory studies are done under controlled conditions to determine whether a treatment has any efficacy compared to a placebo. Pragmatic studies are designed to assess how the intervention performs in everyday real world practice. Pragmatic studies measure practical success but don’t determine actual efficacy: that requires a proper randomized controlled trial (RCT) with an appropriate control. Pragmatic studies have their place, but they can sometimes make an ineffective treatment look good: a phenomenon I have christened Cinderella Medicine.
Dr. Andrew Weil is a rock star in the “complementary and alternative medicine” (CAM) and “integrative medicine” (IM) movement. Indeed, it can be persuasively argued that he is one of its founders, at least a founder of the its most modern iteration, and I am hard-pressed to think of anyone who did more in the early days of the CAM/IM movement, back before it ever managed to achieve a modicum of unearned respectability, to popularize CAM. In fact, no physician that I can think of has over the course of his lifetime done more to promote the rise of quackademic medicine than Dr. Weil. The only forces greater than Dr. Weil in promoting the infiltration of pseudoscience into academic medicine have been the Bravewell Collaborative and the National Center for Complementary and Alternative Medicine (NCCAM). Before there was Dr. Mehmet Oz, Dr. Dean Ornish, Dr. Mark Hyman, or any of the other promoters of IM, there was Dr. Weil.
And why not? Dr. Weil looks like an aging 1960s rock star, and, operating from his redoubt at the University of Arizona, is quite charismatic. For all the world he has the appearance of a kindly, benevolent Arizona desert Santa Claus, an ex-hippie turned respectable dispensing advice about “natural” medicines, writing books, and making himself ubiquitous on television and radio whenever the topic of alternative medicine comes up. Before Dr. Oz told Steve Novella that “Western” science and medicine can’t study woo like acupuncture, Dr. Weil was there, paving the way for such arguments, previously considered ludicrous, to achieve a patina of respectability.
In fact, he’s still at it, doing it far better and far more subtly than the ham-handed Dr. Oz. Unfortunately, it’s the same anti-science message and the same appeal to other ways of knowing built upon tearing down straw men versions of evidence-based medicine (EBM) with gusto. This was brought home last week when Dr. Weil co-authored an opinion piece with Drs. Scott Shannon and Bonnie J. Kaplan for the journal Alternative and Complementary Therapies entitled Safety and Patient Preferences, Not Just Effectiveness, Should Guide Medical Treatment Decisions, an article that was noted at the blog Booster Shots in a credulous, fawning post entitled Dr. Weil says there’s a better approach to evaluating clinical drug trials. In contast, Steve Novella put it far more succinctly (and accurately) in the title of his post: Andrew Weil Attacks EBM. That’s exactly what Weil and company did in this article.
While Steve is absolutely correct, I also see it more as Dr. Weil demonstrating once again that, upstarts like Dr. Oz aside, he is still the master of CAM/IM apologia, much as, even though both were Sith Lords, Emperor Palpatine remained master over Darth Vader until just before the end. You’ll see why in terms of the arguments, both subtle and not-so-subtle, that Dr. Weil and his acolytes make. Moreover, even though his disciple Shannon is granted the coveted first author position, the arguments presented leave little doubt that it’s Weil who’s driving the bus.
Whether it’s acupuncture, homeopathy or the latest supplement, placebo effects can be difficult to distinguish from real effects. Today’s post sets aside the challenge of identifying placebo effects and look at how placebos are used in routine medical practice. I’ve been a pharmacist for almost 20 years, and have never seen a placebo in practice, where the patient was actively deceived by the physician and the pharmacist. So I was quite surprised to see some placebo usage figures cited by Tom Blackwell, writing in the National Post last week:
The practice is discouraged by major medical groups, considered unethical by many doctors and with uncertain benefit, but one in five Canadian physicians prescribes or hands out some kind of placebo to their often-unknowing patients, a new study suggests.
The article references a paper in the Canadian Journal of Psychiatry which, sadly, does not have much of a web presence. The article continues:
This essay is the latest in the series indexed at the bottom.* It follows several (nos. 10-14) that responded to a critique by statistician Stephen Simon, who had taken issue with our asserting an important distinction between Science-Based Medicine (SBM) and Evidence-Based Medicine (EBM). (Dr. Gorski also posted a response to Dr. Simon’s critique). A quick-if-incomplete Review can be found here.
One of Dr. Simon’s points was this:
I am as harshly critical of the hierarchy of evidence as anyone. I see this as something that will self-correct over time, and I see people within EBM working both formally and informally to replace the rigid hierarchy with something that places each research study in context. I’m staying with EBM because I believe that people who practice EBM thoughtfully do consider mechanisms carefully. That includes the Cochrane Collaboration.
To which I responded:
We don’t see much evidence that people at the highest levels of EBM, eg, Sackett’s Center for EBM or Cochrane, are “working both formally and informally to replace the rigid hierarchy with something that places each research study in context.”
Well, perhaps I shouldn’t have been so quick to quip—or perhaps that was exactly what the doctor ordered, as will become clear—because on March 5th, nearly four months after writing those words, I received this email from Karianne Hammerstrøm, the Trials Search Coordinator and Managing Editor for The Campbell Collaboration, which lists Cochrane as one of its partners and which, together with the Norwegian Knowledge Centre for the Health Services, is a source of systematic reviews:
Despite the variety of health systems across hundreds of different countries, one feature is near-universal: We all depend on private industry to commercialize and market drug products. And because drugs are such an integral part of our health care system, that industry is generally heavily regulated. Yet despite this regulation, little is publicly known about drug development costs. But aggregate research and development (R&D) data are available, and the pharmaceutical industry spends billions per year.