Articles

Archive for Medical Academia

Of SBM and EBM Redux. Part I: Does EBM Undervalue Basic Science and Overvalue RCTs?

During the most recent kerfuffle about whether or not Evidence-Based Medicine can legitimately claim to be science-based medicine, it became clear to me that a whole, new round of discussion and documentation is necessary. This is frustrating because I’ve already done it several times, most recently less than a year ago. Moreover, I’ve provided a table of links to the whole series at the bottom of each post*…Never mind, here goes, and I hope this will be the last time it is necessary because I’ll try to make this the “go to” series of posts for any future such confusions.

The points made in this series, most of which link to posts in which I originally made them, are in response to arguments from statistician Steve Simon, whose essay, Is there something better than Evidence Based Medicine out there?, was the topic of Dr. Gorski’s rebuttal on Monday of this week, and also from several of the comments following that rebuttal. Mr. Simon has since revised his original essay to an extent, which I’ll take into account. I’ll frame this as a series of assertions by those who doubt that EBM is deficient in the ways that we at SBM have argued, followed in each case by my response.

First, a disclaimer: I don’t mean to gang up on Mr. Simon personally; others hold opinions similar to his, but his essay just happens to be a convenient starting point for this discussion. FWIW, prior to this week I perused a bit of his blog, after having read one of his comments here, and found it to be well written and informative.

(more…)

Posted in: Basic Science, Medical Academia, Science and Medicine

Leave a Comment (142) →

Answering a criticism of science-based medicine

Attacks on science-based medicine (SBM) come in many forms. There are the loony forms that we see daily from the anti-vaccine movement, quackery promoters like Mike Adams and Joe Mercola, those who engage in “quackademic medicine,” and postmodernists who view science as “just another narrative,” as valid as any other or even view science- and evidence-based medicine as “microfascism.” Sometimes, these complaints come from self-proclaimed champions of evidence-based medicine (EBM) who, their self-characterization otherwise, show signs of having a bit of a soft spot for the ol’ woo. Then sometimes there are thoughtful, serious criticisms of some of the assumptions that underlie SBM.

The criticism I am about to address tries to be one of these but ultimately fails because it attacks a straw man version of SBM.

True, the criticism of SBM I’m about to address does come from someone named Steve Simon, who vocally supports EBM but doesn’t like the the criticism of EBM implicit in the very creation of the concept of SBM. Simon has even written a very good deconstruction of postmodern attacks on evidence-based medicine (EBM) himself, as well as quite a few other good discussions of medicine and statistics. Unfortunately, in his criticism, Simon appears to have completely missed the point about the difference between SBM and EBM. As a result, his criticisms of SBM wind up being mostly the application of a flamethrower to a Burning Man-sized straw man representing what he thinks SBM to be. It makes for a fun fireworks show but is ultimately misdirected, a lot of heat but little light. For a bit of background, Simon’s post first piqued my curiosity because of its title, Is there something better than Evidence Based Medicine out there? The other reason that it caught my attention was the extreme naiveté revealed in the arguments used. In fact, Simon’s naiveté reminds me very much of my very own naiveté about three years ago.

Here’s the point where I tell you a secret about the very creation of this blog. Shortly after Steve Novella invited me to join, the founding members of SBM engaged in several e-mail frank and free-wheeling exchanges about what the blog should be like, what topics we wanted to cover, and what our philosophy should be. One of these exchanges was about the very nature of SBM and how it is distinguished from EBM, the latter of which I viewed as the best way to practice medicine. During that exchange, I made arguments that, in retrospect, were eerily similar to the ones by Simon that I’m about to address right now. Oh, how epic these arguments were! In retrospect, I can but shake my head at my own extreme naiveté, which I now see mirrored in Simon’s criticism of SBM. Yes, I was converted, so to speak (if you’ll forgive the religious terminology), which is why I see in Simon’s article a lot of my former self, at least in terms of how I used to view evidence in medicine.

The main gist of Simon’s complaint comes right at the beginning of his article:
(more…)

Posted in: Clinical Trials, Medical Academia, Science and Medicine

Leave a Comment (56) →

Uff Da! The Mayo Clinic Shills for Snake Oil

A couple of weeks ago, in a review of the Mayo Clinic Book of Home Remedies, Harriet Hall expressed relief that she hadn’t found any “questionable recommendations for complementary & alternative medicine (CAM) treatments” in that book:

Since “quackademic” medicine is infiltrating our best institutions and organizations, I wasn’t sure I could trust even the prestigious Mayo Clinic.

The Home Remedies book may be free of woo, but Dr. Hall was right to wonder if she could trust the Mayo Clinic. About a year ago I was asked to comment on an article in the American Journal of Hematology (AJH), in which investigators from the Mayo Clinic reported that among a cohort of lymphoma patients who were “CAM” users,

There was a general lack of knowledge about forms of CAM, and about potential risks associated with specific types of CAM…

This suggests the need to improve access to evidence-based information regarding CAM to all patients with lymphoma.

No surprise, that, but I couldn’t help calling attention to the paradox of one hand of the Mayo Clinic having issued that report even as the other was contributing to such ignorance:

The Mayo Clinic Book of Alternative Medicine details dozens of natural therapies that have worked safely for many patients in treating 20 top health issues. You may be surprised that Mayo Clinic now urges you and your doctor to consider yoga, garlic, acupuncture, dietary supplements and other natural therapies. Yet the record is clear. Many of these alternative therapies can help you achieve reduced arthritis pain, healthier coronary arteries, improved diabetes management, better memory function and more.

Mayo Clinic cover

Nor could such a paradox be explained by the right hand not having known what the left was doing: Brent Bauer, MD, the Director of the Mayo Clinic Complementary and Integrative Medicine Program, is both the medical editor of the Book of Alternative Medicine (MCBAM) and a co-author of the article in the AJH.

As chance would have it, I had picked up a copy of the latest (2011) edition of the MCBAM only a couple of days before Dr. Hall’s post. Does it live up to its promises? Do its “straight answers from the world’s leading medical experts” respond to “the need to improve access to evidence-based information regarding CAM?” Let’s find out. In some cases I’ll state the implied questions and provide the straight answers.

(more…)

Posted in: Acupuncture, Book & movie reviews, Chiropractic, Energy Medicine, Health Fraud, Homeopathy, Medical Academia, Medical Ethics, Naturopathy, Science and Medicine, Science and the Media

Leave a Comment (23) →

The Cargo Cult of Acupuncture

Bloodletting, of course, was a major aim of early vessel therapy and is frequently described in the Su wen.1

Paul U Unschuld

“Cargo cult” is a metaphor that describes the act of imitating an activity or a practice without any insight into the underlying principles. In the literal sense, it refers to a magico-religious practice observed in tribal societies, where the members ritually imitate the activities of a technologically-advanced society they had contact with, so that they can magically draw their material wealth. For instance, after WWII, indigenous tribes in New Guinea who had come in close contact with cargo planes, started to build landing strips and populated them with plane-like effigies that were made of straw, bamboo, and coconuts, so that they can magically lure the passing planes.2 The term “cargo cult science” was introduced by Richard Feynman in a speech at Caltech in 1974 to describe pseudoscientific studies in which all the superficial aspects of a scientific inquiry are adhered to, but the underlying principles are not scientific. He classified many educational and psychological studies as such, for having the appearance of academic research but lacking the principles of a scientific inquiry.3

Another example of cargo cult science is the plethora of two-arm acupuncture studies that compare a needling regimen using the traditional concepts, and compare it with a non-interventional placebo. These studies might have the appearance of clinical research, but they are inherently flawed and inconclusive, because they do not rule out the possibility that the observed results are mainly due to the painful stimulus and injury caused by a needle, which can occur regardless of the insertion point. Indeed, an acute noxious stimulus from a prickle, heat, or any other painful stimulus – almost anywhere on the skin – can attenuate the perception of pain in another area of the body through a reflex called “counter-irritation,” also called the “pain-inhibiting-pain effect” or “diffuse noxious inhibitory control” (DNIC).4 DNIC was extensively studied by Fauve et al. in the 1980s, who showed in mice that it has an effect equivalent or superior to that of glucocorticoids.5,6
(more…)

Posted in: Acupuncture, Medical Academia, Science and Medicine

Leave a Comment (21) →

Evidence-Based Medicine, Human Studies Ethics, and the ‘Gonzalez Regimen’: a Disappointing Editorial in the Journal of Clinical Oncology Part 2

NB: If you haven’t yet read Part 1 of this blog, please do so now; Part 2 will not summarize it.

At the end of Part 1, I wrote:

We do not need formal statistics or a new, randomized trial with a larger sample size to justify dismissing the Gonzalez regimen.

In his editorial for the JCO, Mark Levine made a different argument:

Can it be concluded that [the] study proves that enzyme therapy is markedly inferior? On the basis of the study design, my answer is no. It is not possible to make a silk purse out of a sow’s ear.

That conclusion may be correct in the EBM sense, but it misses the crucial point of why the trial was (ostensibly) done: to determine, once and for all, whether there was anything to the near-miraculous claims that proponents had made for a highly implausible “detoxification” regimen for cancer of the pancreas. Gonzalez himself had admitted at the trial’s inception that nothing short of an outcome matching the hype would do:

DR. GONZALEZ: It’s set up as a survival study. We’re looking at survival.

SPEAKER: Do you have an idea of what you’re looking for?

DR. GONZALEZ: Well, Jeff [Jeffrey White, the director of the Office of Cancer Complementary and Alternative Medicine at the NCI—KA] and I were just talking a couple weeks ago. You know, to get any kind of data that would be beyond criticism is—-always be criticism, but at least three times.

You would want in the successful group to be three times — the median to be three times out from the lesser successful groups.

So, for example, if the average survival with chemo, which we suspect will be 5 months, you would want my therapy to be at least — the median survival to be at least 15, 16, 17 months, as it was in the pilot study.

We’re looking for a median survival three times out from the chemo group to be significant.

Recall that the median survival in the Gonzalez arm eventually turned out to be 4.3 months.

(more…)

Posted in: Cancer, Clinical Trials, Health Fraud, Medical Academia, Medical Ethics, Politics and Regulation, Science and Medicine

Leave a Comment (30) →

Evidence-Based Medicine, Human Studies Ethics, and the ‘Gonzalez Regimen’: a Disappointing Editorial in the Journal of Clinical Oncology Part 1

Background: the distinction between EBM and SBM

An important theme on the Science-Based Medicine blog, and the very reason for its name, has been its emphasis on examining all the evidence—not merely the results of clinical trials—for various claims, particularly for those that are implausible. We’ve discussed the distinction between Science-Based Medicine (SBM) and the more limited Evidence-Based Medicine (EBM) several times, for example here (I began my own discussion here and added a bit of formality here, here, and here). Let me summarize by quoting John Ioannidis:

…the probability that a research finding is indeed true depends on the prior probability of it being true (before doing the study), the statistical power of the study, and the level of statistical significance.

EBM, in a nutshell, ignores prior probability† (unless there is no other available evidence) and falls for the “p-value fallacy”; SBM does not. Please don’t bicker about this if you haven’t read the links above and some of their own references, particularly the EBM Levels of Evidence scheme and two articles by Steven Goodman (here and here). Also, note that it is not necessary to agree with Ioannidis that “most published research findings are false” to agree with his assertion, quoted above, about what determines the probability that a research finding is true.

The distinction between SBM and EBM has important implications for medical practice ethics, research ethics, human subject protections, allocation of scarce resources, epistemology in health care, public perceptions of medical knowledge and of the health professions, and more. EBM, as practiced in the 20 years of its formal existence, is poorly equipped to evaluate implausible claims because it fails to acknowledge that even if scientific plausibility is not sufficient to establish the validity of a new treatment, it is necessary for doing so.

Thus, in their recent foray into applying the tools of EBM to implausible health claims, government and academic investigators have made at least two, serious mistakes: first, they have subjected unwary subjects to dangerous but unnecessary trials in a quest for “evidence,” failing to realize that definitive evidence already exists; second, they have been largely incapable of pronouncing ineffective methods ineffective. At best, even after conducting predictably disconfirming trials of vanishingly unlikely claims, they have declared such methods merely “unproven,” almost always urging “further research.” That may be the proper EBM response, but it is a far cry from the reality. As I opined a couple of years ago, the founders of the EBM movement apparently “never saw ‘CAM’ coming.”

(more…)

Posted in: Cancer, Clinical Trials, Medical Academia, Medical Ethics, Politics and Regulation, Science and Medicine

Leave a Comment (59) →

Tai chi and fibromyalgia in the New England Journal of Medicine: An “alternative” frame succeeds

It never seems to fail. I go away for a few days, in this case to combine fun with pleasure and pleasure with fun by giving a talk to the Chicago Skeptics and at the same time meeting my brand new (well, by this time three weeks old) nephew for the first time, and something always happens. Before I get to what happened, I just want to point out that the talk actually went pretty darned well. I was utterly shocked that it was pretty much standing room only, with perhaps 50 people there to hear me. Honestly, don’t you people have anything better to do on a beautiful Saturday afternoon in August? But, seriously, the whole thing was a blast, and the assembled skeptics there didn’t even let me off the hook, as at least a couple of them asked some fairly challenging questions, one of which, I must admit, I wasn’t prepared for. In any case, my thanks go out to Dr. Jennifer Newport, skeptical Chicago pediatrician extraordinaire and organizer of my talk and the party at her apartment afterward. Between the two events she raised hundreds of dollars for the vaccination drive going on at DragonCon this weekend, Chicago Skeptics, the Women Thinking Free Foundation, and CFI-Chicago for inviting me and being such fantastic hosts.

Back to business. Science-based medicine (SBM) business, that is.

What happened while I was away could almost be characterized by the New England Journal of Medicine (NEJM) singing “Oops, I did it again.” Three weeks ago, the hallowed pages of the NEJM hosted a truly execrably credulous review article about acupuncture. So bad was the article that it “merited” the incredibly rare triple beat-down from this very blog, with posts by Steve Novella, the ever-irascible Mark Crislip, and myself in rapid succession applying the clue-by-four. As I was preparing to leave for Chicago on Thursday, I happened to look at the very latest issue of the NEJM hot off the presses, and what to my wondering (and watering–it is ragweed season) eyes should appear but an article reporting a study on the use of tai chi in treating fibromyalgia. Entitled A Randomized Trial of Tai Chi for Fibromyalgia, the study comes out of the Tufts University School of Medicine and the Newton-Wellesley Hospital in Boston and was carried out by a team led by Chenchen Wang, MD, MPH. Not surprisingly, the study has gotten a lot of play in the media, for example, in this story in the L.A. Times, which is at least reasonably restrained, probably because it an AP wire story by Marilynn Marchione, who has written some excellent articles about “alternative” medicine before. Even the usually reliable GoozNews seems smitten with this study beyond what it rates, characterizing it as “rare victory for the National Institute of Health’s National Center for Complementary and Alternative Medicine and Sen. Tom Harkin (D-IA), who routinely comes under fire for pushing funding for these types of studies.”

I’m less impressed. You’ll see what I mean in a few minutes, I hope. First, however, let’s look at the study itself.
(more…)

Posted in: Clinical Trials, Medical Academia, Science and the Media

Leave a Comment (80) →

Peer Review and the Internet

Peer-review has been the cornerstone of quality control in academia, including science and medicine, for the past century. The process is slow and laborious, but a necessary filter in order to maintain a certain standard within the literature. Yet more and more scholars are recognizing the speed, immediacy, and openness of the internet as a tool for exchanging ideas and information, and this is causing some to question the methods of peer review. A recent New York Times article discusses this issue.

This issue is very relevant to Science-Based Medicine as this is in part an experiment – an attempt to produce a high quality, editorially filtered, but not peer-reviewed, online journal. Our process here is simple. Outside submissions are reviewed by two or more editors and typically are either accepted with minor revisions or rejected. In addition we have a staff of regular contributors – those who have a proven track record of producing high quality articles. There is no pre-publication review for their submissions, and they are able to post directly to SBM.

Because many of the issues we cover are timely, we emphasize speed of publication. Therefore copy-editing is done post-publication – the notion being that our readers can tolerate a few typos in order to gain access to material more quickly.

(more…)

Posted in: Medical Academia

Leave a Comment (15) →

Does peer review need fixing?

One of the most important aspects of science is the publication of scientific results in peer-reviewed journals. This publication serves several purposes, the most important of which is to communicated experimental results to other scientists, allowing other scientists to replicate, build on, and in many cases find errors in the results. In the ideal situation, this communication results in the steady progress of science, as dubious results are discovered and sound results replicated and built upon. Of course, scientists being human and all, the actual process is far messier than that. In fact, it’s incredibly messy. Contrary to popular misconceptions about science, it doesn’t progress steadily and inevitably. Rather, it progresses in fits and starts, and most new scientific discoveries go through a varying period of uncertainty, with competing labs reporting conflicting results. To achieve consensus about a new theory can take relatively little time (for example, the less than a decade that it took for Marshall and Warren’s hypothesis that peptic ulcer disease is largely caused by H. pylori or the relatively rapid acceptance of Einstein’s Theory of Relativity) to much longer periods of time.

One of the pillars of science has traditionally been the peer review system. In this system, scientists submit their results to journals for publication in the form of manuscripts. Editors send these manuscripts out to other scientists to review them and decide if the science is sound, if the methods appropriate, and if the conclusions are justified by the data presented. This step of the process is very important, because if editors don’t choose reviewers with the appropriate expertise, then serious errors in review may occur. Also, if editors choose reviewers with biases so strong that they can’t be fair, then science that challenges such reviewers’ biases may never see print in their journals. The same thing can occur to grant applications. In the NIH, for instance, the scientists running study sections must be even more careful in choosing scientists to be on their study sections and review grant applications, not to mention picking which scientists review which grants. Biases in reviewing papers are one thing; biases in reviewing grant applications can result in the denial of funding to worthy projects in favor of projects less worthy that happen to correspond to the biases of the reviewers.

I’ve discussed peer review from time to time, although perhaps not as often as I should. My view tends to be that, to paraphrase Winston Churchill’s invocation of a famous quote about democracy, peer review is the worst way to weed out bad science and promote good science, except for all the others that have been tried. One thing’s for sure, if there’s a sine qua non of an anti-science crank, it’s that he will attack peer review relentlessly, as HIV/AIDS denialist Dean Esmay did. Indeed, in the case of Medical Hypotheses, the lack of peer review let the cranks run free to the point where even Elsevier couldn’t ignore it any more. One thing’s for sure. Peer review may have a lot of defects and blindnesses, but lack of peer review is even worse. It’s no wonder why cranks of all stripes loved Medical Hypotheses.
(more…)

Posted in: Medical Academia, Science and Medicine

Leave a Comment (42) →

Dr. Gorski to be speaking at the Chicago Skeptics on Saturday

The week is finally here! Believe it or not, I’m heading back to my old stomping grounds in the 1990s to appear as a guest of the Chicago Skeptics.

This Saturday, August 21, I’ll be giving a talk co-sponsored by Chicago Skeptics, Women Thinking Free Foundation, and the Center For Inquiry-Chicago at the Black Rock Pub & Kitchen. My talk will be on a topic near and dear to my heart (or a topic that fills me with alarm–the two are not mutually exclusive), mainly the infiltration of pseudoscience into medicine. I’ll be sure to touch on a number of issues, and you can be sure I’ll have something to say about the recent acupuncture review that appeared in the New England Journal of Medicine and perhaps a bit about the sort of pseudoscience that’s being practiced at some of our largest and most respected cancer centers. Afterwards, you–yes, you!–can ask me about anything you want if you show up. Anything, including vaccines, skepticism, and even Bill Maher!

If you live in Chicago and want to harass me (not the way Age of Autism harasses me), head on over to the Chicago Skeptics event page and click on the link to RSVP!

ADDENDUM: Holy crap! Someone just informed me that Chicago Comic Con will be in Chicago the same weekend as me. Even worse, William Shatner will be there on Saturday, which is when I’m giving my talk. How on earth can I possibly compete with the Shat?

Posted in: Announcements, Medical Academia

Leave a Comment (1) →
Page 8 of 19 «...678910...»