Archive for Science and Medicine

Does popularity lead to unreliability in scientific research?

One of the major themes here on the Science-Based Medicine (SBM) blog has been about one major shortcoming of the more commonly used evidence-based medicine paradigm (EBM) that has been in ascendance as the preferred method of evaluating clinical evidence. Specifically, as Kim Atwood (1, 2, 3, 4, 5, 6, 7, 8) has pointed out before, EBM values clinical studies above all else and devalues plausibility based on well-established basic science as one of the “lower” forms of evidence. While this sounds quite reasonable on the surface (after all, what we as physicians really want to know is whether a treatment works better than a placebo or not), it ignores one very important problem with clinical trials, namely that prior scientific probability matters. Indeed, four years ago, John Ioannidis made a bit of a splash with a paper published in JAMA entitled Contradicted and Initially Stronger Effects in Highly Cited Clinical Research and, more provocatively in PLoS Medicine, Why Most Published Research Findings Are Wrong. In his study, he examined a panel of highly cited clinical trials and determined that the results of many of them were not replicated and validated in subsequent studies. His conclusion was that a significant proportion, perhaps most, of the results of clinical trials turn out not to be true after further replication and that the likelihood of such incorrect results increases with increasing improbability of the hypothesis being tested.

Not surprisingly, CAM advocates piled onto these studies as “evidence” that clinical research is hopelessly flawed and biased, but that is not the correct interpretation. Basically, as Steve Novella and, especially, Alex Tabarrok pointed out, prior probability is critical. What Ioannidis’ research shows is that clinical trials examining highly improbable hypotheses are far more likely to produce false positive results than clinical trials examining hypotheses with a stronger basis in science. Of course, estimating prior probability can be tricky based on science. After all, if we could tell beforehand which modalities would work and which didn’t we wouldn’t need to do clinical trials, but there are modalities for which we can estimate the prior probability as being very close to zero. Not surprisingly (at least to readers of this blog), these modalities tend to be “alternative medicine” modalities. Indeed, the purest test of this phenomenon is homeopathy, which is nothing more than pure placebo, mainly because it is water. Of course, another principle that applies to clinical trials is that smaller, more preliminary studies often yield seemingly positive results that fail to hold up with repetition in larger, more rigorously designed randomized, double-blind clinical trials.

Last week, a paper was published in PLoS ONE Thomas by Thomas Pfeiffer at Harvard University and Robert Hoffmann at MIT that brings up another factor that may affect the reliability of research. Oddly enough, it is somewhat counterintuitive. Specifically, Pfeiffer and Hoffmann’s study was entitled Large-Scale Assessment of the Effect of Popularity on the Reliability of Research. In other words, the hypothesis being tested is whether the reliability of findings published in the scientific literature decreases with the popularity of a research field. Although this phenomenon is hypothesized based on theoretical reasoning, Pfeiffer and Hoffmann claim to present the first empirical evidence to support this hypothesis.

Posted in: Basic Science, Clinical Trials, Science and Medicine

Leave a Comment (12) →

Medical training versus scientific training

This month I will begin my third year of medical school, after a three-year break for laboratory research. Living alternately in the worlds of med school and grad school has prompted me to reflect on differences between these training programs.

[Obvious disclaimer: I have studied at a single institution, and only for five years.]

I am enrolled in a dual-degree MD/PhD program. About 120 US medical schools have such programs, and the National Institutes of Health funds a third of them (MSTP). The schedule of such programs is generally: 2 years of medical school (culminating in USMLE Step 1), 3+ years of graduate school (culminating in dissertation and PhD), and then the last 2 years of medical school (which I begin this month). The most popular residency choices for MD/PhD graduates are internal medicine, pediatrics, and pathology (match data). Other residencies that attract these graduates include dermatology, neurology, ophthamology, and radiology (survey data). The hopes of those funding the MD/PhD training programs and of those accepting the graduates is that these individuals will become physician-scientists, bridging the divide between lab bench and patient bedside with insights from both. (more…)

Posted in: Medical Academia, Science and Medicine

Leave a Comment (20) →

Human Subjects as Political Pawns

When it comes to “alternative medicine” trials, it seems that the NIH is willing to experiment on people in ways that would be unthinkable for real biomedical research. The federal Office for Human Research Protections (OHRP) has posted a preliminary determination letter, dated May 27, 2009, addressing some of the charges we had made against the politics-driven NIH Trial to Assess Chelation Therapy (TACT).

It is a remarkably damning statement, particularly regarding an NIH study. That is, it found—or the recipients admitted—that each of several charges was valid. Among these are misleading statements and unstated risks in the consent form, and the embarrassing backgrounds of TACT investigators. According to the determination letter,

…investigations revealed multiple instances of substandard practices, insurance fraud, and felony activity on the part of investigators.”


Posted in: Politics and Regulation, Science and Medicine

Leave a Comment (4) →

I get mail–chiroquacktic edition

A long while back, at the original wordpress incarnation of my usual blog, I wrote a piece on the reasons that chiropractic is unscientific nonsense. Because it was popular, I resurrected it. Well, a chiropractor has come to bravely defend his field and left me a comment.

A study in the May 2007 issue of the Journal of Manipulative and Physiological Therapeutics reports health plans that use Chiropractors as Primary Care Providers (PCPs) reduce their health care utilization costs significantly.

The study covers the seven-year period from 1999 to 2005. Researchers compared costs and utilization data from an Independent Physicians Association (IPA) that uses Chiropractors as PCPs and a traditional HMO that doesn’t.

The Chiropractic PCPs had 59 percent fewer hospitalizations, 62 percent fewer outpatient surgical cases and 85 percent lower drug costs compared with the HMO plans.

The patients in the Chiropractic PCP group also reported higher satisfaction with their care than the HMO group. Over the seven-year period, Chiropractic patients consistently demonstrated a high degree of satisfaction with their care that ranged from 89 percent to 100 percent.

Study co-author James Winterstein, D.C. says that patients using Chiropractic PCP health care groups “experienced fewer hospitalizations, underwent fewer surgeries and used considerably fewer pharmaceuticals than HMO patients who received traditional medical care.”

Hmmm…interesting assertions. Let’s examine these assertions and the “article” they come from. (more…)

Posted in: Science and Medicine

Leave a Comment (18) →

“Acupuncture Anesthesia”: a Proclamation from Chairman Mao (Part IV)

The Cultural Revolution

After investigating ‘acupuncture anesthesia’ in the People’s Republic of China in 1973, John Bonica wrote:

From the guarded comments made by several anesthesiologists, I concluded that this disuse [of ‘acupuncture anesthesia,’ after its introduction in 1958 until the Great Proletarian Cultural Revolution began in 1966] was the result of disappointing failures in a significant proportion of patients. During the Cultural Revolution this “negative” trend of not using acupuncture was considered the work of revisionists, and subsequently greater emphasis was given to the widespread use of acupuncture in all hospitals.

Similarly, according to Petr Skrabanek,

Those who dared ask such awkward questions [about ‘acupuncture anesthesia’] were branded as “counter-revolutionary revisionists.”

Skrabanek’s reference for that assertion was this 37-page pamphlet:


Posted in: Acupuncture, Science and Medicine, Science and the Media

Leave a Comment (9) →

The death and rebirth of vitalism

One of the common themes in biology and medicine is the feeling that somehow there must be more. Creationist cults simply know that life must be more than matter, and mind-body dualists (which includes most alternative medicine advocates) are certain that humans are more than an “ugly bag of mostly water” (sorry for the geek reference). If you can stick with me here, I’ll explain to you a bit of the history surrounding this fallacy.

Most of us intuitively feel that we are both a body and a person. In every day life, it makes a certain operational sense to think of our “mind” as being something distinct. From a biological standpoint, however, this doesn’t work as well.

Biology was one of the last of the “natural philosophies” to become a science. It was clear to those who studied chemistry and physics that certain principles seemed to explain the natural world, but those who studied living things were mostly involved in description. Still, biology has become a science in its own right. According to Ernst Mayr, one of the greatest biologists of the last century, a number of events preceded biology being recognized as a legitimate science. One vital event was the recognition that all biological processes were constrained by the laws of physics and chemistry. Another important step was the rejection of two erroneous principles: vitalism, and teleology. (more…)

Posted in: Science and Medicine

Leave a Comment (42) →

Tactless About TACT: Critiques Without Substance Should Be Abandoned

In May 2008, the article “Why the NIH Trial to Assess Chelation Therapy (TACT) Should Be Abandoned” was published online in the Medscape Journal of Medicine. The authors included two of our own SBM bloggers, Kimball Atwood and Wallace Sampson, along with Elizabeth Woeckner and Robert Baratz. It showed that the existing evidence on treating heart disease with IV chelation did not justify further study, and that the TACT trial was questionable on several ethical points. Their ethical concerns were taken seriously enough that enrollment in the trial was put on hold pending an investigation. It has now been re-opened after a few band-aids were applied to the ethical concerns. The scientific concerns were never addressed.

I have seen many critiques of the Atwood study, and not a single one has offered any cogent criticism of its factual content or reasoning. Most of them could have been written by someone who had not bothered to read beyond the title. Their arguments can be boiled down to a few puerile points that can be further simplified to:

(1) I believe the testimonial evidence that chelation works.
(2) Atwood and his co-authors are bad guys.

Now Beth Clay has chimed in with an article entitled “Study of Chelation Therapy Should Not Be Abandoned.” I found it truly painful to read, but even the worst has some value as a bad example. Clay’s article could be used for a game of “Count the Errors.” I will point out some of them below. (more…)

Posted in: Clinical Trials, Politics and Regulation, Science and Medicine

Leave a Comment (10) →

Cranks, quacks, and peer review

Last week, I wrote one of my characteristically logorrheic meandering posts about what turns a scientist into a crank or a doctor into a quack. In a sort of continuation of this line of thinking, this week I’ll turn my attention to one of the other most common characteristics of a crank, be he scientific crank (i.e., a creationist), a quack, or historical crank (i.e., Holocaust deniers), specifically how he views the peer review system.

Not suprisingly, one of the favorite targets of pseudoscientists is, in fact, the peer review system. Indeed, it’s a very safe thing to say that, almost without exception, cranks really, really, really don’t like the peer review system for scientific journals and grant review. After all, it’s the system through which scientists submit their manuscripts describing their scientific findings or their grant proposals to their peers, and their peers make a judgment whether manuscripts are scientifically meritorious enough to be published and grant applications scientifically compelling enough to be funded. Creationists hate peer review. HIV/AIDS denialists hate it. Anti-vaccine cranks like those at Age of Autism hate it. Indeed, as a friend of mine, Mark Hoofnagle pointed out a couple of years ago, pseudoscientists and cranks of all stripes hate it. There’s a reason for that, of course, namely that vigorous peer review is a major part of science that keeps pseudoscientists from attaining the respectability that science possesses and that they crave so.

Posted in: Medical Academia, Politics and Regulation, Science and Medicine

Leave a Comment (73) →

Naturopathic Prescribing: The Dark Side Beckons

I am a terrible Oregon chauvinist.  I think there is no better place to live on the planet. Period.  Great natural beauty, not a lot of people, best beer ever and no pro football team. Oregon is both casual and tolerant.  It is safe to say that dressing up in the Pacific NW means tucking your t shirt into your jeans.  And the citizens of the NW, especially in the Portland metro area, are tolerant of  a diverse number of alternative life styles. What more could you want?

No good deed goes unpunished. The downside of toleration is the proliferation of alternative medicine.  Portland has  a school of chiropractic, a college of oriental medicine and  the country’s oldest school of naturopathy, established in 1956.  It is a year older than me. There are about 850 ND’s in Oregon.  To judge from the number of alternative practitioner offices around my hospital,  most of the graduates stay in Portland.

There are five health care systems in Portland.   Three of the five have hired naturopaths as part of their complementary medicine programs.   My system, as of yet, does not have a scam practitioner on staff, a fact of which I am most proud.  Yet,  I suppose it will come some day. However, if you wonder if a hospital practices evidence and science based medicine, see if they have a naturopath, a chiropractor or an acupuncturist on staff.  If they do, they may be interested in issues other than providing quality health care.

Oregon has had a Board of Naturopathic physicians since 1929 to oversee naturopathic practice.  There has been a long tradition of legislative oversight of naturopathy in Oregon, but they have been able, until recently, to only prescribe medications that are naturally derived.  None of that synthetic nonsense for naturopaths. Natural products only.  Until this month.

In Oregon, naturopaths are no longer limited to natural, herbal and homeopathic concoctions, they can also prescribe substances that actually work.  Recently House Bill 327  was passed by the Oregon legislature to expand the prescriptive privileges of naturopaths.  Drugs can now be added to the naturopathic  formulary just by asking.  The bill was passed by the Senate 22-7 and the House unanimously.  Bummer. If you live in Oregon and want to pester your representative on their profound stupidity, a list is at Send them a link to this post.

As a shill for big pharma and a tool of the medical-industrial complex, I suggest this may not be such a  good idea.  Naturopaths do not have the training, experience  or understanding of medicine to safely prescribe medications. Their understanding of disease and the various therapies taught at naturopathic schools are antithetical to what is required to safely and knowledgeably  prescribe modern medications.

Posted in: Legal, Naturopathy, Pharmaceuticals, Politics and Regulation, Science and Medicine

Leave a Comment (99) →

How do scientists become cranks and doctors quacks?

As a physician and scientists who’s dedicated his life to the application of science to the development of better medical treatments, I’ve often wondered how formerly admired scientists and physicians fall into pseudoscience or even generate into out-and-out cranks. Examples are numerous and depressing to contemplate. For example, there’s Linus Pauling, a highly respected chemist and Nobel Laureate, who in his later years became convinced that high dose vitamin C could cure cancer. Indeed, Pauling’s belief that high dose vitamin C could cure the common cold and cancer fueled the development of a whole new form of quackery known as “orthomolecular medicine,” whose entire philosophy seems to be based on the concept that if some vitamins are good more must be better. In essence, “orthomolecular medicine” is a parody of nutritional science; indeed, its advocates take credit for how some strains of “complementary and alternative medicine” (CAM) so frequently advocate the ingestion of huge amounts of dietary “supplements.” I could even go farther and say that orthomolecular medicine is clearly a major part of the “intellectual” (and I do use that term loosely) underpinning of the various biomedical treatments for autism that Jenny McCarthy and Generation Rescue advcoate.

There are other examples as well, all just as depressing to contemplate. For example, consider Peter Duesberg, a brilliant virologist who in the 1980s was widely believed to be on track for a Nobel Prize; that is, until he became fixated on the idea that HIV does not cause AIDS. True, lately he’s been trying to resurrect his scientific reputation with his interesting and possibly even promising chromosomal aneuploidy hypothesis of cancer, but, alas, true to form he’s been doing it by acting like a crank. Specifically, he sees his hypothesis as The One True Cause of Cancer and disparages conventional thinking as having been so very, very wrong all these years (with his being, of course, so very, very brilliant that he saw what no one else could see). Then there are people like Dr. Lorraine Day, who was a respected academic orthopedic surgeon in the 1980s. In the late 1980s, she started to flirt with AIDS pseudoscience through a scare campaign about catching AIDS from aerosolized blood. Of course, given the mystery and fear over HIV in the early years of the epidemic, such a fear, although overblown, was not so far out of the mainstream as to be worthy of the appellation crank. However, after being diagnosed with breast cancer, unfortunately Dr. Day rapidly degenerated into a purveyor of rank pseudoscience, as well as a New World Order conspiracy theorist, religious loon, and Holocaust denier. And let’s not forget Mark Geier, who, although not a distinguished scientist, did, before his conversion to antivaccinationism, apparently do a real fellowship at the NIH and appeared to be on track to a respectable, maybe even impressive, career as an academic physician. Now he’s doing “research” in his basement, injecting autistic children with a powerful anti-sex hormone drug and abusing epidemiology. There are innumerable other examples.

Posted in: Clinical Trials, Health Fraud, Science and Medicine

Leave a Comment (116) →
Page 62 of 87 «...3040506061626364...»