Science-Based Medicine

When we refer to “science-based medicine” (SBM), it is a very conscious choice to emphasize that good medicine should be based on a solid foundation of science. The name was coined to contrast the difference between the current evidence-based medicine (EBM) paradigm, which fetishizes randomized clinical trial evidence above all else and frequently ignores prior plausibility based on well-established basic science, and the SBM paradigm, which takes prior plausibility into account. The purpose of this post will not be to resurrect old discussions on these differences, but before I attend to the study at hand I bring this up to emphasize that progress in science-based medicine requires progress in science. That means all levels of biological (and even non-biological) basic science, which forms the foundation upon which translational science and clinical trials can be built. Without a robust pipeline of basic science progress upon which to base translational research and clinical trials, progress in SBM will slow and even grind to a halt.

That’s why, in the U.S., the National Institutes of Health (NIH) is so critical. The NIH funds large amounts of biomedical research each year, which means that what the NIH will and will not fund can’t help but have a profound effect shaping the pipeline of the basic and preclinical research that ultimately leads to new treatments and cures. Moreover, NIH funding has a profound effect on the careers of biomedical researchers and clinician-scientists, as having the “gold standard” NIH grant known as the R01 is viewed as a prerequisite for tenure and promotion in many universities and academic medical centers. Certainly this is the case for basic scientists; for clinician-scientists, having an R01 is certainly highly prestigious, but less of a career-killer if an investigator is unable to secure one. That’s why NIH funding levels and how hard (or easy) it is to secure an NIH grant, particularly an R01, are perennial obsessions among those of us in the biomedical research field. It can’t be otherwise, given the centrality of the NIH to research in the U.S.
Continue Reading »

Read meat causes cancer. No, processed meat causes cancer. OK, it’s both read meat and processed meat. Wait, genetically modified grain causes cancer (well, not really). No, aspartame causes cancer. No, this food coloring or that one causes cancer.

Clearly, everything you eat causes cancer!

That means you can avoid cancer by avoiding processed meats, red meat, GMO-associated food (no, probably not), aspartame, food colorings, or anything “unnatural.” Or so it would seem from reading the popular literature and sometimes even the scientific literature. As I like to say to my medical students, life is a sexually transmitted fatal disease that gets us all eventually, but most of us would like to delay the inevitable as long as possible and remain as healthy as possible for as long as possible. One of the most obvious ways to do accomplish these twin aims is through diet. While the parameters of what constitutes a reasonably healthy diet have been known for decades, diet still ranks high on the risk of concerns regarding actions we take on a daily basis that can increase our risk of various diseases. Since cancer is disease (or, I should say, cancers are diseases) that many, if not most, people consider to be the scariest, naturally we worry about whether certain foods or food ingredients increase our risk of cancer.

Thus was born the field of nutritional epidemiology, a prolific field with thousands of publications annually. Seemingly, each and every one of these thousands of publications gets a news story associated with it, because the media love a good “food X causes cancer” or “food Y causes heart disease” story, particularly before the holidays. As a consequence, consumers are bombarded with what I like to call the latest health risk of the week, in which, in turn, various foods, food ingredients, or environmental “toxins” are blamed and exonerated for a panoply of health problems, ranging from the minor to the big three, cardiovascular disease, diabetes, and cancer. It’s no wonder that consumers are confused, reacting either with serial alarm at each new “revelatory” study or with a shrug of the shoulders as each new alarm joins other alarms to produce a tinnitus-like background drone. Unfortunately, this cacophony of alarm also provides lots of ammunition to quacks, cranks, and crackpots to tout their many and varied diets that, they promise, will cut your risk of diseases like cancer and heart disease to near zero—but only if adhered to with monk-like determination and self-denial. (Yes, I’m talking about you, Dean Ornish, among others.)

All of this is why I really wanted to write about an article I saw popping up in the queue of articles published online ahead of print about a month ago. Somehow, other topics intervened, as did my vacation and then the holidays, and somehow I missed it last week, even though a link to the study sits in my folder named “Blog fodder.” Fortunately, it just saw print this week in its final version, giving me an excuse to make up for my oversight. It’s a study by one of our heroes (despite his occasional misstep) here on the SBM blog, John Ioannidis. It comes in the form of a study by Jonathan D. Schoenfeld and John Ioannidis in the American Journal of Clinical Nutrition entitled, brilliantly, Is everything we eat associated with cancer? A systematic cookbook review.
Continue Reading »

I realize that in the question-and-answer session after my talk at the Lorne Trottier Public Science Symposium a week ago I suggested in response to a man named Leon Maliniak, who monopolized the first part of what was already a too-brief Q&A session by expounding on the supposed genius of Royal Rife, that I would be doing a post about the Rife Machine soon. And so I probably will; such a post is long overdue at this blog, and I’m surprised that no one’s done one after nearly three years. However, as I arrived back home in the Detroit area Tuesday evening, I was greeted by an article that, I believe, requires a timely response. (No, it wasn’t this article, although responding to it might be amusing even though it’s a rant against me based on a post that is two and a half years old.) Rather, this time around, the article is in the most recent issue of The Atlantic and on the surface appears to be yet another indictment of science-based medicine, this time in the form of a hagiography of Greek researcher John Ioannidis. The article, trumpeted by Tara Parker-Pope, comes under the heading of “Brave Thinkers” and is entitled Lies, Damned Lies, and Medical Science. It is being promoted in news stories like this, where the story is spun as indicating that medical science is so flawed that even the cell-phone cancer data can’t be trusted:

Let me mention two things before I delve into the meat of the article. First, these days I’m not nearly as enamored of The Atlantic as I used to be. I was a long-time subscriber (at least 20 years) until last fall, when The Atlantic published an article so egregiously bad on the H1N1 vaccine that our very own Mark Crislip decided to annotate it in his own inimitable fashion. That article was so awful that I decided not to renew my subscription; it is to my shame that I didn’t find the time to write a letter to The Atlantic explaining why. Fortunately, this article isn’t as bad (it’s a mixed bag, actually, making some good points and then undermining some of them by overreaching), although it does lay on the praise for Ioannidis and the attacks on SBM a bit thick. Be that as it may, clearly The Atlantic has developed a penchant for “brave maverick doctors” and using them to cast doubt on science-based medicine. Second, I actually happen to love John Ioannidis’ work, so much so that I’ve written about it at least twice over the last three years, including The life cycle of translational research and Does popularity lead to unreliability in scientific research?, where I introduced the topic using Ioannidis’ work. Indeed, I find nothing at all threatening to me as an advocate of science-based medicine in Ioannidis’ two most famous papers, Contradicted and Initially Stronger Effects in Highly Cited Clinical Research and Why Most Published Research Findings Are False. The conclusions of these papers to me are akin to concluding that water is wet and everybody dies. It is, however, quite good that Ioannidis is there to spell out these difficulties with SBM, because he tries to keep us honest.
Continue Reading »

I’m a translational researcher. To those of you who aren’t familiar with what that means, it means (I hope) that I study potential therapies in the lab and try to translate them into actual therapies that will cure patients of breast cancer — or, at the very least, improve their odds of survival or prolong survival when cure is not possible. Translational research is extremely important; indeed, it is the life blood of science-based medicine, with basic science producing the discoveries and clinical research the applications of these discoveries. When it works, it’s the way that science leads medicine to advance. However, sometimes I think that it’s a bit oversold. For one thing, it’s not easy, and it’s not always obvious what basic science findings can be translated into useful therapies, be it for cancer (my specialty) or any other disease. For another thing, it takes a long time. The problem is that the hype about how much we as a nation invest in translational research all too often leads to a not unreasonable expectation that there will be a rapid return on that investment. Such an expectation is often not realized, at least not as fast and frequently as we would like, and the reason has little to do with the quality of the science being funded. It has arguably more to do with how long it takes for a basic science observation to follow the long and winding road to producing a viable therapy. But how long is that long and winding road?

A lot longer than many, even many scientists, realize. At least, that’s the case if a paper from about a year ago by John Ioannidis in Science is any indication. The article appeared in the Policy Forum in the September 5 issue and is entitled Life Cycle of Translational Research for Medical Interventions. As you may recall, Dr. Ioannidis made a name for himself a couple of years ago by publishing a pair of articles provocatively entitled Contradicted and Initially Stronger Effects in Highly Cited Clinical Research and Why Most Published Research Findings Are False, which Steve Novella blogged about a couple of years ago.

Dr. Ioannidis lays it out right in the first paragraph:
Continue Reading »

The science in science-based medicine includes all of science, but relies most heavily on the biomedical literature – published studies that collectively represent our scientific medical knowledge. The scientific basis of medicine is only as good as this body of knowledge and the manner in which it is interpreted and put into practice.

We often discuss on this blog how to evaluate individual studies- the need for blinding, randomization, the importance of study size to meaningful statistical analysis, and other features that distinguish a reliable study from a worthless one. This is important, but only half of the equation. We also at times discuss the medical literature as it relates to a specific medical question or set of related questions – does homeopathy work or are statins beneficial for cholesterol reduction, for example. This requires not only the ability to judge individual studies, but a higher order analysis of the overall pattern of evidence among all relevant studies. Failure to do this, by focusing only on individual studies, results in the failure to see the forest for the trees.

It is this higher order analysis that I wish to discuss in this entry.

Continue Reading »

Everything you know may be wrong. Well, not really, but reading the research of John Ioannidis does make you wonder. His work, concentrated on research about research, is a popular topic here at SBM.  And that’s because he’s focused on improving the way evidence is brought to bear on decision-making. His most famous papers get to the core of questioning how we know what we know (or what we assume) to be evidence. Continue Reading »

David Gorski recently pointed out that Science Based Medicine is going on five years. Amazing. That there would be so much to write about day after day comes as a surprise to me. Somehow I vaguely thought that ‘controversies’ would be resolved. Pick a SCAM, contrast the SCAM with reality as best we understand it, and, once the SCAM was found wanting, it would be abandoned. Why would rational, thoughtful people persist in the pursuit of irrational behavior, contradicted by the universe?

Ha. More the fool me. I would never have guessed that these SCAMs are harder to kill than Dracula (at least one version of Dracula). Stake them and back they come*.

I have tried to avoid repeating repeating information found in prior posts by myself and others, in part because I am lazy and in part because, well, I have said it before. Just look it up. I have come to realize (all too slowly) that each blog entry should be  self contained and that much of the old material is lost in the corn maze (an punning homophone) that is WordPress. Reading my second favorite computer reinforces the realization that each post often needs to be an island universe, complete in itself.

Continue Reading »

‘Tis the season, it would seem, for questioning the scientific method.

You might recall that back in October, I was a bit miffed by an article in The Atlantic entitled Lies, Damned Lies, and Medical Science and expressed my annoyance in one of my typical logorrheic posts. Then, a mere couple of weeks later, Steve Simon wrote a rather scathing criticism of the very concept of science-based medicine, which I ended up answering, again in my usual inimitable logorrheic fashion. Unfortunately, these things often come in threes. Well, maybe not always threes. It’s not as though this “rule” is anything like the count for the Holy Hand Grenade of Antioch, where “Four shalt thou not count, nor either count thou two, excepting that thou then proceed to three. Five is right out.” Except that five isn’t always right out when it comes to these sorts of criticisms of science and/or science-based medicine.

But enough of my pathetic attempt to channel Mark Crislip. The third count in articles expressing skepticism of the scientific method and science-based medicine comes, for purposes of my discussion, in the form of an article in The New Yorker by Jonah Lehrer entitled The Truth Wears Off: Is There Something Wrong With the Scientific Method? Unfortunately, the full article is restricted only to subscribers. Fortunately, a reader sent me a PDF of the article; otherwise, I wouldn’t have bothered to discuss it. Also, Lehrer himself has elaborated a bit on questions asked of him since the article’s publication and published fairly sizable excerpts from his article here and here. In any case, I’ll try to quote as much of the article as I think I can get away with without violating fair use, and those of you who don’t have a subscription to The New Yorker might just have to trust my characterization of the rest. It’s not an ideal situation, but it’s what I have to work with.
Continue Reading »

One of the major themes here on the Science-Based Medicine (SBM) blog has been about one major shortcoming of the more commonly used evidence-based medicine paradigm (EBM) that has been in ascendance as the preferred method of evaluating clinical evidence. Specifically, as Kim Atwood (1, 2, 3, 4, 5, 6, 7, 8) has pointed out before, EBM values clinical studies above all else and devalues plausibility based on well-established basic science as one of the “lower” forms of evidence. While this sounds quite reasonable on the surface (after all, what we as physicians really want to know is whether a treatment works better than a placebo or not), it ignores one very important problem with clinical trials, namely that prior scientific probability matters. Indeed, four years ago, John Ioannidis made a bit of a splash with a paper published in JAMA entitled Contradicted and Initially Stronger Effects in Highly Cited Clinical Research and, more provocatively in PLoS Medicine, Why Most Published Research Findings Are Wrong. In his study, he examined a panel of highly cited clinical trials and determined that the results of many of them were not replicated and validated in subsequent studies. His conclusion was that a significant proportion, perhaps most, of the results of clinical trials turn out not to be true after further replication and that the likelihood of such incorrect results increases with increasing improbability of the hypothesis being tested.

Not surprisingly, CAM advocates piled onto these studies as “evidence” that clinical research is hopelessly flawed and biased, but that is not the correct interpretation. Basically, as Steve Novella and, especially, Alex Tabarrok pointed out, prior probability is critical. What Ioannidis’ research shows is that clinical trials examining highly improbable hypotheses are far more likely to produce false positive results than clinical trials examining hypotheses with a stronger basis in science. Of course, estimating prior probability can be tricky based on science. After all, if we could tell beforehand which modalities would work and which didn’t we wouldn’t need to do clinical trials, but there are modalities for which we can estimate the prior probability as being very close to zero. Not surprisingly (at least to readers of this blog), these modalities tend to be “alternative medicine” modalities. Indeed, the purest test of this phenomenon is homeopathy, which is nothing more than pure placebo, mainly because it is water. Of course, another principle that applies to clinical trials is that smaller, more preliminary studies often yield seemingly positive results that fail to hold up with repetition in larger, more rigorously designed randomized, double-blind clinical trials.

Last week, a paper was published in PLoS ONE Thomas by Thomas Pfeiffer at Harvard University and Robert Hoffmann at MIT that brings up another factor that may affect the reliability of research. Oddly enough, it is somewhat counterintuitive. Specifically, Pfeiffer and Hoffmann’s study was entitled Large-Scale Assessment of the Effect of Popularity on the Reliability of Research. In other words, the hypothesis being tested is whether the reliability of findings published in the scientific literature decreases with the popularity of a research field. Although this phenomenon is hypothesized based on theoretical reasoning, Pfeiffer and Hoffmann claim to present the first empirical evidence to support this hypothesis.
Continue Reading »

Life and medicine generate facts and experiences that require conceptual frameworks that aid in understanding.  It is no good have a pile of facts if they cannot be understood within a broader understanding.

The practice of Infectious Diseases, while certainly aided by understanding anatomy, physiology, microbiology, chemistry and the other sciences that form the core of medicine (referred to in Medical School as the basic sciences), gains a broader  appreciation from the concepts of evolution.  Infectious Diseases, at its most fundamental level, is applied evolution, and understanding evolution often adds greater insight into infectious diseases.  Me find bug, me kill bug, me go home may be my motto, but it is meant in jest.

There have been papers or books that have added conceptual frameworks to my understanding of the natural world and medicine.  Besides evolution, there was Observations on Spiraling Empiricism a classic that all health care providers should read, as it outlines the cognitive errors we all make in prescribing medications; I have discussed this article before.

There is  The Drunkard’s Walk: How Randomness Rules Our Lives by Leonard Mlodinow.  So often the explanation of why something  happens is a shrug of the shoulders; feces occurs. The book formalized my understanding that much of what happens is random and without cause.  The challenge in medicine is trying  find a pattern in the randomness of life upon which to base a diagnosis. It is equally important to recognize when patterns are not there. All too often what is seen as a pattern is our imposing structure on what are random events.  Or maybe that really is a bunny in the clouds.  Clinical study results often occur by chance and having a significant ‘P’ value may still be due to randomness if the study is measuring nonsense.

Continue Reading »