This essay is the latest in the series indexed at the bottom.* It follows several (nos. 10-14) that responded to a critique by statistician Stephen Simon, who had taken issue with our asserting an important distinction between Science-Based Medicine (SBM) and Evidence-Based Medicine (EBM). (Dr. Gorski also posted a response to Dr. Simon’s critique). A quick-if-incomplete Review can be found here.
One of Dr. Simon’s points was this:
I am as harshly critical of the hierarchy of evidence as anyone. I see this as something that will self-correct over time, and I see people within EBM working both formally and informally to replace the rigid hierarchy with something that places each research study in context. I’m staying with EBM because I believe that people who practice EBM thoughtfully do consider mechanisms carefully. That includes the Cochrane Collaboration.
To which I responded:
We don’t see much evidence that people at the highest levels of EBM, eg, Sackett’s Center for EBM or Cochrane, are “working both formally and informally to replace the rigid hierarchy with something that places each research study in context.”
Well, perhaps I shouldn’t have been so quick to quip—or perhaps that was exactly what the doctor ordered, as will become clear—because on March 5th, nearly four months after writing those words, I received this email from Karianne Hammerstrøm, the Trials Search Coordinator and Managing Editor for The Campbell Collaboration, which lists Cochrane as one of its partners and which, together with the Norwegian Knowledge Centre for the Health Services, is a source of systematic reviews:
Despite the variety of health systems across hundreds of different countries, one feature is near-universal: We all depend on private industry to commercialize and market drug products. And because drugs are such an integral part of our health care system, that industry is generally heavily regulated. Yet despite this regulation, little is publicly known about drug development costs. But aggregate research and development (R&D) data are available, and the pharmaceutical industry spends billions per year.
Editors Note: This is a guest contribution from two medical students, one from Chicago and one from Queensland. If you like their work, we’ll consider having them write more for us.
University of Queensland School of Medicine
Igor Irvin Bussel
Chicago Medical School
Rosalind Franklin University of Medicine and Science
In hopes of joining the SBM movement as medical students, we wanted to take aim at a topic that has yet been finely dissected a la Novella or logorrheicly dismembered a la Gorski. Having realized that a fellow medical student, Tim Kreider, is already addressing integrative medicine on campus, we decided that we would attempt to find a controversial topic that has yet to be addressed on SBM. A serendipitous question from a friend sent us on a mission to explore the pseudo-scientific underbelly of the web and science-based rationale of the claim that vitamin C can induce abortion.
The World Wide Web is a stranger place than we can ever imagine. Most users are aware that they can’t believe everything they read on the Internet, yet they often feel like Sherlock Holmes when they find an esoteric and isolated clue to their own unique health puzzle. Recently, we were asked if there was a connection between vitamin C, menstruation and abortion. We were caught off guard by the question, finding it such a strange connection to make. The story, it seems, is that our friend had come down with a cold and taken mega doses of vitamin C to stave it off (another false belief, but not the subject here). A couple of days later her menses began and she was surprised since it was 4 days earlier than normal. She of course turned to Dr. Google and was quickly provided with numerous sources indicating that indeed, vitamin C would induce the start of a menstrual cycle and can even act as a “natural” abortefacient and a substitute for the ‘morning after’ pill. Being a bit more keen than your average Dr. Google user, she was surprised and continued searching, trying to find evidence to contradict these claims. Alas, she found nothingexcept more sites parroting and corroborating the claim. Then she realized she knew a couple of medical students and asked us what we thought. Our literature review turned up a slew of websites using the standard repertoire of trite pseudo-scientific tactics. Any attempt to find a credible source, validated claim, or independent consensus proved futile.
Steven Salzberg, a friend of this blog and Director of the Center for Bioinformatics and Computational Biology at the University of Maryland, is on the editorial boards of three of the many journals published by BioMed Central (BMC), an important source of open-access, peer-reviewed biomedical reports. He is disturbed by the presence of two other journals under the BMC umbrella: Chinese Medicine and BMC Complementary and Alternative Medicine. A couple of days ago, on his Forbes science blog, Dr. Salzberg explained why. Here are some excerpts:
The Chinese Medicine journal promotes, according to its own mission statement, studies of “acupuncture, Tui-na, Qi-qong, Tai Chi Quan, energy research,” and other nonsense. Tui na, for example, supposedly “affects the flow of energy by holding and pressing the body at acupressure points.”
Right. What is this doing in a scientific journal?… I support BMC…But their corporate leaders seem to care more about expanding their stable than about maintaining the integrity of science. Chinese Medicine simply does not belong in the company of respectable scientific journals.
Forming a scientific journal whose goal is to validate antiquated, unproven superstitions is simply not science, whatever the editors of Chinese Medicine claim.
BMC should be embarrassed to be publishing journals that promote anti-scientific theories and otherwise muddy the literature. By supporting these journals, they undermine the credibility of many excellent BMC journals. They should cut these journals loose.
Could a vitamin with proven benefits in one group cause harm to another? That’s the growing concern with folic acid, the vitamin that dramatically reduces the risk of neural tube birth defects such a spina bifida. Studies designed to explore the possible benefits of folic acid for heart disease, stroke and cancer are giving out some worrying signs: At best, folic acid is ineffective, and at worst it may be increasing the risks of some cancers. So what does this say about routine supplementation for the typical healthy individual, and its overall risk and benefit?
Folate (vitamin B9) is an essential nutrient found green, leafy vegetables, broccoli, peas, corn, oranges, grains, cereals, and meats. Folate has important roles in the synthesis of DNA, and consequently cell division. Significant folate deficiency can lead to macrocytic anemia. Folic acid, a synthetic form of folate, is used in multivitamins supplements because it is better absorbed.
Folic acid’s benefits in pregnancy are well documented. Supplementation before conception, and in the first few weeks of pregnancy, significantly and substantially lower the risk of several different birth defects, including neural tube defects (NTDs). The neural tube is the embryonic precursor to the brain and spinal column. NTDs include very serious defects like spinal bifida and anencephaly, birth without part of the brain.
The scientific approach to understanding the world includes the process of carefully separating out variables and effects. Experiments, in fact, are designed specifically to control for variables. This can be especially challenging in medicine, since the body is a complex and variable system and there are always numerous factors at play. We often characterize the many variables that can influence the outcome in a clinical study as “placebo effects” or “non-specific effect” – things other than a specific response to the treatment in question.
A common error to make when interpreting clinical studies is to confuse non-specific effects – those that result from the therapeutic interaction or the process of observation – with a specific effect from the treatment being studied. While this is broadly understood within the scientific medical community, it seems that within certain fields proponents are going out of their way to sell non-specific effects as if they were specific effects of the favored treatment.
This is perhaps most true for acupuncture. As has been discussed numerous times on SBM, the consensus of the best clinical studies on acupuncture show that there is no specific effect of sticking needles into acupuncture points. Choosing random points works just as well, as does poking the skin with toothpicks rather than penetrating the skin with a needle to elicit the alleged “de qi”.
Science-based medicine depends upon human experimentation. Scientists can do the most fantastic translational research in the world, starting with elegant hypotheses, tested through in vitro and biochemical experiments, after which they are tested in animals. They can understand disease mechanisms to the individual amino acid level in a protein or nucleotide in a DNA molecule. However, without human testing, they will never know if the end results of all that elegant science will actually do what it is intended to do and to make real human patients better. They will never know if the fruits of all that labor will actually cure disease. However, it is in human experimentation where the ethics of science most tend to clash with the mechanisms of science. We refer to “science-based medicine” (SBM) as “based” in science, but not science, largely because medicine can never be pure science. Science has resulted in amazing medical advances over the last century, but if there is one thing that we have learned it’s that, because clinical trials involve living, breathing, fellow human beings, what is the most scientifically rigorous trial design might not be the most ethical.
About a week ago, the AP reported that experiments and clinical trials that resemble the infamous Tuskegee syphilis study and the less well known, but recently revealed Guatemala syphilis experiment were far more common than we might like to admit. As I sat through talks about clinical trial results at the Society of Surgical Oncology meeting in San Antonio over the weekend, the revelations of the last week reminded me that the intersection between science and ethics in medicine can frequently be a very tough question indeed. In fact, in many of the discussions, questions of what could or could not be done based on ethics were frequently mentioned, such as whether it is ethically acceptable or possible to do certain followup trials to famous breast cancer clinical trials. Unfortunately, it was not so long ago that such questions were answered in ways that bring shame on the medical profession.
OK, I admit that I pulled a fast one. I never finished the last post as promised, so here it is.
In the last post I alluded to the 2006 Cochrane Laetrile review, the conclusion of which was:
This systematic review has clearly identified the need for randomised or controlled clinical trials assessing the effectiveness of Laetrile or amygdalin for cancer treatment.
I’d previously asserted that this conclusion “stand[s] the rationale for RCTs on its head,” because a rigorous, disconfirming case series had long ago put the matter to rest. Later I reported that Edzard Ernst, one of the Cochrane authors, had changed his mind, writing, “Would I argue for more Laetrile studies? NO.” That in itself is a reason for optimism, but Dr. Ernst is such an exception among “CAM” researchers that it almost seemed not to count.
Until recently, however, I’d only seen the abstract of the Cochrane Laetrile review. Now I’ve read the entire review, and there’s a very pleasant surprise in it (Professor Simon, take notice). In a section labeled “Feedback” is this letter from another Cochrane reviewer, which was apparently added in August of 2006, well before I voiced my own objections:
Following my recent critique here of the book Disconnect by Devra Davis, about the purported dangers of cell phones to health, David Gorski asked me to comment on a recently published “review article” on the same subject. The article is entitled “Risk of Brain Tumors from Wireless Phone Use” by Dubey et al  published in the J. Comput Assist Tomography. At the outset, the same question occurred to both of us: what is a “review article” about cell phones and brain tumors doing in a highly technical journal dedicated to CT scans and CT imaging? While we are both still guessing about the answer to this question, we agreed that the article itself is a hodge-podge of irrational analysis.
As you might surmise, Dubey and his Indian co-authors come to the conclusion that “that the current standard of exposure to microwave during mobile phone use is not safe for long-term exposure and needs to be revised.” But within the conclusion there is also the following: “There is no credible evidence from the Environmental Health and Safety Office (I presume in India) about the cause of cancer or brain tumors with the use of cell phones. It is illogical to believe that evidence of unusual brain tumors is only because of hundred’s of millions of people using cell phones worldwide.” What?! These are opposite and contradictory statements. The main body of the article includes a lot more instances of such inconsistency.