The overall goal of science-based medicine is to maintain and improve the standard of science in the practice of medicine at every level. At the heart of the scientific basis of medical knowledge and practice is a process known as peer-review. We have occasionally written about peer-review on SBM, and once again the process is under the microscope over a specific question – should peer review be open or blinded?
What is peer review
The term peer-review refers to a pre-publication process in which a journal editor will send a submitted manuscript to 2-3 experts in a field relevant to the paper to carefully examine every aspect of the paper. They then provide a detailed analysis of the paper: is the research question relevant and appropriate, did the study design properly address the question, were the methods rigorous, was the statistical analysis appropriate, was the data presented fairly, are the conclusions supported by the data, did the authors account for other publications addressing the same issue, and did they address every possible question or objection?
The reviewers’ reports will make specific recommendations for changes that would be necessary to improve the paper, and also make their bottom-line recommendation: accept as is, accept with revisions, or deny. The journal editor(s) then rely upon those reviews, plus their own assessment, to make a final decision.
The entire segment is premised around a false dichotomy, between excess use of opioids and unproven alternative treatments. It is clear that the reporters didn’t even speak to a pain specialist who relies upon science-based treatments, or if they did the specialist was completely ignored because a SBM approach did not fit into the narrative of the report.
Non-opioid options for pain control
The problem addressed by the segment is real – the current technology of pain control is limited. I don’t want to sell pain management short, we have an array of powerful and effective treatments. There are limitations, however, and many patients are inadequately treated.
Stephanie Seneff first came to skeptical attention when she published a study claiming that vaccines were linked to autism. She trolled through the VAERS database and, as David Gorski noted, “tortured the data until it confessed.” Last year she published a paper in which she claimed glyphosate caused autism, claims which I addressed almost a year ago. Gorski also deconstructed this paper, noting, “In fact, if you look at the slides for Seneff’s talks (e.g., this one, available at her MIT web page), you’ll find a tour de force of confusing correlation with causation…”
Seneff is a computer scientist who apparently is anti-vaccine and anti-GMO. In a stunning example of the Dunning-Kruger effect, she feels she can take her computer expertise and export it to biology. She nicely demonstrates that expertise is not so easily transferable.
From autism to concussions
Last year she also published a paper, which escaped my attention until it was recently pointed out to me, claiming that glyphosate, GMOs, and other modern lifestyle factors are responsible for the recent increase in concussions. Her co-author on the paper is Wendy Morely, who is a “Registered Holistic Nutritionist” specializing in the nutrition of concussion. Neither author has any neuroscience background.
The documentary does well what a good history documentary is supposed to do – put a topic into clear historical and cultural perspective. It is amazing how quickly we collectively forget how things were just a generation ago. Twenty to thirty years is apparently all that is necessary for clever marketing to completely change the way the public looks at an issue.
The origins of integrative medicine
Krainin starts appropriately by reminding the viewer of that historical context. A 1984 congressional report concluded that con artists selling “alternative medicine” were swindling the public out of 10 billion dollars a year. They preyed upon the old, the sick, and the desperate. The situation was considered a “scandal.”
One of our primary goals at SBM is to advocate for high standards of science in medicine. This means that we spend a lot of our time discussing claims and practices that fall short of this standard. This is very useful – exploring exactly why a claim falls short is a great way to understand what the standard should be and why.
An unfortunate consequence of this approach, however, is that many of our articles tend to be negative. We focus on what doesn’t work, on what needs to be fixed, and on why people fail.
But there is a positive side to the story as well that we should not neglect – science is powerful and it works. That is why we are such enthusiastic advocates of science in medicine and why it is so important to get it right. We shy away from overhyping scientific advances in medicine, because the mainstream media does that so well, but every now and then it’s good to acknowledge awesome medical and scientific advances. (more…)
Even more interesting to me than the question of whether or not acupuncture is effective for any particular symptom is the meta-question of how acupuncture proponents have managed to promote a treatment with systematically terrible scientific data. A new study provides a fresh example of this, which I will discuss below.
I think the behavior of acupuncturists reflects the fact that there are subcultures within science, where each community has its own standards, culture, and typical practices. You see this reflected in how they conduct their research and support their claims. Chiropractors, for example, have what is in my opinion a very unscientific culture. Their treatments are not science-based; science is an afterthought cherry-picked to support what is ultimately their philosophy.
The culture of acupuncture
The world of acupuncture has its own culture as well. Within this world there are special, very permissive rules of science that allow acupuncture to work for almost anything. One trend is to look for anything that happens locally in the skin when you stick a needle into it and then declare that a “mechanism for acupuncture.” The rules of the acupuncture culture also allow for a shifting definition of what acupuncture actually is, allowing the definition to conform to whatever the evidence shows. It’s a neat and subtle trick that allows acupuncture proponents to completely subvert the purpose of science.
Science functions best when it is free from any bias or conflict of interest. All those engaged in the process should value what is actually true more than anything else. Unfortunately, there are many sources of bias in science.
Researchers may want their pet theory to be supported. Journal editors want to publish research that will have a high impact. And of course, corporations would prefer that the results of scientific research favor their products and services. A recent round of editorials accuses the Coca-Cola company of trying to put its thumb on the scale of science in order to deflect attention away from sugary drinks as a source of obesity and overweight. What are they doing and what does the science actually say?
According to The New York Times:
The beverage giant has teamed up with influential scientists who are advancing this message in medical journals, at conferences and through social media. To help the scientists get the word out, Coke has provided financial and logistical support to a new nonprofit organization called the Global Energy Balance Network, which promotes the argument that weight-conscious Americans are overly fixated on how much they eat and drink while not paying enough attention to exercise.
Carroll starts out well, essentially pointing out that the division between “conventional” and “alternative” medicine, and the division between “Western” and “Eastern” medicine are false dichotomies. Despite this strong start, he muddles his way through the rest of his editorial.
The primary error he commits is to swing from a false dichotomy to a false equivalency, essentially saying that there is no difference between conventional and alternative practice or practitioners. In order to support this contention, however, he has to distort the facts beyond recognition.
In other words, Carroll commits the less-well-known false continuum logical fallacy. Let me explain.
Recently there was another round of scaremongering headlines and articles claiming that cell phones can cause brain cancer. The Daily News wrote: “The scientists were right — your cell phone can give you cancer.” Many online news sites declared: “SHOCK STUDY: CELLPHONES CAN CAUSE CANCER,” in all caps to make sure you understand that you should be alarmed. None of the mainstream reporting I saw looked past the press release.
The first thing to note as that this is a review article. It does not present any new data. It is not an experiment or observational study. It’s not even a meta-analysis. It is just a group of researchers looking at the literature and proclaiming that it confirms what they already believed.
Truly understanding placebo effects (note the plural) is critical to science-based medicine. Misconceptions about placebo effects are perhaps the common problem I encounter among otherwise-scientific professionals and science communicators.
The persistence of these misconceptions is due partly to the fact that false beliefs about placebos, namely that “the” placebo effect is mainly an expectation mind-over-matter effect, is deeply embedded in the culture. It is further exacerbated by recent attempts by CAM proponents to promote placebo-medicine, as their preferred treatments are increasingly being demonstrated to be nothing but placebos.
One idea that proponents of placebo medicine have tried to put forth is that you can have a placebo effect without deception. The study most often pointed to in order to support this claim is Ted Kaptchuk’s irritable bowel syndrome study. However, this study was flawed in that it told participants that placebos can heal, so it wasn’t exactly without deception. (more…)