One can only imagine how annoyed Twain would have been had he known about opinion polls.
Mark Twain popularized the phrase, “There are three kinds of lies: lies, damned lies, and polls and surveys.” (He may have said “statistics” at the end, but I think this version works as well.)
A new Harris Poll on “alternative medicine” nicely demonstrates some of the problems with polls. The biggest problem is how you frame the questions. You can dramatically affect the results of the poll by exactly how a question is phrased, which other questions come before or after the question, and the overall context of the poll.
In essence a poll is a two-way form of communication. While it is meant to derive information from the subjects of the poll, those subjects are also receiving information from the poll itself. That is the very reason for “push polling” – which is the practice of disguising campaigning messages as a poll. (“On a scale from 1-5, how much are you bothered by the fact that candidate X is a misogynist?”). Yes, Prime Minister has an amusing example of this practice in action.
Interpreting the results of a poll is also not straightforward. You have to know how the subjects of the poll interpreted the questions, and what factors might affect their response. (more…)
As I’ve mentioned before, the single biggest difference between science-based medicine (SBM) and what I like to call pseudoscience-based medicine, namely the vast majority of what passes for “complementary and alternative medicine” (CAM) or “integrative medicine” is that SBM makes an active effort to improve. It seeks to improve efficacy of care by doing basic and clinical research. Then it seeks to improve the quality of care by applying the results of that research to patient care. Yes, the process is complicated and messy, and it frequently doesn’t progress as fast as we would like it to. Sometimes it goes down blind alleys or takes wrong turns, such as when a treatment is adopted too rapidly and determined later to be ineffective. Overall, however, improvement does occur, and it continues to occur. New treatments that work better are discovered. Old treatments that don’t work as well (or that don’t work at all) are abandoned.
There is, however a blurry line between what constitutes medical research and what constitutes quality improvement (QI). A couple of years ago, in one of those unexpected turns that a career can take, an opportunity presented itself for me to become co-director of a statewide quality improvement consortium for breast cancer care in my state. As I’ve alluded to before, it was a case of unexpectedly being in the right place at the right time, of seeing an opportunity and being willing to take it. How I ended up making quality improvement a large part of my career is unimportant. What is important is that it puts me in a unique position among all the other SBM contributors to discuss the interface between science and quality. (It’s also important that I lay down a disclaimer here that this post represents my opinion and my opinion alone; it does not represent the views of the QI with which I’m affiliated, my cancer center, or my university.) In particular, there are ethical considerations that are not obvious, apparently even to someone as brilliant as Steven Pinker, who Tweeted yesterday:
In a recent editorial for the New York Times, Aaron E. Carroll argues, “Labels Like ‘Alternative Medicine’ Don’t Matter. The Science Does.” I agree with this headline thesis, but the details of his argument ironically show the harm that the so-called CAM (complementary and alternative medicine) movement has done.
Carroll starts out well, essentially pointing out that the division between “conventional” and “alternative” medicine, and the division between “Western” and “Eastern” medicine are false dichotomies. Despite this strong start, he muddles his way through the rest of his editorial.
The primary error he commits is to swing from a false dichotomy to a false equivalency, essentially saying that there is no difference between conventional and alternative practice or practitioners. In order to support this contention, however, he has to distort the facts beyond recognition.
In other words, Carroll commits the less-well-known false continuum logical fallacy. Let me explain.
The real differences between SBM and CAM
A man on TV is selling me a miracle cure that will keep me young forever. It’s called Androgel…for treating something called Low T, a pharmaceutical company–recognized condition affecting millions of men with low testosterone, previously known as getting older.
And now for something completely different…sort of.
After writing so much about the latest developments in the ongoing saga of the cancer doctor who is not an oncologist and not a legitimate cancer researcher, plus a rumination on what’s up with President Obama’s nominee for Surgeon General and our favorite form of unscientific medicine, so-called complementary and alternative medicine (CAM), also known as “integrative medicine,” I thought it was time for a change of pace. I wasn’t sure what I was going to write about as Sunday rolled around, but fortunately, as sometimes happens, the New York Times dropped a topic right in my lap, so to speak, both figuratively and literally. It comes in the form of a long article on something that directly concerns men of a certain age, which unfortunately happens to mean men of my age and older. I’m referring to what pharmaceutical company advertising campaigns have dubbed “low T,” short for low testosterone. It’s not clear how the term “low T” originated but Dr. Abraham Morgentaler, founder of Men’s Health Boston, claims to have coined the term when his patients were embarrassed by their difficulty pronouncing the word “testosterone.” Other sources report that it was Solvay Pharmaceuticals that coined the phrase. It doesn’t really matter where the term “low T” came from. The term has stuck, even though the more “correct” medical term would be hypogonadism, as in a man’s testes not working.
I’ll never forget the day when I argued for protecting parents against misleading and false information about the treatment of autism. I was working at a large consumer health organization whose mission was to “empower patients with accurate information” so that they could take control of their health. My opposition was himself a physician who requested that our organization publish an article that advised parents of children with autism to seek out DAN! practitioners and chelation therapy.
I prepared my remarks with the utmost care and delivered them to a committee of our lay executives. I cited examples of children who had died during chelation treatments, explained exactly why there was no evidence that chelation therapy could improve the symptoms of autism and in fact was based on the false premise that “heavy metals” in vaccines were implicated in the etiology of the disease. I concluded that it would be irresponsible for the company to publish such misleading advice/information for parents, and would in fact be counter to our entire mission.
My physician opponent suggested that it was our company’s duty to inform parents of all their options, that we should not be judgmental about treatments, and that I was part of a paternalistic medical establishment that tried to silence creative thinking.
The committee ended up siding with my opponent. I was flabbergasted and asked one of the committee members what on earth they were thinking. She simply shrugged and said that my opponent was more likable than I was.
This experience marked the beginning of my journey towards fighting fire with fire – understanding that being right is not the same as being influential, and that “winning” an argument (where lives are on the line) requires a different skill set than I learned in my scientific training.
And so it was with great interest that I picked up Randy Olson’s book, Don’t Be Such A Scientist: Talking Substance In An Age Of Style. (more…)