Posts Tagged anecdote

The “It Worked for Me” Gambit

Placebo effects help you feel better, they don't make you actually better.

Placebo effects help you feel better, they don’t make you actually better.

It is almost inevitable that whenever we post an article critical of the claims being made for a particular treatment, alternative philosophy, or alternative profession, someone in the comments will counter a careful examination of published scientific evidence with an anecdote. Their arguments boils down to, “It worked for me, so all of your scientific evidence and plausibility is irrelevant.”

Both components of this argument are invalid. Even if we grant that a treatment worked for one individual, that does not counter the (carefully observed) experience of all the subjects in the clinical trials. They count too – I would argue they count more because we can verify all the important aspects of their story.

I want to focus, however, on the first part of that statement, the claim that a treatment “worked” for a particular individual. Most people operationally define “worked” as, they took a treatment and then they improved in some way. This, however, is a problematic definition on many levels.

Placebo effects

Much of the disagreement about how to define “works” comes down to placebo effects. The generally-accepted scientific approach is to conclude that an intervention works when the desired effect is greater than a placebo. When all other variables are controlled for, the intervention as an isolated variable is associated with an improved outcome. That is the basic logic of a double-blind placebo controlled trial.


Posted in: Commentary

Leave a Comment (0) →

The Cancer Cure Anecdote

Dr. Ian Gawler, a veterinarian, suffered from osteogenic sarcoma (a form of bone cancer) of the right leg when he was 24 in 1975. Treatment of the cancer required amputation of the right leg. After completing treatment he was found to have lumps in his groin. His oncologist at the time was confident this was local spread from the original cancer, which is highly aggressive. Gawler later developed lung and other lesions as well, and was given 6 months to live due to his metastatic disease.

Gawler decided to embark on an alternative treatment regimen, involving coffee enemas, a vegetarian diet, and meditation. Eventually he was completely cured of his terminal metastatic cancer. He has since become Australia’s most famous cancer survivor, promoting his alternative approach to cancer treatment, has published five books, and now runs the Gawler Foundation.

At least, that is the story he believes. There is one major problem with this medical tale, however – while the original cancer was confirmed by biopsy, the subsequent lesions were not. His oncologist at the time, Dr. John Doyle, assumed the new lesions were metastatic disease and never performed a biopsy. It was highly probable – the timing and the location of the new lumps following a highly aggressive cancer. But even a diagnosis that is 95% likely will be wrong in 1 patient out of 20 – which means a working physician will have patients with the 5% diagnosis about once a week. The standard of practice today would be to do a biopsy to get tissue confirmation of the diagnosis, and rule out the less likely alternatives.


Posted in: Cancer

Leave a Comment (26) →

When Further Research Is NOT Warranted: The “Wisdom of Crowds” Fallacy

Most scientific research studies have at least one thing in common: the conclusion section ends with, “further research is warranted.” I’d say it’s about as common as the “talk to your doctor” disclaimer in TV ads for pharmaceutical products. And in a way, they both serve the same purpose. They’re a “CYA” move.

What does “further research is warranted” mean in plain English? I think it can be roughly translated: “My research study is not of the size or scope to fully explain all the phenomena described in this article. Therefore, draw conclusions beyond the data and study methods at your own risk. And yeah, my work is important and cool – so people should study it further.”

Of course, the first two sentences are reasonable – we should always remember not to draw conclusions beyond the information provided by the data we’ve collected (even though that’s about as challenging as getting a beagle not to eat a table scrap in an empty room). The real problem is the third sentence. Is the research promising enough to require further investment? How are we to know if further research is indeed warranted? I would argue that it should not be based solely on the subjective opinions of the researchers nor the popularity of the research topic to the general public. (more…)

Posted in: Clinical Trials, Science and Medicine

Leave a Comment (14) →