Articles

Tonsillectomy Indications and Complications

Tonsillectomy remains a common surgical procedure with over half a million cases in the US per year, the most common surgical procedure in children. The indications and effects of tonsillectomy remain a matter of research and debate, as is appropriate. It is also a subject of popular misinformation and alarmism.

A recent article by Seth Roberts raises many of the issues with tonsillectomy, but also reveals the pitfalls of non-experts trying to understand the clinical literature and the effects of bias on evaluating a complex medical question. Throughout the article Roberts displays a persistent bias toward downplaying the benefits and exaggerating the risks of tonsillectomy, while accusing the medical establishment of doing the exact opposite.  The purpose of this post is not to defend the practice of tonsillectomy but to review some of the relevant issues and explore how bias can affect an assessment of the evidence.

Indications for Tonsillectomy

Roberts tells the story of Rachael who was offered tonsillectomy for her son and so did some research on her own. She looked on Pubmed (a good place to start) and found a Cochrane review from 2009.

The Cochrane Review that Rachael found (“Tonsillectomy or adeno-tonsillectomy versus non-surgical treatment for chronic/recurrent acute tonsillitis”) was published in 2009. It describes four experiments that compared tonsillectomy to the care a sick child would otherwise receive. All four involved children like Rachael’s son, and all four had similar results: Tonsillectomies had only a small benefit. (Contrary to what Rachael was told.) During the year after random assignment to treatment — the point at which some children had their tonsils removed, other children did not — children whose tonsils were removed had one less sore throat than children who were not operated on (two instead of three for children like Rachael’s son).

Roberts fails to mention that the benefit described above – one fewer sore throat – was for the mild group only. For those who had a more severe sore throat history, the review found greater benefit:

For more severely affected children adeno-/tonsillectomy will avoid three unpredictable episodes of any type of sore throat, including one episode of moderate or severe sore throat in the next year.

In addition there were fewer sore throat days, 17 with tonsillectomy compared to 22 without, even including the sore throat days from the surgery. One of the main points of Roberts’ article is to criticize the reviewers for omitting information from their review, but while discussing that very point he is omitting information stated plainly in the article abstract.

I find this to be a common source of bias in evaluating medical interventions – conflating the risk/benefit in low risk or severity groups with higher severity groups. Also, Roberts commits other common non-expert fallacies in evaluating the potential risk vs benefit of a therapy, narrowing the field of indications and potential benefits. In his article Roberts assumes that the only indication as a preventive measure for tonsillectomy is for recurrent sore throats. This is certainly a common indication, but not the only one. Tonsillectomy is also performed for acute abscess in the tonsils.

Further, there are complications from tonsillitis that go beyond merely have frequent sore throats, including dysphagia (difficulty swallowing) and sleep apnea (airway obstruction during sleep). In fact these are major considerations in deciding who should get a tonsillectomy.

Published guidelines reflect these additional factors that Roberts ignored – only doing tonsillectomy on severe cases and considering other issues like dysphagia and sleep apnea, and recommending medical management and watchful waiting in less severe cases. Roberts accuses physicians of ignoring relevant information about tonsillectomy, but he ignores published practice guidelines that look much more deeply into the issue than is reflected in his analysis.

Risks of Tonsillectomy

After downplaying the benefits and oversimplifying the indications for tonsillectomy, Roberts then goes on to exaggerate the risks. He seems to take the approach of listing any possible hypothesized risk as if it is established. The links he uses to defend each risk he cites does not support the claims he is making. Once again he is led to the conclusion that doctors are ignoring the risks and morbidity from tonsillectomy, while those alleged risks have not been established.

For example, he lists Hodgkins disease with links to evidence for an association with tonsillectomy. He does link to one article from 1972 and disputes the association, but did not link to a 1987 review that found no association between Hodgkins disease and tonsillectomy. As far as I can see this was the last word on the issue. Roberts still gets to list Hodgkins disease as a scary increased risk from tonsillectomy without fairly representing the state of the evidence.

He also lists variant Creutzfeld-Jacob disease, and links to two articles which do not establish a higher risk of developing the disease from tonsillectomy. Rather, the issue is about whether prion disease has an affinity for the tonsils, and whether tonsillar biopsy can be used for diagnosis. The correlation is about the natural history of prion disease, not a cause and effect from having a tonsillectomy. A deeper reading into the literature is needed in order to see this. Roberts, however, simply searched for tonsillectomy and complications and listed everything he found. In none of the cases is a cause and effect established. He wonders why doctors do not list all of the complications he found – that’s why.

The underlying issue is the effect of tonsillectomy on the immune system. Roberts engages in very simplistic reasoning – the tonsils are part of the immune system, removing them therefore compromises immune function and is a bad idea. He ignores the fact that those tonsils that are removed are unhealthy, and perhaps they have become counterproductive to immune function. From the practice guidelines linked to above here is a summary of the current thinking:

With chronic or recurrent tonsillitis, the controlled process of antigen transport and presentation is altered due to shedding of the M cells from the tonsil epithelium. The direct influx of antigens disproportionately expands the population of mature B-cell clones and, as a result, fewer early memory B cells go on to become J-chain–positive IgA immunocytes. In addition, the tonsillar lymphocytes can become so overwhelmed with persistent antigenic stimulation that they may be unable to respond to other antigens. Once this immunological impairment occurs, the tonsil is no longer able to function adequately in local protection, nor can it appropriately reinforce the secretory immune system of the upper respiratory tract. There would therefore appear to be a therapeutic advantage to removing recurrently or chronically diseased tonsils. On the other hand, some studies demonstrate minor alterations of Ig concentrations in the serum and adjacent tissues following tonsillectomy. Nevertheless, there are no studies to date that demonstrate a significant clinical impact of tonsillectomy on the immune system.

What are the alternatives?

The other display of flagrant bias in the article, reflecting, in my opinion, the successful marketing of “alternative” medical thinking, is Roberts’ endorsement of the naturopathic approach to chronic sore throats. While decrying the lack of evidence-based practice on the part of medical doctors (while not understanding or properly reflecting the evidence), he then offers as an alternative that is blatantly not evidence-based. The naturopath, of course, offered nutritional and herbal treatments. Roberts is correct only in that such interventions are lower risk and less permanent than a surgical procedure, so the threshold of evidence efficacy can be lower, but that does not justify making non-evidence-based recommendations.

He mentions vitamin D. Here the evidence is preliminary and mixed, looking for a correlation with vitamin D levels and chronic sore throats. One study found:

There is no difference between the serum vitamin D level and receptor gene polymorphism among children with recurrent tonsillitis and healthy children. But vitamin D insufficiency is more prevalent in children with recurrent tonsillitis group (18%).

That’s pretty thin. Vitamin C was also raised. Roberts blames Linus Pauling for giving vitamin C a stigma among scientists. This may be true, but it’s quite irrelevant. There have been many studies of vitamin C and infections, and the bottom line is that there is no proven benefit. The same is true for multivitamins.

The naturopath also offered powdered larch bark. Here is a summary of the current evidence there:

Larch arabinogalactan is approved by the US Food and Drug Administration (FDA) as a food additive and fiber supplement. However, available scientific evidence does not support claims that larch bark is effective in treating cancer or any other disease in humans. Early laboratory evidence suggested that larch arabinogalactan may stimulate the immune system. However, a more recent study in mice contradicts this finding. Further studies are needed to identify other uses for larch in humans.

Conclusion:

The point of this post is not to thoroughly review the research and make recommendations regarding tonsillectomy. That is not my area of expertise, and panels of experts have already done that. The point, rather, was to use this question, and Roberts’ article, as an example of how bias and lack of expertise affects how we view the evidence. Several common errors in looking at clinical questions were made here, and I find frequently – taking a simplistic approach to disease and treatment mechanism, failing to consider severity as an indication for a treatment, failing to consider all relevant outcomes when assessing potential benefit, and using different criteria of evidence when assessing alternatives.

I do think that health consumers should avail themselves of information that is available to the public, but it is not easy to do so. Pubmed is a good place to start, but for the non-expert I would also recommend finding practice guidelines or summaries of the evidence prepared for the non-expert. It is also a good idea to use the information found as a basis for a conversation with a real expert, rather than as a means of replacing expert opinion with a “Google University” opinion.

 

Posted in: Clinical Trials, Surgical Procedures

Leave a Comment (79) ↓

79 thoughts on “Tonsillectomy Indications and Complications

  1. Dawn says:

    My eldest had her tonsils and adenoids out, and it was the best thing we did. But then, unlike what Roberts seemed to think, our ENT did not recommend it lightly. She had frequent ear infections, and airway obstruction (wall shaking snoring in a 5 year old is not normal…).

    My insurance of the time required a second opinion, to which we obediently went (and paid for….). That ENT came in, looked at her father and me and made the assumption “yuppie parents, just want the kid’s tonsils out because they had theirs out” (although neither of us had had ours removed, in reality). He took the history, then turned to daughter and asked her to open her mouth and take a deep breath. He jumped back, exclaimed “she’s obstructing! This child needs her tonsils out!”

    My response was simply, “yes….”

    After the removal, no more ear infections, no more sore throats. Best thing we ever did…and medically indicated!

  2. Harriet Hall says:

    Thanks for the excellent summary.

    In the early 20th century, doctors did a LOT of tonsillectomies. Whole families would have them done at once – sort of a package deal. My mother, born in 1920, had frequent throat infections as a child and her parents were told that she would die if she didn’t have a tonsillectomy. She didn’t have the surgery, and she never had another throat infection in her entire life, dying in her 80s of other causes.

    Doctors KNEW tonsillectomies were good for everyone. But even the most “obvious” medical truths require testing, and sober scientists questioned the practice and did controlled studies. Practice changed in response to the scientific evidence.

    This story is a great illustration of science-based medicine at work. It’s a great example of doctors changing their practice in the interest of patients’ welfare rather than sticking to a money-making opportunity in their own best interest. And it stands in stark contrast to the history of alternative medicine, where practices are seldom seriously questioned and no practice is ever discontinued.

  3. DugganSC says:

    {nods} And there probably is a fair amount of backlash from prior times when many procedures were done simply because they were done. Surgery, even minor surgery, is always a risk. It’s generally a manageable risk, but it’s not negligible. People die in the middle of tonsillectomies due to infections or bad anesthesia just like they do for removal of the appendix or open heart surgery. Fewer do, of course, because of the difference in circumstances, but it’s enough that the risks of surgery should always be contrasted against what’s to be gained.

    Personally, it always scares me when there’s a new medical breakthrough involving surgery because I know that many people, being who they are, will buy right into it. Tonsillectomies for everyone, removal of multiple teeth in childhood to “make room”, using braces on almost perfectly straight teeth, flouride treatments every time a child goes to the dentist… it’s all fads to me, and fads which run the risk of harm if we don’t remember that there is a risk.

    It’s kind of like how one of our neighbors who raised dogs said, “Most dogs are friendly and would give their life for you, but not all of them, and never forget that every dog has that little bit of wolf in them that could come out to bite you.”

  4. CarolM says:

    I had mine so young that I have no idea what my situation was prior to that. The one person I know had it as an adult had a hellacious time post-op. So I’m glad mine was done already.

    There is a rumor going around that the op hurts one’s singing voice potential, but apparently it actually improves it by creating a bigger space in the throat.

  5. Calli Arcale says:

    Our church music director had his out recently, as part of a larger procedure to treat his obstructive sleep apnea. It did not impair his singing voice, nor really improve it. Mostly, it *changed* it. I, a non-professional, can’t really tell the difference, but he enrolled in voice lessons before he even had the surgery so he’d be able to start as soon as he was healed and get accustomed to the new space in his throat. He said it was a dramatic difference, but couldn’t judge whether it was better or worse; he was just enjoying getting a good night’s sleep for a change, and quite willing to do the work to adapt to his new “pipes”.

    I’m glad tonsillectomies aren’t done routinely anymore. I have to echo the comment about braces on nearly straight teeth; I’m still not totally clear on why mine were done, nor whether it was worthwhile to do. And I wonder whether routine wisdom tooth extraction will eventually go the same way as tonsillectomies, only done in cases of specific, clinical need rather than just “well, this might cause problems down the road, so let’s get rid of it”.

  6. Regarding tonsillectomy and changing singing voice. I believe velopharyngeal insufficiency can be one risk of tonsillectomy (or other surgeries of the palate area, depending upon what they are altering).

    http://emedicine.medscape.com/article/873018-overview

    Basically, by changing the muscle or space that effect velopharyngeal closure, problems with hyper or hypo nasality can occur (too much or too little air escaping from the nose during speech). I’m speculating, but an adult with previous good articulation could have a change in voice quality with VP, but the risk of losing clarity of speech would be less than with a child who hasn’t yet acquired good articulation.

    A child with VP can struggle a lot with articulation. They have a hard time with consonants since they can’t develop enough pressure within the mouth to correctly produce the sounds. Also they can develop compensatory misarticulations like glottal stops.

    No idea how much of a risk this is with tonsillectomy, I doubt it’s high, otherwise more speech therapists would be familiar with treating it. I’m only familiar with the concept because velopharyngeal insufficiency is associated with cleft palate and my son had it before his second to last surgery.

    Sorry to be obscure, I’ll return you to your regular discussion.

  7. Seth Roberts says:

    There are many factual mistakes in your comment on my article.

    1. “Roberts displays a persistent bias toward downplaying the benefits and exaggerating the risks of tonsillectomy while accusing the medical establishment of doing the exact opposite.” I do not say that the Cochrane reviewers exaggerate the benefits of tonsillectomies. Your post does not contain a single example where I say the medical establishment exaggerates the benefits.

    2. “Roberts fails to mention that the benefit described above – one fewer sore throat – was for the mild group only.” That’s wrong. I do mention it. Rachael’s son would have been in the mild group. I wrote “(two instead of three for children like Rachael’s son)”.

    3. “In his article Roberts assumes that the only indication as a preventive measure for tonsillectomy is for recurrent sore throats.” Nowhere do I assume that. You give no examples.

    4. “while not understanding the evidence.” I failed to find any examples in your post of my “not understanding the evidence.” I omitted some of the evidence for benefit because that was not the main point of my post. Omission of evidence does not equal misunderstanding of evidence.

    5. “Roberts accuses physicians of ignoring relevant information about tonsillectomy, but he ignores published practice guidelines that look much more deeply into the issue than is reflected in his analysis.” I said that doctors ignore relevant information about tonsillectomies when making statements to parents about the risks of tonsillectomies. For example, they fail to tell parents that tonsils are part of the immune system. What you say about practice guidelines does not contradict that.

    6. “Roberts engages in very simplistic reasoning – the tonsils are part of the immune system, removing them therefore compromises immune function and is a bad idea.” Because tonsils are part of the immune system, removing them runs a considerable risk of compromising immune function, just as removing part of the brain runs a considerable risk of causing brain damage. I say this in the post. It is common sense.

    7. “He ignores the fact that those tonsils that are removed are unhealthy.” What you call a “fact” is not a fact — it is a theory. As you say yourself in the next sentence (“perhaps they have become counterproductive”).

    8. The “summary of current thinking” is a theory of unknown validity. By failing to say this, you overstate your case.

    9. “There are no studies to date that demonstrate a significant clinical impact of tonsillectomy on the immune system.” Since practitioners of evidence-based medicine refuse to take seriously evidence that doesn’t come from a placebo-controlled double-blind experiment, there will surely never be — for ethical reasons — a study that “demonstrates a significant clinical impact of tonsillectomy on the immune system.” To say that one’s point is supported by the absence of evidence that cannot exist is not serious argument.

    10. “Roberts’ endorsement of the naturopathic approach to chronic sore throats. . . . he then offers as an alternative that is blatantly not evidence-based.” I suggest you look further at the Vitamin D evidence I link to. There is evidence that Vitamin D supplementation helps. Rachael searched for evidence. My post includes a long paragraph about her search. To call what she did “blatantly not evidence-based” is like saying white is black.

    11. A biased view of evidence is shown by the following which you quote approvingly: “However, available scientific evidence does not support claims that larch bark is effective in treating cancer or any other disease in humans. Early laboratory evidence suggested that larch arabinogalactan may stimulate the immune system. However, a more recent study in mice contradicts this finding.” In other words, there is some evidence supporting the value of larch bark (“early laboratory evidence”) and some evidence (“a more recent study in mice”) not supporting the value of larch bark. Given this, to say “available scientific evidence does not support claims . . .” is false. An accurate statement is that some evidence does and some evidence doesn’t.

    12. You recommend “summaries of the evidence prepared for the non-expert.” The Cochrane review that I discuss included a summary of the evidence for non-experts. That summary — the only summary for non-experts I found — was grossly misleading. Unless you can point to a tonsillectomy summary for non-experts that is not misleading, the available evidence (one bad summary, zero good summaries) contradicts your recommendation.

    13. Here are your most serious factual mistakes: “In none of the cases [of bad side effects] is a cause and effect established. He wonders why doctors do not list all of the complications he found – that’s why.” I listed more than a dozen bad side effects associated with tonsillectomies (polio, obesity, etc.). The associations — association is not causation — were made by the authors of the articles and PubMed indexers, not me. Mistake One: I did not claim these associations reflected cause and effect. Mistake Two: I did not “wonder why doctors do not list all of the complications he found” — I simply pointed out that the Cochrane reviewers did not list them.

    These are serious factual mistakes because they reflect a poor understanding of risk and what parents (and other people contemplating medical treatments) want to be told. They go to the heart of my objection to how “evidence-based medicine” is practiced. Consideration of evidence is good, of course. Omission of evidence is bad. Omission of evidence of danger, when telling parents about the pros and cons of a treatment, is very bad. Right here you endorse exactly that — omission of evidence of danger. You are not alone, of course. “Evidence-based medicine” practitioners endorse omission of evidence over and over and over. If it were called “some-of-the-evidence-based medicine” it would be less deceptive.

    Risks are inherently uncertain. In contrast to benefits, risks are usually low probability or hard to measure. It will always be difficult (and sometimes impossible) to go beyond association to causation. To wait for certainty of causation before mentioning them is to deprive parents of useful information. Parents do not want to wait for certainty before being told of the risks of tonsillectomies. If tonsillectomies have been associated with obesity, they want to be told this. They do not want this information to be omitted simply because it hasn’t yet been established that the association reflects causation. If tonsillectomies have been associated with a 50% increase in heart attacks, they want to be told this. They don’t want this information to be omitted because it is uncertain whether it reflects causation. Yet you endorse that omission.

  8. BillyJoe says:

    Harriet,

    “In the early 20th century, doctors did a LOT of tonsillectomies. Whole families would have them done at once – sort of a package deal.”

    Even in the second half of 20th century!
    At the age of two, my brother attended with his first episode of tonsillitis/sore throat and was immediately booked for a tonsillectomy. I was done as a “package deal” – without ever having had a single episode of tonsillitis.

    (Granted we were living in Queensland – a backwoods state of Australia at the time, where kerosene heaters kept us warm, blocks of ice were the only option to keep food cool, milk was delivered into pails left out on the verandah, and toilet pans were still in use.)

  9. Chris says:

    I apparently had a very bad case of tonsillitis. Since my dad was commuting between two states, and then we moved to California to he could go to Vietnam my symptoms were kind of ignored. I mostly remember trying to sleep as a six year old and listening to the ringing in my ears. It may not help that I really don’t complain much about pain, mostly because I am not as sensitive as others.

    The first person to notice was my new first grade teacher in California who noticed I was not reacting to noise. I was so stuffed up that I could not hear very well (like a continual ear infection).

    So I got to go to the old Fort Ord Army Hospital for a tonsillectomy. I mostly remember the ward full of kids, getting presents, the open hallways between buildings, and grumpy Army medical techs who had to get me out of the bed so I could use the restroom. Oh, and how much my throat hurt after the surgery after I was given my favorite drink, grape juice.

    Then we got home, and all was well until I needed to blow my nose, over and over again. Then I looked at the tissue and saw it was red. I don’t remember much else until I woke up in the hospital again. My brother said he had to clean up the blood in the back of the car where I was unconsciousness as my mother drove me from Pacific Grove back to Fort Ord. I’ve been told that my mother’s reaction was epic when the medical tech tried to treat me for just a “nose bleed.” I was kind of disappointed I didn’t get any presents for that stay in the hospital.

    My hearing returned, the school district gave me some speech therapy (back when California had money for schools), and I had to repeat the first grade.

  10. cellculturequeen says:

    I got caught between two doctors at age 14: An ENT who (after a thorough examination) recommended a tonsillectomy, and the family pediatician who (after a brief look with a spatula) advised against it and accused the ENT of money-grabbing.
    I had never had a “classic” tonsillitis in my life, but suffered from chronic respiratory issues since age 7, including sleep apnea and a constant feeling of having lumps in my throat. The pediatrician was on the anti-surgery backlash side, recommending it only for kids who had been bed-ridden with fever and sore throat multiple times over the last months. Chronic but less dramatic cases like mine apparently didn’t count.

    I still remember the ENT reading out the pathology report after surgery: The tonsils were covered in pus, severely scarred and infected with fungus. Instead of supporting my immune system, they had probably been overwhelming it for years.
    Without them I still had the occasional sore throat, of course, but at least I could sleep through the night and talk without constantly clearing my throat.

  11. weing says:

    I’m surprised that Mr. Roberts didn’t invoke the authority of Obama who accused us of doing tonsillectomies so we could make more money.
    http://www.youtube.com/watch?v=ZhNeGYYPgIE

  12. Weing, eehhh? is that what he was saying? Not really what I heard.

  13. David Gorski says:

    A biased view of evidence is shown by the following which you quote approvingly: “However, available scientific evidence does not support claims that larch bark is effective in treating cancer or any other disease in humans. Early laboratory evidence suggested that larch arabinogalactan may stimulate the immune system. However, a more recent study in mice contradicts this finding.” In other words, there is some evidence supporting the value of larch bark (“early laboratory evidence”) and some evidence (“a more recent study in mice”) not supporting the value of larch bark. Given this, to say “available scientific evidence does not support claims . . .” is false. An accurate statement is that some evidence does and some evidence doesn’t.

    No, Seth.

    Note two words Steve used, “in humans.” Steve was quite correct. If there is only a preliminary animal study, even if positive, that does not support the efficacy of larch bark in humans. Ditto preclinical laboratory studies. Your response to Steve, however, demonstrates quite clearly that he was spot-on correct about your biases. Physicians and scientists know that most animal and lab studies don’t pan out in humans. They serve mainly as a screening tool to determine whether a therapy might work in humans. If it works in animals, it does not mean the therapy will work in humans, only that it might. Ditto lab studies. More importantly, animal and studies are not evidence for efficacy in humans, as you imply they are. That’s why we need to do clinical trials. There’s a difference between studies suggesting “value” and studies supporting claims of efficacy. Physicians don’t generally claim efficacy in humans unless there is some good clinical trial evidence to suggest that there is efficacy in humans. You appear to accept Steve’s characterization of the preclinical evidence for larch bark as conflicting. Here’s a hint: Conflicting preclinical evidence can’t be convincingly used as evidence supporting efficacy in humans and to imply that it does is stretching too far.

    In addition, one thing Steve didn’t emphasize in his post nearly as much as I thought he should have: You are relying on naturopaths. In fact, for that reason alone, I was seriously tempted to write a post about your Boing Boing post myself, but Steve beat me to it. In any case, naturopathy is quackery, and we’ve posted abundant evidence for this right here. You can either search for the word “naturopathy” using the search box on this blog or peruse a few examples I’ve picked out:

    http://www.sciencebasedmedicine.org/index.php/naturopathy-and-science/
    http://www.medscape.com/viewarticle/471156
    http://www.ncbi.nlm.nih.gov/pubmed/14745386
    http://www.sciencebasedmedicine.org/index.php/open-letter-to-dr-josephine-briggs/
    http://www.sciencebasedmedicine.org/index.php/naturopaths-and-the-anti-vaccine-movement-if-you-cant-dazzle-em-with-real-science-baffle-em-by-getting-a-law-passed/
    http://www.sciencebasedmedicine.org/index.php/ontario-naturopathic-prescribing-proposal-is-bad-medicine/
    http://www.sciencebasedmedicine.org/index.php/another-state-promotes-the-pseudoscientific-cult-that-is-naturopathic-medicine-part-1/
    http://www.sciencebasedmedicine.org/index.php/another-state-promotes-the-pseudoscientific-cult-that-is-naturopathic-medicine-part-2/
    http://www.sciencebasedmedicine.org/index.php/another-state-promotes-the-pseudoscientific-cult-that-is-naturopathic-medicine-part-3/
    http://www.sciencebasedmedicine.org/index.php/another-state-promotes-the-pseudoscientific-cult-that-is-naturopathic-medicine-part-4/

    That there is some conflicting data regarding vitamin D and sore throats does not make your assertions in your Boing Boing post “evidence-based.” In medicine, “evidence-based” has a fairly specific meaning. It doesn’t mean “based on evidence I cherry picked from PubMed because they agree with my bias.”

    Believe it or not, I’m not entirely in disagreement that too many tonsillectomies are done. Certainly that was true when I was a child, and it’s probably still true now. However, that doesn’t mean tonsillectomies are never indicated or that the procedure wasn’t indicated in the child you described. Moreover, I found it rather insulting that you treated the revelation that the tonsils contain lymphocytes and are part of the immune system as though doctors are “ignoring high school biology.” You put that in the title, presumably to be provocative; so you shouldn’t be surprised that you got a negative reaction from physicians. Moreover, your lecturing on how science didn’t have a good handle on the function of lymphocytes 50 years ago didn’t help. Should we be similarly held responsible for things said about genetics that turned out to be wrong simply because the genetic code wasn’t worked out until around 50 years ago?

    Finally, I can’t help but notice that your discussion of polio and tonsillectomies seems to echo those I’ve found in crank websites that turn up when I Google “tonsillectomy and polio.” For instance:

    http://www.whale.to/vaccine/polio_and_tonsillectomies.html (an all-purpose conspiracy theory website)
    http://www.whale.to/vaccine/polio4.html (an all-purpose conspiracy theory website)
    http://housewifepi.blogspot.com/2010/11/link-between-tonsillectomies-polio.html
    http://insidevaccines.com/wordpress/2010/10/10/polio-causes-and-effects-part-ii/ (an antivaccine website)

    Moreover, in the era of polio vaccine, even if this association held up (and, perusing the evidence myself I wasn’t particularly convinced), it’s nearly 50 years out of date and irrelevant to current practice. Bringing it up serves no purpose other than fear mongering because, even if it were true, given the polio vaccine and how rare polio is in the U.S. now, fear of bulbar polio risk from tonsillectomy would no longer be a reason not to do tonsillectomies.

  14. Seth – thanks for taking the time to write, but your reply is as filled with errors and fallacies as your original post. I won’t argue back and forth about what you said or implied – readers can read your original article and see if my characterization is accurate.

    Let me address a few of your specific points:

    3 – You never mention other benefits, like treating dysphagia or apnea. If you were actually aware of these benefits and omitted them from your discussion then you are not guilty of making poor assumptions (for which I gave you the benefit of the doubt) but of deliberate dishonesty. Saying this was not part of your point is not a justification, given the conclusions of your article.

    6 – The brain analogy is apt, but you draw the wrong conclusion. Neurosurgeons do sometime operate to remove part of the brain. Sometimes the brain is damaged or abnormally formed so that it is not contributing to function but is just causing problems, like seizures or pressure. That is a good analogy to removing chronically infected tonsils.

    7 – The “theory” gambit is all too familiar to us. Some tonsils, when removed, are little more than bags of pus, bacteria, sometimes fungus. The exact net effects this has on immune function may be uncertain, but that these tonsils are unhealthy is pretty clear – one might even call it “common sense.”

    9 – Wrong. We consider all forms of evidence. It is a straw man to say we only admit double-blind placebo controlled trials. But all kinds of studies have strengths and weaknesses. We have written about that issue extensively here (for example:http://www.sciencebasedmedicine.org/index.php/evidence-in-medicine-correlation-and-causation/). Also, it’s not just the absence of evidence because no one has looked – there have been many studies and no net negative effect on immune function has been found.

    10 – I did look at the evidence for vitamin D. It’s mixed, perhaps suggestive of a possible benefit, but at this point “thin”, as I said. Evidence for vitamin C – negative. Multivitamins – negative. So much for the naturopathic approach.

    11 – and- Larch barch – evidence is preliminary and mixed, as David already pointed out. My point remains – while you were decrying the lack of evidence-based practice, you endorsed practices that were blatantly not evidence-based.

    12 – I lilnked to practice guidelines that are very accessible – more so than the Cochrane review, and broader in scope than the review. Cochrane reviews often are focused on a specific clinical question – they are a good source for info, but not one-stop shopping.

    13 – Here is what you are missing. The kind of evidence you are referring to is the typical noise that we see for every clinical intervention. We find correlations all over the place. We would be paralyzed into complete inaction if we took every preliminary thin possible correlation at face value. Most do not pan out when further better evidence is collected. We take risks very seriously, but have to put evidence into context.

    And my main point here was – the evidence for efficacy that you reject is better than the evidence for risks that you accept. You accuse physicians of being biased and acting out of self-interest, when in reality you just don’t understand the nature of medical evidence and how to synthesize it.

    Finally – you make lots of assumptions about what doctors are telling patients, but you do not support those assumptions. The practice guidelines I linked to establish the standard of care – what doctors should be doing, including for informed consent.

    We tell patients about established risks and even possible risks, but not highly speculative or disproven risks. There has been a great deal of discussion in the medical community about where to draw the line of reasonable informed consent. We do not wait (another straw man on your part) for absolute proof of risk.

    Seth – I am willing to assume that you mean well and are trying to be a good skeptic when looking at this surgical procedure. But you have allied yourself with the wrong side – naturopaths and the like – and as David pointed out, they are hopeless cranks and pseudoscientists. Our goal here is to take as thoughtful an approach to evidence and science in medicine as possible. Sorry I had to use your post as an example of bad and biased reasoning in medicine, but you have to be prepared for that when you publish your thoughts publicly. These are serious medical questions, and you have taken a serious responsibility onto yourself by advocating publicly for certain medical decisions. Part of that responsibility is due diligence, and part of answering for your opinions.

  15. David Gorski says:

    Steve,

    I just perused Seth’s blog, and it seems he’s a bit prone to woo, at least in certain areas. For instance, he seems to be too impressed with Jock Doubleday’s “vaccine challenge” that skeptics and scientists have dismantled time and time again:

    http://blog.sethroberts.net/2009/06/10/how-safe-are-vaccines/#comment-314092

    Our very own Harriet Hall discussed this “challenge”:

    http://www.sciencebasedmedicine.org/index.php/the-150000-vaccine-challenge/

    Seth also declares that he is “quite unsure” whether mercury in vaccines causes autism, attacking a straw man version of Paul Offit’s arguments as to why mercury in vaccines doesn’t cause autism and sneeringly asking “Is this the best they can do?”

    http://blog.sethroberts.net/2009/01/13/vaccine-safety-is-this-the-best-they-can-do/

    Finally, Seth likens science to religion and complains that scientists are too “dismissive”:

    http://blog.sethroberts.net/2009/01/14/the-power-law-of-scientific-dismissiveness/

    I fear we won’t get very far.

  16. DugganSC says:

    I’m glad to see Seth Roberts commenting on the review. I sometimes wonder if the doctors we criticize are aware that people are posting negative reviews. I feel like he handled his objections in a firm and reasonable manner, explaining where he’s coming from and where he feels he’s been misrepresented. It’s only when we acknowledge what the other person is saying that we can have a dialogue.

  17. Scott says:

    Physicians and scientists know that most animal and lab studies don’t pan out in humans. They serve mainly as a screening tool to determine whether a therapy might work in humans. If it works in animals, it does not mean the therapy will work in humans, only that it might. Ditto lab studies.

    Something of a tangent, but I would presume that the reverse is also true. There are probably therapies which would work in humans, but don’t in animal models. It seems kind of unfortunate that we don’t have a good way to determine what they are. But since we can’t really justify subjecting people to the risks involved in human trials without good evidence supporting the likelihood that it’ll work, and lab/animal studies are the ways we have to produce that good evidence, I don’t see an alternative.

    But it’s really kind of depressing to think that there probably exist quite a few safe and effective therapies, which nobody’s yet been able to come up with a workable way to find.

  18. Chris says:

    After I hit “Submit Comment” on my tonsillectomy anecdote (short version, I really needed one but I almost bled to death), and rushed off to pick up a kid, it occurred to me that there may be a reason for fewer cases these days.

    My mother was a very heavy smoker. My dad did smoke cigars, but not terribly often (plus he was gone half the time due to being in the Army during the Cold War). I remember a blue haze that was often present in the house. Since I am very allergic to tobacco and smoke, I was probably in a constant state of an allergic immune response. My tonsils, and entire upper respiratory system were being stressed.

    Fortunately I spent a great deal more time outdoors in early elementary school, since there were always hoards of baby boomer kids around (after my dad returned from Vietnam we moved on base to Ft. Ord, now during some Mythbusters episodes I can reminisce about where I ran around in 2nd and 3rd grade, though the part I lived in is still in use as part of the Navy Language School, which is how my school, Stillwell Elementary is being used).

  19. teeps29 says:

    Seth Roberts apparently doesn’t remember what he writes from one paragraph to the next. First he says “I omitted some of the evidence for benefit because that was not the main point of my post,” then 10 paragraphs later, it’s “Omission of evidence is bad.” Hoist by his own petard, I’d say. (I just like saying that.)

  20. Seth Roberts says:

    I’d like to respond to your various points.

    1. I say that a study in rats is evidence supporting the use of larch bark with humans. DG (David Gorski) disagrees: “If there is only a preliminary animal study, even if positive, that does not support the efficacy of larch bark in humans.

    I assume that “support the efficacy of X” means “make more plausible the efficacy of X” (not “ensure the efficacy of X” — of course not). What DG says would be true if there were no correlation between the effect of a treatment on rats and the effect of the same treatment on humans. In other words, if learning what Treatment X does to rats tells us nothing about what Treatment X does to humans. This is false, of course. The rat/human correlation is why rats are used as screening tests. Rats and humans share a lot of DNA; the correlation is no surprise.

    2. DG: “You are relying on naturopaths.” Huh? The woman I write about (Rachael) saw a naturopath who made various recommendations. Rachael then examined the evidence for and against those recommendations. That is not blind trust (“relying”).

    3. DG: I “cherry-picked” evidence. It is Rachael, not I, who examined the evidence for and against Vitamin D, which was one of the naturopath’s recommendations. How you know that she “cherry-picked” that evidence is beyond me.

    4. DG: The link between polio and tonsillectomy is “irrelevant to current practice.” That would be true if no viruses like the polio virus still exist. We don’t know that. We don’t know with certainty why polio and tonsillectomies were repeatedly associated. Until we know why, it remains plausible that tonsils protect against polio. If they protect against polio, it is plausible that they protect against other viruses — because of similarities between viruses. The polio/tonsillectomy association may be telling us something that can help us understand what tonsils do. What tonsils do is relevant to current practice.

    5. Steven Novella (SN) complains that I “never mention other benefits” of tonsillectomies. My article had two main points: 1. By looking at evidence, a woman came to a different conclusion than her doctors. 2. A Cochrane Review about tonsillectomies left out a mountain of anti-tonsilletomy evidence. My post was not meant as an overall evaluation of tonsillectomies.

    6. SN: “The brain analogy is apt, but you draw the wrong conclusion.” Actually, I think we agree here. As I say in the post, in extreme cases removal of part of one’s brain is reasonable. I meant that in extreme cases, tonsillectomies could be reasonable, too.

    7. SN: “We consider all forms of evidence.” I stand corrected. You just don’t want outsiders, such as parents, to consider them? One of the main points of my post, as I said, was that the Cochrane Review (aimed partly at outsiders) omitted a mountain of negative evidence.

    8. SN: “There have been many studies [of the effect of tonsillectomies] and no net negative effect on immune function has been found.” This ignores two findings that may be due to a “negative effect on immune function.” One is the association with polio. The other is the association with heart attacks. Until you can explain with certainty why these two associations were observed — and no one can — I think you should be less confident of your “no negative effect” conclusion.

    10. SN: The evidence for Vitamin D is “mixed . . . So much for the naturopathic approach.” Huh? First, naturopaths vary widely — I wouldn’t draw such broad conclusions from one case. Second, the evidence for many mainstream medical treatments is mixed. This does not lead me to conclude “So much for mainstream medicine.”

    11: SN: “You endorsed practices that were blatantly not evidence-based.” I endorsed a woman (Rachael) searching for evidence rather than accepting blindly what she was told. She did this twice: 1. With what she was told by a conventional doctor. 2. With what she was told by a naturopath. Let me repeat: I endorse examination of evidence, as opposed to blind trust.

    12. SN: “The kind of evidence you are referring to is the typical noise that we see for every clinical intervention.” I disagree. The polio/tonsillectomy association is not “typical noise.” At least twenty different studies found that association. The evidence showing what lymphocytes do is also not “typical noise” — it isn’t noise at all.

    14: SN: “The evidence for efficacy that you reject is better than the evidence for risks that you accept.” That “evidence for efficacy that [I] reject” is what? You’re sure that pro-tonsillectomy evidence I failed to mention is better than the research what showing lymphocytes do? I’d love to know what that evidence is.

    15. SN: “The practice guidelines I linked to establish the standard of care – what doctors should be doing, including for informed consent.” I found no statement in those practice guidelines about what parents should be told about risks.

    16. SN: “You have allied yourself with the wrong side – naturopaths and the like – and as David pointed out, they are hopeless cranks and pseudoscientists.” This is wrong. To say it again: I support people like Rachael, who was equally skeptical of naturopaths and regular doctors. That’s my position: skepticism (which does not mean rejection) of all expert advice. I endorse always looking at the evidence yourself, no matter what any supposed expert says.

    Thanks to both of you (DG and SN) for considering the issues I raised and allowing me to defend my views.

  21. Seth – Our position with regard to informing oneself regarding healthcare issues is more nuanced, and was one of the points I was making in my post. It is fine to search for information yourself, and no one here is advocating “blind trust” in anyone. We are all activist skeptics.

    But it is folly to substitute one’s own opinion for that of experts who have spent years mastering a subject. You and Rachel inadvertantly provide ample evidence of the pitfalls of this approach. You are clearly trying to understand the evidence, but continue to make basic errors and lack perspective.

    I want to reply further to one point, number 6, you seem to agree to my interpretation of the brain analogy. But you don’t acknowledge that his is the whole point. If it is indicated to take out tonsil’s in “extreme” cases than all we need to define is what constitutes an “extreme” case. I linked to guidelines that spell it out operationally – specific criteria, that you have not been able to dispute and that stands in contrast to your characterization of mainstream medical practice.

    If we define “extreme” cases as meeting published guidelines, then you are essentially agreeing with the current standard of care on tonsillectomy, which certainly is not the bottom line impression left by your original article.

    Related to this is the point that one of the criteria for removing tonsils, at least theoretically, is that they are so adversely affected that they are no longer providing a useful immune function. If true, then all of your talk about the immune consequences of tonsillectomy do not apply. This includes all the historical data about polio, taken from 50 years ago when the threshold for doing tonsillectomy was very different from today. Also – the studies do not establish whether or not tonsillectomy causes susceptibility to any condition, or if severe chronic tonsillitis leads to both tonsillectomy and susceptibility to other diseases.

    I have no problem with questioning the practice of tonsillectomy. My major criticism of your article was the straw man you drew of mainstream medical thinking and practice, and the different treatment you gave to mainstream practices vs those recommended by a naturopath.

    My main problem with your points in the comments is your treatment of medical evidence. It simply does not make sense and is not in accord with how evidence is used in medicine. For example, you seem to be arguing that because the correlation between rats and humans is not zero that we can base clinical decisions on rat data. This makes no sense. The correlation is not zero, but it is pretty weak. It is just enough to use as a screen for later research, but not useful in making clinical decisions. Most treatments that look promising at the animal data level do not work in humans. This is therefore not evidence that can be legitimately used to support a treatment in humans. It’s bad medicine and bad science.

  22. BillyJoe says:

    I don’t think you are going to convince him.

    As a result of what I have learned from these science-based medicine blogs and other science and medicine websites and discussions over the past 5-10 years, I understand what is wrong with Seth Roberts’ article and I understand Steven Novella’s criticism of it. I don’t think Seth Roberts, in his present state of knowledge, is capable of understanding his error and I wonder if he is prepared and inclined to put in the effort to understand it.
    Without the necessary background knowledge would he be prepared to offer advice to a pilot with engine trouble?

    The whole point is that, when it comes to specialised areas of interest, we mostly have no choice but to rely on the consensus of people who spend their whole working lives gaining expertise in these specialised areas of interest.

  23. JPZ says:

    Argh! I can’t believe I am speaking up considering how much I have been criticizing Seth’s same (and a previous) post on Boing Boing (Nutrition Industry, Seth). Seth is a conspiracy theorist plain and simple.

    But (argh again):

    Anything less than studies worthy of a clinical practice guideline can (and perhaps will) be dismissed on SBM if it is on their poorly-defined “black list” of “stuff we don’t like.” The goal posts move from “early evidence is great” for some topics to “I wouldn’t recommend that to a patient; therefore, there is no actual evidence” when they don’t like the topic (evidence presented here previously). But, n.b., an unlicenced and perhaps untrained person should not recommend an herbal product for pediatric use based solely on animal study data. That just isn’t wise.

    That said, pretty much everything else David and Steven said was spot on. I am surprised that they were so patient with Seth considering the other contents of his blog. Perhaps that is a good sign for SBM.

  24. Harriet Hall says:

    @JPZ
    “Anything less than studies worthy of a clinical practice guideline can (and perhaps will) be dismissed on SBM if it is on their poorly-defined “black list” of “stuff we don’t like.” The goal posts move from “early evidence is great” for some topics to “I wouldn’t recommend that to a patient; therefore, there is no actual evidence” when they don’t like the topic ”

    Are we talking about the same SBM blog? The one I’ve been reading and writing for has never dismissed anything because we “don’t like” it. And the sequence is “There is no actual evidence; therefore, I wouldn’t recommend that to a patient.”

  25. kathy says:

    Seth said: “The goal posts move from “early evidence is great” for some topics to “I wouldn’t recommend that to a patient; therefore, there is no actual evidence” when they don’t like the topic ”

    Does Seth understand how complex and multi-level full-scale clinical trials are? Before a doctor gets to the point of recommending a treatment or drug unreservedly (aot saying: “It might be helpful”), it has jumped through a whole variety of hoops. In vitro (cell culture) and animal-based studies are not even called “clinical trials”, they are “pre-clinical”. To say anything other than “these results are promising” or “early evidence is great”, is unethical.

    It’s not that anyone has moved the goalposts, but just that running halfway to the goal line doesn’t score any points. It’s promising, yeh, it will excite all the spectators, maybe even the coach, but the move must be completed to add to a team’s score.

    Seems to be a common problem out there, especially as newspapers/websites/TV shows report long before a trial is finished … maybe they want to be the first to cover a possible medical breakthrough? … don’t want to wait. They don’t realize, or don’t mention to their audience, that this is just a start … maybe a promising start, but no goal is scored until it’s scored.

  26. Seth Roberts says:

    Uh, Kathy, you are quoting JPZ, not me.

    SN: “It is fine to search for information yourself, and no one here is advocating “blind trust” in anyone. We are all activist skeptics. But it is folly to substitute one’s own opinion for that of experts who have spent years mastering a subject.” I see. No matter what evidence the patient (i.e., Rachael) finds in her search, the expert is always right?

    SN: “It is folly to substitute one’s own opinion for that of experts who have spent years mastering a subject. You and Rachael inadvertantly provide ample evidence of the pitfalls of this approach.” After Rachael questioned what her ear nose and throat doctor said, and took another approach, her son had no more sore throats. How this “provide[s] ample evidence of the pitfalls of [her] approach” escapes me.

    SN: “You seem to be arguing that because the correlation between rats and humans is not zero that we can base clinical decisions on rat data. This makes no sense. The correlation is not zero, but it is pretty weak. It is just enough to use as a screen for later research, but not useful in making clinical decisions.” So if Drug X causes cancer in rats that is “not useful in making clinical decisions” (such as whether Drug X should be prescribed)? And if rat research leads to an understanding of what tonsils do (as actually happened), that too is irrelevant to clinical decisions (about tonsillectomies)?

  27. David Gorski says:

    SN: “It is folly to substitute one’s own opinion for that of experts who have spent years mastering a subject. You and Rachael inadvertantly provide ample evidence of the pitfalls of this approach.” After Rachael questioned what her ear nose and throat doctor said, and took another approach, her son had no more sore throats. How this “provide[s] ample evidence of the pitfalls of [her] approach” escapes me.

    And that, sir, is a perfect demonstration that Steve is correct and you do not understand clinical evidence. Single anecdotes are not evidence of efficacy. Too many confounders can make correlation seem to be causation, such as confirmation bias, regression to the mean, confusing the natural history of a problem with a treatment effect. At most, anecdotes can be indicators that research into a therapy might be worthwhile. At worst, they are extremely misleading, because we are pattern-seeking animals and, as such, very quick to impute causation to correlation. That you either don’t understand or don’t accept Steve’s explanation (take your pick) only reinforces his very point: The folly of substituting your non-expert understanding for deeper knowledge of a subject, which is what you did and continue to do.

    You’re attacking a straw man, anyway. Steve wasn’t just referring to your anecdote. He appeared to have been referring to the totality of your post, including your many mistakes, your obvious biases, and your clear misunderstanding of clinical evidence, all topped off with your arrogant lecturing of physicians as though they don’t understand basic biology and your trying to imply that because physicians’ poor understanding of the function of lymphocytes 50 years ago implies anything about our understanding of lymphocyte function now. In any case, Steve also gave multiple examples from your post and from your responses.

  28. Seth wrote: “So if Drug X causes cancer in rats that is “not useful in making clinical decisions” (such as whether Drug X should be prescribed)? And if rat research leads to an understanding of what tonsils do (as actually happened), that too is irrelevant to clinical decisions (about tonsillectomies)?”

    This is yet another example of not understanding the nature of medical evidence. You are confusing three types of uses of animal data – screening for potential risks, indications of possible efficacy, and elucidating basic mechanisms and biology. You are jumping among these three very different uses of evidence as if they were equivalent.

    What I am saying is that animals data is a very poor indicator of efficacy in humans. At best in might point the way for later research.

    Animal data for risks may also be poor, but to err on the side of caution we require that new drugs, for example, not show any significant toxicity in animal models prior to exposing humans to new agents.

    Animal data is perhaps best used for looking at and understanding basic biology, and this of course can be used to generate hypotheses (but not conclusions) about clinical effects.

    David already pointed out your reliance on anecdote – which we have written about here extensively as well.

  29. WilliamLawrenceUtridge says:

    So if Drug X causes cancer in rats that is “not useful in making clinical decisions” (such as whether Drug X should be prescribed)? And if rat research leads to an understanding of what tonsils do (as actually happened), that too is irrelevant to clinical decisions (about tonsillectomies)?

    Um…if Drug X causes cancer in rats it’s still in the preclinical stage and would probably be discarded. Nobody would prescribe a drug that is being tested in rats because it’s not a prescription medication yet.

    Results in animals is tested with research on humans. The present approval process in the US, UK and Canada at least would preclude clinical use in humans based on animal studies. At best, you would move on to studies in humans to see if it is safe, effective and ultimately has a favourable risk-to-benefit ratio. Put another way – if “Drug X” causes cancer in rats, it’s probably not a drug, it’s probably a potential drug. Results in rats are not useful for clinical decisions because the drug is not at a clinical testing stage. In fact, based on causing cancer in rats, the drug in question probably wouldn’t make it to humans.

    The words “in rats” should be an article-killer for newspapers and the rest of the media because anything that happens “in rats” is many years, millions of dollars and considerable amounts of research away from being considered relevant to humans. Anytime you see “in rats”, immediately downgrade the story from “interesting” to “preliminary at best, misrepresentation at worst”.

    Seth, you should read up on the actual process of drug investigation and clinical trials. If you think that “in rats” research is relevant to humans, you probably shouldn’t be giving advice. The fact that naturopaths and other nutters are willing to make recommendations based on “in rats” research is one of the reason they are held in such contempt on this blog. Doctors don’t give drugs out based on “in rats” studies, and there has been a steady accumulation of restrictions and guidelines over the past couple centuries governing what is necessary before a drug becomes prescribable. It’s very restrictive and places heavy emphasis on the precautionary principle. Unlike, say, naturopathy.

  30. JPZ says:

    @HH

    As I have said many times, I have found no issues with your posts on SBM that I have seen. In fact, upon hearing a recent talk of yours on YouTube (not quite sure which it was, but you said “They didn’t hear it because they weren’t expecting to hear it.”), I have quite a lot more admiration for your POV.

    You have wisely chosen not to speak on behalf of your colleagues who have chosen not to respect science on topics they do not like. I have seen it in criticism of chiropractery that jhawk and others have done an incredibly poor job of defending (and I could give a good stinking-damn about the topic – since they have repeatedly insulted me). I hear SBM proponents fixated on “subluxations” while bumbling, newer chiropracters like jhawk seem to be saying, “Yeah, it is still there, but we really don’t see that as the future of chiropractery.” I am not the expert like many others here, but I do know good science. So when I am directed to a Canadian Chiropractic College website that includes (among the woo): 1) studies to establish a better sham control for a neck procedure, 2) application of chiropractic methods to design a better seat cushion for long distance truckers, and 3) improving imaging techniques to identify anatomical characteristics that (may or may not) correspond to chiropractic issues and their resolution. Sounds like a drop of actual science in a pool full of woo. But, SBM can’t respect that. Just repeat subluxation, subluxation, subluxation until the annoyance goes away (i.e. HEAD-ON, apply directly to the forehead, repeat).

    But in my own field of expertise (nutrition), I have seen a shocking disregard for science – especially on the part of Scott Gavura and occasionally on Steven Novella’s part. I have laid the major points out in detail before in earlier posts, but there was never a reply involving science. I will reprise only one point. Scott dismissed the efficacy of probiotics based on one “systematic” review. I pointed out that the review was fatally flawed because it compared apples and oranges, i.e. you cannot conflate the results of one family, genus or species of probiotic bacteria with another. I provided links to EU regulations that you can’t conflate the results of unrelated bacteria to another as well as authoritative guidelines from the scientific association that studies probiotics. His answer was that the authors of the review called it a systematic review – so they must be right, i.e. if I don’t like it, I won’t hear evidence about it. This is one of many examples I cited in my earlier condemnation of skeptic bias on SBM.

    I am glad that you maintain a positivist attitude about the mission of SBM. Perhaps you can influence others.

    @kathy

    Everything you said supports EBM. If actual science is meaningless to you until it reaches the level of a practice guideline, then you can take comfort in the current system and wait for your professional organization to tell you what to think. It seems that SBM approves.

    1. Harriet Hall says:

      @JPZ,
      I see no need to “influence others” about the mission of SBM. I stand behind every author on this blog. The accusation that any of our authors have shown a “shocking disregard for science” is ludicrous. I think it’s more likely that you are viewing their writings through the distorted spectacles of your own beliefs.

  31. JPZ says:

    @WLU

    Really? Your discussion centers on toxicology, not drug efficacy. Seth is amazingly misinformed and perhaps self-delusional. But, if a toxicology study “in rats” finds evidence of harm, then the human equivalent dose is calculated (based on body surface area) and regulatory agencies can and often do proscribe a maximum acceptable dose for humans. You seem to be proposing that if the rat study finds evidence of harm, we should test it on humans just to make sure. Really?

  32. Seth Roberts says:

    David Gorski wrote: “And that, sir, is a perfect demonstration that Steve is correct and you [Seth] do not understand clinical evidence. Single anecdotes are not evidence of efficacy.”

    I said nothing about “evidence of efficacy”. I said Rachael’s story WASN’T evidence (of something else). Here’s what I wrote: “After Rachael questioned what her ear nose and throat doctor said, and took another approach, her son had no more sore throats. How this “provide[s] ample evidence of the pitfalls of [her] approach” [as Steve Novella said] escapes me.”

    Here you seem to have misunderstood me. Another point in my last comment seems to have been ignored. Here’s what I wrote:

    “SN: “It is fine to search for information yourself, and no one here is advocating “blind trust” in anyone. We are all activist skeptics. But it is folly to substitute one’s own opinion for that of experts who have spent years mastering a subject.” I see. No matter what evidence the patient (i.e., Rachael) finds in her search, the expert is always right? ”

    Let me elaborate. I am saying experts (in this case, doctors) are sometimes wrong. Non-experts, such as Rachael, can sometimes figure this out by searching for evidence themselves. The Internet makes this much easier, which is why I believe stories like Rachael’s have value. What she did, others can do. Where she went, others can follow. This is the fundamental message of my piece: “Don’t be passive. You (non-expert) can help yourself.” In my opinion, it is wise, not “folly”, to search for evidence oneself, as Rachael did. If the evidence you find contradicts expert advice, go with the evidence.

  33. BillyJoe says:

    …the point is that you need to know how to evaluate the evidence.
    That requires lots expertise on top of lots of background knowledge.
    Unfortunately, the university of google does not provide that expertise and knowledge.

  34. weing says:

    “How this “provide[s] ample evidence of the pitfalls of [her] approach” [as Steve Novella said] escapes me.”

    It escapes you because you do not understand the law of small numbers.

  35. David Gorski says:

    No, Seth. I did not misunderstand you at all. You presented Rachel’s anecdote as a counterpoint to Steve’s assertion of the folly of non-experts trying to interpret the medical literature. In fact, when you say, “if the evidence you find contradicts expert advice, go with the evidence,” you reinforce Steve’s point about that very folly even more. After all, Steve and I (and others) have just expended considerable effort pointing out exactly how non-experts can misinterpret the evidence, as Rachel and you both did. Steve’s point, in fact, is that it’s very, very difficult for a non-expert to tell what the evidence actually shows, particularly through the prism of a bias that led her to consult with quacks. (Yes, naturopathy is quackery.) It’s the University of Google effect that led Jenny McCarthy to become an antivaccinationist.

    This does not mean that we are telling patients not to bother or to passively do what the experts tell them. If we thought that, then none of us would bother to write this blog. In fact, we encourage people to seek out information, but in a form that they can digest. However, there is such a huge amount of background information that’s necessary to know to begin to interpret the scientific literature on a topic that it’s unlikely that a lay person without background in the relevant science will find what the evidence means on her own—or even understand most of the evidence. That’s why it’s necessary to find lay-accessible summaries of the current literature.

    Think of it this way. I have two advanced degrees (an MD and a PhD), but it would never cross my mind to try to figure out, for example, the current state of string theory by delving into the peer-reviewed physics literature. I don’t have the background knowledge, and I haven’t studied advanced calculus in nearly 30 years.; without such knowledge and understanding, trying to figure out advanced physics from the physics literature would be folly. So, unless I want to spend several years learning the background knowledge and re-learning long-forgotten calculus and differential equations, I am forced to rely on interpreters. So why is medicine considered different? Why do people without a background in the relevant sciences think that they can interpret peer-reviewed medical literature? Why do they think that their analyses are any better than my analysis of advanced physics based on the peer-reviewed literature in physics would be?

    We are not saying that science is never wrong or that experts are never wrong. However, we are saying that if we weigh two likelihoods (i.e., that of an expert being right versus that of a non-expert without the requisite background knowledge being right), the odds are much more favorable that the expert is probably correct. Your analysis and that of Rachel fell right into line with those probabilities, too. The difference between you (and Rachel) compared to me is that I recognize that I’m just as prone to the Dunning-Kruger effect as any other human being looking at an area of knowledge in which I am not an expert and try to take steps to make sure that I’m not tripped up the way you and Rachel were.

  36. JPZ says:

    @HH

    Allow me to recap the one of several unscientific conclusions that lead me to believe in SBM (skeptic) bias. It may be in several parts to allow the use of sufficient references. If it does not go though, I would hope you could prevent it from being suppressed:

    In this SBM discussion:

    http://www.sciencebasedmedicine.org/index.php/constipation-myths-and-facts/

    Scott Gavura’s comment on probiotics in constipation was:

    “Probiotics, covered in depth by Mark Crislip already, are live microorganisms administered with intent of a therapeutic effect. If you like yogurt that does double duty, you’ll probably see brands that include Bifidobacterium and Lactobacillus, and, depending on your national regulator, there may be vague health claims about intestinal “wellness” on the label. The idea of probiotics for constipation is at least plausible, as probiotics have the potential to disrupt the colon’s bacteria ecosystem – if only to a very limited extent, as Mark noted in his post. For constipation, their effectiveness hasn’t been demonstrated though. A systematic review published in 2010 examined the data supporting their use in adults and children. Five high quality trials were identified and the results were unimpressive:

    Data published to date suggest that adults with constipation might benefit from ingestion of B. lactis DN-173 010, L. casei Shirota, and E. coli Nissle 1917, which were shown to increase defecation frequency and improve stool consistency. However, in some cases, even if there was a significant difference in results, their clinical relevance is unclear.

    Their conclusion:

    Until more data are available, we believe the use of probiotics for the treatment of constipation condition should be considered investigational.

    Overall, not encouraging. And little reason to recommend their use. That’s the opinion of some regulators, too. The European Food Safety Authority has largely rejected general health claims for probiotics.”

    To which I replied:

    “The Chmielewska and Szajewska (2010) review you cited reviewed five different probiotic strains. If one strain of probiotic is efficacious in one indication, it does not mean that any other probiotic will be effective as well (http://www.worldgastroenterology.org/assets/downloads/en/pdf/guidelines/19_probiotics_prebiotics.pdf). Probiotic strains differ by mechanism, gut survival, replication in the gut, ecological niche and elimination. This was not a “systematic review” it was a review of five independent studies. Four out of the five studies showed improvement mostly based on NIDDK criteria of “Constipation is defined as having a bowel movement fewer than three times per week.” The one study that showed no efficacy was a children’s study using Lactobacillus GG as an adjunct to lactulose (i.e. a different study question that did not test the direct effects of probiotics). Moreover, there were no adverse effects reported among the five trials.

    “The European Food Safety Authority has largely rejected general health claims for probiotics.”

    Not so fast. EFSA rejected 170 of 180 submitted claims for insufficient information. But, the EFSA process does not define what constitutes sufficient information and communication between the review panel and the petitioner is not allowed during the review process. Only after the claim is rejected do you find out what was needed – and then the panel may change to different experts with different biases when you resubmit. The International Scientific Association for Probiotics and Prebiotics submitted their reaction as well (http://www.isapp.net/docs/ISAPP_responds_to_EFSA_oct09.pdf). This is the professional organization for scientists in this field – the industry one is Internation Probiotics Association.”

    I forgot to say that it is like saying cows and elephants can’t do the same thing so that proves whatever theory you have about mammals is wrong. So, Scott’s review paper is fatally flawed based on the references provided, i.e. written by the top probiotics professional organization. His opinion about the EU process is wrong based on the reference provided. His dismissal based on lack of clinically-significant outcomes is false based on NIDDK criteria for constipation (for some of the studies).

    Um, yay SBM for “seeing though” the evidence from government labs, professional organizations, and, well, oddly enough, facts. Remarkable insight for… well, true believers.

    HH, I am a bit surprised that you would defend anything your colleagues would say without qualifications. Can we focus on evidence rather than bias? Or do you think there is no such thing as skeptic bias?

  37. JPZ – It’s OK to disagree with someone’s analysis of the evidence without calling them unskeptical, biased, and “true believers.” I think what Harriet is objecting to categorically is your over-the-top characterization of the opinions of SBM authors you have disagreed with.

    For example, you are faulting Scott for agreeing with a review of the evidence that concluded the 5 quality studies of different probiotic strains in constipation is “plausible” but “unimpressive” and therefore deserves more research. This is a “true believer” position?

    It is certainly worth pointing out that the different studies looked at different strains, but this would actually count in favor of Scott’s conclusion – because then we only have one good study for any particular strain. One study, especially with modest outcomes, is hardly impressive. It is fair to consider this preliminary at best, and not sufficient to justify marketing claims.

    The burden of proof lies with those making marketing claims for probiotic products that they have the benefits they claim. I think it’s reasonable to conclude that they largely have not done this. You have said nothing to contradict it.

    Similarly, you have made some extreme claims of bias and being unskeptical, and you have not made your case. If anything you have shown the bias to lie with you.

  38. Harriet Hall says:

    @JPZ,
    “HH, I am a bit surprised that you would defend anything your colleagues would say without qualifications. Can we focus on evidence rather than bias?”

    Please don’t put words in my mouth. I didn’t say I would defend anything they would say without qualifications. I said “I stand behind every author on this blog.”

    I think the only “bias” demonstrated on SBM is a bias in favor of rigorous science and reason. Every author on this blog is on the same wavelength about science-based medicine and I respect their approach and their integrity. That doesn’t mean they never say anything that other science-based thinkers might legitimately disagree with.

  39. JPZ says:

    @Steven Novella

    Perhaps we could review the evidence for this one-of-many (IMHO) points supporting skeptic bias here (and this pains me deeply because I consider myself a skeptic as well, but I don’t accept the closed mindedness of the philosophy here).

    Based on my post, you seem to assume that there are only five studies of probiotics to evaluate; therefore, the “systematic” review Scott chose (apparently with an Ouiji board) must be right. Again, you assume that any field of study that you automatically dismiss could not contain more studies than those in the one article that supports Scott’s POV (many, many more articles don’t support his POV, but I can only post two at a time – it is a great way for SBM to quiet dissent). You assume then that there is only one article that supports each Genus/Family of organism if I (well, and the EU and the top professional organization in this field, you know) say the studies cannot be combined for a “systematic” analysis. Even if that were true (which hilariously it isn’t), 3 out of 4 efficacious studies fulfilled NIDDK criteria for efficacy, and the one study showing no efficacy did not pose a valid hypothesis to be including in the “systematic” review (unimpressive?? to whom? Oh yeah, this was an amazingly insightful unscientific review that SBM-bias loves). I have to question why two researchers such as us are having this conversation based on the evidence. I mean seriously, Steve.

    Putting five studies together with some word in common does not make them “systematic” nor coherent. Your trust in dismissal of fields of studies you don’t like shows great loyalty, but simply isn’t true.

    I am afraid I have not heard an opinion from anyone on “S”BM (other than the one I hear now from you) calling for more research – only dismissal.

    “Bias lies within me?” ORLY? Please define such. Scott’s total dismissal of probiotics for constipation provided exactly two references – 1) a review that I have repeatedly provided authoritative references for how it is fatally flawed and 2) a ridiculous inference that certain EU rulings reflect their support of his claims (which I provided a link to disprove). I dismissed both of his points of support with science/evidence/etc., and it still seems that you and others pooh pooh contrary evidence you don’t like (hmm, sounds like SBM-bias). Do I need a note from some deity to break through this SBM bias? (LOL, yeah, I think we all get that funny one).

    I don’t get it. POV – Evidence = Dismissal if you folk don’t like the particular topic. Why do I need to keep saying the same thing over and over if you are ACTUALLY looking at the evidence and weighing the science? Mostly, I have found this SBM bias to be a tragic misinterpretation of the Baysean prior (to confirm the suspicions – I done minoritied in dat statistical stuff in m’ own doctorialating), but probiotics tend to have a bit more evidence considering the pharmaceutical and major food companies funding those studies.

    Are you seriously going to brush a science-based criticism off like some alchemist’s ravings? Does it sooth your conscience to laugh off something your bias doesn’t understand?

    @HH

    Please allow me to challenge these self-agrandizing concepts of SBM “science” and “reason” before you pass judgement. Also, I tend to inject humor when explaining difficult concepts. Please consider that the humor is for their myopic benefit, not yours.

    @Others

    I imagine the wingnuts are flying around looking for vulnerabilities to exploit about SBM (based on a couple of emails I received). To them I say, kids, this is how the big boys and girls talk when you are not around. You aren’t getting any hate of skeptics from me.

  40. JPZ says:

    Wow, it posted!

  41. Quill says:

    @JPZ: Bias is a tricky thing indeed. Since the bloggers here post their affiliations and COIs, I must ask: Are you in the business of marketing or selling nutritional products?

  42. JPZ says:

    @Quill

    I have been very open that I work in the nutritional products industry, I am currently a consultant for a number of companies. As an industry insider, I have posted on this blog several times about how this industry works.

  43. Quill says:

    @JPZ

    Thank you for that. I’m still a new reader here so I will have to go explore more in the archives. (As always, thanks to the doctors and insiders that post here. I appreciate the time y’all take to make sense of things for non-medical people.)

  44. Seth Roberts says:

    “Steve and I (and others) have just expended considerable effort pointing out exactly how non-experts can misinterpret the evidence, as Rachel and you both did. Steve’s point, in fact, is that it’s very, very difficult for a non-expert to tell what the evidence actually shows, particularly through the prism of a bias that led her to consult with quacks. (Yes, naturopathy is quackery.)”

    Strange but true: I have yet to grasp how I or Rachael have misinterpreted “the evidence”. Whereas you have written:

    “Note two words Steve used, “in humans.” Steve was quite correct. If there is only a preliminary animal study, even if positive, that does not support the efficacy of larch bark in humans.”

    Actually, if Statement X is true in mice, it makes Statement X more likely to be true in humans. See any physiology text, for example.

    As for your belief that “naturopathy is quackery,” I can only say that my limited experience contradicts this. Consultation with a naturopath led Rachael to a treatment (Vitamin D) for which there is some evidence. Likewise, I wrote for Boing Boing about a woman with terrible migraines who benefited more from the one naturopath she saw than all of the many doctors she saw:

    http://boingboing.net/2011/07/25/finding-the-source-of-migraines-and-fifty-useless-migraine-drugs.html

    Sure, naturopaths must often prescribe (a) treatments that don’t work and (b) treatments for which there is no evidence. But conventional doctors also do both of these quite often.

  45. Chris says:

    My anecdote has been completely ignored due to a drama emperor. Sniff sniff. Never mind, carry on. ;-)

  46. kenny5277 says:

    “Sure, naturopaths must often prescribe (a) treatments that don’t work and (b) treatments for which there is no evidence. But conventional doctors also do both of these quite often.”

    I think it’s time for an episode of “Name that logical fallacy”.

    Seth, Steve’s first response to you in the comments section gave you the benefit of the doubt regarding your intentions and motivations here, and with that the opportunity to learn something from people more expert than yourself. It’s a shame that you decided to double down and dig yourself in deeper.

  47. weing says:

    Seth,
    You should learn about the law of small numbers as it applies to sample size.

  48. WilliamLawrenceUtridge says:

    Actually, if Statement X is true in mice, it makes Statement X more likely to be true in humans. See any physiology text, for example.

    Yes, which is why we test on animals, but “more likely” does not mean “absolutely is true and we should base treatment decisions on rodent tests”. Testing in mice is one step in demonstrating safety and efficacy in humans. We still have to test, y’know, in humans.

    Rats can eat garbage with relative impunity. Should we accordingly re-stream our nutritional guidelines on that basis?

    As for your belief that “naturopathy is quackery,” I can only say that my limited experience contradicts this. Consultation with a naturopath led Rachael to a treatment (Vitamin D) for which there is some evidence.

    You do know what an anecdote is, right? And the difference between an anecdote and science? And why science is superior? And that vitamin D is currently being actively investigated to determine if current dietary requirements are too low? And that the naturopath’s recommendation, if based on science, is completely redundant to what a real doctor would recommend? And that many of the “explanations” given by naturopaths are mutually contradictory? And that the test they use like hair analysis aren’t actually validated for the purpose they are used for?

    Sure, naturopaths must often prescribe (a) treatments that don’t work and (b) treatments for which there is no evidence. But conventional doctors also do both of these quite often.

    First, [citation needed] for your last sentence. This has been addressed before:

    http://theness.com/neurologicablog/index.php/how-much-modern-medicine-is-evidence-based/

    http://scienceblogs.com/insolence/2007/11/how_much_of_modern_medicine_is_evidenceb.php

    Unless you’re getting that particular CAM talking point from a newer survey, that bit of data is from 1961 (51 years ago if you’re counting) from a sample of country practitioners in northern England.

    To address your first sentence – you’re actually saying that naturopaths prescribe treatments that don’t work, and that’s OK????? Because “doctors do it”? So if Bill Clinton cheated on his wife, that makes it perfectly morally acceptable for me to do it? As long as someone else in the world has done something stupid or wrong, that means it’s perfectly fine for everyone else to do so? Great, I need some money for lunch – I’ll go beat up a teenager and take theirs. And if you’re talking about evidence-based actions, that’s still stupid – in South Africa for a long time it was widely advertised that HIV didn’t cause AIDS and antiretrovirals should be abandoned. Well, I guess that means the FDA should withdraw aproval for AZT!

    Look up the logical fallacy tu quoque and see if it applies here. Individual treatments stand or fall on their evidence base. If a treatment is found ineffective, doctors abandon it. Naturopaths do not. And if it is found effective, it doesn’t matter by who, doctors adopt it – and if it fits their worldview, CAM practitioners will co-opt it. Like they did with diet and exercise – pretending doctor’s don’t counsel patients to lose weight and eat some vegetables.

  49. Chris says:

    WLU:

    Rats can eat garbage with relative impunity. Should we accordingly re-stream our nutritional guidelines on that basis?

    You mean Ratatouille was not a documentary?

    Seriously, I lost count of the times that a study on rats and mice have been posted claiming “X causes Y! Run for the hills! Stop your evil ways! We are doomed!”

  50. JPZ says:

    Let me set aside my dumbfounded and aghast disbelief that members of the SBM editorial staff would give credence to unsupported argumentation by its own editorialists. Much like the old question for woo believers goes, “What level of evidence would it take to change your mind?”

    There are only two pieces of evidence to support Scott’s opinion on probiotics and constipation: 1) a review by Chmielewska and Szajewska and 2) Scott’s interpretation of recent ESFA regulatory actions regarding probiotics and health claims.

    The review compares five different probiotic organisms for their efficacy on constipation. Their goal is to first ask “Are probiotics effective” and then ask “If so, which one’s work.” Based on scientific guidelines from the ISAPP and EFSA, if one genus/species of probiotic works for one indication, it tells you nothing about whether a different genus/species works for the same indication. So, if four out of five probiotic strains have no effect on constipation, it says nothing about probability of whether other strains of probiotics work (hasty generalization?). You can’t test apples and oranges (pears, bananas, etc.) for vitamin C, compare vitamin C across different fruit tests, and make a statement that, since most of these fruits don’t contain much vitamin C, all fruits contain unimpressive amounts of vitamin C.

    Moreover, if these are just five independent studies to be discussed, the results still indicate that three out of five of these probiotics work for constipation. NIDDK makes their best guess at a guideline for constipation diagnosis (it is terribly more complex than that) which is <3 stools/week. So, rather than have folk resort to their personal definition of “clinical significance” to dismiss papers they don’t like, let’s accept this as an authoritative guidelines. Two of the papers (Mollenbrink and Yang) had a baseline below this guideline which the probiotic increased above the guideline (sig better than placebo too). Koebnick started out with a median at the guideline that increased significantly above the guideline (sig better than placebo too). So, ALL adult studies provided statistically and clinically significant relief – how is that unimpressive?

    For the two pediatric studies, one was an adjunct treatment study with lactulose that did not include a probiotic only control. That is not a well-conducted study as Steve assured us they all were, and it should have been excluded from this review. The final pediatric study had n = 18 treatment and n = 9 control, and found no effect.

    Finally, Scott said, “That’s the opinion of some regulators, too. The European Food Safety Authority has largely rejected general health claims for probiotics.” As I pointed out, of the 180 claims submitted at that point, 170 were rejected on incomplete paperwork. It is false for Scott to extrapolate this regulatory action to indicate EFSA regulators support Scott’s opinion. I have spoken with two EFSA Special Panel members, and they do not agree with Scott either (I am going to guess that is two more than Scott).

    Steve says, “I think it’s reasonable to conclude that they largely have not done this. You have said nothing to contradict it.”

    JPZ says, “Here’s the evidence (citations posted previously, but I can post them again two at a time). Now the burden is on you to prove your point.”

    Is it “reasonable to conclude” that you will duck this point as it has been ducked every time it comes up here? I have never received a scientifically-valid reply, only opinion about me and woo in general. I have yet to receive a reply of any form that shows that someone actually looked at the papers. I believe an unwillingness to address scientifically valid criticisms in areas unpopular on this blog constitutes SBM-bias (aka “skeptic bias”).

    Steve says, “Similarly, you have made some extreme claims of bias and being unskeptical, and you have not made your case. If anything you have shown the bias to lie with you.”

    Back at you.

  51. JPZ – I jumped into this conversation not to offer an independent opinion about the efficacy of probiotics but to address your claims of bias and your tone. You continue to demonstrate a shrill tone in your criticisms of SBM that seems very out of proportion to the alleged deficiencies you cite.

    For example, you wrote:
    “Based on my post, you seem to assume that there are only five studies of probiotics to evaluate; therefore, the “systematic” review Scott chose (apparently with an Ouiji board) must be right. Again, you assume that any field of study that you automatically dismiss could not contain more studies than those in the one article that supports Scott’s POV (many, many more articles don’t support his POV, but I can only post two at a time – it is a great way for SBM to quiet dissent). You assume then that there is only one article that supports each Genus/Family of organism if I (well, and the EU and the top professional organization in this field, you know) say the studies cannot be combined for a “systematic” analysis. Even if that were true (which hilariously it isn’t), 3 out of 4 efficacious studies fulfilled NIDDK criteria for efficacy, and the one study showing no efficacy did not pose a valid hypothesis to be including in the “systematic” review (unimpressive?? to whom? Oh yeah, this was an amazingly insightful unscientific review that SBM-bias loves). I have to question why two researchers such as us are having this conversation based on the evidence. I mean seriously, Steve.”

    When did I assume this. This is an absurd conclusion on your part that to me reflects an emotional/biased response to this issue. I never wrote or suggested any such thing. I would point out that Scott references Mark Crislips more thorough review and then gave a couple examples. So – you really should be addressing Mark’s review, not my reference as a side point in a comment to Scott’s brief examples.

    You assume that I dismiss the evidence for probiotics. When have I ever done that? You assume I have some a-prior bias against probiotics. Where did you ever get that idea:? Gut flora play an important role in GI function and we are learning more and more about their role in health. I think that research into manipulating gut flora is a very promising new avenue of therapeutics. I also think that simply eating one or a few strains of bacteria is a crude intervention and we are learning that the GI flora is a complex and stable ecosystem, not easily altered. I am following the evidence with interest, and my bias, if anything, is that the future will bring increasing GI flora-based interventions. I have no bias against probiotcs.

    Your assumption that I have a bias is in itself a reflection of your own bias. Your expression of being “aghast” at our behavior makes it increasingly difficult to take you seriously.

    You assume we are trying to quiet dissent by limiting links in comments (really – we are one of the most open medical blogs out there). Rather – we are overwhelmed with spam and need to take reasonable measures to limit it. And – the multiple links just put your comment in moderation, and it was quickly approved. So, really?

    Regarding the implications of the review – you are making a serious logical error in your argument. You are falsely assuming that Scott was arguing that the evidence shows probiotics do not work for constipation. He did not. He concluded that there is insufficient evidence to support the conclusion that they do work. This is not a subtle difference. Your argument, that evidence for one species of bacteria says nothing about other species, applies in the former case but not the latter – as I pointed out, but you apparently missed. Scott is not applying data from one species to another. He is saying that the evidence for any particular probiotic formula for the treatment of constipation is underwhelming.

    I am not an expert in this field and have not done a thorough review myself. But I have done enough of a review to see that Mark’s opinion seems reasonable, and the issues with plausibility also seem reasonable.

    Here is the most recent review I could find on PubMed for childhood constipation: http://www.ncbi.nlm.nih.gov/pubmed/21949142

    “There is some evidence that fiber supplements are more effective than placebo. No evidence for any effect was found for fluid supplements, prebiotics, probiotics, or behavioral intervention. There is a lack of well-designed RCTs of high quality concerning nonpharmacologic treatments for children with functional constipation.”

    Hey – maybe this review is wrong also. I find that reviews tend to overcall rather than undercall efficacy, and of course there is the well-documented publication bias and researcher bias that pushes studies in the false-positive direction. So even if flawed, the negative reviews certainly are reason for caution in concluding that any probiotics are effective for constipation.

    (Another issue is that some probiotics seem to increase stool frequency in normal individuals, but this may not translate to function constipation, so we need to be cautious in making inferences from this data.)

    JPZ – I am perfectly willing to be convinced by the evidence that probiotics work for any particular GI syndrome. But your shrill accusations and bias are not convincing.

  52. JPZ says:

    Thank you Steven for a thoughtful but still 10,000 foot view of the issue I raise. I will do my best to return the same courtesy. The concern I express is about “skeptic bias” on SBM, and I present Scott’s probiotics comments as the first example. The second example will focus on one of your comments. Nevertheless, I feel this comment thread is relevant to your POV either way.

    Steven: “When did I assume this.”

    JPZ: In this earlier post in this same thread, “…this would actually count in favor of Scott’s conclusion – because then we only have one good study for any particular strain. One study, especially with modest outcomes, is hardly impressive.” Thus, it seemed that you supported Scott’s conclusion because one study is hardly impressive (we can discuss “modest outcomes” in a minute), but failed to take into account that there may be more studies to evaluate and as such it may be too early to draw any conclusion.

    Steven: “I would point out that Scott references Mark Crislips more thorough review and then gave a couple examples.”

    JPZ: Mark’s review provided no citations other than the WSJ article and did not deal with constipation. I wish to address Scott’s conclusions about probiotics and constipation as evidence of skeptic bias.

    Steven: “You assume that I dismiss the evidence for probiotics.”

    JPZ: As far as I can tell, my only comment that can be interpreted that way is, “Your trust in dismissal of fields of studies you don’t like shows great loyalty, but simply isn’t true.” If you don’t dislike the field of probiotics, I would not think this applies.

    Steven: “Your assumption that I have a bias is in itself a reflection of your own bias. Your expression of being “aghast” at our behavior makes it increasingly difficult to take you seriously.”

    JPZ: Why would I be aghast, Steven? I strongly indentified with this group (and to some extent still do) until I began to see more and more examples of skeptic bias. When I bring it up, it is denied. Skeptics should be able to self-examine their beliefs IMHO.

    Steven: “You assume we are trying to quiet dissent by limiting links in comments…”

    JPZ: No, I am just PO’ed that science-based discussions that can often require more references than text are limited to this extent here. I would like to reply with a list of references long as my arm, but I can’t and the moderators can. That is not limiting dissent so much as it is giving yourself the higher handicap. If someone can instruct me how to put links on here without making them actual links (only text), I can adapt. Moderation can also take days (personal experience) by which time the thread is stale.

    Steven: “He did not. He concluded that there is insufficient evidence to support the conclusion that they do work. ”

    JPZ: ALL three adult probiotic papers showed efficacy according to NIDDK constipation criteria. ALL of the adequately controlled RCTs in adults in a review show clinically-relevant and statistically-significant efficacy, how is it possible to conclude “insufficient evidence” without a priori bias? The fact he didn’t know how to properly review probiotic data is immaterial to how he reached his conclusion.

    Steven: “Hey – maybe this review is wrong also.”

    JPZ: That review only quotes the exact same two pediatric studies in Chmielewska and Szajewska (2010), one adjunct therapy study with no probiotics-only control and one with n=9 control subjects. It is hard to see how they could reach different conclusions.

    Steven: “Another issue is that some probiotics seem to increase stool frequency in normal individuals, but this may not translate to function constipation, so we need to be cautious in making inferences from this data.”

    JPZ: I haven’t seen any evidence of probiotics inducing diarrhea, and, in fact, there is evidence that they may be a useful treatment for diarrhea (refs on request).

    Steven: “I am perfectly willing to be convinced by the evidence that probiotics work for any particular GI syndrome”

    JPZ: Let’s start with whether Scott’s conclusions about probiotics and constipation are evidence of skeptic bias. All of the adult studies were efficacious (clinically and statistically)in the review he cites, but he concludes “unimpressive.” Can we say this is cut and dried skeptic bias and move on to the next example?

    Steven: “But your shrill accusations and bias are not convincing.”

    JPZ: I intend to prove most if not all of my “accusations” if you will hear me out, and, if my tone sounds shrill, I apologize. I was only going for angry.

  53. Quill says:

    Steven Novella wrote in reply to JPZ: “Your expression of being “aghast” at our behavior makes it increasingly difficult to take you seriously.”

    Indeed. Aghast means filled with horror or shock. It is a very strong word to denote a severe emotional reaction and state. Attempting to put humor or even “going for angry” in a web post could be a constructive thing, but since some degree of precision is necessary in any science-based discussions, it might be better to slack on the hyperbole. :-)

  54. JPZ says:

    @Quill

    Wouldn’t that depend on how strongly I identified with the approach on SBM? If you feel comfortable with someone’s philosophy and strongly identify with their point of view (skepticism), then they do the exact thing you thought they most opposed (unfounded bias), and refused to conder evidence that they did (e.g.1 my previous post) – would “aghast” be an unwarranted word?

  55. weing says:

    “The concern I express is about “skeptic bias” on SBM…”
    OK. I admit to “skeptic bias”.
    And “skeptic bias” is bad because…?

  56. Chris says:

    Now kind of dragging this back on topic, though a bit obliquely. This morning I saw this post from an advocate of Natural Hygiene claiming:

    Sadly, most of our children’s friends have had their tonsils out (unnecessarily), their appendix out (ditto above), are on puffers ( ditto)& have regular doses of anti-biotics. How tragic! They look at our children , who have all experienced acute illnesses the same as their friends, & wonder why they are bodily intact & drug-free. I wonder??

    Now I know that tonsillectomies are not done as much as they were forty years ago, but I doubt they are being done as much as this person claims. I tried looking for the different rates of tonsillectomies in the past decades, but my PubMed-fu failed me.

    Can someone point out a review with the rates of tonsillectomies in the past few decades? Thanks much.

    I am not even touching the issue of a community where most of the children have had appendectomies.

    (SBM topic suggestion: Natural Hygiene)

  57. JPZ says:

    @weing

    Like any bias, it prevents you from seeing facts in the face of your bias.

  58. weing says:

    @JPZ,

    I’m skeptical of that. Show me.

  59. Chris, I know exactly one child who has had an appendectomy, after accute appendicitis. I don’t even know that many ADULTS who have had their appendix* out. One that I can think of…after his appendix burst on an airplane to Hawaii. I wonder, what is the natural hygiene approach to peritonitis?

    I also don’t know many kids that have had tonsils out, although, since they have discovered some of the negative effects of sleep apnea in children, tonsillectomies as a treatment for sleep apnea may be on the rise.

  60. JPZ – you didn’t really address my main points. You assume I have been dismissive of the evidence for probiotics, but did not make this case. You assume I have some bias against probiotics, but there is no reason for this assumption and it does not reflect my actual position. Further, you still fail to properly represent Scott’s position, by failing to see the difference between – evidence of lack of efficacy, and insufficient evidence of efficacy.

    Focusing down on just one issue, you wrote:

    “ALL three adult probiotic papers showed efficacy according to NIDDK constipation criteria. ALL of the adequately controlled RCTs in adults in a review show clinically-relevant and statistically-significant efficacy, how is it possible to conclude “insufficient evidence” without a priori bias? The fact he didn’t know how to properly review probiotic data is immaterial to how he reached his conclusion.”

    and

    “Let’s start with whether Scott’s conclusions about probiotics and constipation are evidence of skeptic bias. All of the adult studies were efficacious (clinically and statistically)in the review he cites, but he concludes “unimpressive.” Can we say this is cut and dried skeptic bias and move on to the next example?”

    Hardly cut and dried. First, you can be wrong without being biased, and you have not established bias. You assume it. Second, three studies is not impressive, even if all positive, and you yourself point out that they were of different strains, and therefore can’t be combined. You also refer to there being other studies, but this review discussed the only three they felt were sufficiently well-designed.

    It’s not cut and dried when the only reviews I can find published all come to the same conclusion – insufficient evidence. Here is another review: http://www.ncbi.nlm.nih.gov/pubmed/20220625

    “A limited number of studies that are available on the effectiveness of probiotics on constipation have shown conflicting results, but promising results have been found for certain strains.”

    And another: http://www.ncbi.nlm.nih.gov/pubmed/21382583
    “While a scientific basis for a role for these approaches in the management of constipation continues to develop, evidence from high-quality clinical trials to support their use in daily practice continues to lag far behind.”

    And from the review in question:
    “gain, although the results were statistically significant, the overall effects were clinically modest. All of the conclusions are based on single studies, some of which had a very small number of participants and methodological limitations; thus, the conclusions should be interpreted with great caution. Repeat studies with the probiotic strains that have been proven effective are needed. A paucity of data did not allow us to conclude whether any particular probiotic is more effective than another.”

    So, yes, the result were positive and statistically significant, but these best studies (according to the reviewers) were modest in effect, and small studies with methodological limitations.

    If anything seems clear cut to me it’s that the data is unimpressive at present – a small number of randomized controlled trials that are themselves small with modest (if positive) results.

    You seem to have proven that the bias lies with you, and yet you think it’s cut and dried the other way and are “aghast” that we disagree with you, and can only assume we are hopelessly biased.

    Also – I am not cherry picking. I am simply searching for “probiotics” and “constipation” and looking at every review that comes up in reverse chronological order. If I am missing any relevant reviews, or good studies not included in these reviews, please link to them.

  61. therling says:

    From reading the post and subsequent exchange in the comments section, as well as sampling parts of Mr. Robert’s blog (e.g., “New Support for Prenatal Ultrasound Cause of Autism” blog.sethroberts.net/category/autism/sonograms-cause-autism/ , “…as I’ve said, medical research is almost entirely bad.” http://blog.sethroberts.net/2011/11/06/testing-treatments-eight-questions-for-the-authors/, etc.), I initially thought Mr. Roberts was a well-meaning person who had insufficient experience in scientific methodology and interpreting journal articles. Furthermore, his support of “naturopathy” made me skeptical about his qualifications to render opinions about medical science.

    With that, I was rather surprised that after clicking on the link to his book, “The Shangri-La Diet,” on Amazon, that he is a professor of psychology at UC-Berkeley. Or should I say “is or was” because it’s a little unclear. The “About the author” section states that he is a “tenured professor at UC-Berkeley,” but in the biography that he supplied himself he refers to his being a “professor emeritus at UCB.”

  62. JPZ says:

    @Steven

    I would like to say this proves my point, but it is unsatifying considering this forum holds CAM supporters to higher standards than you have just demonstrated. If you cannot see that the review in question stating that the results are “clinically modest” is unscientific and false when the NIDDK (please provide a higher authority if I am citing the wrong one) definition of clinically significant constipation was fulfilled at baseline and was no longer fulfilled at study completion, then I really don’t know what to say. We aren’t discussing whether probiotics work for constipation – there are literally dozens of studies on the topic, and it would takes weeks in a forum to debate their relative merits and demerits. And you sounded like you were saying that Scott got it right by accident, which I don’t think of as a compliment. If you would like to start that probiotic/constipation discussion as a separate thread, I would be happy to contribute references and insight – BUT…

    We are discussing whether there is evidence that various SBM contributors exhibit sufficient “skeptic bias” to cause them to ignore scientifically valid studies (or other independently-verifiable facts) and reach unscientific conclusions. If Scott’s sole cited sources for his conclusion are the review in question and a false statement about EU regulators, then there is no basis for his conclusion (you seem to keep skipping over the concept that the review in question can be measured against an authoritative definition to establish its quality and veracity – you guys love authoritative sources to smash CAM efficacy studies, why this exception?). If he wishes to say that he only read the abstract and didn’t understand what the actual definition of constipation entails, let him say so, and I am sure that most people here can appreciate an honest mistake – I do. He had an opportunity, but his only reply to my fairly detailed critique was to school me on the Baysean prior and refer me to Mark’s probiotic review that doesn’t contain references. If a woo-meister directed you to a webpage with no references, would you then use it as the index review for the subject (to be honest Mark’s review is pretty good, but could use more than a little bit of a reference-based facelift)?

    Well, to your first point, I answered you with the comments that you made that led me to criticize them. I hardly think that you can look into my mind and decide if my impression of your comments that elicited my reaction “fails to make the case.” It is: 1) You said something, 2) I reacted to it, 3) you said I am wrong, 4) I explained my reaction, 5) you clarify your position, and 6) after some back and forth, we come to an understanding – you seem stuck at 5. And, as I said in my previous posts, some of your comments don’t match what I said – the ever popular “putting words in my mouth” aspect of SBM.

    You really didn’t address my main points either. Perhaps we are both intent on making two unrelated points, or perhaps I would like to say that you have a blind spot due to skeptic bias, but I am introspective enough to question whether I have communicated the science and the gap between science and expressed SBM opinion clearly enough before concluding you have a full-fledged blind spot. And, the other bias examples could have been interesting discussions as well. Unless someone can give me some insight on the missing link in communication here, I don’t see any alternative to both of us walking away assuming the other one is blinded by bias – which is both stupid and sad for two skeptics who should be used to a little self-examination.

  63. weing says:

    @JPZ,
    Now you are getting to see how difficult it is for medications to show more than a “clinically modest” effect. Just because we aren’t impressed, doesn’t me we ignore the studies. I get the impression that you want us to jump on the probiotic bandwagon as if they were a panacea. Sorry, but that is too hard to do. I studied in Eastern Europe, and we utilized lyophilized lactobacilli when I was a medical student, I have recommended probiotics to my patients whenever I prescribed antibiotics since 1984.

  64. JPZ says:

    @weing

    If “clinically modest” is the same as “clinically efficacious by authoritative standards” then you are missing the point – or avoiding it. By what standard is it “clinically modest” except for in the authors eyes?

  65. weing says:

    “By what standard is it “clinically modest” except for in the authors eyes?” What do you think? By what standard would “they are the greatest thing since sliced bread” be except for in the authors eyes? Why is it so important to you that the “clinically modest” description is taken as an affront?

  66. JPZ says:

    @weing

    I have no idea why you are playing this little game for your own amusement. Please supply me with an authoritative standard for a “clinically significant” change in constipation status, and we can apply it to the review in question. Full stop. Supply a standard or be deemed blinded by “skeptic bias.”

    Why is it an “affront”? Well, because you are ignoring data – you know evidence – that thing you glossed over in the dozens of posts before this one. If your personal opinion trumps evidence, I can only guess that you are a surgeon.

    Fun joke

    “The Pope goes to heaven, and St. Peter is showing him around the pearly gates and the streets paved with gold. St. Peter takes the Pope down a couple of side streets and shows the Pope his new home – a nice brownstone on a quiet street.

    The Pope says, “But I was the voice of God on Earth, why am I denigrated to this condition?”

    St. Peter says, “We value humility in Heaven.”

    The Pope replies, “Yes, of course. My thanks to God.”

    St. Peter then takes the Pope to Heaven’s cafeteria. The Pope immediately walks to the front of the line, but St. Peter says, “We are all holy in the sight of God, and we should wait and show humility to our brethern.”

    The Pope takes his tray and waits in line.

    Suddenly, someone dashes into the room, rushes to the front of the line, throws his gloves in the face of those nearby, grabs food from the line with his bare hands and eats it as he rushes out the door.

    The Pope says, “I thought we were all equal here!”

    St. Peter replies, “Sorry, that was God. Sometimes he thinks he is a surgeon.”

  67. weing says:

    Let’s define constipation as patients with mild abdominal discomfort with at least 2 of the following:
    1) 3 spontaneous BMs per week and improvement in the other symptoms compared to placebo over, let’s say, 3 months. A p < 0.05 would be significant.

  68. weing says:

    Sorry,

    I guess parentheses and math symbols screw things up.

    Let’s define constipation as patients with mild abdominal discomfort with at least 2 of the following: less than 3 spontaneous BMs per week, hard stools, moderate to severe straining at defecation.

    Then see what percentage has 3 or more spontaneous BMs per week and improvement in the other symptoms compared to placebo over, let’s say, 3 months. A p less than 0.05 would be significant.

  69. weing says:

    BTW, I am not a surgeon, but good joke anyway.

  70. JPZ says:

    @weing

    Cite your sources for this clinical standard

  71. JPZ says:

    Yep, Steven walked away. Stupid and Sad, really (as I said). I am tempted to be a gadfly on so-called “S” BM (funny, considering the constipation discussion we didn’t have). Continually pointing out the unscientific assumptions of the folks on this board infected with “skeptic bias.” But if Steven “Mr. Skeptics Guide to the Universe” doesn’t get legitimate science-based criticism of his collaborators, then maybe there is no science here, only faith in comments you do not like. Can I hear an “Amen” here for the faithful who only believe in science when it is convenient? LOL

    Yep, gadfly is sounding good. I’ll start compiling PubMed references. I wish this could have been a reasonable conversation, though I tried hard. Now, play ball.

  72. weing says:

    @JPZ

    The link I provided is for the paper that led to the approval of Lubiprostone. It is the source of the “clinical standard” that I gave you as it was used by the FDA and the authors.

  73. JPZ says:

    @weing

    You do realize that the reduction in “science based medicine” oh, wait “significant bowel movements” in the study you cite thresholds at the 3 BM standard I cited based on NIDDK criteria. I mean the pro-SBM (lol, given the study cited by weing) crowd seems particularly blindsided by science that contradicts their pre-determined world-view. And they won’t come out and play when someone can throw hard balls as well as or better than they can. Well, I will enjoy being their gadfly.

  74. weing says:

    @JPZ,

    I gave you a study that the FDA used for approval of an effective drug. Do I think that drug’s effect is “impressive” and the greatest thing since sliced bread? No! It was good enough for the FDA. Have the probiotics, that you are so enamored with, been shown to even meet the standard used by the FDA for approval? Stop throwing hard balls of crap around and show us some “well formed” data.

Comments are closed.