Articles

H. Pylori, Plausibility, and Greek Tragedy: the Quirky Case of Dr. John Lykoudis

Mark Crislip is on vacation, but through an arduous series of shakings and succussions (beating his head against the wall?) we have channeled part of his essence: This post mostly concerns itself with infectious diseases, thanks to several recent posts on SBM that discussed the plausibility of health claims† and that touched on the recent discovery that most peptic ulcer disease (PUD) is caused by a bacterium, Helicobacter pylori. Several comments and statements quoted in those posts reveal recurrent questions regarding both plausibility itself and the history of the H. pylori hypothesis. In this post I’ll attempt to answer some of those questions, but I’ll also insert some new confusion.

Plausibility ≠ Knowing the Mechanism

Let’s first dispense with a simple misunderstanding: We, by which I mean We Supreme Arbiters of Plausibility (We SAPs) here at SBM, do not require knowing the mechanism of some putative effect in order to deem it plausible. This seems so obvious that it ought not be necessary to repeat it over and over again, and yet the topic can’t be broached without some nebbishy South Park do-gooder chanting a litany of “just because you don’t know how it works doesn’t mean it can’t work,” as if that were a compelling or even relevant rebuttal. Let’s get this straight once and for all: IT ISN’T.

Steve Novella explained why at the Yale conference and again here. We talked about it at TAM7 last summer. For a particularly annoying example, read the three paragraphs beginning with “Mr. Gagnier’s understanding of biological plausibility” here.

OK, I’ll admit that I’m beginning to learn something from such frustration. Perhaps we’ve not been so good at explaining what we mean by plausibility. The point is not that we don’t know a particular mechanism for homeopathy, for example; the point is that any proposed mechanism would necessarily violate scientific principles that rest on far more solid ground than any number of equivocal, bias-and-error-prone clinical trials could hope to overturn. The same is true for “energy medicine” and for claims based on non-existent anatomical structures (iridology, reflexology, auricular acupuncture, meridians, chiropractic “subluxations”), non-existent physiologic functions (“craniosacral rhythms“), or non-existent anatomic-physiologic relations (“neurocranial restructuring,” “detoxification” with coffee enemas, dissolving tumors with orally administered pancreatic enzymes). The spectrum of implausible health claims euphemistically dubbed “CAM” is full of such nonsense.

Reader daedalus2u proposed a useful way to clarify the point:

I think the idea of prior plausibility should actually be reframed into one of a lack of prior implausibility. It isn’t that one should have reasons to positively think that something is plausible before testing it, but rather that one should not be able to come up with reasons (actually data) why it is fatally implausible.

Some of what We deem implausible will not be fatally so, of course. Implausibility can be based not only on established physical and biological knowledge, but also on studies, as is the case for sticking needles into people, injecting them with chelating agents, or claiming that autism is caused by childhood immunizations.

Plausibility and History

A second theme, though not as simple, concerns the historical role of plausibility. Reader anoopbal introduced the point:

am not sure if you can apply biological plausibility to every situation. It is usually considered as a weak criterion because it is limited by our knowledge.

If we used plausibility 300 years back, nobody would have used citrus fruits for scurvy nor people would have believed mosquitoes are linked with black water fever.

I think that daedalus’s “reframing” proposal deals with that objection to a large extent. I also don’t think that anoopbal’s examples are all that revealing. It seems to me that empiricism was the main source, other than myth, for plausibility at the time. Plausibility in the biomedical sense is not something that can be usefully discussed for the period prior to about the mid-19th century, when enough was finally known about biology and chemistry to hatch science-based medicine in its full form. Prior to that, most useful diagnostic and treatment methods had been discovered empirically (accidentally).

This is not to say that someone living before the mid-19th century could not have applied plausibility to a medical question—obviously that could happen at any time—but that to attempt to do so, when so much was still mysterious (how cowpox pus worked, microbiology, Avagadro’s number, energy flux in living organisms, physiology, pharmacology, etc.) or ‘explained’ by magic (the Vital Force, miasmas, sympathetic magic, the 4 humors, etc.) would have meant very little by today’s standards. And I do believe that there is a fundamental difference—not merely a foolish conceit about modernity—between what we know today and what we knew 300 years ago. Thus I don’t think that biological plausibility is a weak criterion now, even if it was then.

To give anoopbal his or her due, he seemed to partially agree when he later noted:

And that‘s exactly the limitation of biological plausibility. It is limited by what we currently know. Centuries back our knowledge about earth was limited, and you can’t blame them for believing the earth [was] a flat disc.

On to H. Pylori

Daedalus offered another interesting take on plausibility:

An idea does not have low prior plausibility if it does not agree with prior explanations, it has low prior plausibility if it does not agree with prior data.

Many (most?) scientists make this confusion too. That is because they are thinking on the level of the explanations, not on the level of the data that led to those explanations. The explanations may be wrong, the data that led to them is not.

Given the presumption that the data are accurate, we would all probably agree with this. Daedalus, however, then got a little tripped up:

The idea of using antibiotics to treat ulcers was incompatible with the idea that ulcers were due to too much acid. It was not incompatible with any of the data surrounding ulcer treatment.

So did Harriet Hall:

The idea of treating ulcers with antibiotics was not incompatible with any of the data about ulcers; it was only incompatible with the idea that ulcers were caused by too much acid.

At the time that Barry Marshall and Robin Warren proposed their bacterial hypothesis, there were data suggesting that ulcers were caused by too much acid: acid neutralization or suppression of acid formation resulted in better than 90% healing of peptic ulcers, compared with about 30% for placebo. If such therapies were discontinued after healing, the ulcers typically relapsed, only to be healed again by renewed acid suppression. This did not rule out the possibility of some other factor also being involved, of course, but it would seem to have come pretty close to throwing down the gauntlet of Ockham’s Razor.

Why not just Treat with Antibiotics?

What might it have taken prior to 1984, short of what was subsequently done, to convince the world that peptic ulcer disease could be effectively treated with antibacterial agents? A reader sent just such a question to Steve Novella:

What would Science Based Medicine do if H. pylori was not known, but a study showed that antibiotics given to patients with stomach ulcers eliminated symptoms? I assume that SBM wouldn’t dismiss it outright saying that it couldn’t possibly be helping because antibiotics don’t reduce stomach acid. I assume a SBM approach would do further studies trying to discover why antibiotics work. But, in the meantime, would a SBM practitioner refuse to give antibiotics to patients because he doesn’t have a scientific explanation as to why it works?

A straightforward answer is as follows. Although the question may raise an interesting general point about plausibility, the example is not a good one. Antibiotics are not one medicine but many. They all have side effects, some quite serious. Bacteria are also not one species but many; they have widely differing sensitivities to various antibiotics. Which antibiotic(s) would the study have used, and on what basis? Responsible MDs would not have accepted such a scheme for PUD, because they would have needed to know what they were treating and how to treat it (H. pylori turns out to require three different antibiotics given simultaneously).

A predictable rejoinder to this is that many physicians routinely treat upper respiratory tract infections, most commonly caused by viruses, with an antibiotic. Without going into detail, let me assure you that this does not refute my point: in many cases MDs should not be treating these URIs with antibiotics, and in cases where it makes a bit of sense to do so it is done with a single, short-term antibiotic with a benign risk/benefit profile, known to be effective against the most common community-acquired bacterial culprits of the respiratory tract. This is quite different from attempting to treat a mysterious bacterium that might not even exist, for a disease that already has effective treatments that are safer and have fewer side effects than antibiotics.

Mikerattlesnake got the point:

I think it’s a wise addendum to directly address the logical misstep in the question you received. Those who understand SBM would get the answer from the broad approach taken in your post, but those people aren’t the ones likely to parrot the fallacy.

To put it simply: finding that an antibiotic was effective against an ulcer would indicate a bacterial cause for ulcers that would warrant further study. The reason for that has entirely to do with prior plausibility. Antibiotics are known to fight bacteria. If an antibiotic cures ulcers, it gives us a plausible answer for the mechanism causing ulcers. The questioner makes the mistake of assuming that we would never abandon the assumed cause of ulcers, but SBM looks for mechanisms of action for ailments as well as cures.

So did BillyJoe, with the appropriate caveat:

I like it.

Only one thing though: this could never have happened. As I understand it, the treatment involves taking three different tablets – two antibiotics and an acid suppressing drug – twice a day for a week. How likely is that to have happened by chance?

Discoveries Require Context

Some might assume that Robin Warren and Barry Marshall were the first to discover bacteria apparently living in the human stomach and duodenum, and the first to propose that the bacteria might be involved in diseases of these tissues, but this isn’t the case. Such bacteria were first observed in the 19th century. Over subsequent decades there were sporadic reports of similar bacteria, but they were not necessarily associated with diseases and their presence could not be reliably reproduced. The table of contents of Helicobacter Pioneers: Firsthand Accounts from the Scientists who Discovered Helicobacters 1892 – 1982, edited by Barry Marshall, gives a hint of just how close some investigators came to the truth:

  1. Helicobacters were discovered in Italy in 1892: An episode in the scientific life of an eclectic pathlogist, Giulio Bizzozero. Natale Figura and Laura Bianciardi
  2. The discovery of Helicobacter pylori in Japan. Yoshihiro Fukuda, Tadashi Shimoyama, Takahashi Shimoyana and Barry J Marshall
  3. An early study of human stomach bacteria. A. Stone Freedberg
  4. Gastric urease in ulcer patients in the 1940′s: The Irish connection. Humphrey J O’Connor and Colm A O’Morian
  5. How it was discovered in Belgium and the USA (1955 -1976) that gastric urease was caused by a bacterial infection. Charles S Lieber
  6. A personal history of giving birth to the cohort phenomenon of peptic ulcer disease. Amnon Sonnenberg
  7. John Lykoudis: The general practitioner in Greece who in 1958 discovered the etiology and a treatment of peptic ulcer disease. Basil Rigas and Efstathios D Papavassiliou
  8. How I discovered helicobacters in Boston in 1967. Susumu Ito
  9. How we discovered in China in 1972 that antibiotics cure peptic ulcer. Shu-Dong Xiao, Yao Shi and Wen-Zheng Liu
  10. Helicobacter pylori was discovered in Russia in 1974. Igor A Morozov
  11. The discovery of Helicobacter pylori in England in the 1970′s. Howard W Steer
  12. We grew the first Helicobacter and didn’t even know it!. Adrian Lee, Michael Phillips and Jani O’Rourke
  13. The Dallas experience with acute Helicobacter pylori infection. Walter L Peterson, William Harford and Barry J Marshall
  14. The discovery of Helicobacter pylori in Perth, Western Australia. J Robin Warren
  15. The discovery of Helicobacter pylori, a spiral bacterium, caused peptic ulcer disease. Barry J Marshall
  16. Helicobacter pylori treatment in the past and in the 21st Century. Peter Unge

Prior to Marshall and Warren, human gastric bacteria were not only inconsistently seen, but were never cultured and hence never characterized in a useful way (for a more basic treatment of this topic, please see my 2004 essay in Skeptical Inquirer). At least two distinct technological advances were necessary to set the stage for the discovery and characterization of H. pylori in humans: first, a simple and safe method for obtaining gastric mucosa specimens from live patients had to be devised; second, the field of bacteriology had to appreciate the existence of highly fastidious organisms and devise methods for growing them in culture. The first of these requirements was satisfied only by the late 1970s, when flexible, fiberoptic endoscopy became widely available.

I am not enough of an historian of bacteriology to state, with certainty, when H. pylori might have first been cultured, if only its existence had been fully appreciated, but it is doubtful that it could have occurred much sooner than it did. Helicobacter Pioneers reports that the first successful culture of any helicobacter species—isolated from mice—occurred in 1968. Other examples of fastidious bacteria have also been characterized relatively recently: mycoplasma pneumoniae and chlamydophila pneumoniae, two organisms that cause atypical pneumonia, were still thought to be viruses until the 1960s; several new species of helicobacter and campylobacter, some of which are human pathogens, have been discovered only in the last 10-15 years.

The culture requirements of H. pylori, moreover, are esoteric: it grows best in an atmosphere of 5% oxygen and is helped by the presence of certain antibiotics to discourage overgrowth by more hardy contaminants, which are almost impossible to avoid when collecting specimens from the stomach via the mouth. Helicobacter takes much longer to grow than most bacteria, and but for serendipity Warren and Marshall might have missed it. They abandoned their first 34 culture attempts (or, more precisely, “junior microbiology staff” abandoned them) in spite of multiple variations of media and temperatures, after no growth had occurred within 48 hours. It was only after a five-day Easter vacation, during which the 35th attempt was left undisturbed, that tiny, transparent colonies appeared.

John Lykoudis: the Real Galileo of PUD?

An intriguing story in Helicobacter Pioneers is found in chapter 7: John Lykoudis: The general practitioner in Greece who in 1958 discovered the etiology and a treatment of peptic ulcer disease. You can read most of this chapter at the Google Books website. If it is accurate, it makes the answer to the question that the reader posed to Dr. Novella not so straightforward as Mikerattlesnake, BillyJoe, and I argued above: it inserts the “new confusion” that I promised at the beginning of this post. Lykoudis’s story has all the necessary tragic elements:

a general practitioner in a small, isolated town in Greece, prompted by a single clinical observation, developed on his own the concept that PUD and gastritis had an infectious etiology. As if this was not enough, this most unlikely student of PUD proceeded to devise an apparently effective treatment, based on the antibiotics of his time.

Lykoudis’s treatment, apparently developed by trial and error, consisted of 3 antibiotics (2 quinolines and streptomycin, for you microbiology/infectious disease enthusiasts out there) and vitamin A, taken orally. He patented this regimen in a pill that he named Elgaco, “from the Greek word for ulcer (= elkos), gastritis and colitis” (for which he also asserted that his treatment was effective). He eventually claimed to have treated 30,000 patients with nearly perfect results and no toxicity. According to the authors of the chapter,

The success of Elgaco cannot be quantified from extant notes on thousands of patients, because the outcome of each patient is not recorded. We have concluded, however, that his treatment was successful, based on the following considerations. First, our current understanding of the etiology and treatment of PUD makes it plausible that his treatment was effective. Second, there is the written testimony (some of it sworn, as explained later) of many of the patients who were treated by Lykoudis. All report prompt responses to his therapy. In some cases, patients even detail that radiographically proven ulcers were cured following treatment with Elgaco and that such cure was confirmed by repeat radiological series. Third, Lykoudis had a large following and despite fierce opposition from the establishment, patients flocked to him from all over Greece.

In spite of this, his attempts to make his discovery known to the world were rebuffed at every turn:

He encounter[ed] formidable obstacles in convincing the medical establishment, the Greek regulatory authorities and the pharmaceutical industry. In fact, Lykoudis spent the rest of his life engaged in incessant activity to propagate his treatment of PUD and gastritis. His archives, some made recently available by his family, make it clear that he was fully aware of the importance of his discoveries. They also convey an almost suffocating sense of frustration…

[He was] completely shunned by the medical establishment of his time, or at best, considered an eccentric provincial physician…

In 1966, Lykoudis attempted to publish his observations in the Journal of the American Medical Association, but his manuscript entitled “Ulcer of the Stomach and Duodenum” was rejected…Unfortunately, no copy of this manuscript survives for re-evaluation in the light of current knowledge.

Lykoudis did, however, publish his own booklet, “The Truth about Gastric and Duodenal Ulcer.” In it he wrote:

There is no doubt that gastritis and duodenitis, which have gastric and duodenal ulcer as their complication, are inflammations due to an infectious agent…

Lykoudis made numerous attempts to get his remedy approved by the Greek Drug and Pharmacies Administration, to no avail. He even managed to enlist the aid of influential politicians:

In 1967, Lykoudis succeeded in getting the attention of the Prime Minister’s office. His correspondence with the Minister of Health on 21 August, 1967, a sad document indeed, is revealing. He registers his frustration that medications with apparently no effect on PUD were approved, whereas Elgaco was repeatedly rejected. He proposes, in essence, a phase III trial: 100 PUD patients to be treated at a State hospital by the eminent professors, 50 with conventional treatment and 50 with Elgaco. ‘Their refusal to approve it is understandable, but their refusal to test it is not!’ he writes.

Lykoudis continued:

If the study proves them correct, they will be vindicated and I will become a laughing stock…It is dramatically urgent to clarify this issue…Too much, endless talking, which leads nowhere, while it is simple to resolve this in a practical way. Only facts constitute the truth.

Yet again he was refused. Lykoudis also tried, unsuccessfully, to interest several drug companies in his regimen. The final insults were these:

…he was referred for disciplinary action to the Athens Medical Association, of which he was a member, ‘because (a) he prepared and distributed an unapproved medicinal preparation…and (b) he made his method publicly known to attract patients’…On 6 November 1968…the Disciplinary Committee, presided over by a neurology professor, fined him 4000 drachmas…

A more serious problem for Lykoudis was his indictment in the Greek Courts…

In the latter instance numerous former patients came to his support; one of them testified that Lykoudis “treated also many poor ulcer patients free of charge.”  We are not told the outcome of the indictment.

Lykoudis died in 1980 without knowing that he would soon be vindicated. His story is disturbing because it is an almost perfect hybrid of two entirely different possibilities: on the one hand, a legitimate innovator who is unfairly rejected and persecuted, in spite of heroic efforts over more than 2 decades to prove his theory; on the other, a classical example of unwitting foolishness, bordering upon quackery or sociopathy.

It is only in hindsight that we can grant that there is a good chance that it wasn’t the latter. Consider the striking parallels, however, to Nicholas Gonzalez, whose main arguments have consisted of patient testimonials and case reports selected by himself, who claims that his regimen is nontoxic, who claims to treat some patients for free, who was hounded by regulatory boards for a time, who found political allies to help defend him, and who for years pleaded that all he wanted was a chance to test his regimen:

I believe in research. I don’t want this to be out there until we prove it works by the strictest standards of orthodox medicine. What I have wanted from the day I began researching this under Dr. Goode at Cornell in 1981, was to do appropriate clinical trials.

Again: Discoveries Require Context

My sense, reading the story of John Lykoudis, is that he was treated unfairly, and I think that most people would agree. A major caveat is that the authors of the chapter are clearly sympathetic to him, and it’s quite possible that another account would read differently.

Whether fair or not, it seems to me that the major weakness in Lykoudis’s case is that he never characterized the putative bacteria in any way: he didn’t see them, he didn’t provide direct evidence of them for others to examine, and he didn’t culture them.

Even his sympathetic biographers recognized this. Although they attributed his failure to “his lack of academic credentials” and even more to “his thesis [being] contrary to established, albeit unsubstantiated, dogma,” they also observed:

Unfortunately, when he was compelled to identify these elusive organisms, particularly when dealing with regulatory agencies, he meandered around known pathogens, unable to build a strong case for any of them. His main argument, and the strongest one he could marshal in all his writings in favor of the infectious etiology of these clinical entities, was the response to treatment that he had witnessed.

A good argument can be made that characterizing the “bug” not only is, but ought to be a sine qua non for treating a putative infectious disease with drugs. This is true because the drugs are not benign or universally effective, as argued above, but also because there are precedents suggesting that to do otherwise opens the door to mistreatments. Bacteria or viruses are frequently offered as potential etiologic agents for all sorts of diseases whose causes are poorly understood, particularly when there is an inflammatory component (that is one reason that the H. pylori hypothesis didn’t represent a new “paradigm”). Osteoarthritis and rheumatoid arthritis are examples, but these have, so far, eluded attempts at proof.

When I first learned about sarcoidosis and Crohn’s disease in medical school in the 1970s, I would have bet dollars-to-donuts that they, and rheumatoid arthritis and a few other diseases for that matter, would eventually be shown to have infectious etiologies. I would still almost make that bet, my only hesitation being that after 30+ more years of investigations and impressive advances in microbiology (including vastly more powerful methods of detecting well-veiled foreign invaders, from electron microscopy to nucleic acid amplification), no apparent culprits have feen found.

During that same time, moreover, not only peptic ulcer disease but also Lyme disease, Legionnaire’s disease, and Toxic Shock Syndrome were shown to have bacterial origins, and AIDS was shown to be caused by a virus. Thus on the basis of what has been learned about infectious diseases it can’t be argued, with a straight face, that biomedical progress is hampered by stodginess or petty jealousies or dogmatic thinking or conflicts of interest or any of the other usual suspects, even though they certainly all exist among individual scientists. What hampers progress, in cases such as Likoudis’s, is what hampers all scientific progress: the context is not prepared.

Instances in which at least some people have become convinced of a spurious infectious etiology, on the other hand, have not been pretty. In the early 20th century, prior to the discovery of antibiotics, some psychiatrists became convinced that “insanity” was caused by bacteria in the mouth and that the appropriate treatment was “surgical bacteriology”: tooth extractions, tonsillectomies, and in intractable cases removal of “testicles, ovaries, gall bladders, stomachs, spleens, cervixes, and especially colons.” More recently a putative bacterial cause of atherosclerosis has spawned a small quack industry.

Even if any of the diseases mentioned above is eventually found to be infectious in origin, this will not necessarily vindicate those whose premature exuberance put patients in harm’s way. Such exuberance ought to motivate legitimate investigations, not half-assed, ill-conceived treatments. Still, I seem to hear a ghostly voice in my ear, speaking Italian with a thick Greek accent…

E pur si muove!

………………

† The Prior Probability, Bayesian vs. Frequentist Inference, and EBM Series:

1. Homeopathy and Evidence-Based Medicine: Back to the Future Part V

2. Prior Probability: The Dirty Little Secret of “Evidence-Based Alternative Medicine”

3. Prior Probability: the Dirty Little Secret of “Evidence-Based Alternative Medicine”—Continued

4. Prior Probability: the Dirty Little Secret of “Evidence-Based Alternative Medicine”—Continued Again

5. Yes, Jacqueline: EBM ought to be Synonymous with SBM

6. The 2nd Yale Research Symposium on Complementary and Integrative Medicine. Part II

7. H. Pylori, Plausibility, and Greek Tragedy: the Quirky Case of Dr. John Lykoudis

8. Evidence-Based Medicine, Human Studies Ethics, and the ‘Gonzalez Regimen’: a Disappointing Editorial in the Journal of Clinical Oncology Part 1

9. Evidence-Based Medicine, Human Studies Ethics, and the ‘Gonzalez Regimen’: a Disappointing Editorial in the Journal of Clinical Oncology Part 2

10. Of SBM and EBM Redux. Part I: Does EBM Undervalue Basic Science and Overvalue RCTs?

11. Of SBM and EBM Redux. Part II: Is it a Good Idea to test Highly Implausible Health Claims?

12. Of SBM and EBM Redux. Part III: Parapsychology is the Role Model for “CAM” Research

13. Of SBM and EBM Redux. Part IV: More Cochrane and a little Bayes

14. Of SBM and EBM Redux. Part IV, Continued: More Cochrane and a little Bayes

15. Cochrane is Starting to ‘Get’ SBM!

16. What is Science? 

Posted in: Basic Science, History, Science and Medicine

Leave a Comment (25) ↓

25 thoughts on “H. Pylori, Plausibility, and Greek Tragedy: the Quirky Case of Dr. John Lykoudis

  1. Draal says:

    I think the full story behind Dr. Marshall’s work is just as interesting. He too was frustrated by dismissal of his hypothesis by the medical community to the point he ingested H. pylori himself.
    http://www.metamath.com/math124/statis/Marhelio.htm
    http://science.education.nih.gov/home2.nsf/Educational+ResourcesResource+FormatsOnline+Resources+High+School/928BAB9A176A71B585256CCD00634489

  2. anoopbal says:

    “Plausibility in the biomedical sense is not something that can be usefully discussed for the period prior to about the mid-19th century, when enough was finally known about biology and chemistry to hatch science-based medicine in its full form. ”

    Maybe in the 26th century they might say the same about the “period prior to the mid 21st century”

    “The point is not that we don’t know a particular mechanism for homeopathy, for example; the point is that any proposed mechanism would necessarily violate scientific principles that rest on far more solid ground than any number of equivocal, bias-and-error-prone clinical trials could hope to overturn.”

    This is not plausibility. This is the Hill’s criteria of Coherence. Dr. Hill says, “What is biologically plausible depends upon the biological knowledge of the day. On the other hand, the cause-and-effect interpretation of our data should not seriously conflict with the generally known facts of the natural history and biology of the disease (termed as coherence).”

    So what you try to say as biological plausibility is :
    1. Biological plausibility and
    2. Coherence

    Both concepts are important I think. I guess you just had both into one.

  3. Scott says:

    I think of the “plausibility” in this sense as being more analogous to a Bayesian prior probability.

    Maybe in the 26th century they might say the same about the “period prior to the mid 21st century”

    Possible, but I think unlikely. There would have to be a revolution in our means of understanding of the world similar in scope to the scientific method.

    And even if such did happen, it wouldn’t invalidate the point that there’s been such a profound shift in the nature of medicine between then and now that such comparisons are simply meaningless.

  4. “To give anoopbal his or her due, he seemed to partially agree when he later noted:

    And that‘s exactly the limitation of biological plausibility. It is limited by what we currently know. Centuries back our knowledge about earth was limited, and you can’t blame them for believing the earth [was] a flat disc.”

    Actually, it sounds to me like anoopbal is implying that our current knowledge is always to future knowledge as the belief that the world is flat is to the understanding that the world is approximately a gravitationally relaxed oblate spheroid.

    I tend to disagree with that. I don’t think it’s modern conceit to think we now have a pretty firmly established understanding of the basic workings of physics and the universe. Sure, there are gaps in there, and some of those holes are pretty big, such as dark matter & dark energy, but it’s still pretty unlikely that any gaps we fill in are going to fundamentally alter the rest of what we already understand. We’re not going to solve the dark energy question and figure our we are really immaterial beings and start traveling by mental projection to Deepak Chorpra’s home world.

    Consider that even as big a leap as special relativity was, it didn’t make Newtonian physics obsolete, it just showed us that Newtonian physics is really just an extremely accurate approximation (essentially much more accurate than our ability to measure the inaccuracy) until velocity gets near the neighborhood of the speed of light

  5. It’s key to note that there is a sometimes subtle distinction between plausibility and probability, and that the two are often very closely intertwined. The basic concept that H. Pyori causes ulcers was not inherently implausible by itself, that is it wasn’t a particularly implausible biological mechanism, but was improbable in the context of the existing data regarding ulcers and acid reduction.

  6. daijiyobu says:

    @Karl Withakay per “distinction between plausibility and probability [...] the two are often very closely intertwined.”

    Perhaps a Venn diagram is in order?

    -r.c.

  7. Kim – thanks for extending the discussion of plausibility, which is so central to SBM. The idea of differentiating “plausibility”, “prior probability” and “coherence” is interesting. I will have to think about that. I resist increasing complexity unnecessarily, unless it is deserved by greater precision of practical consequence.

    Meanwhile, the de facto definition of plausibility as it is applied in medicine includes all of those concepts – what does the basic science say, is there a possible mechanism, are any laws violated, is it consistent with what we already know, and what is the status of existing empirical clinical evidence? In short, how much prior scientific data will have to be overturned if the new claim is true, and how much existing scientific data already supports the claim? (And – what is the pattern of that data?)

  8. Re: Lykoudis – this is indeed an interesting and disturbing story. I wonder if it is at least partially apocryphal.

    But taken at face value, I think the real lesson is that process is more important than whether or not one turn’s out to be correct. Science is about process.

    The problem with Lykoudis is that his behavior was indistinguishable from the myriad quacks and charlatans that existed then and exist today. That in hind sight one turned out to be on the right track is not all that surprising, and their contemporaries should not be faulted for their inability to predict the future.

    The question is – what did Lykoudis do to convince the scientific community of his claims. Did he perform carefully controlled double-blind placebo-controlled trials? Did he attempt to enlist the help of a microbiologist to try to isolate the organism? Or did he just expect people to take his word for it?

    What did he do to deserve being taken seriously? Being right in the hindsight of history is not enough.

  9. pmoran says:

    I share Steve’s instinct that there were probably multiple factors behind Lykoudis’ difficulty in gaining immediate acceptance of his theories.

    Perhaps I am just getting old, but I find certain distinctions that are being made here hard-going. Are not plausibility, prior probability and coherence all saying much the same thing — that ALL the relevant evidence has to be considered and given the weight it deserves when evaluating a scientific claim or hypothesis?

  10. tmac57 says:

    If the Lykoudis story is essentially true ( it does seem just a little too ‘on the nose’), then it it might be considered as a near perfect example of someone being right for mostly the wrong reasons. The story will undoubtedly be used to show how science has “got it all wrong”, and why the unproven and improbable ‘cures’ for all diseases are getting short shrift from mainstream researchers.

  11. pmoran says:

    Is it yet known precisely how a diffuse bacterial infection in the stomach gives rise to discrete ulcers in the duodenum? I don’t see how previous experience in microbiology would have led anyone to look for a bacterial cause of duodenal ulcers within the stomach.

  12. @ pmoran-

    from Dixon MF. Patterns of inflammation linked to ulcer disease. Baillieres Best Pract Res Clin Gastroenterol, 2000. 14 (1): 27–40.:

    “Peptic ulcers are accompanied by different patterns of chronic gastritis and duodenitis that generally run parallel to the topography of colonization by Helicobacter pylori (H. pylori). Duodenal ulcers arise on a background of a gastroduodenitis; the gastritis is antrum-predominant while the duodenitis requires acid-induced gastric metaplasia in the duodenal mucosa before bacterial colonization can occur. The colonized and inflamed metaplastic areas in the duodenum (and inflamed pre-pyloric antrum) are the initial sites of ulceration. Proximal gastric ulcers arise in a diffuse (pan-) gastritis or a corpus-predominant H. pylori gastritis when the weakened gastric mucosa (especially in the antrum-body transitional zone) is susceptible to ulceration even in the presence of subnormal acid production. These distinctive patterns of gastritis are sufficiently consistent for them to be used to predict ulcer risk.”

    Another example that I can think of off the top of my head (although a little different) is a discrete superficial infection (carbuncle, etc) that is caused by a common skin colonizer (like staph aureus). It seems that the environment in the stomach/duodenum affects the colonization pattern of h. pylori, and ultimately determines where most of the inflammation, and eventually ulceration, would occur. This is not completely out of line from what is known about the rest of the field of microbiology. I’m not sure, however, what the level of knowlege in the field of microbiology was at that time.

  13. Peter:

    In addition to DoR’s note above, take a look at Warren and Marshall’s Nobel lectures. Marshall reports that in an early series of 100 consecutive endoscopies with biopsy (ca. 1981-2), they were surprised to find that those “with bacteria” included 13/13 with DU, 18/22 with gastric ulcer, 23/42 with gastritis, and 8/16 with normal endoscopic findings. This was the first indication that the bacteria seemed to be important in ulcer, particularly DU, rather than merely in gastritis.

    In Warren’s lecture near the end he discusses DU as follows:

    We were surprised to find duodenal ulcer so closely related to Helicobacter. However, further investigation shows that most duodenal ulcers can be viewed as distal pyloric ulcers. They are in the duodenal cap and the pyloric mucosa normally extends through the pylorus (figure 12). Biopsies from the proximal border of all duodenal ulcers in this study showed either gastric mucosa or scarred mucosa, consistent with a gastric origin and with no apparent Brunner’s glands, as seen in duodenal mucosa.

    The pyloric mucosa is very mobile and easily moves some distance through the pylorus. When the stomach contracts, a mixture of food fragments and corrosive gastric juice squirts through the pylorus. Perhaps it is not surprising that ulcers are so common here, especially when the epithelium is damaged by infection and active inflammation.

  14. Regarding some of the other comments above:

    There is no useful difference between plausibility and the “coherence” suggested by anoopbal, as is clear from the quotation attributed to Dr. Hill: “the biological knowledge of the day” must include “the generally known facts of the natural history and biology of the disease.”

    There is a definitional difference between plausibility and Bayesian prior probability: the latter is a number greater than or equal to 0 and less than or equal to 1. It is a project in quantifying plausibility.

    Actually, it sounds to me like anoopbal is implying that our current knowledge is always to future knowledge as the belief that the world is flat is to the understanding that the world is approximately a gravitationally relaxed oblate spheroid.

    That was also my interpretation of anoopbal’s comment, which is why I rejected the notion of “a foolish conceit about modernity.” When I wrote that anoopbal “seemed to partially agree” I was attempting to nudge him (and others who insist that all knowledge is “relative”) toward noticing the fallacy illustrated by the very example that he used: is the earth a flat disc or not?

    If we can reasonably assert that we now have more certainty about the shape of the earth than we did hundreds of years ago, how is that different from asserting that we now have more certainty about anatomy, physiology, and biochemistry than we did hundreds of years ago?

    The only alternative, it seems to me, is to assert that “maybe in the 26th century” there is a more than an infinitesimal chance that the world will once again be “proved” to be a flat disc.

  15. anoopbal says:

    “There is no useful difference between plausibility and the “coherence” suggested by anoopbal, as is clear from the quotation attributed to Dr. Hill: “the biological knowledge of the day” must include “the generally known facts of the natural history and biology of the disease.””

    Agreed. But is there a difference if Hill had it as 2 concepts?

    “hat was also my interpretation of anoopbal’s comment, which is why I rejected the notion of “a foolish conceit about modernity.” When I wrote that anoopbal “seemed to partially agree” I was attempting to nudge him (and others who insist that all knowledge is “relative”) toward noticing the fallacy illustrated by the very example that he used: is the earth a flat disc or not? ”

    My point is not earth being flat or whatever. The bottom line is that there might be areas in science which might see a paradigm shift in the future. It is not at all a “foolish conceit about modernity”. For example, the plasticity of the brain concept was laughed off until the last decade. And it is the most important discoveries in brain science. Will that happen in every feild every decade? No. Will that happen in a few fields? Maybe. The greatest virtue of a scientist is some healthy skepticism.

  16. JMB says:

    Very interesting and educational article. I wonder if there may be a corollary in the rediscovery of preventive effects of aspirin for heart attacks and stroke?

    I would speculate that EBM, as practiced by the Cochrane Collaboration would have rejected Lykoudis’s work because of methodological errors. Perhaps an SBM approach would have taken greater notice. Rejection of his submissions from medical journals is not an indictment of an SBM approach. SBM does not discount clinical experience as much as EBM (although your definition of EBM may vary). I would not say that Cochrane Collaboration’s embrace of CAM is a rational correction of past mistakes.

    There are significant pitfalls in using a prior probabilities in either the translation of medical science into medical practice, or the rational allocation of resources for direction of research. The biggest pitfall is the size and completeness of our a priori information. When there is a heavy insistence on certainty based on statistical significance and experimental design, we tend to filter a lot of a priori information out. Bayes strategy is based on probabilities large and small. If measure of uncertainty is treated more as a probability term, rather than all or none based on statistical significance, then we will probably see fewer paradigm shifts, but more updates on factors in the equations. Such an expansion of a priori information is now possible with the information age. I think it is significant that the web was really developed by physicists seeking more rapid and complete communication of information. I think medicine would do well to update its scientific methods to account for these changes. Of course, there already are changes that are significant, greater use of Bayes methods that are computationally intensive, greater use of population databases as opposed to sampling statistics, and greater access to medical literature through services like PubMed.

    Now, if only we could get free access to the entire article on the web.

  17. cloudskimmer says:

    Hasn’t the current treatment for ulcers undergone a complete change from the idea that they were caused by excess stomach acid to being caused by bacteria? And doesn’t this show that science-based medicine changes based on the evidence? Can anyone say the same for acupuncture, chiropractic, naturopathy, or any other of the quack treatments? They never abandon anything, carrying on with the same rubbish for years, decades, and with homeopathy, a couple of centuries without recognizing that the fundamental precepts on which they are base have been completely discredited. Instead of an example of the failure of modern medicine, isn’t this a success: discarding a failed treatment in favor of another which does better? Are there any effective ulcer treatments in the panoply of SCAM? Can acupuncture, chiropractic, naturopathy, homeopathy, or any of the others cure an ulcer better than a placebo? Did any of them even do as well as the acid-reduction treatment? Where are the studies to support any contention that other disciplines (it hurts to call SCAM that) have any capacity to treat ulcers?

  18. daedalus2u says:

    I find myself disagreeing with both Dr Atwood and Dr Novella. A correct idea never actually lacks prior plausibility. If a prior plausibility assigning method assigns a low value to a correct idea, that is a problem of the plausibility method being used, not of the idea.

  19. Instead of an example of the failure of modern medicine, isn’t this a success: discarding a failed treatment in favor of another which does better?

    Yes, absolutely so. That’s how I got interested in the H. pylori topic to begin with, as explained here.

    daedalus, I agree with you (and I suspect Steve does as well, though I’ll leave that to him). My citing of data supporting the ‘acid’ hypothesis was merely to play the devil’s advocate. My opinion, the very first time that I heard about the H. pylori hypothesis, was that it was utterly plausible. I tried to make this clear in the Skeptical Inquirer essay linked in the previous paragraph, and also in my report of the recent Yale Symposium:

    The notion that bacteria might cause an inflammatory lesion was entirely plausible, of course, and even if some physicians were surprised to learn that bacteria can adapt to an acidic environment, bacteriologists were not. [etc.]

  20. DREads says:

    Sorry if this is a little off topic. This has nothing to do with peptic ulcers, H. pylori, Lykoudis, plausibility, or the philosophy of prior belief. If there was a SBM blog posting on the application of statistics to medicine (or just statistics), this comment might be more on topic. However, statistics is rarely a primary topic of blog posts here.

    Just to be clear, Frequentists and Bayesians agree on Bayes Theorem, measure theory, and the laws of probability. Invoking Bayes Theorem does not necessarily put an analysis into a Bayesian framework. This is a common misconception and an important one to correct. Frequentists assume the parameters of a distribution are fixed. Bayesians do not.

    Where does Bayesian statistics start to come into the picture? By applying a Bayesian interpretation of probability, we can place a density p(\theta) on the parameter space (called a prior density) because parameters are not fixed. A likelihood function f(x|\theta) quantifies the likelihood of the data given a particular \theta. If we then invoke Bayes Theorem,

    p(\theta|X) = c*f(X|\theta)p(\theta)
    posterior = likelihood * prior

    where c is a normalizing constant, we get a posterior distribution p(\theta|X). The prior and the posterior are both distributions over \theta with the difference being that the posterior incorporates the data (with a likelihood function) whereas the prior does not. A common saying is “Yesterday’s posterior is today’s prior”. Bayes Theorem can be used to update the posterior given some new data Xnew,

    p(\theta|Xnew,Xold) = c*f(Xnew|\theta)p(\theta|Xold)
    new posterior = likelihood * old posterior (new prior)

    Credible intervals are analogous to frequentist confidence intervals. When integrating a posterior from a to b gives 95%, we say that the 95% credible interval is the parameter \theta from a to b. However, unlike confidence intervals, credible intervals can be interpreted as probabilities, i.e. there is a 0.95 probability \theta lies between a and b.

    The weights of the likelihood and prior can be used to judge the influence of the prior vs. the data on the posterior. As the number of samples gets large, eventually the posterior is mostly determined by the data.

    If the prior and likelihood are of specific functional forms, then one can sometimes analytically derive a posterior that comes from the same family of distributions as the prior. Such a prior is called a conjugate prior. For example, a normal likelihood with a gamma prior is a gamma posterior so we say that a normal likelihood is conjugate to a gamma prior. By analyzing the posterior hyperparameters, one can assess the influence of the likelihood vs. the prior on the posterior as the number of samples, the prior parameters, or the sample parameters vary. It’s a fun exercise (also fun to derive!). For example, given a Bernoulli likelihood and a Beta(alpha,beta) prior, the posterior is a Beta(alpha’,beta’) with parameters alpha’=alpha+n*xbar and beta’=beta+n-n*xbar where xbar is the sample mean and n is the sample size. Once n exceeds alpha/xbar, the alpha parameter of the posterior is more determined by the data than the prior. Sometimes choosing a prior poorly doesn’t matter because there is enough data to swamp it.

    When the sample size is small and prior knowledge is limited, one can avoid overspecifying the prior with a non-informative prior such as a uniform distribution, maximum entropy distribution, or Jeffrey’s prior.

    Prior and posterior distributions are just the surface of Bayesian statistics. However, claiming to take a Bayesian viewpoint solely by the use of Bayes Theorem on simple probabilities ignores much of what Bayesian statistics has to offer to data analysis.

  21. JMB says:

    DrEads,

    Since you may be correcting me in my use of sciency terms, I would certainly appreciate any education you have to offer. My views tend to combine different areas of study, but often in incorrect ways. So here is my strange approach, please feel free to critique it.

    To me, the basic model in medicine is that we have an individual, the patient, that has several observable features (a set). They usually have a chief complaint (which may be a single feature or group of features, a subset). We ask questions to determine how those features have changed in time. The first issue the doctor tries to determine is what is bothering the patient, and how the set of bothersome features has changed in time. The second issue is to predict how the features will change in the future. So fundamentally, we are always dealing with how the observable features will change in time.

    The second issue begins to incorporate medical science. For our medical science we take groups of people with similar features, and determine how the distribution of features changes with time. So given the example of an ulcer in the bottom of the stomach or first part of the duodenum, we would observe that over time that group of patients who had similar features would change into a different distribution of features. Some will spontaneously resolve. Some will continue with no change. Some will develop what are new features not initially observed, anemia due to blood loss, infection or pancreatitis due to perforation, or even death due to blood loss or perforation. So now, armed with past observations, with have an expected change in distribution over time, in the untreated patient.

    For the third issue we consider how to change the expected future distribution to a more desirable distribution. Now we have various interventions that may or may not result in a change in the expected distribution over time. The measured difference between the expected result and the observed result after intervention may be considered information in the information theory formulation.

    In the natural course of the disease, we can consider the expected change in distribution of features to be that observed in a population of those without the disease. The information about the disease is how the observed change in the diseased population over time compares to the expected changes (which are the observed changes in the nondiseased population). Disease here is used to refer to the set of features in an individual in which we can find others with similar sets of features (a classification problem).

    So what we know a priori is both observed data (the features), but also some information (how the expected distribution of features will change in time whether we are talking about the group that does not have the disease, the group with the untreated disease, and the group with the treated disease.

    In my discussion, I tend to talk about a priori information, which includes observed data, and information concerning observed and expected (a type of causality). I tend to think that information theory lies closer to the Bayesian viewpoint than the frequentist viewpoint. Uncertainty, in my viewpoint, can arise from many sources (measurements, unidentified factors, chaos, population variation), but ultimately uncertainty affects our ability to predict how a patients features will change in time.

    So for the fourth issue, the practicing doctor must deal with the uncertainties of medical science in trying to decide the best intervention to recommend to the patient.

    My complaint about EBM emphasizing RCT above all other forms of a priori information is that RCTs break apart the initial set of observations on each individual patient, and perform statistics on the distribution of the observation, relying on the randomization process to avoid the ignored data skewing the result. To my way of thinking, they are discarding much useful information. Now the inefficiency of information use may be justified to prove (as much as can be proved) causality. So RCTs may be the best pure science. But when a doctor must make a decision about how to treat a patient, some of the potentially useful information has been discarded in the name of pure science. In terms of information theory, we have reduced information content in the pursuit of determining cause and effect. The information content we have discarded could still be useful in a decision made by a patient.

    In the case of John Lykoudis, he presented much information that would have been valuable to doctors in treating patients with PUD, but the information he tried to disseminate in medical science publications was not up to the standards of medical science. Consequently, the information was not widely disseminated. What I would suggest is that with the availability of the internet, more such information could be disseminated without the filter of peer review (for example, the SEER cancer database). However, we would have to train doctors in the use of such information and develop consortiums to make the data available and protect the privacy of the patient. For example, the Breast Cancer Consortium contains a wealth of (anonymized) data. If doctors were educated in how to use that data to make decisions, and given that a causal connection between screening mammography and breast cancer mortality reduction has been established by RCTs, then the properly trained doctor might be able to give a better prediction of harms versus benefits for an individual patient, than the prediction suggested by meta-analysis of RCTs. Although John Lykoudis lacked a full scientific explanation of benefit, there was enough a priori information to recommend it as a therapy until endoscopy and bacterial culture methods were developed to complete the scientific concept.

    So there you have it, my picture muddied by throwing in a bit of probability theory, information theory, and even some implicit computer modeling, to end up with a lot of brown. I do think that distributions all change with time. The only way we get a static distribution is if we observe it in a short enough time frame (a snapshot). I would assume that means that I adhere to the Bayesian viewpoint. My critique of EBM is that the methods are focused on reducing the uncertainty of cause and effect, not on reducing the uncertainty of a decision for an individual patient. If SBM expands the methodology of EBM, then hopefully the expansion will allow further reduction in the type of uncertainty faced when trying to give the best estimate of harms versus benefits for an individual patient.

    Another thought, if we are really to use a Bayes approach, then we should maintain the set of observations for an individual patient in our recording and reporting of data, as well as preserve it in any mathematical approach to assessing the data. The higher order terms in a Bayes probability calculation should not be discarded (at least now that we have fast computers with large memories).

  22. JMB says:

    Just to muddy the waters more, I kinda have my own definition of information in medicine. In traditional information theory, information content of a message is measured by the negative log probability of the message (log base 2). I would generalize information theory to medicine to be defined as the negative log probability of changes in measures of the state of an individual over a specified time interval. That concept or intervention that gives us the best prediction of a change in state then has the greatest information. Homeostatic mechanisms normally produce predictable variations which represent the base state of the individual. Disease processes produce variations from the base state that are lower probability, so identification of the disease process represents an increase in information content. If we can recognize the information that an intervention will result in a transition from less probable values back to the base state, then we have increased the information content of our knowledge. Informational efficiency in experimental design is the the power of the design both to classify which patients may benefit from an intervention, as well as predict the amount of benefit from an intervention. This changes the focus of experimental design from “is there scientific validity of cause and effect” to “how much information can we derive that will allow us to predict whether a given patient will benefit from an intervention.” If gene sequencing really becomes successful (if it becomes cheap enough and has sufficient predictive power), then I wonder if we won’t have to change our emphasis in medical scientific research to something more like information theory, reducing the importance of British empiricism.

    The Bayesian viewpoint would accommodate such a change in approach, but maybe the frequentist viewpoint would as well.

  23. DREads says:

    Hi JMB,

    Thanks for your very thoughtful and informative post. I did not mean to point fingers. My comment was not in response to your post but was meant to be a general comment. It is quite refreshing to see doctors like yourself (I am assuming you are a doctor) independently appreciate the limitations of certain statistical abstractions used to validate medical practice. Assuming a distribution is fixed over time has serious limitations, especially given that the real world is dynamic and changing.

    To me, the basic model in medicine is that we have an individual, the patient, that has several observable features (a set). They usually have a chief complaint (which may be a single feature or group of features, a subset). We ask questions to determine how those features have changed in time. The first issue the doctor tries to determine is what is bothering the patient, and how the set of bothersome features has changed in time. The second issue is to predict how the features will change in the future. So fundamentally, we are always dealing with how the observable features will change in time.

    Kudos for using the word “feature”. It is used in pattern recognition and machine learning in the same manner you use it. Patients are instances (sometimes called examples, points, or samples) and features are a set of variables to describe the instances. Features can be real valued (e.g. oral temperature in Celsius), binary (e.g. positive test result), ordinal (e.g. pain severity from 1 to 5), or nominal (e.g. male vs. female), or much more complicated entities such as time series (e.g. an EKG), imagery (e.g. an x-ray), or a volume (e.g. constructed from a CT scan). Some features may be missing (e.g. BP might not always be taken), subjective (e.g. patients interpret their pain ratings differently or describe their symptoms in different ways), or objective (e.g. blood-counting equipment calibrated to give counts within a certain error tolerance).

    There are hundreds of algorithms out there for doing prediction, modeling, and analysis. Each one deals with different data modalities and makes different assumptions. An algorithm is only useful if it makes guarantees about its performance such as bounds on the expected error. Just as the medical establishment is striving towards methodological coherency and consistency, people in my field are striving to develop and validate new tools to facilitate prediction and analysis for more complicated modalities. Unfortunately, people outside statistics, machine learning, information theory, mathematics, and computer science are often unfamiliar with modern statistical tools (like ones developed after 1960, no offense). It is refreshing to see doctors like yourself take interest. With more cross-disciplinary interaction, I hope EBM will eventually open up to adopting more modern methodologies without compromising its integrity.

    The second issue is to predict how the features will change in the future.

    There are multiple ways to view prediction when the distribution of features changes over time. In the 19th century, a mathematician by the name of Markov developed the notion of a stochastic process, which is an indexed collection of random variables. A generalization of stochastic processes, random fields, are so remarkably general that they can be used to describe distributions over the space of all images, the space of all videos, the space of all utterances, the space of all EKGs, the space of all CT volumes, etc. They often require a high level of detailed formalism and care that sometimes proving even simple results is difficult. However, their importance for validating new modeling and analysis tools cannot be understated. They will serve as a fundamental theoretical building block for modeling and prediction for many decades to come. People sometimes talk about the use of random fields/stochastic processes for all modeling and prediction tasks as a Holy Grail. In the mean time, simpler representations are often used so we can develop tools to tackle real world problems sooner.

    So, how do we predict when the subset of features of a patient change in complicated ways over time? Stochastic processes! Random fields! J/k. Online machine learning is a subfield of machine learning that assumes the distribution over features changes over time. In the case where the distribution is static, there is a notion of *risk*, which is just the expectation of the loss with respect to the distribution. In the online setting, we measure how well a predictor does by its regret, which quantifies how much lower the risk could be had a better decision/prediction been made. As new data comes in, a prediction is made, and the online learning algorithm updates the model to improve its future prediction performance.

    Just as medicine is driving itself towards consistency/coherence, theoreticians in online machine learning are striving to develop algorithms which provide guarantees such as regret bounds or bounds on the expected number of mistakes within a given period. Some online learning algorithms to take a look at include mixing past posteriors or weighted majority. I’d be happy to write more about them and their applicability to various prediction problems but I’ll save it for a future posting.

    My complaint about EBM emphasizing RCT above all other forms of a priori information is that RCTs break apart the initial set of observations on each individual patient, and perform statistics on the distribution of the observation, relying on the randomization process to avoid the ignored data skewing the result. To my way of thinking, they are discarding much useful information. Now the inefficiency of information use may be justified to prove (as much as can be proved) causality. So RCTs may be the best pure science. But when a doctor must make a decision about how to treat a patient, some of the potentially useful information has been discarded in the name of pure science. In terms of information theory, we have reduced information content in the pursuit of determining cause and effect. The information content we have discarded could still be useful in a decision made by a patient.

    Very good point. One major drawback of many RCTs is that they are oversimplified. Clinical trials are expensive and sometimes harm patients (within the limits of what is ethically acceptable, of course), so why not maximize their usefulness and informativeness? How can this be done without compromising the validity of a study’s conclusions?

    There is a rich landscape of new prediction and modeling tools out there. Many of them could be used to make optimal use of patient data from clinical trials. For example, clustering algorithms could be used to find the k-most similar patients to the patient in front of you.

    A single feature that changes over time can be represented as a time series. Patients with similar patterns to a given patient could be found with a with time series clustering or classification algorithm. Part of my dissertation explores some theoretical limitations of time series classification and proposes some algorithms for large time series data sets. One useful aspect of time series clustering is it enables you to discover classes in extremely large data sets, something that would be difficult for to do with just a human eye. You can also generate a dendrogram (like a phylogenetic tree) of time series so you can visualize how patients cluster together.

    What I would suggest is that with the availability of the internet, more such information could be disseminated without the filter of peer review (for example, the SEER cancer database). However, we would have to train doctors in the use of such information and develop consortiums to make the data available and protect the privacy of the patient. For example, the Breast Cancer Consortium contains a wealth of (anonymized) data.

    Good point. Hoarding data can greatly inhibit scientific progress. Unfortunately, since data takes money to collect, institutional barriers are often put in place to prevent sharing it.

    The Los Alamos HIV database is a good example of how widely disseminating information can help science. The database catalogs genotypes from samples collected around the world, and this has done wonders in learning more about how the virus mutates geographically. Without this information, ARV therapy would not be as effective. Some of the people involved in this project and the HIV vaccine are good friends/colleagues of mine. It is refreshing to know and see firsthand groups in medical science embrace technology to tackle elusive problems like HIV.

    If gene sequencing really becomes successful (if it becomes cheap enough and has sufficient predictive power), then I wonder if we won’t have to change our emphasis in medical scientific research to something more like information theory, reducing the importance of British empiricism.

    From my vantage point as a modeling scientist, the fields of computer science, statistics, information theory, mathematics, algorithms, and machine learning are pretty blurred these days. Today, to be an effective data analysis/prediction researcher, you must be very familiar with the latest contributions in all or most of these fields (analysis and modeling fields). Information theory is just one building block, and an important one, but not a unified solution for everything.

    As technology progresses, most fields in science are becoming increasingly drowned with data. Many analysis methodologies in use today were developed when science was starved of data and analysis could be carried out on scratch paper. Today, this is no longer true. If the discovery of novel science is to keep pace with technology, we will need to adapt the way we analyze our data. If EBM is to keep pace, it will need to start adopting more modern modeling, prediction, and analysis tools.

    Cheers,

    Damian

    PS: I’m still a Mr.–the DR in my screenname are my first two initials. Although I will graduate soon, I won’t be a medical doctor so perhaps I should change my screen name so others don’t accidentally interpret my posts as medical advice. :)

    PPS: In case you’re interested, the other focus of my dissertation is machine learning for object detection, which studies how to automatically learning how to predict the location of objects in images. The quality of predictions made by an object detector is ill-defined–it depends on what the object detector is being used for. Very often, people do not carefully define what a good object detector means for their problem. In my research, I showed that if you change the definition of an object detection problem slightly, the ROC curve (illustrates the true positive vs. false positive rate) can change significantly. This motivates the need to define your problem well, choose a criteria that’s well matched with the problem, optimize on that criteria so you have a chance of getting the best solution possible, and validating on that criteria to show you haven’t overfitted.

  24. JMB says:

    Thank you very much for your very informative response. I am a radiologist with a background in medical decision making and digital image analysis/manipulation. I have experience with programming the Fast Fourier Transform for calculation of holograms, programming probit analysis in machine language (ugh), programming expert systems, Bayes strategy for decision making, nearest neighbor analysis, neural networks, Markov Chain Monte Carlo simulations, deconvolution techniques, computer aided diagnosis, computer assisted education, and quality assurance data collection. I have a broad experience partly because I was mostly self taught (my only successful hologram was a simple tetrahedron). Although I have had assistance from experts at key points (such as the production of the hologram onto film), I lack formal classwork in most of those areas. Consequently, my ideas tend to be pretty rough around the edges. I appreciate the responses that help me refine away those rough edges.

    Using the SEER cancer database, I was modeling how various experimental designs in RCTs would result in a distribution of sampling results compared to the population result. I would do a randomized selection of cases from the SEER database using a criteria drawn from published results (for example, the size/grading of breast cancer in a particular age group), and then using the recorded outcomes for the individual patients in the database calculate the simulated results of their experimental design. Repeating this simulation over and over using the large database resulted in a distribution of results from the sampling process. I could test how different decisions (such as inclusion/exclusion), and simple luck of the draw at the outset of the experiment would affect how close to the population statistic, the sampled statistic would be. It was an eye opener to see how often our statistics and experimental design could lead us to a wrong conclusion. The 95% confidence interval failed more than 5% of the time. So I would definitely agree with you that our methods are out of date. It would be wise to use computer modeling to test the experimental design of RCTs before they begin, because of the expense involved.

    I got out of academics partly because I was told that my work in AI/Medical Decision Making wasn’t going to attract any research money, so I needed to focus more on those projects that would be funded (in my case, data compression of medical images and reports).

    Back then, doctors weren’t interested enough in how we could modify medical research and medical practice to be more efficient in advancing healthcare (by changing experimental design, data encoding, and statistics/modeling). Perhaps times have changed.

    At least one of my ideas got some notice. I advocated using coded reports to establish a nationwide database to assess outcomes of breast cancer screening. My idea was to identify mammogram interpreters who were substandard, and try either further training or restriction of practice. At least the idea was cited in a Surgeon’s General Report as one more reason to adopt the BIRADS classification (I did not write the BIRADS classification, I did promote the idea of using the ROC strategy of classification that matches BIRADS I-V). Unfortunately, the national database was never established.

    So I hope you will have more success than me. I am glad you take an interest in statistical methods in medicine, but it will take patience and perseverance to make a difference.

Comments are closed.