Articles

Homeocracy

Recent comments on homeopathy again resulted in references to the 1994 Pediatrics paper by Jacobs et al on use of homeopathic remedies for childhood diarrhrea.  The authors of that paper concluded from their blinded study that homeopathic remedies, tailored to the individual infants and children, were effective in reducing the number of diarrhea stools and thus in shortening the illness.  The paper has been widely referenced and reported to have proved  homeopathy efficacy, and the critiques have been argued by homeopaths as irrelevant, as has been done by Mr. Ullman.

In this short series I want to recount my experience with the report, its predecessor, and its two major sequellae, as well as its effect on systematic reviews of homeopathy.  I will begin with a description of the first two studies, concentrating on their methods. Then I will discuss the results of the 1994 Pediatrics report and the authors’ interpretation of the results. Then (I hope last) I will discuss the third paper which the authors claimed supported findings of the first two and the meta-analysis which combined data from all three. If you are imagining why this  series interests me, and imagining the worst, you are probably right. The sequence will  help to reveal how some of the information in “holistic” and “alternative” systems become published, and despite critiques and disproofs through a sort of systematic deconstructions, still develop wheels of their own and enter the fund of general knowledge seemingly forever. Or, at least for several decades, until a social belief switch is finally turned off.

These effects have in turn, political and economic effects, that are of more urgent concern now as promoters lobby to have sectarian methods covered under insurance and federal medical plans.  Probably as important, is that these reports will illustrate a different way of looking at the literature of implausible propositions.  To destroy the suspense and let you in on the idea, the idea goes something like this: We usually look at positive reports on quacky methods to find falsification or fatal errors or hidden or absent data that disprove the thesis.  Those are satisfying  when present. But there is an alternate route;  disqualify or at least diminish the meaning of a study because of the way it was set up, carried out, and/or reported, and the way that the worthless outcome data were subsequently used (the garbage in side.)  The garbage pile as it accumulates, results in higher degrees of unbelievably – as it lacks both authenticity and validity.  The focus must then turn to the statistical methods and how they are used to (mis)interpret the pile of data.

What often happens here is that  the authors apply statistics designed to analyze data that are collected from experiments that are carried out in uniform ways, with reliable methods and instruments, in order to  distinguish or characterize plausible and measurable phenomena.  When one measures imaginary phenomena, then the results do not tell anything about the phenomena, but about the experiments themselves – and the errors in the experimental and measuring system.  That is where I’m going with this. My analysis will show that the data collected were so heterogeneous and random, and the measuring so subjective, that the results were meaningless (also garbage.)

The affair began with Jacobs’s report of a pilot study published in the British Homeopathic Journal in 1993 (1). I was unaware of this report until I read the next one in Pediatrics, 1994. (2).  I, with co-author Bill London, a health educator with the Natl. Council against Health Fraud wrote a five page analysis of the second paper. That came about when a nursing educator in the Pharmacy dept. at the teaching hospital where I taught medical oncology caught me in a hallway and asked me to examine the report which concluded that homeopathy worked for childhood diarrhea. She and  our head pharmacist knew of my interest in medical quackery,. She did not know how to explain the report to her students.   So I bit on it.

It was more difficult than I thought, for several reasons. First, I was familiar with homeopathy, but not as familiar with its modern practice. I learned that homeopaths use a computerized system for selecting remedies. More problems came after those.

I contacted the editor and asked if he would consider publishing a critique, considering the potential impact – treatment of millions of children in underdeveloped nations, and the implausibility that homeopathy worked. The editor was prompt, replying yes, partly because – as we later learned – he felt his job was on the line for having accepted the Jacobs manuscript, so vigorous was the negative response  from readers.

As for the trial setups, we assumed they were the same for all three trials, although the amount of details reported varied. We found unanticipated problems. The diagnoses – other than diarrhea – on which remedies were selected were criteria based on symptoms only. Here are sample criteria from the table.  They were divided into those of “mind,” “general,” and “stool” (quality).

Mind:  Great anxiety and restlessness, fearful,
Capricious; irritable; quarrelsome, nothing p[leases, striking out, moaning, frenzied…
Slow, lethargic, weakness of memory (in a 1-2 year old?)
Diarrhea after mental excitement. Fidgety and restless.
Irritable, indifferent, weeping.
Stools:  Acrid, burning, excoriating, worse after midnight,
Green, slimy, offensive, chopped grass, … smelling like rotten eggs, …
Slimy, scanty, lienteric, bloody. Acrid, corrosive, frothy, yellow. Sour odor.
Profuse, frequent, gushing,  painless, watery. Bloody with green mucus, offensive. Rectal
prolapse. Exhaustion after stool…
Worse at night, after milk, involuntary, sudden expulsion. Worse 5 AM (sic) Red ring around
anus. Offensive, acrid stools. Painless, sour; thin; watery. Odor of rotten eggs.
General: [only the first criterion for each collection of criteria for brevity)
Prostrate,
one cheek red, the other pale;
Chills/heat;
gagging/ retching;
Cold sweat face and feet

Each combination (of each category - first to fifth - of the above called for a different homeopathic remedy.

Not mentioned was how each criterion combination / remedy was determined. Apparently by prior “provings.”  But “proving” have never been proved to our knowledge, by repeated trials - blinded or not.  So already we are in deep trouble. The criteria overlap, are subjective and made by a second party - often a relative - in answer to specific questions. Each answer depends on memory for the prior 1-2 days, powers of observation, sense of small, frequency of diaper change, and so forth.

Thus we have:

  1. A glut of information, all highly subjective and subject to multiplicity of error sources.
  2. A data base not proved for validity or consistency.
  3. Overlapping criteria (admitted by the authors).

The next problem is the selection of timing.  The initial visits, had to have been after 2 days of illness, but there was no specification of at which point the questions were asked.  Thus, the quality of the stools, the state of “mind“ and the general appearance could have been at any time after the inclusion period. Given the large variation in those qualities from hour to hour and day to day,  the chosen remedy was based also on the time of the answer, a “snapshot in time” upon which the next few days’ remedies were chosen and given. One can imagine the wide descriptive differences obtained, yet all were treated as homogeneously obtained. Yet these data were treated with statistics as if those descriptions and remedies were tightly controlled.

The potency of all selected remedies was 30C…that is, 1X 10/\ -300.  How come?  We have not even begun to deal with the problem of the homeopathic dosage system yet.  Don’t need it for this part of the discussion.  In order to grasp the significance of what the homeopaths did in these clinical trials, I got the idea yesterday to analogize the problem in physics terms.  With apologies to my physics profs and associates, I tried it this way (and please feel free to critique this as I have not run this by anyone else…instant peer review!)

I imagine a physics experiment done under similar conditions.  Test the influence of variable doses of electricity on a series atomic particles - electrons, protons, whatever - with their energies and “spins” varying to hundreds of variations, with the “dose” of the electric field chosen from a previously set of criteria which had been tested only once or twice for effects on a different set of particles - the effects recorded and entered into a book  (The Organon) and a modern computer data base. Then select a random “dose” or power, charge, etc., to be applied to each test particle at random times after the particle had traveled a minimum distance from its source.  Then measure the effect on each particle at four defined times, record all of the values as a group at each time, then analyze them with a statistic.  If a pattern shows up, would it mean anything?

I guess homeopaths are right in a sense when they claim that standard science cannot measure what homeopathy does. Nothing can. Especially as their computer cannot organize the data, laden with close to randomized bits.

Homeopaths are caught on the point of their own petard here. They claim to be able to select remedies based on previous proving as recorded in “The Organon” in its several iterations, and the information placed in a computer program based on that information. The information going in is in reality “disorganized” or “unorganized” to play on a few words, and no amount of excusing or rationalization can organize it.

Thus, the outcomes are nothing more than random bits of information, even if carefully measured. And therefore, any emerging patterns do not indicate efficacy or harm, but only normal variations of random patterns.  They measure only the noise in the system.

Enough mental exercise for this time. Next I’ll look at the second paper’s outcome data and what the authors did with it, and move on to the third paper’s adventure in the journal system, then on to the meta-analysis.

REFERENCES:

  1. Jacobs J, et al, Homeopathic treatment of childhood diarrhea. Brit Homeop J, 1993;82(2):83-86.
  2. Jacobs J et al. Treatment of Acute Childhood Diarrhea with Homeopathic Medicine: A Randomized Clinical Trial in Nicaragua. Pediatrics, 1994;93(5):719-724.

Posts in the series:

  1. Homeocracy
  2. Homeocracy II

Posted in: Basic Science, Clinical Trials, Homeopathy

Leave a Comment (25) ↓

25 thoughts on “Homeocracy

  1. Dr Benway says:

    If a pattern shows up, would it mean anything?

    HAMLET: Do you see yonder cloud that’s almost in shape of a camel?

    LORD POLONIUS: By the mass, and ’tis like a camel, indeed.

    HAMLET: Methinks it is like a weasel.

    LORD POLONIUS: It is backed like a weasel.

    HAMLET: Or like a whale?

    LORD POLONIUS: Very like a whale.
    ____________________________

    The brain is infinitely clever at finding patterns in noise. Ergo, science’s requirement that one predict the pattern first before looking at the data.

  2. overshoot says:

    The physics example isn’t particularly good — among other things, you can’t apply “electricity” to elementary particles.

    Chemistry, offhand, seems a better bet.

  3. DVMKurmes says:

    Interesting. This comment is slightly off topic, but a group of aussies and brits just published this paper on PLoS;

    http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0005192

    they have developed a mathematical model that explains why ineffective treatments are likely to spread in a population while effective treatments may not.

  4. Joybobington says:

    Would that work with regular drugs?

    Random drugs, random dosages?

  5. Wallace Sampson says:

    Dr. Benway: That’s close to what I used to use to demonstrate random patterns to a class, then pass around a small bottle of nonpareils. On target, pertinent – thanks.

    All: Apologies – I had a brain collapse. 30C is not 1X10/\-300 but 1X10/\-3 000. Well, does it matter?

    overshoot: Sorry for the “electricity” flub. I meant to express a setup like the quantum experiments that alter the spin on one of a pair of particles by passing it through a polarizing mechanism. Or whatever its called at bad hours and hurried thinking. I can try to think up a chemistry analogy…can’t… think..now… Problem is that the data for the data base for the computerized answer were themselves little more than random.

    DVM Kurmes: Thanks for the ref. plos has a wealth of stuff, and should be on everyone’s review list – if there were time. Next project: How to understand the math of their model.

    joybobington: Would that work with regular drugs? Random drugs, random dosages?
    I guess just as well. In creating analogies for understanding I think it helps to use subjects more removed from the one under discussion…but one always loses a degree of similarity with each step. Or did you mean “really work?”

  6. Wallace Sampson says:

    Also,someone will also recognize the alternate explanation for reporting significance in the study… selection of end point or fabrication or even adulteraton of the tst matrials. Dealt with in the next installment.

  7. overshoot says:

    I had a brain collapse. 30C is not 1X10/\-300 but 1X10/\-3 000. Well, does it matter?

    Alas, no. It’s 10^-60

    Way, way, WAY past Avogadro’s number of course but 10^-300 is beyond the number of particles in the Universe by a good bit, too. 10^-3000 is, if you’ll pardon me, a whole ‘nuther order of magnitude in narishkeit.

    By all means get some rest. If you want to toss a chemistry (or other) analogy around we can do it on-board or off.

  8. DanaUllman says:

    Wallace Sampson’s blog is a classic example an inadequately informed and misinformed statement about homeopathy and some of its research.

    First, Sampson assumes to know something about homeopathy, but as he acknowledges, he is “not as familiar with its modern practice.” I appreciate that honesty, though it is astonishing how little he seems to know about homeopathy, despite many decades of debate with me (and others) on this subject.”

    The good news is that Sampson accurately acknowledges that homeopaths do not necessarily give the same drug to each patient with the same disease. He acknowledges that we also seem to have some interest in the patient him/herself, though he makes this actually sound “weird” as he notes that we ask about different characteristics of the person’s “mind,” “general” symptoms and “stool.” Excuse me? What IS the problem here? This is simply good medicine. Do you NOT want to know about your patient?

    Further, there is a logic behind looking at different SYNDROMES of disease. It quickly becomes obvious that people with the same disease have idiosyncratic versions of it. More important, there is a certain logic to then finding a drug that would actually cause the similar “syndrome” of symptoms in healthy people as a way to help provide immunological support for helping the sick person. Instead of using drugs to SUPPRESS symptoms, the homeopathic method is to find a drug that would mimic them. Homeopaths then observe that patients are hypersensitive to substances that have the capacity to cause the similar symptoms that they are experiencing.

    This next part is where a 1st grade education in homeopathy is missing. Sampson asserts, “Not mentioned was how each criterion combination / remedy was determined. Apparently by prior “provings.” But “proving” have never been proved to our knowledge, by repeated trials – blinded or not.”

    First, homeopathic textbooks called “material medica” (Latin for “materials of medicine”) are based on toxicology reports AND “provings.” Hmmm…”provings” (as called “drug provings”) are single- or double-blind trials in toxicology, where human volunteers are given a crude OR potentized dose of a medicinal agent until symptoms become manifest. These symptoms are then recorded in material medica (and now on expert system software).

    Wally really goes in for the kill when he questions the ability of the patient’s intelligence. He notes, “Each answer depends on memory for the prior 1-2 days, powers of observation, sense of small, frequency of diaper change, and so forth.”

    Thanx…but not every clinician has such disdain for patients.
    Wally again shows us how little he knows when he says, “Homeopaths are caught on the point of their own petard here. They claim to be able to select remedies based on previous proving as recorded in “The Organon” in its several iterations, and the information placed in a computer program based on that information.”

    Sorry…but “The Organon of the Medical Art” was Hahnemann’s seminal work on the philosophy and methodology of homeopathy. It has NO material medica in it. “The Organon” went through 6 editions as he continued to refine his understanding and to record his experiences.

    As for the research about which he was critiquing…first, Sampson shows that he has not kept up on this research by not acknowledging this meta-analysis of their work of three studies.

    Jacobs, J, Jonas, WB, Jimenez-Perez, M, Crothers, D,
    Homeopathy for Childhood Diarrhea: Combined Results and Metaanalysis from Three Randomized, Controlled Clinical Trials, Pediatr Infect Dis J, 2003;22:229-34.

    And for the record, the study in PEDIATRICS (1994) actually found that children with diarrhea who were found by the laboratory to be infected with a known pathogen were found to have a statistically significant effect, as compared with those given a placebo. Further, they show that children get over their diarrhea a full day earlier. The research on Tamiflu shows at best a one-day improvement over placebo. Are you not going to recommend that a parent help his/her child who is suffering?

    And by the way, several million children in 3rd world countries die each year from diarrhea. My public health background tells me that we need simple and safe solutions like homeopathy…

  9. Citizen Deux says:

    Childhood deaths from diarrhea in third world nations are in a wholly different class from developed nations. The identified causes of diarrhea range from pure viral / bacteriological vectors to excess of Vitamin C.

    The point, well made by Dr. Sampson, is that the study had no control method to distinguish the presntation of symptoms and provided no evidence of efficacy based upon a course of treatment.

    Please provide proof that homeopathy works on even one of the most basic causes of diarrhea, say giardia, and can do so reliably. As a public health expert, you are no doubt aware of the impracticality of conducting an exhaustive examination of patients when widespread dysentery strikes a community, thus an effective regimen is required.

    What does homeopathy offer in this arena?

    How to Make a Homeopathic Remedy

  10. David Gorski says:

    Dana, Dana, Dana….

    I’ve dealt with that whole nonsense about homeopathy and childhood infectious diarrheal diseases before:

    http://www.sciencebasedmedicine.org/?p=82

    Wally is also more than familiar with these studies. In fact, he’s more familiar with them then even I am.

  11. Scott says:

    The inability to recognize that someone who says “I was not familiar with X” is NOT saying that they are currently unfamiliar reflects quite poorly on a person’s arguments.

    If Dana can’t comprehend simple English, why should anybody place any credence in his other claims?

  12. Mjhavok says:

    Homer: Not a bear in sight. The Bear Patrol must be working like a
    charm.
    Lisa: That’s spacious reasoning, Dad.
    Homer: Thank you, dear.
    Lisa: By your logic I could claim that this rock keeps tigers away.
    Homer: Oh, how does it work?
    Lisa: It doesn’t work.
    Homer: Uh-huh.
    Lisa: It’s just a stupid rock.
    Homer: Uh-huh.
    Lisa: But I don’t see any tigers around, do you?
    Homer: Lisa, I want to buy your rock.

  13. Dr Benway says:

    I don’t see the point in reading Mr. Ullman’s posts until he proves to the chemists that homeopathic remedies can be distinguished in some way from mere water.

  14. Chris says:

    After looking at the archives (here and elsewhere) and how he avoids answering direct questions, and him pulling out the same tired stuff that even after it has been discredited: I assume you will never read his stuff again.

    You should be interested to know that Dana Ullman now claims that homeopathy can help with the head trauma of the type that killed Natasha Richardson.

    It is being discussed on the Bad Science Forum. The reaction there is less than complimentary.

  15. Dr Benway says:

    Wow. What a cunt.

  16. Dr Benway says:

    Oops. I just tried to lower the discourse but was thwarted by the filter.

    2nd try…

    Wow. What a cunning bunt.

  17. Eric Jackson says:

    Took a look over at the Bad Science Forums, and found one particular post that was too much to resist, and quite applicable to this current entry:

    “It’s a pilot study. Homeopaths seem to do nothing but pilot studies. At the end of these studies, instead of assessing and improving their method, estimating an effect size and working out the required power for the real study or selecting an appropriate null and alternative hypothesis they just get on and make conclusions appropriate to the full study. There is then no actual study to follow up the pilot.

    The pattern seem to me to be as follows.

    1. Do a small, rubbish study, publish it as a pilot study so none can really complain its rubbish.
    2. Draw tentative positive conclusions that are not really born out by the data but couch them in speculative terms.
    3. Release a press release announcing new and conclusive evidence for the efficacy of homeopathy.
    4. Add study to list of positive evidence for homeopathy and circulate to Hpath students, practitioners and true believers.
    5. Complain these studies are not included in meta-analysis. Use this as further evidence of conspiracy against homeopathy.
    6. Do another small rubbish study in an unrelated area to the first study.
    7. as 2 ”

    Thank Derrik for this gem: http://badscience.net/forum/viewtopic.php?f=3&t=8649#p168734

  18. pec says:

    “A randomized double-blind clinical trial comparing homeopathic medicine with placebo”

    “the treatment group had a statistically significant (P < .05) decrease in duration of diarrhea”

    You have not explained why a difference was found if homeopathy doesn’t work.

  19. Eric Jackson says:

    pec: Actually he explained that quite clearly. The next post was also to go into detail on the data itself.

    The basic gist of it, is if you collect data with nonsensical collection criteria, from a badly defined population, and then administer treatments that overlap, you have not only compounded the error, but made it unimaginably larger. Add in parental memory as a collection method and you’ve just made an enormous mess.

    If you use statistical tests on bad data, what you get out doesn’t mean anything. The old computer phrase Garbage In, Garbage Out very much applies (http://en.wikipedia.org/wiki/GIGO)

  20. pec, your University of Michigan website link is just a recounting of the same article. How does it help the conversation? But even in that website, they make a singular comment that both you and the infamous Dana, will never be able to explain:

    If indeed effective, there is still no proven mechanism of homeopathy.

    Zero molecules in solution make it just water. Since there is no physics, chemistry, biochemistry or any other science that can explain how zero molecules of anything can have any effect (well, maybe the water itself does), everything else is irrelevant, including this study. It’s kind of humorous (no, just plain hysterical) that the total clinical proof that homeopathic potions have any usefulness is one weak study published 15 years ago.

  21. David Gorski says:

    Ugh.

    I graduated from U. of M. for both undergrad and medical school. It deeply shames me that any U. of M. faculty would say there’s any role whatsoever for the One Quackery To Rule Them All (homeopathy) in any disease and actually publish it on a university website called “evidence-based pediatrics.”

    The pain…the pain.

    Now I know how Kim feels about Harvard.

  22. But UofM occasionally plays good football. Oh wait.

  23. Badly Shaved Monkey says:

    Linguistic point: a petard is a bomb for breaching walls. Being hoist by one’s own petard means being blown into the air by it when it goes off prematurely. It is not a spiked weapon such that being hoist by it means being lifted up by one of those spikes.

Comments are closed.