Articles

Ginkgo biloba – No Effect

Another one bites the dust.

The National Center for Complementary and Alternative Medicine (NCCAM) is generally a waste of taxpayer money, but they have sponsored several well-designed large trials of popular herbal supplements. And one by one these studies have shown these popular products, such as echinacea for the common cold, to be ineffective.

To add to the list, published in JAMA this week are the results of the largest and longest trial to date of Gingko biloba for the improvement of cognitive function and to treat, prevent, or reduce the effects of Alzheimers disease or other dementia. The results of the study are completely negative.

The study was very rigorous – a consensus trial designed to address all the criticisms of prior smaller studies. It was a direct comparison of Gingko biloba at 120mg twice a day, double blind, randomized, multi-center trial involving 3019 subjects aged 72-96 for a median of 6.1 years. Subjects were followed with standardized tests of cognitive function.

The results are easy to report – every measure showed no difference between G biloba and placebo. There was no difference in cognitive function, risk of developing dementia, rate of progression of dementia or normal cognitive decline with aging. Usually such studies involve some random noise in the results, especially when several outcomes are measured. But with such a large study, random fluctuations should average out, and that is exactly what happened.

Gingko biloba has been used for centuries as a medical herb, and the most popular claim made for its use was to enhance cognitive function. The justification for this claim was always thin – G biloba has a mild blood thinning effect. It was therefore claimed that the herbal drug would enhance blood flow to the brain and improve brain function.

However, this is not a plausible mechanism. The brain exquisitely regulates its own bloodflow, and suboptimal perfusion results in a widening of the blood vessels to increase flow. This autoregulation would not be enhanced by mild blood thinning in a healthy individual.

In someone with severe atherosclerosis or narrowing of the arteries, to the point where autoregulation of blood flow is not able to optimize perfusion of brain tissue, the result is typically stroke-like symptoms. In this situation a blood thinner may improve perfusion, and generally drugs like aspirin are prescribed.

This, of course, implies that if simple blood thinning could improve cognitive function, then aspirin would be more effective at this than G biloba. In any case, this putative mechanism was never very plausible.

More recent studies have found that G biloba has some anti-oxidant effects. However, anti-oxidants as a class have not been found to be effective for cognitive function or any other clinical outcome. So this too lacks plausibility.

It was also found that G biloba can reduce amyloid aggregation. Amyloid plaques build up in the neurons of patients with Alzheimers disease, so this is a plausible mechanism for slowing the progression of some forms of dementia. However, as we now know this does not translate into a measurable clinical effect.

The lessons from this study and the lack of effect for Ginkgo biloba should learned and generalized.

Historical use of an herbal drug is not sufficient evidence for its effectiveness.

Preliminary, small, or poorly designed studies are unreliable, and often result in false positives. Only large definitive trials are reliable.

Finding a potential mechanism for a drug, herbal or otherwise, is not a sufficient basis for a clinical claim – you need clinical trials with actual people to support such claims. Further, if researchers go looking for potential mechanisms to explain a putative action of a drug or supplement, it is not surprising that they will find some. Drugs typically have many biochemical actions in the body, and finding an effect is not surprising. There is also likely confirmation bias and the file-drawer effect at work – favoring the publishing of interesting and positive studies.

In the end – all the ancient wisdom, small studies, and putative mechanisms meant nothing. They were all trumped by a large and impeccably designed study that shows Gingko biloba is of no measurable benefit for cognitive function.

These results call into question the practice in many countries of allowing pharmacological agents like G biloba to be marketed as supplements with health claims prior to being adequately studied. The European and US markets for G biloba are in the hundreds of millions of dollars per year. It will be interesting to see what happens following this study.

The study did find that G biloba was generally safe. However, it should be noted that G biloba, although sold in the US as a supplement, should be considered a drug. It does have an anti-platelet blood-thinning effect and should not be taken prior to surgery. However, because many people think of herbs as supplements and not drugs, patients rarely disclose their supplements to their doctors, and doctors fail to take a supplement history. Safety is therefore still an issue.

Herbs and botanicals have been and can be a valuable source for useful pharmacological agents. However, regulating and using them as supplements has many flaws – as the history of Gingko biloba once again highlights.

Posted in: Herbs & Supplements

Leave a Comment (59) ↓

59 thoughts on “Ginkgo biloba – No Effect

  1. windriven says:

    “It will be interesting to see what happens following this study.”

    I suspect the answer will be: not much. Wishful thinking dies hard. The take will be that the study was flawed: the dose was wrong or the tests used to evaluate cognitive function were biased. Or simply that Big Pharma is out to get the sweet, green, crystal-gazing purveyors of magical herbs and powders.

    According to USA Today gingko biloba is about a $98 million business in the US. Big Herba isn’t going to cede that without a fight.

    The supplements business has at least one journal of its own: Nutrition Business Journal. I was going to subscribe to get an insider’s view of the industry. But my curiosity couldn’t scrape together the $1200 for an annual subscription. I don’t think that Buffy and Feather Dawn, packaging herbal remedies in their basements are springing $1200 a year to keep their fingers on the pulse of the industry.

    Ignorance may be the seed of quackery but $ is the water and fertilizer that makes it grow.

  2. Plonit says:

    And one by one these studies have shown these popular products, such as echinacea for the common cold, to be ineffective.

    +++++++

    Is echinacea the right example to use, given that there do appear to be sound scientific studies that echinacea reduces the period of cold symptoms?

    http://www.library.nhs.uk/rss/newsAndRssArticle.aspx?uri=http://www.library.nhs.uk/resources/?id=262678

  3. windriven says:

    @Plonit

    The Shah study that you cite is a meta-analysis and the abstract alone isn’t enough to judge the strength of the underlying studies.

    At a minimum I would argue that the use of echinacea in treating the common cold is controversial. For instance Barrett, et. al. published an RCT, n=148. in Annals of Internal Medicine (Ann Intern Med. 2002 Dec 17;137(12):I18. ) and concluded that “echinacea provided no detectable benefit…”

    One can find studies showing benefit and studies showing none. Absent a large scale RCT with a preparation of known purity, how is one to judge? And think about the difficulties of such an RCT. Echinacea purpurea is a plant. I am not aware of any published studies clearly identifying the supposed active compound in echinacea so how are the researchers to assure the potency of the preparation? And if researchers cannot, how do the manufacturers or, for that matter, the consumers?

  4. Plonit says:

    If you have access to the Lancet Infectious Diseases (I do) then you can read the whole study. It seems a fair study to me, though there are letters to the editor raising questions – addressed by the authors, you can judge for yourself.

    There are similar findings in the Cochrane review

    http://mrw.interscience.wiley.com/cochrane/clsysrev/articles/CD000530/pdf_fs.html

    (again, I have access to the full review, though I gather this is not available in the US).

    When it comes to echinacea for the prevention or treatment of common colds, the evidence of benefit is not overwhelming, and the authors of these metanalyses don’t claims beyond that which the evidence supports, e.g. from the Cochrane review

    “We reviewed 16 controlled clinical trials investigating the effectiveness of several different Echinacea preparations for preventing and treating common colds. Two trials investigated whether taking Echinacea preparations for 8 to 12 weeks prevents colds but found no clear effect. The majority of trials investigated whether taking Echinacea preparations after the onset of cold symptoms shortens the duration or decreases the severity of symptoms, compared with placebo. It seems that some preparations based on the herb of Echinacea purpurea might be effective for this purpose in adults, while there is no clear evidence that other preparations are effective or that children benefit. Side effects were infrequent but rashes were reported in one trial in children.”

    But weak evidence of benefit is quite different from stating that it has been shown to be ineffective – which is what the post here states.

    It is perhaps useful to bear in mind that there is nothing inherently implausible about plants having some biological action (unlike, say, homeopathy). We are, after all, just talking about a form in which chemicals are delivered to the body – which may or may not work, and which may have various interactions and toxic effects.

  5. CW says:

    Echinacea was never intended to treat the common cold though. A Swiss supplement manufacturer was mistakenly told that Native Americans used it to treat colds. But Native Americans used it for pain relief, and only because they observed elk or bison eating it when they suffered from some wound. There’s not even accurate anecodotal evidence to support the echinacea-cold treatment claim, let alone several lines of evidence-based studies.

  6. Plonit says:

    Echinacea was never intended to treat the common cold though. A Swiss supplement manufacturer was mistakenly told that Native Americans used it to treat colds. But Native Americans used it for pain relief, and only because they observed elk or bison eating it when they suffered from some wound.

    ++++++++++

    Ermmmm, can I quote the Novella on this – “Historical use of an herbal drug is not sufficient evidence for its effectiveness.”

    How echinacea came to be regarded as useful for treatment of the common cold is irrelevant to whether it is or is not effective. The point is that it has not been “proven ineffective” – which was Novella’s claim above.

    Obviously, it is for the proponents of echinacea (of which I’m not one) to demonstrate not only that it is “not proven ineffective” but that it is actually safe and effective (a different thing entirely).

    The same is true of proponents of oral pseudoephedrine, of course.

  7. Joe says:

    Plonit- you cited newspapers, that won’t cut it. The latest and best RCT (Turner, Ronald B., et al NEJM Volume 353(4), 28 July 2005, pp 341-348) showed no benefit. See also commentary by Sampson, Wallace NEJM Volume 353(4), 28 July 2005, pp 337-339. Sampson’s comment concerns something relevant to today’s post- Echinacea was not traditionally used to treat colds, and I doubt Ginkgo was traditionally used to treat dementia.

    James A. Duke has an enormous ethnobotany database and it does not list dementia as an indication for Ginkgo http://www.ars-grin.gov/cgi-bin/duke/ethnobot.pl One could wonder whether Alzheimers was known to the ancients. As a result, pleas to the antiquity of an herb do not support many uses in promoted by modern herbalists.

    When herbalists turn to ethnobotany to support the use of an herb, there is a further problem of identifying what the shamans use it for. One cannot go into the Amazon and ask a shaman about any particular disease, they don’t have the terminology or diagnostic, physiologic and anatomic understanding. If a shaman does describe a disease very clearly, the botanist is unlikely to recognize anything other than the most common ailments.

    In short, claims to ancient and traditional uses of herbs have to be held in suspicion beyond the fact that we know they were not properly studied.

  8. windriven says:

    @Plonit

    “It is perhaps useful to bear in mind that there is nothing inherently implausible about plants having some biological action…”

    Absolutely. Acetylsalicylic acid was first prepared from, as I recall, willow bark. Many plant derived, biologically active compounds have been identified. But this is part of the problem with echinacea. What specifically is the compound supposed to be responsible for ameliorating cold symptoms? If that is not known, how can echinacea be tested? Not all plants are the same – ask any marijuana smoker. The THC level varies. How can one assess the value of a therapy if one cannot standardize it?

  9. Echinacea is a perfect analogy to Gingko.

    The preliminary data is mixed and weak at best.

    There is even less prior plausibility than Gingko – not because herbs cannot work, they are drugs and potentially can have useful effects. But because, as noted, the claim that echinacea treats the common cold was essentially made up out of whole cloth – it would be an amazing coincidence if it actually worked for the cold.

    There have been two large NCCAM funded clinical trials of echninacea – both negative.

    So we have low quality mixed evidence, and the best evidence is negative, for a claim with low prior probability.

    The fact that proponents have not given up is of no consequence to the scientific claim.

  10. Plonit says:

    Actually, I linked to the NHS Evidence Health Information Resouces “Hitting the headlines” summary, which included discussion of the way in which the Shah’s meta-analysis was presented media. The purpose of those summaries is described here http://www.nhs.uk/news/Pages/SirMuirGraysBiography.aspx (“Hitting the headlines” has been replaced by “Behind the headlines”). As I’m not in the cold business, I would not have any reason to read Shah’s meta-analysis except for the purpose of arguing the toss here. However, I don’t have a particular reason to be strongly skeptical about the summary provided by the NHS EHR.

    Shah’s meta-analysis is easily located at The Lancet Infectious diseases. Ditto the Cochrane review that I also cited.

    How is the fact that echinacea was not traditionally used to treat colds relevant to today’s post?

  11. Plonit says:

    The preliminary data is mixed and weak at best.

    +++++++++++

    Is that a different claim from “proven ineffective”? I would think so, but perhaps there is some logical step I’m missing.

  12. FelixO says:

    Reading the reaction to Plonit’s original post it does concern me that that the general impression of the replies is one of ‘dismiss’ rather than ‘discuss’.

    WINDRIVEN:
    “The Shah study that you cite is a meta-analysis and the abstract alone isn’t enough to judge the strength of the underlying studies.”

    I think this comment is a red herring.
    Isn’t the purpose of the meta-analysis to allow the authors to summarise (i.e. in the abstract) the totality of the evidence whilst taking in to account the strength of the underlying studies?

    CW:
    “Echinacea was never intended to treat the common cold though”

    Another red herring as explained by Plonit.

    JOE:
    “Plonit- you cited newspapers, that won’t cut it.”

    Joe, failed to notice that the link was discussing the treatment of the Shah study by the newspapers for the purpose of determining whether they had accurately reported it. Whereas Plonit was using the link to direct people to the study itself (or at least discussion of it by a reputable source i.e. the NHS web site).

    WINDRIVEN:
    “What specifically is the compound supposed to be responsible for ameliorating cold symptoms? If that is not known, how can echinacea be tested?”

    Another red herring!
    It clearly can be tested since there have been dozens of studies (of varying quality) and a number (3+) of meta-analyses.

    Steven Novella:
    “The fact that proponents have not given up is of no consequence to the scientific claim.”

    Red herring!
    Plonit didn’t argue that and furthermore (s)he is not a proponent.

    ====

    Remember Plonit’s original point was:

    “Is echinacea the right example to use, given that there do appear to be sound scientific studies that echinacea reduces the period of cold symptoms?”

    Based on a quick flick through the Cochrane meta-analysis and not yet having read the Shah meta-analysis, my response to Plonit’s concern would be as follows:

    “The Cochrane meta-analysis you cite is a 2009 update of a 2006 update of a 1999 study. It does not include the more recently available evidence including Turner 2005 which was a large trial which showed no effect beyond placebo.”

    I think that would have been a reasonably phrased response.

    (If I had access I would read Shah and also Schoop 2006 which is quoted in the Cochrane meta-analysis as supporting echinacea (and having taken account of Turner 2006 to do so).

    Please try to address the substance of a post when replying!

    Happy New Year!

  13. Plonit: “Is that a different claim from ‘proven ineffective’?”

    Sure, there’s a distinction … but not an important one. If an intervention works well, it won’t produce “weak and mixed data” when studied thoroughly, as echinacea has been. Such data shows that it either doesn’t work at all (proven completely ineffective), or it “works” so underwhelmingly that the debate is reduced to much ado about nothing (proven to be at least mostly ineffective). While the latter may not be exactly the same thing as the former, it’s close enough for me.

    Just how lame does the data have to get before we get on with more interesting work? Really, what are the odds that echinacea is suddenly going to wow us in the next trial?

  14. windriven says:

    @FelixO

    ““What specifically is the compound supposed to be responsible for ameliorating cold symptoms? If that is not known, how can echinacea be tested?”

    Another red herring!
    It clearly can be tested since there have been dozens of studies (of varying quality) and a number (3+) of meta-analyses.”

    Another red herring my butt. How can one assure dosage of an unknown and botanically derived substance when one can’t even say what it is? As an analogy, consider the sugar content of maize kernels – another botanical product.

    “Kernel sugar content. In standard sweet corn cultivars, such as ‘Silver Queen’, kernel sugar content peaks at 5 to 11 percent. Peak levels last in the field only 2 days at 80 degrees F or 5 days at 60 degrees F before sugar is converted to starch. Even if ears are picked at peak sugar content, quality decreases rapidly after picking because of loss of sugar.” (www.ncsu.edu/sustainable/profiles/botcorn.html).

    5 to 11% is a wide variation. Does the active component in echinacea vary this much? No? How do you know that if you don’t know what the active component even is? How can you ever hope to conduct a meaningful trial when you have no way of assuring consistent dosage?

  15. Plonit says:

    Thank you FelixO. I think you “get me”.

    If I were being argumentative, my response to yours would be that the Cochrane reviewers are aware of Turner 2005, but give some grounds for it falling outside their inclusion criteria ["We included studies if participants were...individuals with unspecific viral upper respiratory tract infections (URTIs) with a clinical diagnoses of common cold, influenza-like syndrome, or viral URTI (it was not possible to apply a standard definition of common cold across all trials)... We did not include studies of individuals suffering from other URTIs with a defined etiology (for example influenza) or a more specific symptomatology (for example acute sinusitis, angina tonsillaris).]

    As you note, they do refer to Schoop 2006, which considers Turner 2005 alongside some other studies said to be similar to Turner 2005 (i.e. studying induced rhinovirus colds). So, like you – I’d want to see Schoop 2006 before coming to a view about “shown to be ineffective” (as opposed to “no convincing evidence of benefit”).

  16. Joe says:

    When it comes to “AltMed” the Cochrane reviews can be unreliable; they often leave the inmates in charge of the asylum. Moreover, you really need to go to the original literature to make a point, don’t rely on a review when there is a dispute (Feynman told us that). Finally, no meta-analysis of inferior studies can make a good study, Turner’s 2005 study of Echinacea trumps the poorer work.

  17. My point was that PRELIMINARY evidence is mixed and unconvincing – and then we have two fairly large studies (not preliminary) which are negative and trump the earlier evidence. That is a close analogy to Gingko.

    The fact that the claim echinacea treats colds was made up by a manufacturer is absolutely relevant. It gets to prior probability, which is important to SBM (if not EBM – remember what blog you are reading).

    If I make up at random a clinical claim for an herb or drug or other substance, the probability that the random claim will turn out to be true is less than if I base a clinical claim on something – anecdotal experience, or putative mechanism of action.

    In the end, the clinical evidence trumps all – but how much clinical evidence is necessary to settle an issue is determined partly by prior plausibility.

    One other small point – stop talking about “proof” – a negative study “shows” a treatment to be ineffective. That is reasonable colloquial short hand – and is not meant to imply “proof”. A negative study is more that lack of evidence – it is evidence for lack of efficacy.

  18. Plonit says:

    If the primary literature is underpowered, then – usually – you need meta-analysis (or, alternatively, more studies) to make a point.

    Meta-analysis is not about pooling all the data, regardless of quality. The goal of meta-analysis is to exclude methodologically rotten studies, give less weight to less reliable studies (i.e. large confidence intervals), while giving greatest weight to more reliable studies (i.e. small confidence intervals). I totally agree that garbage in = garbage out. But if you want to dismiss that a particular meta-analysis is garbage in, then you have to show that it is based on garbage and not simply assume it because you dislike the conclusion.

    Similarly, perhaps it is true that Cochrane leaves inmates in charge of the asylum (insofar as it relies on people with an interest in a field to undertake systematic reviews). On the other hand, in order to dismiss a particular review that complaint would have to be made about particular authors and the means by which this had biased the review would have be shown in the particular case rather than simply assumed on basis of some more generic suspicion about Cochrane reviews. Can be unreliable, but ain’t necessarily so.

    Incidentally, Schoop of Schoop 2006 seems to work for BigHerba – if that adds grist to your mill.

    Anything suggest that Shah et al are asylum inmates?

  19. FelixO says:

    @Windriven:

    “Another red herring my butt. How can one assure dosage of an unknown and botanically derived substance when one can’t even say what it is?”

    If testing Echinacea in various forms shows that is has some effect, then the unknown compostion of the dose in individual trials could be the cause of the mixed results.

    On the other hand, serious scientists might wish to not expend energy on identifying the active ingredient until the some level effectiveness has been demonstrated.

    I don’t think your point is relevant to the discussion of whether Echinacea has been shown to be effective/ineffective.

  20. Plonit says:

    Quite right, my mistake for the colloquialism. A negative result may show lack of efficacy – or may show an underpowered study. The power calculation is dependent on judgments about what constitutes a clinically significant effect.

    Assuming CW is right that echinacea was used for its supposed analgesia properties, that would appear to give some prior plausibility for use as a cold remedy. (The active ingredients of the most commonly-used OTC medications marketed for colds are, as far as I’m aware, paracetamol, pseudoephedrine and caffeine – i.e. analgesia, decongestant of debated efficacy when taken orally, and a stimulant).

    The tentative findings of the Cochrane review are that only Echinacea purpurea shows any effect, whereas Turner 2005 is a different plant (E. angustifolia).

    So, I guess that anyone interested in showing that Echinacea (unqualified) is ineffective would need to reference negative studies for each species. Alternatively, you could argue that one does not need to show that Echinacea is ineffective, but only to put the onus on proponents to show safety and efficacy (which is where the onus always was, of course).

  21. Jeff says:

    According to a researcher quoted in this 2005 article the active components in echinacea are known:

    “What determines the effectiveness of any herbal product is its ability to deliver an effective dosage of active compounds. The specific components of Echinacea responsible for its immune-enhancing effects are the polysaccharides, alkylamides and the cichoric acid,” says Dr. Murray. “While each of these components is effective alone, the greatest degree of effectiveness occurs when the three active components are combined and at a specific ratio.”

    Some echinacea products are definitely better than others. this 2007 Lancet meta-analysis found one specific brand of echinacea to be effective reducing the incidence and duration of colds.

  22. Those test non-results are pretty damning. The one potentially indicative effect, with the p value of .06, goes contrary to the hypothesis.

    One valid criticism is the drop-out rate. I really see no argument supporting the possibility of failure to detect a true effect, but it is possible that there was some kind of differential drop-out effect, such as where those in placebo dropped out because their cognitive performance was declining and so they guessed they were in the placebo group.

    The power analysis incorporates this degree of drop-out, however. This drop-out issue will be where the Ginckcko Bonobo people will say this study is worthless.

    It is pretty clear that if there were some profound effect that all of these GB advocates have been raving about, it would have at least whispered somewhere in this data. The drop-out portion is nowhere near the size to threaten the effect size that should be there, judging from the advertisements and such.

  23. –I should have added: I am familiar with most of those tests, and have used them clinically in older adults to strive to answer referral questions on dementia – usually – is forgetfulness due to depression or dementia?, or is the recent memory problem more likely due to delirium or dementia. I have read the manuals for these, I have been supervised by a seasoned expert as I was trained on these, and I recently helped write a peoposal for a similar study, looking for clinically significant cog decline in a sample of 2000 older adults. The MMSE is lousy for detecting subtle efects, but the CVLT and others noted would definitely be powerful enough to catch any clinically significant difference.

  24. –I was hasty in my first comment – the block design borderline sig diff is confounded by decreases in motor speed that accompany aging, as would trails A / B to a lesser extent. The next whispers of a finding – the diff on Rey Osterreith and on animal naming each favor placebo, although result is not clin or stat significant.

  25. Echinacea is particularly a problem with “evidence of lack of efficacy” because proponents typically dismiss negative studies as wrong part of wrong species for wrong virus. Apparently, we would need to test every part of every plant against every potential cold virus to adequately demonstrate lack of efficacy.

    That is the craziness of allowing products with health claims without the need for evidence for efficacy. For practical purposes, it reverses the burden of proof, which results in a ridiculous situation.

    What we can say from the evidence so far, is that the chance of a consumer pulling an echninacea product off the shelf and getting an effective treatment for their particular cold is negligible.

  26. Zoe237 says:

    “What we can say from the evidence so far, is that the chance of a consumer pulling an echninacea product off the shelf and getting an effective treatment for their particular cold is negligible…

    Isn’t that true of any cold remedy?

    I don’t know who’s right without reading the studies and weighing the evidence, and have no desire to (since I’m not taking echinacea or ginkgo biluba), but I’m glad research is being done even on “natural” remedies.

  27. CW says:

    Just to clarify, it wasn’t my intention to use the argument because the history of the echninacea-common cold was invented by a mistake that it couldn’t be proven to later have that link.

    My point was, as Dr. Novella stated, the fact that its claim was a complete fabrication may be used in conjunction with the lack of empirical evidence.

    If there was an anecdotal claim of a link between echninacea and treating rattlesnake bites (which is how the Native Americans used it), I could at least understand why the belief would still exist today. Although I would still want to see evidence by scientists/researchers to prove the link in a double-blind study.

  28. Plonit says:

    What we can say from the evidence so far, is that the chance of a consumer pulling an echninacea product off the shelf and getting an effective treatment for their particular cold is negligible.

    ++++++++++

    Which is indeed what should be said, and what the Cochrane review says. The standard of evidence for claims about herbal remedies should be as high as for any other OTC (or prescription) remedies. That’s a given. (Of course, we would then have to tackle those parts of the existing formulary for which there is scant evidence).

    To restate, I’m not a proponent of echinacea. In fact, have never taken the stuff, and continue taking large doses of oral pseudeophedrine in my proprietry cold remedies (how’s that for irrational). I am a proponent of placing the burden of proof in the right place. The statement absolute “Echinacea has been shown to be ineffective” pricked my ears only because it is listed as a herbal remedy supported by good evidence in Singh & Ernst _Trick or Treatment_ (of Chiropractic Libel fame). Hence, I looked for the studies and found Shah in the Lancet, which seemed on first pass a sound paper, though with critics (as I noted above).

    But the statement “echinacea has been shown to be ineffective” on the basis of the existing contradictory evidence is surely to buy-in to the reversed burden of proof, no? You don’t actually need to make such an absolute (and possibly incorrect) statement, if you place the burden of proof where it really ought to be (that is, that individual formulation of echinacea to demonstrate its efficacy in relation to each claim it makes).

  29. wertys says:

    A study by Franklin Miller et al showed that lack of efficacy is not enough to affect supplement sales, but well-publicized evidence of lack of safety is. In effect what Joe (or Josephine) Six-Pack is doing is making a risk-benefit calculation and coming up with the old gambler’s fallacy that ‘if there’s no downside and there might be an upside, I’d be crazy not to take it.’

    I suggest that the evidence-based strategy for skeptics is to emphasize the known pharmacological effects of GB, ie its anticoagulant effect and point out that the benefit for cognitive performance, if any, is too small to justify even a relatively small risk. Having said that, if you need emergency surgery and you’re on an anticoagulant the outcome is definitely adversely affected.

    Reference is
    Does the evidence make a difference in consumer behavior? Sales of supplements before and after publication of negative research results.

    Tilburt JC, Emanuel EJ, Miller FG.

    J Gen Intern Med. 2008 Sep;23(9):1495-8. Epub 2008 Jul 10.

  30. windriven says:

    @Plonit

    “I don’t think your point is relevant to the discussion of whether Echinacea has been shown to be effective/ineffective.”

    Then your thinking process is flawed. As you concede, “If testing Echinacea in various forms shows that is (sic) has some effect, then the unknown compostion (sic) of the dose in individual trials could be the cause of the mixed results.”

    If the unknown composition of the dose in individual trials could be the cause of mixed results then any other unidentified confounding factor could just as readily be the cause of mixed results, specifically of the rather meager positive results in a few studies. Ergo, the studies prove bubkes.

    You can dress a pig in a polka dot bikini but that doesn’t make it less of a pig.

  31. Plonit says:

    @windriven, Just so you know, that’s FelixO you are quoting.

  32. windriven says:

    @Jeff

    Interesting post. But the first article that you mention is from a supplements group and the Dr. Murray you cite is the “Director of Education” of a supplements manufacturer. Further, it is not clear just what sort of doctor we’re looking at here. Is Dr. Murray an MD?, DO?, chiropractor?, podiatrist?, naturopath? You see my point… Nor is it clear on what he bases his assertion that the substances he mentions appear in echinacea or that they have any clinical impact on cold symptoms alone, in combination, or in the combination he imagines to be ideal.

    The Lancet that you cite is the same Shah meta-analysis discussed elsewhere.

  33. windriven says:

    “@windriven, Just so you know, that’s FelixO you are quoting.”

    My sincere apologies Plonit.

  34. Plonit says:

    The Lancet that you cite is the same Shah meta-analysis discussed elsewhere.

    +++++++++

    Has anyone actually got any substantive objections to this paper? (Other than feeling it contradicts the Turner study which they regard as definitive).

  35. Scott says:

    Has anyone actually got any substantive objections to this paper? (Other than feeling it contradicts the Turner study which they regard as definitive).

    Being contradicted by larger, higher-quality studies that it did not consider is more than ample reason to discount a meta-analysis. So I’m not sure what the point of your question is.

  36. windriven says:

    Yes Plonit, I do. The Shah study is a house of cards built of questionable data. If you were going to test the effectiveness of acetylsalicylic acid as clotting inhibitor would you grind up willow bark and feed it to your subjects? No, because there is no way to assure consistency of dosage and because there are likely to be a variety of bioactive compounds in the powder that might confound the results of the test.

    Further, meta-analyses never rise to the level of a single, well-designed, double-blinded RCT. Until something better comes along, Turner stands as the gold standard.

  37. Plonit says:

    Being contradicted by larger, higher-quality studies that it did not consider is more than ample reason to discount a meta-analysis.

    +++++++++++++

    If you read the Shah systematic review, you will see that Turner 2005 is one of the 14 included studies.

  38. windriven says:

    Plonit, you miss my fundamental point: Turner, Shah and the other extant echinacea tests are castles built of sand because the DUT is unknown and uncontrolled.

    If Big Pharma said, “we have this really cool stuff that we squeeze from shrimp carapaces that we think mitigates symptoms of influenza, we don’t know how it works, we can’t guarantee that whatever the active ingredient is is consistent from dose to dose, but hey, trust us, it is really cool stuff and we think it works!” would you have the same impulse to accept it? What if the record of trials was mixed at best but they had a recent meta-analysis that showed the compound had promise? What if your life depended on it?

    Nobody’s life is going to depend on echinacea one way or the other. So perhaps you are embracing the notion that echinacea doesn’t seem to have a downside and there could conceivably be an upside, so why not try it? Fine, you are entitled to make that choice. But good science doesn’t draw that distinction. Either there is good evidence or there isn’t.

  39. Plonit says:

    windriven, my last comment was in response to Scott, who doesn’t appear to have read Shah 2007 and therefore was not aware that Turner 2005 was an included study.

    I’m neither proponent or user of echinacea (as stated upthread). I also don’t think I should be entitled to choose *any* medication that has not been shown to be safe and effective – i.e. this is a matter for regulation not individual chocie (also stated upthread). It therefore follows that I don’t think that I (or anyone else) is entitled to make the choice to take echinacea outside of the context of clinical trials.

    I merely questioned whether the absolute statement that it had been *shown to be ineffective* was accurate, given the current state of knowledge.

  40. Scott says:

    Plonit,

    I was referring to the Snitz study, not Turner.

  41. Plonit says:

    Since it is the first time that Snitz is referenced by name, would you be kind enough to link to the study? Or at least provide a citation. Many thanks.

  42. Scott says:

    It’s the one the original post is about. Silly me for assuming people would have clicked through to it already *wink*.

  43. Plonit says:

    But that is about Ginkgo biloba (about which I have made no comment whatsoever – see upthread if you want to check).

    My comments have been entirely related to Novella’s assertion that echinacea had been shown to be ineffective (again, check upthread for confirmation of this).

    If you thought that references to Shah meta-analysis, Turner 2005, etc… were concerned with Ginkgo biloba, I’m afraid you were mistaken. *wink*

  44. Plonit says:

    In fact, I’d be rather concerned if Shah 2007 *did* include Snitz, given that they concern different agents (accepting windriven’s comments on the difficulty of confirming this) and the prior implausibility of time travel etc….

  45. Scott says:

    Bleh, sorry. I got myself all confused and turned around, and freely acknowledge my error.

  46. Plonit says:

    No worries, apologies accepted.

  47. Mark P says:

    If someone said to me that echinacea has been shown via reliable studies to have a small chance of producing a small benefit for a cold, I would not consider that it was worth taking it.

    Science may yet prove this of echinacea, but in the real world, what’s the point? I want drugs that have a realistic chance of having a sizeable benefit. I’m not shelling out my money for possible mild alleviation of a not particularly serious problem.

    The problem is, of course, that alt-med only need the slightest positive to trumpet the effectiveness. They don’t care if in pratical terms it is useless.

    All this time and money trying to prove a small effect for echinacea is money that could be spent on trying to find an actually effective medicine. From what I have seen of the arguments so far, echinacea is useless in real terms, even if perhaps it has some mild benefit in some cases.

  48. Plonit says:

    I don’t disagree with any of that. So, lets say that (“possible mild alleviation of a not particularly serious problem” – if that’s what the science says – of course that is disputed) rather than that echinacea has been *shown to be ineffective*. Of course, private corporations are free to spend their R&D money how they like – and if the manufacturers of herbal remedies were held to the same standard as other medications (as I argued they should be, upthread) then they would have to invest to demonstrate safety and efficacy or go out of business (and probably would go out of business in the absence of evidence to support their products). And if, spending their own money, they provided evidence to support their health claims, fair enough.

    As to where public money should be spent – that’s a different argument.

    Of course, it is also the case that much of what we throw at colds is useless in practical terms (special “cold & flu” remedies sold in the UK are little more than paracetamol souped-up with pseudeophedrine, so why not just take paracetamol given lack of evidence for oral pseudoephedrine?) – there’s a significant price differential too between plain old analgesic, antipyretic paracetamol and what is marketed as a special “Cold & Flu” formulation.

Comments are closed.