Articles

Systematic Review claims acupuncture as effective as antidepressants: Part 1: Checking the past literature

acupuncture

A recent systematic review in PLOS One raised the question whether acupuncture and other alternative therapies are as effective as antidepressants and psychotherapy for depression. The authors concluded

 differences were not seen with psychotherapy compared to antidepressants, alternative therapies [and notably acupuncture] or active intervention controls

or put it differently,

antidepressants alone and psychotherapy alone are not significantly different from alternative therapies or active controls.

There are clear messages here. To consumers: Why take antidepressants with their long delay and uncertainty in showing any benefits–but immediate side effects and potential risks–when a few sessions of acupuncture work just as well? To promoters of acupuncture and alternative therapies: you can now cite an authoritative review in the peer-reviewed PLOS One as scientific evidence that your treatments is as effective as scary antidepressants and time-consuming psychotherapy when you make appeals to consumers and to third-party payers.

The systematic review had five co-authors, of whom three have been involved in previous meta-analyses of the efficacy of antidepressants. However, fourth author Irving Kirsch will undoubtedly be the author most recognizable to consumers and policymakers, largely because his relentless media campaign claiming antidepressants are essentially worthless, no better than placebo. For instance, in an interview with CBS 60 Minutes Irving Kirsch: The difference between the effect of a placebo and the effect of an antidepressant is minimal for most people.

Irving Kirsch: The difference between the effect of a placebo and the effect of an antidepressant is minimal for most people.

Lesley Stahl: So you’re saying if they took a sugar pill, they’d have the same effect?

Irving Kirsch: They’d have almost as large an effect and whatever difference there would be would be clinically insignificant.

The article opens up a new front in the antidepressant wars: Past claims, notably from Irving Kirsch, were that that any advantage of antidepressants over pill placebo was clinically trivial. Now he and his co-authors assert that antidepressants have an advantage over placebo, but only are equivalent to psychotherapy,or alternative treatments such as  exercise and acupuncture.

At first glance, this article proceeds methodically and consistently to its conclusions. I’m sure that many consumers and journalists will be persuaded unless they have strong preconceptions or an intimate familiarity with the existing literature. However, when I gave the article repeated close reads and checked citations against what was actually said in cited articles, I was struck by the

  • Inconsistencies with what is said in high quality available literature.
  • Unorthodox use of meta-analysis and voodoo statistics.
  • Idiosyncratic use of common terms that usually mean something else.
  • Ideologically driven conclusions, i.e,  a persistent commitment to previously strongly stated opinions in the face of contradictory evidence.

The article lacks transparency—i.e., the inclusion of basic details of what was done with what studies– and leaves readers either forced to accept the authors’ conclusions or to undertake themselves a labor-intensive independent search of the literature without the benefit of sufficient clues from the article as to where to look. Exploring these problems together, we can learn something about the use of systematic reviews as propaganda and marketing tools. Journalists covering the antidepressant war, please take note.

Yet,  I was also left with unsettling concerns about the vulnerability of the PLOS journals in the editing and peer review of cleverly packaged poor science, and also the inability of the PLOS journals to give proper recognition to post publication peer commentary and correction. Lapses in editing and peer review happen all the time, even in high impact traditional journals, but the issue is how readily and prominently they can be corrected. I am an Academic Editor of PLOS One and donate my time because I recognize the fallibility of prepublication peer review and my commitment to post publication peer commentary. I think, though, that this article highlights the need to fix some key aspects of the PLOS journals’ current editorial policies so that proper recognition can be given to post publication peer review in indexed, citable commentary, not just in blogs or elsewhere– something that does not happen now.

Here’s how I will build my case in two blog posts that this article presents a distorted view of antidepressants versus acupuncture for depression and that it makes unwarranted conclusions. In this first blog post, I will compare what is said about what is known about antidepressants and acupuncture in this article to what I found in key sources. In a second blog, I will examine the manner in which this systematic review was conducted and reported, and the deviations from best practices by which the authors reaching their foregone conclusions.  In the second blog I will also draw upon points that I extracted from the literature discussed in the first blog.

One of the many advantages of open access journals like PLOS One is that anyone in the world with Internet access can readily obtain copies of articles without charge. This allows bloggers to have to provide lots of details in order for readers to evaluate independently their claims. I will capitalize on this in encouraging readers to go directly now to the open access article and form their own opinions. In the first of two blogs on this topic, I mainly summarize here how the authors frame the research questions in the introduction and discuss their findings in their conclusion in terms of the fit with the existing literature. But, please, don’t depend solely on what I say, when you can readily look for yourself at the open access article and, if you’d like, follow up on the citations there.

What the article says

The Introduction claims

a number of recent articles have emphasized the inability of antidepressant medication to consistently demonstrate superiority to placebo pills.

As for psychotherapies for depression, they

have also come under scrutiny for the inability to demonstrate substantial superiority to various treatment controls as opposed to waiting-list (no treatment) controls. Similarly, “alternative therapies such as acupuncture and exercise have shown promise of individual published studies…, [But] the profile is less impressive according to independent review such as Cochrane reviews… and those conducted by the National Institute of Health and Clinical Excellence.”

Where does this leave us?

Given this level of ambiguity, it is unclear if pharmacological treatments are any better or worse and psychotherapies or psychotherapies are any better than non-traditional treatments such as exercise and acupuncture.

Thus, the groundwork is laid for the current systematic review.

The Discussion restates its context:

Much has been made of the inability of antidepressants to demonstrate clinically significant superiority to placebo in antidepressant clinical trials.

Then, the key conclusion of the study

Although antidepressants alone and psychotherapy alone did differ significantly from placebo controls, treatment-as-usual and waiting list controls, they did not different from alternative therapies such as exercise and acupuncture or active treatment control procedures.

However

Although combination therapy did not statistically separate from exercise and acupuncture in the blinded trials, these alternative therapies themselves were not statistically superior to placebo… This may be due to the small number of trials evaluating these.

And getting provocative,

Our results indicate that in acute depression trials using blinded raters the combination of psychotherapy and antidepressants may provide a slight advantage whereas antidepressants alone and psychotherapy do not significantly different [sic] from alternative therapies such as exercise and acupuncture or active intervention controls such as bibliotherapy or sham acupuncture.

Consumers, journalists, and promoters of alternative therapies please take note that what conclusions come from a comprehensive review of the scientific evidence.
antidepressants
Comparison with the existing literature: Antidepressants

Let’s start with the phrase in the introduction “Inability of antidepressant medication to consistently demonstrate superiority to placebo”? and its echo in the discussion, the “inability of antidepressants to demonstrate clinically significant superiority to placebo in antidepressant clinical trials.”

The authors cite four sources, two of them by the authors of this PLOS One article. The first source that is cited, “Evidence b(i)ased medicine – selective reporting from studies sponsored by pharmaceutical industry…” was important in demonstrating publication bias in antidepressant studies submitted to the Swedish drug regulatory authority, but it’s irrelevant to evaluating the efficacy of antidepressants in clinical practice. The Swedish authority evaluates whether antidepressants achieve a 50% reduction in scores on the Hamilton Depression Rating Scale in four-to-eight week trials. In such short trials, placebo effects are maximized and drug effects often have not yet occurred, although side effects are occurring. So, this article really can’t be entered into evidence as to the inability of antidepressant medication to demonstrate efficacy.

The second cited source, “Listening to Prozac but hearing placebo…” was co-authored by Irving Kirsch. The most relevant and provocative claim in the article is

a confident estimate that the response to inert placebos is approximately 75% of the response to active antidepressants medication. Whether the remaining 25% of the drug response is a true pharmacological effect or enhance placebo effect cannot yet be determined…

Citation of the article, as opposed to others, particularly a later high profile article in PLOS Medicine by Kirsch is odd. I have blogged about the cited article (1,2) elsewhere, and I will shamelessly draw on those blog posts in the following comments.

In one of my blog posts, I stated

The article is quite dense and difficult to follow in its twists and turns in logic and sudden declarations of definitive conclusions. Kirsch and Saperstein claimed [their] interpretations were based on rather complicated meta-analyses, which were described in a way that it is unlikely that anyone else could replicate them. Their examination of past reviews and computer searches had yielded 1500 publications, but only 20 were retained as meeting inclusion criteria. What was important technically was that calculation of within-condition effect sizes was nonstandard, and involved calculating for measures of depression the mean posttreatment score minus the mean pretreatment score, divided by the pooled standard deviation.

The article was published in a soon-to-be defunct Internet journal, Prevention and Treatment that was suffering from a lack of submissions. The action editor, psychologist Annette Stanton later conceded on the SSCPnet listserve that she was over her head in selecting reviewers and making editorial decisions about the article. The article was accompanied by scathing, but authoritative commentaries. One of the commentaries was by psychiatrist Don Klein who resigned as founding co-editor of Prevention and Treatment because of its editorial policies. He did not mince words in describing the Kirsch article as relying on

a miniscule group of unrepresentative, inconsistently and erroneously selected articles arbitrarily analyzed by an obscure, misleading effect size… The attempt to further segment the placebo response, by reference to psychotherapy trials incorporating waiting lists, is confounded by disparate samples, despite Kirsch and Saperstein’s claim of similarity.

He concluded by labeling the article as representing a failure of peer review.

Psychologist Robyn Dawes’ brief critique of Kirsch’s article was almost as scathing as Klein’s and zeroed in on an odd way of calculating effect sizes:

“the simple posttreatment minus pretreatment difference in an outcome variable for treatment group over placebo group does not define either a treatment effect or a placebo effect, even for groups randomly constructed. Such differences must be compared with the difference obtained from a (randomly selected) no-treatment group in order to evaluate the effect of treatment or placebo… Effect involves a comparative judgment, not just a pre-post one. And even when a legitimate effect is found for placebo, it often (almost always?) makes little sense to talk of a proportion of a treatment effect has been accounted for by a placebo effect. The logic of Kirsch and Saperstein (1998) is the seriously flawed. Science (like art and life) is not that easy.”

Dawes didn’t let up

the unusual—to use a kind word—nature of this analysis is best illustrated in the first sentence of the discussion section… “No-treatment effect sizes and effect sizes for the placebo response were calculated from different sets of studies.” There’s no such thing as a no treatment effect size.

Dawes ends with:

If knowledge were as simply attained as implied by Kirsch and Saperstein’s article – by just defining effect sizes as a pre-post comparison—it would not even be necessary to randomize. We would know enormous amounts more than we currently do. Unfortunately, we don’t.

In my next blog in which I get into the specifics of the acupuncture versus antidepressant question, we will find it is subject to the same criticisms of the earlier Kirsch study provided by Don Klein and Robyn Dawes. I haven’t been able to find other examples in the literature except for studies by Kirsch in which summary effect sizes are calculated separately for active treatments and placebos and then compared. But will be finding more flaws than just that. But back to the literature review….

The second article that is cited is by Fournier and colleagues and is available open access here. The article is notable in claiming

The magnitude of benefit of antidepressant medication compared with placebo increases with severity of depression symptoms and may be minimal or nonexistent, on average, in patients with mild or moderate symptoms. For patients with very severe depression, the benefit of medications over placebo is substantial.

It ends with a complaint that in efforts to promote antidepressant treatment

There is little mention of the fact that efficacy data often come from studies that exclude precisely those MDD patients who derive little specific pharmacological benefit from taking medications.

The article has received considerable media attention, although a lot less than the work of Irving Kirsch. However, it is also been subject to withering criticism. Among the most telling criticisms:

  • The systematic search of the literature identified 2164 citations, of which 281 were retrieved, but only six studies were included in the analysis because of the need for individual patient-level data.
  • The analysis was restricted to only trials of two antidepressants, imipramine and paroxetine. Imipramine is not a first-line antidepressant, and use of paroxetine is restricted because of concerns about side effects and a withdrawal syndrome, and discouragement of its use with pregnant and nursing women.
  • Only a small proportion of the patients entered into these trials scored in the in the mild range of depression and so even the authors concede that results may not generalize to mild depression.
  • The claim of “minimal or nonexistent” benefits for antidepressant was based on arbitrary criterion that to be otherwise, effect size of at least .5 had to be achieved.

Harriet Hall has already blogged about the effect sizes of antidepressants and provided an excellent link to a discussion of effect size (ES), and so I won’t reproduce her arguments here. However, I will point out that that the criterion of ES  > .5 originally came from Jacob Cohen’s proposed cut off for a large effect size which he warned (page 567) was quite arbitrary and ran the risk of being misunderstood. I asked psychiatrist Simon Gilbody, who has conducted key meta-analyses of enhancements of care for depression what he thought about this criterion and he replied:

“I never believe any study these days which finds an ES > 0.5. it usually means the science is wrong and we haven’t found the bias(es) just yet.”

The third of the four articles is co-authored by Erick Turner, a psychiatrist and former FDA reviewer of antidepressant data submitted by Pharma.  The NEJM article involved a meta-analysis of published and unpublished antidepressant trials submitted to the FDA as part of the drug approval process.

According to analyses of data obtained from the FDA, only half of the trials were positive, but according to the published journal articles, almost all of the trials were positive. In almost a dozen cases, trials that were negative or questionable according to the FDA were published as if they were positive. Once unpublished reports were taken into account, the overall effect size (ES) for antidepressants relative to placebo lowered from 0.41 to 0.31, where ES is the standardized mean difference in improvement. However, in all cases, antidepressants were significantly superior to pill placebos. So, Turner and colleagues found that antidepressants were not they are cracked up to be, but they did not dismiss them as ineffective. As Turner was later quoted, “the glass is far from full but far from empty.”

The fourth source cited in opening statements of the antidepressant/acupuncture meta-analysis is by two of its authors, Khan and Brown. It involves a study of the severity of depression in response to antidepressants using the FDA administrative database. It concludes that severity of initial symptoms affects clinical trial outcomes. While doesn’t seem to present an overall effect size for antidepressants versus placebo, it notes that 48% of clinical trial showed the superiority of investigational antidepressants over placebo, and 64% of the clinical trial showed superiority for established antidepressants relative to pill placebo.

It’s noteworthy which meta-analyses by the authors of the antidepressant/acupuncture meta-analysis were not acknowledged in that article. A PLOS Medicine meta-analysis by Kirsch is among the most highly cited articles in that journal and has been widely discussed in the media. However, its claim that the relative advantage of antidepressants versus placebo are minimal and insignificant would seem to contradict the claims in the present antidepressant/acupuncture article that the effects of antidepressants are significantly superior to pill placebo, although no better than acupuncture. Furthermore, a 2005 study by two of the co-authors of the antidepressant/acupuncture meta-analysis involved 329 depressed patients from their antidepressant clinical trials center who had been entered in 15 multicenter trials. Effect sizes were .51 in the low/moderate group, .54 in the high/moderate group, .77 in the moderately severe group, and 1.09 in the severe group. Effect sizes obtained in multisite trials are known to vary across specific sites, but these relatively strong effects would seem to undermine claims that antidepressants are no better than placebo or no better than acupuncture.

In sum, this seems to be a marked difference in the portrayal in the antidepressant/acupuncture article of the past literature concerning antidepressants versus placebo, and even between the past work of the authors of this article and what they say here. A commentary co-authored by Erick Turner in the British Medical Journal on the discrepancy between Kirsch and his interpretation of similar results aptly notes “we agree that the antidepressant ‘glass’ is far from full, we disagree that it is completely empty.

Acupuncture<

The PLOS article comparing acupuncture and antidepressants states

“Alternative therapies such as acupuncture and exercise have shown promise… [But] the profiles are less impressive according to independent reviews”.

Let’s see how much less impressive. An updated Cochrane review (2010) found insufficient evidence to recommend acupuncture for depression, and insufficient evidence of a consistent beneficial effect from acupuncture compared to wait list or sham acupuncture control. There were obvious difficulties that could not be readily overcome in the blinding of providers and patients as to whether a medication or acupuncture were received. There was further a high risk of bias across studies. Overall, the study of acupuncture for depression was characterized by a lack of consistently rigorous scientific study and poor reporting. There was unexplained heterogeneity, i.e., inconsistency of results across studies they could not readily be interpreted. Most of the studies were from China, including all of the antidepressant/acupuncture comparisons, with participants largely recruited from inpatient populations. Very few of the American RCTs comparing antidepressants to pill placebo entered into meta-analyses involved in patient samples.

A more recent systematic review of systematic reviews of acupuncture for depression found that all of the positive reviews and most of the positive primary studies originated from China and that there was reason to believe that the reviews were less than reliable. The authors concluded that acupuncture was essentially unproven as a treatment for depression.

A systematic review and meta-analysis of Chinese randomized trials concerning the antidepressant venlafaxine concluded that all trials were of low quality and underpowered.

Yet another systematic review revealed an astonishing publication bias to randomized trials of acupuncture more generally: no trial published in China, or for that matter Russia/USSR had found an acupuncture treatment to be ineffective relative to a comparison treatment.

An interview study with the authors of 2235 apparently randomized trials published in the approximately 1100 medical journals help published in China found that less than 10% adhered to accept a methodology for randomization and only 6.8% could be deemed authentic randomized controlled trials.

To say the least, these reviews suggest the need for caution in accepting data from China and in synthesizing and comparing data from RCTs conducted in China with those conducted in the West without some consideration of quality of the trial and risk of bias. Given the quality of the literature, we are not even in a position to evaluate acupuncture for depression, much less make statements about its efficacy relative to antidepressants or placebo.

In summary, I don’t think that someone would anticipate what is said in the existing literature from how it is portrayed in the acupuncture/antidepressant meta-analysis. Furthermore, we can anticipate some formidable challenges to integrating data from antidepressant versus pill placebo trials with acupuncture trials and acupuncture-compared-to-antidepressants. There appear to be few head-to-head comparisons and a lack of comparability of the control conditions in acupuncture versus antidepressant trials. Both patients and providers are well aware of treatment assignment in ways that cannot be blinded and so the repeated references in the antidepressant/ acupuncture article to controlling for blinding stands out as odd. Finally, there is good reason to doubt the quality and credibility of acupuncture studies conducted in China, where most studies are conducted. In my next blog post, we will see how these problems were addressed or ignored.

Posted in: Acupuncture, Neuroscience/Mental Health, Science and the Media

Leave a Comment (32) ↓

32 thoughts on “Systematic Review claims acupuncture as effective as antidepressants: Part 1: Checking the past literature

  1. windriven says:

    “Where does this leave us?”

    Apparently not in a good place if one suffers depression. If antidepressants or psychotherapy are not significantly better treatments than acupuncture then the depressed seem well and truly hosed.

    My takeaway after this half of the post is that the benefits of mainstream treatments are barely separable from the background noise. Is this a damnation of current therapies for depression or a damnation of researchers’ abilities to conduct and report a sound trial?

  2. James Coyne says:

    I would like to think that a careful review of the claims in this PLOS article yields a damnation of five authors who merrily drove past the data in a hasty rush to a pre-planned destination. But stay tuned, another blog post is on its way.

  3. Janet says:

    Will your next post explain how acupuncture and exercise get lumped together as “alternative” treatments?

    I’ve always thought exercise could be helpful, at least for mild depression, although I cannot say how I got that idea except general reading–which I have now come to regard as suspect. Even so, I would not call it “alternative” except in the sense that it’s not a pharmaceutical intervention.

  4. James Coyne says:

    @Janet: There are a number of problems in the lumping and splitting in these analyses, not only of acupuncture being lumped with exercise in bibliotherapy, but in pill placebo being lumped with waitlist control and sham acupuncture. What a mess!

    The literature concerning exercise for depression is quite mixed, including a recent large-scale disappointing trial in the UK. I think the evidence is strongest for exercise having short-term effects on depression, particularly among the mild or moderately depressed.

    http://cat.inist.fr/?aModele=afficheN&cpsidt=24138075

    I think it’s great has a part of a stepped diagnosis/stepped care program in which mildly depressed patients from primary care are initially assigned to get an exercise program and then monitored for improvement, before trying anything more.

    I think exercising in groups can also have a social activation effect for the mildly depressed.

  5. kb says:

    Excellent post and I look forwards to the next one!

    In psychology research there has been an intensive debate about the need to evolve beyond the flawed method of null-hypothesis significance tests and classic p values. For example, Killeen has suggested the statistic P-rep http://homepage.psy.utexas.edu/homepage/class/Psy391P/Killeen.2005.pdf predicting the probability of replication. It is not without faults (e.g., doesn’t take prior probability into account http://psyphz.psych.wisc.edu/web/reference/macdonald_PsychSci2005.pdf ), but it least there is acknowledgement of the problem and discussion of alternatives. Is any of this reaching science-based medicine?

    a minor point: A few times you say “blog” when you mean “post”, as in “In a second blog, I will examine the manner in which this systematic review…”.

  6. James Coyne says:

    Thanks, kb, especially for the link.And I will try to catch my confusion of “posts” with “blogs.”

  7. Becky says:

    And where does that, or any research on anti-depressants, leave those of us with major depressive disorder, recurrent, severe? I only find research showing how useless anything is. For me a vagal nerve stimulator and seroquel seem to be keeping the wolves away, but when I read this, I think maybe I should just swing a dead cat by the tail at midnight and get the same effect.

  8. James Coyne says:

    Maybe, Becky, but I think a fairer conclusion is that when it comes to the effectiveness of treatments for depression, the glass is not full, but it is far from empty. We can’t predict ahead of time what treatment will work best for a particular patient, but on average, the treatments are significantly better than pill-placebo in clinical trials. And when depressed persons find the right treatment, their response is better than the average response in clinical trials where switching treatments and more than minor adjustments in treatment are not typically allowed.

  9. Alia says:

    And I’m just left wondering about one thing. If we wanted to be really scientific and properly test all modalities – how on earth could we do double-blinded study on psychotherapy? It seems impossible to me, but then perhaps I’m not imaginative enough.
    Becky – from my own experience (several of my friends suffer from depression, OCD or bipolar disorder) antidepressants combined with therapy seem to be working. It’s anecdata, of course, but then again, my sample is quite big.

  10. mousethatroared says:

    Maybe someone has mentioned this in the comments. I haven’t had time to review. Many people assume the research on anti-depressants and depression can be generalized to anti-depressants effect on things like anxiety disorders. Not so. One would have to look at the evidence for the individual anxiety disorder and the medication.

  11. Janet says:

    Thanks for the link and for restoring my confidence in choosing responsible sources for my reading. :-)

  12. fxh says:

    I have problems with many of the studies on anti -depressants and depression in general – I don’t have access to most of them at the moment but over the years have had a good look at very very many of these studies. I have also worked at all levels in the field of serious to severely psychiatrically ill and disabled persons.

    Most of the studies, I’m making a sweeping statement here but largely I believe I’m justified, focus only on reduction in symptoms according to a standard bunch of surveys/questionnaires. Many of the survey points seem too close together – 4 – 6 weeks to capture much change in such a short time with medication. Some drugs are barely beginning to kick in by 4 week.

    But the bit which becomes important is the focus on self reported symptomology. I’m not suggesting that some symptomology, especially negative ones, would be good to see diminished/ improved. However in the last 10 years at least that I’m aware of many Support Groups (not clinicians) Carers Groups ( family and friends) Work Skills groups and Living Skill teachers who all all pitch in without a specific clinical orientation , but a practical everyday living orientation. They work with hospitals a clinics but they work with the Client/patient on measurable changes in coping with everyday life. Stuff that can be seen and measured BY OTHERS.

    So that Bridget Mary Depression may improve in her ability to buy the groceries each week, pay the rent monthly, not make noise in the apartment building and get the cops called, eat reasonably well, do some exercise by going to the gym paid for as part of the plan, monitor her diabetes and BP regularly and have reasonable interactions with her parents, helpers and useful meetings with a care team that includes clinical but NOT solely focused on self reported CLINICAL measures.

    In fact Bridget Mary Depression may be functionally highly as observed by others on a range of accepted criteria in daily life. That is acting (and being) Normal. But she may still report the same clinical symptoms as when she stared to commence the practical program. – So symptom surveys at 2,4 6 12 weeks may show little change – BUT ON OBJECTIVE functioning measures, by those who live and work with here, those few weeks may show a remarkable improvement in the skills of everyday living.

    There’s many who argue, with some care and thought, that the ability to be judged by others as functioning well and in fact be functioning well, is a far far preferable measurement than just focussing on symptom reduction. That is you may still have your harsh voices, but be able to attend work, converse in a normal way, do shopping, attend well to personal care tend, clean apartment and pay rent – all the while still having symptoms.

    I know this is a bit of a unfocused rant – but to put it briefly – a multi focused (modal) approach, of drugs, counselling, practical sup[port with daily living and not focused too much on symptoms may ( I would say it does) give us a better fix on what works and what doesn’t.

  13. James Coyne says:

    You make an excellent point, fxh, antidepressants or any treatment for depression are not really evaluated with patient-oriented measures that get at improvement in functioning. The Hamilton Depression Scale used by the FDA is particularly poor in this respect. The FDA approves drugs on their ability to produce changes in Hamilton Depression Scale scores, which does not necessarily translate into improvement in patients’ sense of being depressed or their ability to function.

  14. Harriet Hall says:

    Another concern is that patients are paid to participate in studies, and if they want in, that tends to bias their answers on depression scales and other measures. The studies may be looking at some subjects who don’t really meet the criteria for mild/moderate depression, and that muddies the waters.

  15. James Coyne says:

    Totally agree, Harriet, and with institutions being paid for accruing patients who meet criteria to industry sponsored trials, there is often another layer of deception and unreliability of initial depression scores. When institutions fall short in their recruitment of sufficiently depressed patients from among those volunteering, messages get passed to recruiters and raters to find patients with higher scores–and somehow they do.

  16. brescd01 says:

    I appreciate the thoughtful analysis but I find more interesting the subtext and motivations of the frequent attacks on antidepressant efficacy. Is this a manifestation of the anti-psychiatry movement? Prejudice against mental illness? Reaction to the perceived windfalls of Big Pharma? An understandable discussion given the popularity of anti-depressants?

    The funny thing is, as a practicing neuropsychiatrist and sleep physician who is presumably expert in anti-depressants, the clinical utility of any intervention is only distantly related to its effect size in any particular study. So if antidepressants have a “small effect size” whatever that means when one is generalizing and I am not sure one meaningfully can, they are clinically very useful agents. A lot of internists report prescribing anti-depressants is their most satisfying intervention.

    But there has to be a larger story here, that there is such relentless questioning of anti-depressants’ efficacy and not of sedatives’ or anti-psychotics’ or even mood stabilizers’.

  17. James Coyne says:

    You are on to something significant, # brescd01. I agree that legitimate questions and been raised about the efficacy and safety of antidepressants. But they are often been raised without any objectivity, and some of those voicing concerns are quite resistant to data. “Antidepressant wars” are raging in which some people judge data solely in terms of whether they fit preconceived notions or who produces the data. Appearances of conflicts of interest are important, but we need to extend the notion to people having staked out clear ideological positions, as well as those who have had some involvement with Pharma.

    A lot has been written on the limited relevance of FDA style clinical trials, with their dependent solely on changes in Hamilton Depression Scale Scores to individual patient response.

  18. mousethatroared says:

    brescd01 ” A lot of internists report prescribing anti-depressants is their most satisfying intervention.”

    HeHe – How nice for them. Personally, when I took Paxil, I think I would have used the word “helpful”. “Satisfied” is not a word that I would have associated with the medication. ;)

  19. markjones says:

    Is this science-based medicine?
    http://www.youtube.com/watch?v=8F7mixarLoo
    Where’s the so called science in psychiatric drugs. It’s all subjective, no blood test, no diseases. If you are writing about quackery, why are you not including Psychiatrists?

  20. Marc Stephens Is Insane says:

    markjones:

    That video you posted is scientology propaganda. The CCHR is a scientology front group.

    The scamatology philosophy on health works so well at Narconon, doesn’t it? The bodies continue to pile up as addicts are “treated” with hours of saunas and overdoses of niacin, overseen by cult member chiropractors, naturopaths and DOs. That’s what the CCHR promotes.

    Praise Xenu!

  21. Narad says:

    The scamatology philosophy on health works so well at Narconon, doesn’t it?

    I take it that you caught the name association.

  22. James Coyne says:

    Scientology has played a substantial, unnoticed role in the “Antidepressant wars,” financing product liability suits, phoney antidepressant-survivor support groups etc. As they have done on my other blogs, their spamming is showing up here. I must be doing something right.

    Forbes put this article up last week and then took it down, probably because the major source was Citizens Commission On Human Rights International (CCHR), a Scientology front group.

    http://webcache.googleusercontent.com/search?q=cache%3Ahttp%3A%2F%2Fwww.forbes.com%2Fsites%2Flawrencehunter%2F2013%2F01%2F14%2Fpsychiatric-drugs-not-a-lack-of-gun-control-are-the-common-denominator-in-murderous-violence%2F

  23. Marc Stephens Is Insane says:

    James,

    Have you ever heard from a scienotology/OSA sockpuppet who posts under the name Louanne? Any time there’s any news coverage of her cult (as there is right now, and will be on CNN on AC360 tonight!!) she bombards the comments section defending her cult and attacking any critics. As Hubbard said, “Always attack, never defend…”

    The CCHR has a multi-million dollar museum on Hollywood Blvd. called “Psychiatry: An Industry of Death.” Cult members like Travolta and Alley have been shilling for the CCHR (and Narconon) for years.

    Our quack friend Dr. Julian Whitaker, while not a scientologist himself, is on the board of the CCHR. He is a credit to the medical profession: beside being one of Burzynski’s only supporters he is anti-vax and anti-psychiatry.

    Scientologists who are denied proper medical and mental health care die. There is a long list of casualities the cult is responsible for. The CCHR is one of their worst front groups. Jan Eastgate, from the Australian branch of CCHR, once advised a 10-year-old girl to deny her scientologist step-father was sexually abusing her.

    So yeah, I like to call out anyone who posts any pro-CCHR propaganda. Glad to see publications like Forbes and The Atlantic respond to feedback.

  24. sigh.
    Coyne is a great scientist to be active in this discussion.

    One muddied issue in all of this is the question of whether antidepressants (generally, or any of the leadingly prescribed SSRIs or SNRIs) for depression (major depression or the range of elevated scores) provide a worthwhile benefit above placebo, generally.

    I have read scores of efficacy trials. I see the same graph over and over. Depression scores drop from baseline to eight week follow-up for the drug, and drop nearly as low for the placebo. Finally, toward the end of the study period, the drug eeks out a statistically significant advantage over placebo.

    I was home yesterday, observing the MLK holiday by pretending to be the antidepressant demographic: watching television mid-day.

    Sure, I am aware antidepressants give a good enough benefit to some that they sustain the prescription. but the trials report averages, and on average, the placebo does pretty darn well compared to the drug.

    The lines are so close that, in my opinion, there is not a “clinically meaningful difference.”

    So, I agree generally with Kirsch that drug is hardly better than placebo.

    If anyone wants to challenge this conclusion, provide a study (a decent study), and I will go dig it out, determine the depression scale used, determine how many points of difference there must be to be clinically meaningfully different, then I will calculate the number of points, on avg, between drug and placebo (at the statistically sig 8-wk data point most likely), and if that clin sig diff is not broached, then I will conclude that placebo is essentially about as effective as drug, and I will post the relevant info here.

    Kirsch is essentially correct on this point, whether he is a scientologist or not.

  25. James Coyne says:

    Kirsch is not a scientologist, even if an ideologue. In a comment at my other blog, though, one of his co-authors on the highly cited PLOS Medicine paper proclaimed that scientology is perhaps the best thing that ever happened to psychiatry.

    I have a forthcoming article in Psychological Medicine that meta analyzes all comparisons we could find of pill-placebo and psychotherapy. The differences are essentially the same as for pill-placebo vs antidepressants.
    ,
    Stay tuned, MedsVsTherapy, tonight or tomorrow night, I will have another posting on this acupuncture vs antidepressant comparison.

  26. evilrobotxoxo says:

    @medsvstherapy: I’ve prescribed a lot of antidepressants, and my main clinical activity right now is ECT (e.g. treating patients on whom antidepressants didn’t work), so I could argue both sides of it. I would argue that antidepressants work well for some people but not others, so they’re worth using even though the average performance isn’t that great. I remember one time that I had two patients come in, on the same day, with almost identical clinical presentations. I started them both on Lexapro, and one had a really dramatic response, and the other one had no response at all. The point is that many people who do respond respond well, a bunch of the others don’t, and the average (in a self-selected non-representative patient population composed of semi-professional serial clinical trial enrollees) shows a modest but measurable improvement over placebo. The best evidence of this comes from randomized discontinuation trials where they take people who did respond to an antidepressant, then taper half of them off in a blinded fashion and see what percentage relapse over the next year. In those trials, the rate of relapse in the non-medication group is around three-fold higher (IIRC), so the effect size is huge in that population. As far as I’m concerned, antidepressants do have a clinically significant effect size in some individual patients, period. There’s plenty of evidence to back this up, and it’s consistent with the clinical experience of myself and the people in the field who I’ve talked to.

    However, it is definitely true that there’s a huge number of patients for whom they don’t work very well, the pharmaceutical companies have definitely oversold their effectiveness, and some PCPs overuse them when they should be referring for CBT and etc. There have been meta-analyses trying to determine the NNT (number needed to treat) for antidepressants, and most of those that I’ve seen have a number less than 10. Kirsch would probably argue that if antidepressants only help 1 patient in 6, then that barely beats placebo. I would argue that the polio vaccine only helped 1 patient in 50, so 1 in 6 actually isn’t that bad. They have an extremely low rate of serious side effects, so the risk/benefit ratio is clearly in favor of using them. Of course, that should be in conjunction with psychotherapy, exercise (but not acupuncture), sleep hygiene, etc.

    MedsVsTherapy: you’re a therapist, right? I suspect maybe you have a lower opinion of the efficacy of SSRIs for depression because you intrinsically see a biased population, i.e. people who are either treatment-refractory or are referred to you because they have axis II pathology. That said, I’ve been trying to get a ketamine infusion protocol approved at my institution, and I might become involved in a clinical trial of deep brain stimulation for depression, so I obviously believe that we need to improve treatments for depression substantially.

  27. tnpcanada says:

    fxh makes several important points about the socio-cultural aspects of mental health treatment. It also seems important for studies to clarify if patients are in their first,second, third or multiple relapses, or equally on their first, second, etc medication. Never mind, what about co-morbidity? Mental illnesses rarely come solo. It seems that part of the problem is the definition of depression itself and how its severity is determined. To use fxh’s more generous criteria could make for more accurate information, but it’s a massive task to come up with protocols that are scientifically acceptable and funding friendly.

  28. in response to the robot – i pretty much agree with everything you posted. however, your line of thinking does not match up with the line that has been most common in antidepressant research – there, a group of people meeting dx criteria receive either drug or placebo, and outcomes are assessed at perhaps 8 wks.

    there is no subgroup analysis.

    i could not locate a study in my files despite browsing for 15 minutes – it was illuminating: in an ssri drug vs placebo trial for depression, the typical outcome was observed – the decreasing severity across time for both drug and placebo, with drug eeking out a point oh five. sold. sign me up.

    when further analyzed, the drug group showed 2 groups: those with great response, a bit more profound than the drug group overall, and a non-responder group, whose lack of change was avgd in with those who received benefit.

    i recall this study being pub in the recent five yrs – it could be a bit older – however, it is ridiculous that at this point in time, with all of us clinicians having real world knowledge, that the science has not advanced to routinely investigate and report outcomes by this distinction.

    if you take the responders off of one ssri, will they respond to another? if you respond to citalopram, are you more likely to respond to escitalorpam than to paroxetine?

    there are factors limiting “us” academic-minded people from moving on to more intelligent questions which certainly can be addressed. some of tese mor penetrating studies have been done.

    but the fda criterion of efficacy drives our thinking to be focused on whether a drug (to soon be under patent and mass-marketed) works or not; we don’t even care about why/causality. just a black box there.

    that model gets promoted to providers, and works its way into expert advice and sadly into healthcare performance criteria. by HEDIS, your patients with a depressive dx should be Rx an antidepressant.

    depression outcomes have been shown to vary by preference for talk therapy, drugs, or both – when preference is matched, outcome is best. so, ideally a pt ought to have some interactive decision-making experience, and not be steered toward drugs just because it is easier for our healthcare system to deliver drugs vs talk therapy; along with that, we should somehow eventually be able to identify who will, physiologically, benefit from antidepressants – of course, the dream is to ultimately know which drug for which person, if a person is a drug candidate.

    Yes, I am a therapist, and have been addressing depression in my various work experiences before and after training going back in time to before we could listen to prozac.

    in private settings, my client base has been those interested in talking. in public settings, i have been fortunate to be in a setting where we could offer either or both, but the bias has been motivated by access factors and not so much whether a person desires meds or talk therapy – so i have delivered plenty of care where a client truly had no perference – for many, the depression was detected by the doc or screening, so there was no patient-seeking-care involved.

    this is a great discussion, and there are many angles, and a lot of promising findings in various studies.

    one study, Furmark J Neuroscience dec 2008, found stronger placebo-induced relief of social anxiety in some people – some with one serotonin-gene profile versus others – so some people may be more physiologically susceptible to placebo effects.

    a review of affinity for the H1 histamine receptor in ssris, antipsychotics, and other drugs that aleviate depression suggests that the placebo effect might be working through sedation or other histamine phenomena.

    if so, it might be wiser to trial an antidepressant, or antipsychotic, against diphenhydramine, maybe at a low but detectable dose, rather than an inert drug. if you do a web image search for ‘remeron affinity’ you will see that, yes, it affects some 5ht receptors, but has much greater affinity for H1. so, if remeron works for depression, you could believe it is affecting serotonin, but then again it might be better to consider the H1 binding. any such drug would be a great candidate as a nocebo for depression.

    sure, it may be that all of these hypothesis fail to pan out when decently studied. nonetheless, they are worth considering, and are far more advanced than the black box works/doesn’t work FDA approval model.

    i have a bias toward talk therapy, and i have no hesintancy in declaring this. i think the healthcare system should be engineered to deliver talk therapy before drugs – simply based on efficacy and side effects and pt preferences. if i can get obama to appoint me to some new panel, i will work my world domination plan according to my biases. the physicians hav already done this, and we counselors are lousy at it.

    i recognize, though, that what is needed to see if the healthcare system should bend to my will is whether the evidence is on my side. sure, i favor articles such as those by Kirsch, but i try to be open to the evidence, wherever it leads. as a therapist, there are so many problems in this world, i will never be out of business – as long as there is a funding mechanism and a health care delivery system to support the practice of talk therapy. and as long as i an others licensed providers can protect our turf.

Comments are closed.