Articles

Is acupuncture as effective as antidepressants? Part 2. Blinding readers who try to get an answer

This is the second blog post about a recent PLOS One review claiming that alternative therapies such as acupuncture are as effective as antidepressants and psychotherapy for depression. The article gives a message to depressed consumers that they should consider alternative therapies as a treatment option because they are just as effective as conventional treatments. It gives promoters of alternative therapies  a boost with apparent evidence from a peer-reviewed journal that can be used to advertise their treatment and to persuade third-party payers that alternative treatments are just as effective as antidepressants and should be reimbursed.

In my first post, I could not reconcile what was said in this article with the citations that it provided. The authors also failed to cite some of their own recent work where it would have been embarrassing to arguments they made in the review. Most importantly, other meta-analyses and systematic reviews had raised such serious concerns about the quality of the acupuncture literature that they concluded that any evaluation of its effectiveness for depression would be premature

I will show in this blog post that the review article is misleading junk science, but that it also demonstrates the need for PLOS journals to publish citable critiques when they inevitably have let bad articles be published. Editorial oversight and prepublication review can never be perfect, but lapses in quality should be correctable in citable articles with the same status as the work they are correcting. Readers who find junk science in PLOS should not be left with the only options of blogging or commenting in a thread that requires handsearching to find, as they are now.

I will be criticizing the review for not providing readers with a means of independently evaluating what was being claimed, the idiosyncratic use of common terms that typically mean something else in other contexts, the unorthodox analyses integrating data across studies with voodoo statistics, and the ideologically driven conclusions.  Having read and re-read the article numerous times, I can find no better critique that what Don Klein said of an earlier article by one of the review article’s authors, Irving Kirsch:

a miniscule group of unrepresentative, inconsistently and erroneously selected articles arbitrarily analyzed by an obscure, misleading effect size

The article lacks transparency in not providing the basic background and data that one needs to interpret a systematic review and should expect to find there. Quantitative data for statements in the abstract and discussion sections are largely missing from the results section, except for what is presented in a bar graph claiming to make comparisons among 10 possible treatment and control conditions. But just try to figure out from the article which studies of what treatments went into constructing the bars.

Trust our authority, we did a systematic review.

Trust our authority, we did a systematic review.

 

Playing sleuth with obscure and misleading clues

I thought I knew was meant by “alternative treatments” because the introduction stated simply “alternative treatments such as acupuncture and exercise” and the method section said “we conducted a similar search for controlled trials of alternative therapies (exercise and acupuncture) for depression.” I couldn’t find any further formal definition of alternative treatments for the rest of the article.

However, when I accessed appendix S2 (I wonder how many readers actually go to the trouble of doing this), I found that the alternative therapies evaluated in combination with antidepressants for the “combination therapy” bar in the graph included sham acupuncture (1 study),  verum acupuncture (1), brief dynamic therapy (1), nurse led problem-solving (1), social skills training (1), relaxation training (2), and one cognitive behavioral session plus task assignment (1). This appendix also noted that the treatments that went into the bar graph as “alternative therapies” were aerobic training (3), autogenic training (1), exercise (8), running (2), acupuncture (3), and laser acupuncture (1). Active intervention control groups included acupuncture control (3) bright light therapy (1), inactive laser acupuncture [what’s that?] (1), low intensity stretching (1) massage therapy (1), relaxation training (2), and weightlifting (1). These were further lumped together with 16 other conditions including bibliotherapy, counseling, client centered therapy, parenting education, brief cognitive therapy, and marital therapy in order to form the “active intervention controls” group summarized in the another bar in the graph.

Some of this mixed bag of “active intervention control groups” are evidence supported therapies for depression, others are evidence supported, but not for depression, and others are simply untested or unproven treatments lacking a scientific rationale. But they are all simply lumped together in a single summary statistic.

The text of the article gave no citations for which study provided what treatment, so I went to Appendix S1 where all of the 115 treatment trials included in the analysis were listed (Again, who would bother to do this? Figuring out what-is-being-said-about-what should not be so difficult.) I could not consistently tell simply from the titles which articles evaluated which of these treatments or provided which control group or when acupuncture was being considered an active treatment or an active control.

There were only five acupuncture studies listed and so I looked them up. Three were from a single American research group, two from China. All were methodologically flawed, underpowered studies explicitly labeled as preliminary or exploratory. Only one involved antidepressants and only combined with acupuncture. None involved a test of acupuncture against psychotherapy. With no head to head comparisons provided by these studies, I do not understand why these authors did not give up and simply declare that there was insufficient quality literature for a comparison of acupuncture versus antidepressants or psychotherapy. Instead, the authors continued undeterred with their integration and meta analysis. I guess that they had a conclusion to get to.

The legend for the bar graph indicates the results for unblinded treatments are in red and blinded treatments are in blue. Try to figure out what is meant by “blinded,” because it is probably not what you came to the article expecting. Investigators evaluating psychotherapy are faced with the problem that their raters of outcomes might be biased if they knew which treatment patients had been receiving. Trials are commonly considered blinded if the raters do not know what treatment patients got, or if evaluations of outcome came directly from patients. There is evidence that patient self-report is actually a conservative means of evaluating interventions for depression, relative to interviewer ratings.

In contrast, this review considered ratings coming directly from patients as unblinded, i.e., inferior, and lumped self-report with results obtained in studies where raters who knew to which treatments patients they were assigned. There is no way of unpacking the studies considered unblinded because of patient self-reported outcomes versus those considered unblinded by the usual definition. So there is no way of comparing the results of this review with the existing literature. Readers relying on their familiarity with the psychotherapy literature to interpret these results will be misled.

journal.pone.0041778.g002-1

A meta-analysis by any other name….

The article had already gotten off to a bad start with the title and abstract identifying it as a systematic review, but not labeling it as a meta-analysis. I had a lively debate with colleagues about whether this article can even be considered a meta-analysis. We all agreed that it violates basic rules in conducting and reporting a meta-analysis and would yield unreliable results.

My opinion is that the systematic review is a meta-analysis because it involves integrating and analyzing  results from different clinical trials (therefore, meta-analyzing). There are established ways for systematically amassing and integrating data from clinical trials and well-known problems when alternative methods are used. Authors should not escape from these standards by simply failing to call their work a meta-analysis. The authors of the present “systematic review” should know better, because three of them have published peer-reviewed meta-analysis.

Meta-analyses are supposed to be the new gold standard for evaluating evidence from clinical trials. Best evidence was once the blinded randomized controlled trial, but such trials are often methodologically flawed and inconsistent in the results, and the failure of positive results to replicate is notorious. Meta-analysis supposedly holds the promise of being able to integrate data from across trials in ways that overcome the flaws of individual trials and resolves the apparent contradictions in their results.

But investigators conducting meta-analysis are often arbitrary in deciding what trials are included, how differences in their individual quality are taken into account, and how the various trials are lumped or split in integrating a set of trials and arriving at an overall interpretation. The ability of readers to determine exactly what was done and to evaluate for themselves whether conclusions are appropriate depends on authors adhering to standards with at least a minimal level of transparency. Authors must give readers enough information to decide for themselves. In the case of this article, Caveat Lector, unless you are willing to simply accept the authors’ sometimes outrageous claims, you going to have to either simply dismiss them or take the time to do your own literature search without much help from what is presented in this article.

A standard for evaluating meta-analyses and systematic reviews

The ‘Assessment of Multiple Systematic Reviews’ (AMSTAR) is a brief validated checklist for evaluating meta-analyses and systematic reviews, and can be accessed here. Following the numbering of the AMSTAR items, the article is flawed in in the following ways if:

3. Was a comprehensive literature search performed? The article claims there was a comprehensive search of the literature and even provides a flow chart of searches. But I could not map the flow of searches into the small number of articles ultimately included in the analyses, as seen in numbers in Appendix S2. And things were otherwise confused and confusing. The citations included in the text for the first search column indicates that the search largely overlapped with the searched indicated in the second column. Different search strategies were used for exercise and acupuncture than for psychotherapy. The authors started only with a Cochrane Collaboration review of exercise for depression and a Cochrane Collaboration review of acupuncture for depression, ignoring other meta-analysis of these treatments. In contrast, the author searched the literature for meta-analyses of psychotherapy and drew on them for studies to be included in their review.

5. There was no list of excluded studies. Readers are thus not in a position to evaluate whether the exclusion was appropriate and therefore whether the meta-analysis is balanced or biased. Obviously, lots of studies were excluded, but we are given no reason why. For instance, the flow chart says the authors started with an updated Cochrane Collaboration review of acupuncture and depression. I checked and found that Cochrane Collaboration review identified 41 trials and retained 30 for meta-analysis. Are we to believe that in only retaining five acupuncture studies , these authors of this review were wiser and had stricter standards than the Cochrane group? Maybe, but that should require some explaining.

Usually, peer reviewers insist on a table of excluded studies, and with an open access, internet journal, there is no worry about page limitations restricting what can be provided to readers.

6. Basic characteristics of the included studies were not provided. These data are critical for a reader to be able to evaluate the appropriateness of inclusion, as well as getting a clinical sense of the appropriateness of lumping or splitting studies for analyses. Again, why didn’t the PLOS One editor or peer reviewers insist on this?

7. There was no assessment of the scientific quality of the included studies. The authors note that some of the results included in the meta-analyses were limited to patients who completed the trial (but they do not say which or how many), while other studies were intention-to-treat analyses, i.e., all patients who were randomized included, regardless of whether they were full exposure to the treatment.

Consider this: it is meaningful information that substantial numbers of patients assigned to a treatment are not around to be assessed at the end of treatment. Not being available for follow up assessment is seldom random, and often occurs because of dissatisfaction with the treatment received. The purpose of a randomized trial is thus defeated if only data from completers are analyzed. Intention to treat analyses are the gold standard and we could expect that trials that did not rely on them were biased, and perhaps by other methodological flaws.

The scientific quality of included studies varied, depending on whether they were evaluations of alternative treatments like acupuncture versus antidepressants or psychotherapy. I already noted the poor quality of the acupuncture studies, as the Cochrane Collaboration and others have done. In contrast, the FDA Registry of antidepressant trials has rather strict methodological requirements, and so these trials should be of better quality, particularly after unpublished trials were accessed, as these authors did. To investigate the issue of systematic variability in the quality of trials being considered, I took another quick look at the titles of psychotherapy studies listed in Appendix S2.

Some appeared not even to be the main reports of a clinical trial, but post hoc secondary analyses. A number of the titles indicated that the studies were preliminary, exploratory, or pilot studies. Anyone familiar with the psychotherapy literature knows the authors often so label their studies after the fact, especially when something has gone wrong so they were unable to complete the study as planned or when they want to pass off statistically improbable positive findings from an underpowered study as valid. So, we need to keep in mind that type of intervention being evaluated is probably confounded with quality and bias in ways we really can’t readily disentangle.

8. The article’s provocatively stated conclusions did not take into account the scientific quality of research that was reviewed. It’s appropriate to qualify conclusions of systematic reviews and meta-analyses with acknowledgment of the limitations to the quantity and quality of available studies. For instance, it’s common for Cochrane reviews to declare that the quality of studies is insufficient to decide if an intervention is effective, as was the case in the updated Cochrane review from acupuncture. In contrast, authors of this review boldly and confidently stated conclusions when there are ample reasons to be skeptical and even to avoid yet making any conclusions.

9. Combining the findings of studies was patently inappropriate and there was no test of whether it should have been done. There was no quantitative evaluation of whether it was statistically appropriate to combine studies (statistical heterogeneity) nor consideration of whether the specific lumping of studies and conditions made sense clinically (clinical heterogeneity). As we saw, lots of questions could be raised about the lumping and splitting, particularly in the creation of the group of active intervention control conditions, which don’t at all seem to belong together.

10. There was no conventional assessment of publication bias, yet based on the past systematic reviews I covered in my last post, there was good reason to suspect rampant publication bias.

Combining results from different clinical trials

The authors’ strategy for abstracting results from different trials was flawed, produced biased results, and, like other aspects of this article, should have been caught by the PLOS editor and reviewers.

The standard way of summarizing results from trials is to calculate pre-post differences for the contrasting groups, take the difference in change between the groups, and standardize it.  So, we calculate how much patients in the intervention group changed, how much patients in the control group changed, the difference between the two groups, and then standardize this so that results have the same metric and can be combined with results of other trials.

Think of it: We might expect the results for interventions to vary with the trial in which they were evaluated, depending on how depressed the sample of patients were that was being studied and whether results for all patients who had been randomized were analyzed. We also should expect differences in the performance of antidepressants whether the drugs are administered in a trial advertised as recruiting patients for a comparison of antidepressants versus psychotherapy versus a trial advertised as comparing antidepressants versus acupuncture. Patient characteristics including their preferences and expectations matter. These are just some of many contextual factors that might influence the differences in effects that are found.

Thus, treatments do not have effect sizes, only comparisons of treatments do, and analyses have to take into account that the differences that occur are nested within trials and vary across trials. And the assumption that a set of trials are sufficiently similar in their results to be combined can be examined with statistical tests of heterogeneity—whether results of studies taken to be similar are sufficiently homogeneous or, if they are too different—heterogeneous–what the source of this heterogeneity  might be. The conclusion of many meta analysis is that results observed in a set of trials are too variable for the results to be meaningfully integrated.

So, if authors intend to lump under the heading “combination treatment,” antidepressants plus acupuncture, antidepressants plus nurse delivered problem-solving, and antidepressants plus relaxation training,  they are assuming that these are all equally representative of alternative treatment and results from them being added together generalize back to the individual treatments. In conventional meta analysis, whether this is reasonable can be tested, but it was not in this review.

What the authors did was radically different, simple, but nonsensical. They calculated standardized pre-post differences for each intervention separately and then averaged them into the numbers for the groups represented in the bar graph. They thus compared summary effects of particular broad groups of treatment to each other and to various lumpings of control conditions. I have never seen  this done except in other meta-analyses conducted by one of the authors, Irving Kirsch, and he routinely gets roundly criticized for his idiosyncratic method. Critics have noted that this method produces different results than the conventional method applied to the same data.

The authors’ method of calculating the effects of treatments ripped the treatments from the context in clinical trials and ignored all differences among those trials. They considered the treatments going into a group as if they were equivalent, and then made comparisons with other treatments and control groups.  If the authors approach were valid, we would not even need to conduct randomized clinical trials. Rather all we would have to do is simply recruit groups of patients, expose them to treatments, and collect and compile the results.

The authors also ignored whether the comparisons they were making were head-to-head, i.e, actually occurred in the clinical trials, or were indirect, artificially constructed by comparing the results for  treatments from very different trials. There is ample evidence that head-to-head comparisons are more valid.

In summary, the authors conclude that antidepressants and psychotherapy were not significantly more effective than alternative therapies such as acupuncture and exercise or even active control conditions. They arrived at this conclusion having only one flawed preliminary study from China comparing acupuncture to antidepressants and no comparisons between acupuncture and psychotherapy. They arrived at this conclusion when other meta-analyses were concluded that the poor quality of the existing acupuncture literature for depression did not allow any statements about its efficacy and when other meta-analyses of exercise for depression were concluding that it had at best short-term effects. A recent large-scale study failed to find effects of exercise for depression.

[I think an evidence-based case could be made for exercise as the first step in a stepped care approach to mildly and moderately depressed persons, or maybe the first step in a stepped diagnosis of persons who are having mood problems. If they recover and their improved mood continues, then a diagnosis of major depression and more intensive care are not appropriate. But the authors of this review are not interested such subtleties.]

They are quite impressed with their conclusions and state even more boldly

Our results also suggest the interpretation of clinical research evaluating relative efficacy of depression treatments using the randomized, double-blind paradigm is problematic. With the exception of waiting-list control and treatment-as-usual, it is difficult to differentiate active treatments from “treatment controls” in adequately designed and highly blinded trials.

Nonsense. I think that what they’ve really shown is that if one adopts idiosyncratic approaches to selecting trials and summarizing the data derived from them, one can arrive at conclusions that are quite different from more conventional syntheses of clinical trial data.

Postscript

At about the same time that this article was being accepted at PLOS One, one of its authors, Irving Kirsch was hyping in his CBS 60 Minutes News interview his conclusions from an article he had previously published in PLOS Medicine. His message was that any differences between antidepressants and pill placebo were clinically trivial. Even though the PLOS Medicine paper is among the most accessed and cited of any PLOS Medicine articles to date, the systematic review did not cite it, perhaps because of the apparent contradiction that its conclusion and the one being made in the review that antidepressants are better than pill placebo, but no better than acupuncture or exercise.

In either case, there is junk science and misleading mischief being perpetrated with the credibility of publishing in a PLOS journal attached. Kirsch’s PLOS Medicine article received lots of scathing commentary in that journal, but these are not citable nor directly accessible with electronic bibliographic databases such as ISI Web of Science or PubMed. Readers must have the persistence to get past the misleading abstract posted at these resources, go back to the actual article in the PLOS journal, and scroll down through the many comments. A number of frustrated commentators ultimately, and often much later, published extended critiques elsewhere. But readers should be able to find critiques going directly to PLOS.  I think the readers of PLOS journals deserve active post-publication debate in these journals that is citable and accessible through electronic bibliographic sources. If misleading abstracts of PLOS articles appear in those electronic resources, and they inevitably will, abstracts of critiques ought to appear there also. Let readers decide which side to believe.

It’s time for PLOS journals to stop allowing themselves to be used for dissemination of junk science and propaganda without the opportunities for debate and correction. The reputation of PLOS journals is at stake.

 

 

Posted in: Clinical Trials, Neuroscience/Mental Health

Leave a Comment (129) ↓

129 thoughts on “Is acupuncture as effective as antidepressants? Part 2. Blinding readers who try to get an answer

  1. Exilapotekare says:

    Not that long ago, when open access publishing was just starting off, I looked into publishing in PLoS One due to the great work that had been done (and still is being done) with PLoS Medicine and PLoS Biology. Thankfully something stopped me – and it was not the publication fee… It was the fact that PLoS One explicitly told everyone that the manuscripts would not be reviewed for quality, only methodology and whether the methodology agreed with discussions and conclusions. PLoS One would publish any investigation that is methodologically sound, with the effort to act as a co-financer of the more prestigious journals of PLoS (at that time Medicine and Biology)!
    Over the years this has become evident with an increasing number of articles without real merit in PLoS One, especially in the fields of medical interventions. I think we still only see the beginning of “worthless peer-reviewed publications” and the development of a multitiered journal hierarchy. That something can be found in the “scientific literature” has historically been a problem for “grey sources” (unpublished theses, obscure journals, small languages etc), soon it will be a problem of “(semi)predatory publishers”… PLoS needs to rethink their publishing model soon, or else they will be seen as nothing more than a scientific vanity press.

  2. James Coyne says:

    # Exilapotekare, I think you misconstrue my criticism of PLOSOne. I applaud its willingness to publish all methodologically sound articles because this is an important corrective to for profit journals publishing only positive results and leaving on published the results of negative trials. As Ben Goldacre has spelled out in Bad Pharma selective publication in the for profit journals leads to dangerous and ineffective drugs being promoted with citations of publications in high impact journals they convey a distorted view of how they work.

    We need to have all results of all trials available in print, but we need some assurance that these results are presented accurately and transparently so that readers can form their own opinions. Such a process will sometimes let some bad science and bad medicine into print, and that is why I am calling for citable post publication peer review.

  3. Xplodyncow says:

    I have severe treatment-resistant MDD. If it weren’t for this blog and excellent posts like this one, I could easily fall prey to alternative medicine — hell, I’ll try anything at this point — especially since Khan et al conclude, “These data suggest that type of treatment offered is less important than getting depressed patients involved in an active therapeutic program.”

    So thank you very much.

    P.S. potential COI: I work in evil Big Pharma; also, one med (bupropion) did wonders for me until it stopped working — so I take issue with those who claim that antidepressants are no better than placebo.

  4. Narad says:

    inactive laser acupuncture [what’s that?]

    I presume the reference is this. It specifies neither the manufacturer nor the wavelength used. This fellow appears to recommend the near-IR, which would be invisible.

    (Also: Seriously, PLoS is including MS Word files as supplementary material?)

  5. Narad says:

    P.S. potential COI: I work in evil Big Pharma; also, one med (bupropion) did wonders for me until it stopped working

    Been there. Gotta love it when the term of art is in fact “poop-out.”

  6. pmoran says:

    If I am reading all this correctly, the overriding problem is that you cannot compare these “effect sizes”, contrived as they are out of before-and-after comparisons within single arms of studies of different treatment methods, unless you knew that every detail of the study populations and their management was the same — which you can’t, even when pooled, this being the very reason for the R (randomization) and all the blinding in the usual double-blinded RCT. Without such an insecure footing we cannot say with confidence what this data means..

    OTOH, if the effects of antidepressants are, as posited, in substantial part due to placebo and nonspecific influences then it is not clear how better studies could settle the question of relative effectiveness of some of these treatment programs, so as to support sound informed consent.

    For example, it would be unethical now to perform a study comparing acupuncture and antidepressants in patients with severe depression. One suicide would be too many. Yet with mild depression the results could probably be easily manipulated either way. It is almost inevitable that at least some studies would favor acupuncture in cost/risk/effectiveness terms especially once drug side effects were taken into account.

    How would we react to that? On our answer hangs the justification for our determination to oppose acupuncture under all circumstances.

    There remain good reasons for the mainstream choosing not to endorse acupuncture, but they are clearly not based upon the application of a “gold standard” of evidence, once you start to consider “acupuncture” as the sum of a variety of nonspecific (but therapeutic) influences, as many indeed do nowadays.

    This statement from the paper thus probably has a substantial grain of truth at least wiht mild cases “These data suggest that type of treatment offered is less important than getting depressed patients involved in an active therapeutic program

  7. James Coyne says:

    #Pmoran This is an excellent summary and reaction to my blog post and gave me lots of food for thought. You hit the nail on the head in articulating why these authors’ calculation of effect sizes was an accurate and misleading.

    A previous paper by one of the authors of the acupuncture/antidepressant paper, Irving Kirsch, is also an author of what is among the most highly cited PLOS Medicine paper so far

    http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0050045

    and it thoroughly confuses the issue of whether the difference between an active condition and a placebo control group in some way identifies the size of the mechanism of the active treatment. Harriet Hall has blogged about this particular paper and effect sizes, maybe more than once

    I think that it is useful to compare an intervention and a pill placebo condition, under appropriate circumstances, to determine the relative efficacy of an intervention. However, that efficacy of an intervention could be due to mechanisms very different than what enables a pill placebo to have an effect. So, unless we have testable ideas about mechanism, we can’t assume that an intervention incorporates the same mechanism as a pill placebo in an additive way. Efficacy of an antidepressant is not necessarily equal to some biologic effect plus the effect observed with pill placebo in any simple additive way.

    I’ve had to think a lot about acupuncture as treatment contact with for depression. I think we can agree that there is only a nonsensical claim of any plausible biological mechanism and at least some semblance of a claim of testable mechanisms for psychotherapies. When we get to the clinical level of administering a treatment for depression to an actual patient, we often need to think about adapting the treatment to that patient, particularly when the initial treatment is not having satisfactory effects. Conventional psychotherapies allow a lot of leeway for therapists’ problem-solving. For instance cognitive behavior therapy can be adjusted if a depressed persons actually getting more ruminative when there’s so much focus on their thinking. The focus can sift to behavior activation or collaborative empiricism that gets the depressed person off their butt and out testing their ideas in the real world. That can be accommodated within the theory of mechanism. I don’t know what you would do if a patient is it responding to acupuncture. If you really believe in the mechanism, maybe you could move the needles or poke the person more often, but that gives limited options. What you think about that?

  8. David Gorski says:

    There remain good reasons for the mainstream choosing not to endorse acupuncture, but they are clearly not based upon the application of a “gold standard” of evidence

    Not really. They are definitely based largely on application of the “gold standard” of evidence, because, as we’ve pointed out time and time again on this blog, the better the quality RCT, the more indistinguishable acupuncture effects are from placebo interventions. It doesn’t matter where you stick the needles, nor does it even matter if you stick the needles in. Add to that the complete lack of anything even resembling a plausible physiological rationale for why acupuncture might “work” for the various conditions for which it is touted, and there is plenty of reason based on a combination of “gold standard” evidence and lack of biological plausibility to dismiss acupuncture as nonsense.

  9. James Coyne says:

    David, We are agreement on acupuncture. But, however ineptly, I was trying to point to the dilemma of someone who advocated it has a treatment, justifying it because all treatments are supposedly equal in depending heavily on a ritual and a rationale. As Khan/Kirsch argue in the PLOS article, even sham acupuncture is suitable.

    I don’t buy that, but suppose for the sake of argument, we assume that someone is going to acupuncture for depression and getting treatment, whether sham or otherwise. What Irving Kirsch and a lot of people don’t acknowledge is that many persons have to get some adjustment in what ever they are receiving for depression before they get a benefit. So, treatment can’t be a matter of a rigid routine, even with the rationale. My point is what would an acupuncturist offer under the circumstances? I think they would be stuck moving needles around. Psychotherapists, in contrast, would be guided by their sense of mechanism to try something different than they had been trying.

  10. Alia says:

    James Coyne: “Psychotherapists, in contrast, would be guided by their sense of mechanism to try something different than they had been trying.”

    And in some cases psychotherapists will admit their limitations and suggest that the patient should go to another therapist working within different framework. I’ve seen it happen from time to time. Or suggest going to a psychiatrist to augument psychotherapy with antidepressants.

  11. James Coyne says:

    Totally agree, #Alia, at least I would hope that they would. Quite difficult to predict ahead of time what evidence-based treatment will work for the individual depressed patient. And therapy + meds on average does better than either alone, particularly with more depressed patients. But you raise an interesting question: if acupuncture is obviously not producing improvement in a patient, where would the practitioner make a referral? Somehow I doubt it would be to a conventional mental health professional. Rather, to some a practitioner of other woo.

  12. nybgrus says:

    Dr. Coyne,

    Thank you for the post. It is from posts like these that I learn the most. I am still trying to make sure I fully understand the nuances since I have not had the time to really read it in detail.

    The difficulty I see with the application to acupuncture, as pmoran posits, is beyond the fact (as Dr. Gorski has pointed out) that “gold standard” evidence actually does show it to be nothing beyond placebo. Because, as I have argued here before, when it comes to psycotherapy many things can reasonably be considered an “active” intervention. Literally everything about simply being in a study, interacting with all the people involved, etc are actively affecting the pathological organ – the brain. The same cannot be said for actual disease of other organs; acupuncture will not affect the beta cells of your pancreas and fix your diabetes.

    The difficulty comes from scratching one micrometer past the veneer of the “therapeutic ritual” that is acupuncture.

    A patient comes to me as a physician, and is depressed. I offer:

    …pharmacotherapy. “How is that going to help me?” “Well, there is data… neurotransmitter theory of depression… corrects imbalance… has been shown to be reasonably effective” “How does it do that?” “By inhibiting the reuptake of serotonin… etc”

    …talk therapy. “How is that going to help me?” “Well, there is data… neural plasticity… working out problems verbally and addressing them can actually help restructure brain pathways… has been shown to be reasonably effective.” “How does it do that?” “Your brain can actually change connections and restructure pathways if you give it the chance… talking through your problems can actually help literally change your brain directly, which is reflected by your slowly changing thoughts and attitudes…”

    …acupuncture. “How is that going to help me.” “Well, there is data… which shows that other stuff that works might not be better than acupuncture… and some that shows it can maybe help people who have failed other therapy…” “How does it do that?” “Well, the needles don’t actually do anything… you just have to believe that they will… because really all that is going on is non-specific interactions between you and some nice old Asian man… and that can restructure your neural pathways…”

    “…so why are you telling me to get needles stuck in me again?”

    Of course, your point about adjusting therapy is just as good, if not better, a reason. But really, you just have to either believe the needles do something yourself or hope that your patient doesn’t ask the simple question of how it works. So you are either incompetent or unethical. Because the moment you admit it has nothing to do with needles, you are stuck trying to justify how that is a better option than a day at the spa, a concert, or actual psychotherapy. Or you have to tell your patient that it will realign his meridians to allow the chi to flow properly and adjust his attitude. Which means it doesn’t matter if it is an acupuncturist who will refer you to a naturopath or a physician doing the acupuncture. It’s just a non-starter.

    Tomorrow, when I have some more time and a fresh head, I want to re-read and delve more deeply into your critiques as a learning exercise. But I felt it incumbent to point out that if the rationale for acupuncture in depression is the ancillary patient-practitioner interaction, then psychotherapy is a much better and more ethical option, despite the fact that I would agree that the non-specific aspects of acupuncture in this case actually are active interventions directly treating the disease process (though in a roundabout way via lying – by ommission or otherwise – about the nature of acupuncture).

  13. pmoran says:

    James Coyne: What Irving Kirsch and a lot of people don’t acknowledge is that many persons have to get some adjustment in what ever they are receiving for depression before they get a benefit.

    I am not sure how you know that. Are there studies? It may merely mean that “tincture of time” or unknown incidental happenings in the patient’s life have at last worked some magic.

  14. James Coyne says:

    # nybgrus I’m not sure I could’ve reticulated your points better than you did and I certainly agree with them.

    Having almost obsessively read this PLOS one article in order to provide the detailed critique I have across these two blog posts, I’m convinced that these authors had no particular confidence in acupuncture as a treatment for depression. Rather, they were trying to stay on message in making the point that one of them, Irving Kirsch, has made so often before, that all that we do for depression is simply a matter of placebo. Unfortunately in the case of this PLOS one article, they had to really torture the data that really didn’t support this point.

    #Pmoran evidence-based psychotherapy for depression is highly manuallzed and a therapist who is not achieving the intended result can simply follow the manual and shift gears.

    Practice guidelines for antidepressant treatment of depression recognize the patient should be showing improvement at five weeks and if there is no improvement, some action should be taken. Actions might involve encouraging adherence with patient education, adjusting dosages, or shifting medication. Unfortunately, in primary care treatment of depression, where most treatment occurs, there often isn’t an assessment at five weeks and so overall outcomes are quite poor and substantially worse than what is achieved in clinical trials.

  15. This is a great deconstruction of an article, and a great investigation into what really helps for depression.

    I want to add two aspects that others are addressing, but not in the way I as a psychologist see them.

    I have been trained in a very mainstream, normal way, including with an eye toward evidence. We psychologists (some of us – there are a lot of off-the-map psychologists out there) recognize that one factor that works fairly reliably for depression is “behavioral activation.” Doing something rather than nothing.

    With a behavioral approach, specific behaviors would be determined to counter specific aspects of the depressive experience, but overall a certain level of activity would be encouraged, and also “pleasurable experiences” would be encouraged, since often a depressed person has a notable imbalance in daily life of “unpleasurable” to “pleasurable” experiences. It is nice to prescribe someone to enjoy an ice cream per day, or a relaxing bath per day.

    “Behavioral activation” is the concept in my mind when I see “exercise,” weight-lifting,” and even “bright light therapy.” Doing something is better than nothing, being proactive may boost self-efficacy, and other various modest effects may be involved, such as social involvement. The effect may be a “fake it til you make it” effect, with self-efficacy and increased decent social interaction experienced as a side effect of increased activity.

    So, hopefully this gives others a concept for seeing the plausibility of a varied range of interventions that otherwise seem pretty random, and seem like they are interventions somehow parallel to therapy or medications. I just see them as “behavioral activation.”

    But I also mention this to destroy the idea that these are “alternative therapies.” They are components of a decent cognitive-behavioral intervention. While exercise is not itself a “psychotherapy,” your counselor should be assessing your daily activities and interactions, and figuring out where you might be lacking, and making it a priority to convince you to get active, to encourage/push/motivate, and to problem-solve your failure to get this going. That is part of decent psychotherapy for depression: the therapist should be helping you fake like you are in the swing of things, until you do get the normal drive or desire back to do this. It is a real boot-strapping thing.

    Also, my second comment is that one man’s placebo effect is another man’s “nonspecific therapeutic factors.”

    Irwin Yalom’s book on group psychotherapy is the classic source to grasp the influence of “nonspecific therapeutic factors” as active ingredients in therapeutic change. Yalom was considering group psychotherapy, so he does not discuss “depression” so directly as he discusses progress and change generally.

    You can probably google “nonspecific therapeutic factors” to find the list, but they include things such as “instillation of hope” and “universality.” In clients who begin to improve, these factors are endorsed as being perceived by the client. Those not improving don’t perceive these phenomena.

    Along with Yalom’s nonspecific therapeutic factors, Carl Rogers came to the opinion that “unconditional positive regard” is a necessary condition for improvement. Again, he did not focus so closely on the medical model concpet of “depression,” but addressed a range of mental health difficulties. Unconditional positive regard means that the therapist, no matter what, has a favorable opinion of the client, and conveys this simply by a decent, respectful interactional style. This includes our listening skills: nodding and saying “um-hummm” to show you are following a discussion, parroting, reflection, summarizing, posture, etc., are the techniques or ingredients that sum up to convey the attitude of “unconditional positive regard.” you can google that, also.

    For us psychotherapists, these “nonspecific factors” are bread and butter therapeutic phenomena.

    Outside of psychotherapy, others don’t quite conceptualize these in this way, and they are comprehended in a vague way, and labelled “placebo effects.”

    By theory and evidence, change happens by the practice of specific (identifying and challenging depressogenic thought patterns, behavioral activation, relaxation, autogenic training, etc.) and nonspecific factors (mentioned just above.

    These ingredients will often work even if not delivered in a psychotherapeutic context. Why would they not? If they work, they work. Hence, a support group or good friend can provide a certain dose of these.

    When a client goes to an acupuncture session, or goes through the detection-and-intervention process for antidepressant medication from GP or psychiatrist, plenty of these nonspecific factors are involved.

    So, I encourage people to grasp these concepts and learn about them. “Behavioral activation” and “nonspecific therapeutic factors.” I believe it will help you see what is happening when a person shows a good response to acupuncture or placebo drug, or “autogenic training.”

    As an example, this helps me understand how acupressure can yield the same degree of effect as acupuncture, and ends up not being such a good “control” condition for an acupuncture trial.

    JC, thanks for all the work on this topic.

  16. “Blinding” readers is funny: the quadruple blind – pt, provider, assessor, and study reader.

  17. mousethatroared says:

    MedvsTherapy – If I had the option, I would “like” your comment.

  18. James Coyne says:

    Me too, and yours too, # MedsVsTherapy.An update on the effects of this pair of blog posts.

    It has stimulated a lot of exchange within the huge editorial board of PLOS One, with all of the responses highly positive and supportive. It is generally acknowledged that my critique exposed an article that had escaped adequate peer review. But the dilemma is that detailed commentaries such as I provided in my blog posts do not yet have a citable, indexable in ISI web of science, outlet in PLOS one. Everyone seems to be admitting that that’s a problem, but has someone said, changing that policy of a huge Journal with tens of thousands of reviewers and over 15,000 papers published last year, is like trying to change the course of the Titanic.

    I’ve been invited to do a third posting on this article at the PLOS blog Mind the Brain that specifically takes the perspective of a bewildered depressed person trying to choose between psychotherapy and acupuncture on the basis of what they read in the scientific literature. Stay tuned…

  19. pmoran says:

    MedVstherapy, As an example, this helps me understand how acupressure can yield the same degree of effect as acupuncture, and ends up not being such a good “control” condition for an acupuncture trial.

    Yes! Those studies are not truly testing efficacy, and certainly not overall effectiveness within practical medicine. They are testing out aspects of Chinese medical theory and practice that we would normally not bother considering, regarding them as having no important, or unique, physiological significance or plausibility.

    Nevertheless, in SBM-speak the failure of minimal variants to obviously outperform each other somehow morphs into a bald “‘it’ doesn’t work!”. What is it that isn’t working, when we are also obtaining those unusual effect sizes from such variants?

    Nybgrus: The difficulty I see with the application to acupuncture, as pmoran posits, is beyond the fact (as Dr. Gorski has pointed out) that “gold standard” evidence actually does show it to be nothing beyond placebo.

    You may not have noticed that Dr Gorski’s response was not to the point. The critical context-setting ending to this sentence of mine (highlighted below) was cut out out.

    There remain good reasons for the mainstream choosing not to endorse acupuncture, but they are clearly not based upon the application of a “gold standard” of evidence, once you start to consider “acupuncture” as the sum of a variety of nonspecific (but therapeutic) influences, as many indeed do nowadays.

    For one thing, as is now made clear, there is no body of quality head-to-head comparisons to go on. It is not even quite clear what “quality evidence” would consist of in such a study, siince either side could easily boost reported outcomes by maximizing various non-specific influences and their accompanying element of reporting bias.

    The answer probably is that there is no answer at present, and there probably won’t be until or unless we can somehow focus upon more objectively measurable outcomes, or develop objective surrogates for true treatment efficacy.

  20. Narad says:

    But the dilemma is that detailed commentaries such as I provided in my blog posts do not yet have a citable, indexable in ISI web of science, outlet in PLOS one. Everyone seems to be admitting that that’s a problem, but has someone said, changing that policy of a huge Journal with tens of thousands of reviewers and over 15,000 papers published last year, is like trying to change the course of the Titanic.

    I don’t think I get this. The publication strategy is already haphazard. The addition of citable rebuttals only represents another layer. Does Thomson Reuters even care?

  21. nybgrus says:

    I’m probably going to regret getting into this again so I will do me best to be brief.

    You realize, pmoran, that you are now in essence arguing that there is no data that we can actually parse to answer the question of acupuncture and the “non-specific therapeutic ritual” of it when we just had it out about how you claim there are piles of data to support these “surprisingly large effect sizes” from acupuncture.

    So which is it? Do we have data demonstrating these effect sizes or do we not?

    You then go on to say that what we have is essentially a giant hodge podge of data from studies that aren’t designed to give us the info we are seeking on the topic, so we can’t really draw our “skeptically dogmatic” conclusions… yet you can somehow use that to infer utility of acupuncture?

    And you completely gloss over the fact that the vast majority of research on acupuncture is done by people who actually are looking at the purported putative mechanisms of it… not “as the sum of a variety of nonspecific (but therapeutic) influences, as many indeed do nowadays.” It is we skeptics who are the ones seeing it as the sum of a variety of nonspecific influences. Not the people actually studying it!

  22. James Coyne says:

    #Narad. With respect to original research, the publication strategy of PLOS One is coherent and represents a strong corrective to confirmatory bias and selective availability of positive studies in conventional print journals. Studies have to methodologically adequate and transparent about their limitations. This cannot occur perfectly and thus the necessity of citable, indexable corrective commentaries.

    http://www.plosone.org/static/reviewerGuidelines#criteria

    Criteria for Publication

    To be accepted for publication in PLOS ONE, research articles must satisfy the following criteria:

    The study presents the results of primary scientific research.
    Results reported have not been published elsewhere.
    Experiments, statistics, and other analyses are performed to a high technical standard and are described in sufficient detail.
    Conclusions are presented in an appropriate fashion and are supported by the data.
    The article is presented in an intelligible fashion and is written in standard English.
    The research meets all applicable standards for the ethics of experimentation and research integrity.
    The article adheres to appropriate reporting guidelines and community standards for data availability.

  23. Narad says:

    With respect to original research, the publication strategy of PLOS One is coherent and represents a strong corrective to confirmatory bias and selective availability of positive studies in conventional print journals.

    Yes, but I said haphazard, not incoherent; the intended target was breadth, which goes to the size of the “editorial board.”

    Studies have to methodologically adequate and transparent about their limitations. This cannot occur perfectly and thus the necessity of citable, indexable corrective commentaries.

    Again, I get this. What I’m not getting is the actual obstacle to implementing the goal. I take it that such commentaries, if indexed as part of the journal, should be interpretable as being up to the standards embodied by the journal. Is this the intersection, in that the model doesn’t have a method to apply an imprimatur to a response that the input gantlet doesn’t expect to see?

  24. pmoran says:

    Nybgrus: I’m probably going to regret getting into this again —

    No, you’ll be fine, eventually. It’s simply a very difficult notion to us to entertain with our conventional medical background and its (largely understandable) biases, that, to make the point quite clear, a strong collection of placebo and other non-specific influences could match weakly active pharmaceuticals when caring for patients with certain illnesses.

    But it could be true. There already studies suggesting this possibility (including the present rather flawed one), but for the reasons that I have mentioned it is difficult to interpret them. We do need to be able to respond to them in a more sensible way than referring to those rather irrelevant differences between nominally “real” and “sham” acupuncture.

    This is uncharrted territory for all of us.

  25. evilrobotxoxo says:

    Most of the recent posts on this forum seem to miss the fact that most of the “placebo” response in antidepressant trials does not come from a real placebo effect, but from spontaneous remission. Depression is by its very nature a cyclical disorder. Most people will eventually remit spontaneously even without treatment, and the goal of antidepressants is to speed remission and to prevent relapse later on. There have been trials where they compare people in the “placebo” arm of a trial to people who are on a waiting list to be enrolled in a trial, and they’ve seen indistinguishable remission rates between the two groups. This isn’t something that’s been studied well enough for us to know how much of the effect in the placebo arm is due to nonspecific therapeutic effects vs. spontaneous remission, but the data we do have suggests that spontaneous remission is the vast majority of it. There’s an underlying narrative that depression is particularly susceptible to the placebo effect because it’s “all in your head” or somehow not a real disease, but it’s more likely due to the fact that it’s simply cyclical.

    The second thing is that antidepressant trials are extremely suspect because it’s become nearly impossible to recruit a suitable patient population. It used to be that there were lots of depressed people who were untreated, but now the vast majority of them are treated by their primary care provider, so whoever you have left is treatment-refractory, or more likely, an unemployed person who will say anything or feign any symptoms to get a check. A lot of these people are enrolled in 5-6 clinical trials simultaneously. It’s hard not to be too cynical, but the studies we have are generally of such low quality (or studying things in such an unrealistic way) that it’s hard to make treatment guidelines based off of them.

  26. evilrobotxoxo says:

    p.s. the most compelling evidence of antidepressant efficacy is not in treatment of acute depression, but in prevention of relapse. Those studies show that people who stay on antidepressants vs people who are tapered off to placebo have something like three-fold lower relapse rates. The evidence of SSRIs in anxiety disorders is also more compelling. So Kirsch and other antidepressant denialists are playing a game similar to creationists who keep talking about some weak spot in the fossil record while ignoring all of the more compelling genomic data.

  27. nybgrus says:

    evilrobotoxxx:

    Thanks for that. I think I knew that somewhere in my head, but it wasn’t something at the forefront of my conscious thoughts on the matter. I think it is quite helpful to augment my thoughts on this topic.

    pmoran:

    I don’t know what you are on about. I’ve even said it again in this very thread – I have absolutely no reservations about so-called placebo having a direct, tangible, and measurable benefit in depression (and other psychiatric disorders to varying degrees). I also explained why I think it reasonable to consider them active treatments and they are mislabled as placebo effect, except insofar as that we lump everything that is not the dependent and independent variables into that category.

    And I’ve also said it innumerably – if I have a bias it should be in the favor of CAM and placebo. I was educated to believe they were both valid constructs and powerful. It was after doing ever more reading in depth that I was forced to modify my outlook to be in line with the evidence.

    Now the only leg you may have to stand on regarding this particular topic is if the nonspecific effects of acupuncture/pressure/magic joo joo are actually more consistently beneficial and/or with a much larger effect size than psychotherapy. However, there is absolutely no a priori rationale for this to be the case and zero evidence to support the notion. So it is very unscientific to insist strict agnosticism towards the topic – we can at best say we are agnostic but it is unlikely. And even if it were true, then it would be our goal to tease out why and remove that from the BS that is acupuncture itself. Because as I said in my first comment here, at some point in some way you must be lying to your patient in order to treat their depression with acupuncture. So no matter the effect size or consistency demonstrated at some future time, our hands are bound.

  28. mousethatroared says:

    @evilrobotxoxo – good comment too! I’m curious, would you say then, based on the other information we have – wait list compared to placebo – that the benefits found from accupunture are more likely due to cyclic remission than nonspecific therapeutic influences?

  29. evilrobotxoxo says:

    @MTR: yes, I’m saying that the “placebo” arm should be called the nonspecific effect arm. Nonspecific effects include spontaneous remission (usually referred to as “regression to the mean” for conditions that aren’t necessarily cyclical), nonspecific therapeutic influences (i.e. the “placebo effect”), dishonesty/unconscious bias (patients and researchers artificially inflating a patient’s symptoms to get them enrolled in the trial, then artificially deflating them out of a desire to see people get better), etc. The problem with the concept of the “placebo arm” of a trial is that it implies that the apparent treatment responses in untreated people are due to the “placebo effect,” when in reality they’re due to a bunch of different factors.

  30. akhan says:

    This is Arif Khan, MD, the primary author of this manuscript. I am happy to note that this manuscript has generated discussion which as you all know is the point of scientific publication. First and foremost, my intention (in part representing all of the authors) was to present depression research data, a journey analogous to the one taken by Alice in her wonderland. There are a lot of advocates and believers, but relatively few critical thinkers.

    Having said I would like to make two further points. Specifically, conducting a meta-analytical synthesis is not easy or the data adequate to come to firm conclusions. However, these data represent a common sense and clinical problem that physicians and allied health professionals face everyday. Here is my version that I couldn’t put in a manuscript, but can address in a setting like this.

    Of every hundred human beings who develop the depressive syndromes, about 10 will ‘improve’ with time alone, the so-called waiting list, spontaneous remission etc being the descriptors.

    Of the 90 patients left, about 20 to 30 will get better whether they see an African shaman, Indian Sadhu or get fake accupuncture. There are several descriptors that people have used, with of course placebo being the most misused and abused word. As many of you on this website have stated it is the non-specific therapeutic effect.

    Besides these so-called controls, it is likely that another 5 to 10 will ‘improve’ if they see somebody who has more of an expertise with mood disorders. Specifically, forms of psychotherapy and antidepressants. For both of these, you have to pay the piper. Psychotherapy can go wrong and is expensive and time consuming. Antidepressants are simple, foster magical thinking, are easy to take, although you are left with the side effect burden, which luckily in my own life time has changed for the better.

    Further intensity and intrusion such as combined medication and psychotherapy, medication combinations, light therapy, ECT, TMS all will all enhance the proportion of those doing better by another ten or so.

    So, here is the crux of the problem. If you have added right, we still have more than 40 of the original group of depressives not ‘improving’.

    The take home message from me as the author and clinician is that our understanding and treatment of depression is not as good as it should be. At best, the emperor is partly clothed, not naked as some would like to point out. But, hopefully, we can get him clothed and in better shape, but this is only possible if we appreciate what is missing. Arif Khan, MD

  31. Dirk Stool says:

    Has anyone ever considered here, or has any scientific research been presented to show that possibly a change in a depressed persons individual circumstances may be most effective. You know… having a more fulfilling job, more money to pay the bills, a better relationship or even a nicer society to live in. Has any research on these issues been done? Or are we stuck with defining this as a medical condition to be ‘cured’ by pills, therapy, acupuncture or placebo?

  32. Alia says:

    The problem is, it’s not always the case of individual circumstances. Take my sister – she had a fulfilling, well-paid job, interesting hobbies, some close and not-so-close friends with whom she could talk and have fun, support in her family (that’s me). And depression. She went through group therapy, which helped her a lot and after 10 years I hope we can safely assume that her episode of depression is over.
    Or take a friend of mine – she’s been at least mildly depressed throughout her life, always sad, never felt true joy. All that was until she went to a psychiatrist who found a right antidepressant for her. A few months ago she said something like “Now I finally know what it’s like to feel happy”.
    So I guess we can sum it up “It’s complicated”. Yes, in some cases a change in individual circumstances will help. In some it won’t.

  33. Dirk Stool says:

    Alia says ‘Take my sister – she had a fulfilling, well-paid job, interesting hobbies, some close and not-so-close friends with whom she could talk and have fun, support in her family (that’s me).’

    That is your view. What was hers?

  34. Alia says:

    Hers was unhappy love afair. And OK, maybe that’s individual circumstance – but you must admit that it’s sometimes hard to find someone to happily fall in love, just like that. And barring that, group therapy was the second best thing for her.

  35. Narad says:

    You know… having a more fulfilling job, more money to pay the bills, a better relationship or even a nicer society to live in. Has any research on these issues been done?

    Research such as somehow “prescribing” these items?

  36. bs king says:

    @Dirk Stool

    As a trained (but not practicing) therapist,, I can say there actually has been at least some research dedicated to this topic. The ones that spring to mind are in regards to income….while there’s evidence that incomes over certain amounts do not do much to increase happiness ($45k/year for one person is the number if I remember correctly), amounts below that most certainly increase depression risk (can’t find the study now, I’ll have to look for it later).

    As for a nicer society:
    http://www.ncbi.nlm.nih.gov/pubmed/23349294

    I think some of the problems with treatment come in when either 1. there’s no contributing problem the patient can point to or 2. the contributing problem is unsolvable (or very difficult) or 3. an underlying mental illness makes a lousy life circumstance harder to deal with. I dealt a lot with #2 when I ran grief groups through a hospice. The loss of a loved one was not fixable, but we did work hard to help people find appropriate resources to help them deal with the practical matters around their loss before referring them to a doctor to talk about medication.

    It must be noted though that (at least in my state) a master’s level therapist who does not inform a patient that medication is available and could help can be found negligent and liable for damages if the patient later goes on medication and finds it works. I think this duty to inform can sometimes be construed by people as “pushing” meds…..and of course there are some providers on all levels who do actually push meds more than I think they should.

    I’m sure I can dig up more studies on the topic if you’re interested.

  37. mousethatroared says:

    I remember the first time I sought help from a therapist/psychiatrist who prescribed the Paxil I was taking and CBT. I talked to my brother afterwards. He said “How can you be depressed. You have a good job you love, live in a great town. It sounds like you have great friends…what could be so wrong?” I said, “Yeah, that’s kinda the point. I’ve got all that, but I still feel bad.”

    I’m sure life circumstances can send people into depression, In fact I know it’s happened to me. But sometimes things just change, hormones-something in your brain- whatever and you feel very different about basically the same circumstances. Kinda hard to explain to someone who hasn’t experienced it.

    The thing is, assuming that your feelings are always dependent upon circumstances can be very problematic, because you keep trying to fix something that’s basically not broken to make yourself feel better. Sometime, the reality is, your life is fine, your brain (hormones, whatever) need work. :)

  38. pmoran says:

    evilrobotxoxo:Most of the recent posts on this forum seem to miss the fact that most of the “placebo” response in antidepressant trials does not come from a real placebo effect, but from spontaneous remission

    That ‘s true. Spontaneous remissions will indeed tend to dampen any differences found when treatments are compared and it is one obvious factor in why so many different treatments seem to have about the same order of “effect” upon depression. I suspect that the SR rate is considerably more than Arif’s 10% in the population that the usual family doctor will be asked to treat.

    However, as you go on to allow, there should be similar rates of spontaneous remission in a waiting list or “usual care” group. In most studies where comparisons are made to those, complex, patient-involving and ongoing interventions like acpuncture outperform those groups by a substantial margin (0.5 effect sizes) — and with many conditions, not only with depression.

    Yet –

    There have been trials where they compare people in the “placebo” arm of a trial to people who are on a waiting list to be enrolled in a trial, and they’ve seen indistinguishable remission rates between the two groups

    Presumably those studies used a pill-placebo, which may merely have performed poorly under the conditions of those particular studies. Wouldn’t most anti-depressant researchers do their best to reduce placebo responses?

    The present question concerns more active, involving and complex interventions in an almost certainly more promising therapeutic environment.

    I state again that have no particular brief for acupuncture, and I have nothing against antidepressants (– well, some tiny nagging suspicions, perhaps — but who wouldn’t have those?). I simply feel we should be being more careful in what we are saying on certain matters. As Arif implies, we cannot yet be quite sure what our own evidence means.

    So I see no grounds for changing usual medical practices, but grounds for less hostility to other claims, for now, when treatments are safe.

  39. Dirk Stool says:

    Alia

    ‘Hers was unhappy love afair. And OK, maybe that’s individual circumstance – but you must admit that it’s sometimes hard to find someone to happily fall in love, just like that. And barring that, group therapy was the second best thing for her.’

    Richard P Bentall has continually pointed out that therapy – any therapy where the ‘patient’ believes someone is actually caring will cause an improvement in depression. This is why acupunture may work. This is why antidepressants may work. But I do not believe it. Improvement in personal circumstances will work and will result in less relapse. How this condition (depression) has become a ‘medical’ problem and not a socialogical problem is beyond belief to me. Maybe you can convince me otherwise.

  40. Dirk Stool says:

    # mousethatroared

    ‘I’m sure life circumstances can send people into depression, In fact I know it’s happened to me. But sometimes things just change, hormones-something in your brain- whatever and you feel very different about basically the same circumstances.’

    Maybe. I am only asking to see the science behind this. Where is it?

  41. Dirk Stool says:

    # bs king

    “As a trained (but not practicing) therapist,, I can say there actually has been at least some research dedicated to this topic. The ones that spring to mind are in regards to income….while there’s evidence that incomes over certain amounts do not do much to increase happiness ($45k/year for one person is the number if I remember correctly), amounts below that most certainly increase depression risk (can’t find the study now, I’ll have to look for it later).2

    Yes I am aware of this study. But not sure I understand your point here. So everyone who does not earn $45k per year is suffering from a medical condition called depression to be cured by pills? Do you think this?

  42. mousethatroared says:

    Dirk Stool – Oh well, I haven’t yet seen your evidence that depression is caused by life circumstances, either. Why is that your default?

    Are you assuming that ALL mental disorders are caused by life circumstances or do you think that depression is distinctly different from anxiety disorders, schizophrenia, bipolar, etc?

  43. Narad says:

    Improvement in personal circumstances will work and will result in less relapse.

    How do you propose implementing this? Take two bootstraps and call me in the morning?

  44. mousethatroared says:

    A second response

    Dirk Stool “Maybe. I am only asking to see the science behind this. Where is it?”

    I think it’s published in books and journals and stuff…maybe you should start in the library.

  45. Dirk Stool says:

    I would just like to point out that the ‘placebo effect’ only reports on the subjective experience of the patient. So for depression the HAM-D checklist is used. There may be many reasons (Milgram Experiment etc) why people may present false information to a doctor. There is also a report comparing patients reports to the efficacy of ECT to the doctor and another person which shows major discrepancies. I actually do not believe in the placebo effect.. mind over matter?.. based on such research as this http://europepmc.org/abstract/MED/8942043/reload=0;jsessionid=hgCStEFaCEitO6FEBwe9.2. I could cite other research that reaches similar conclusions. I think a further blog is required here rather than just random comments…

  46. Dirk Stool says:

    # mousethatroared

    ‘Dirk Stool – Oh well, I haven’t yet seen your evidence that depression is caused by life circumstances, either.’

    OK. So you think depression is not caused by life’s circumstances but by something else. I am not sure I agree with you here. What do you think the ‘else’ is? Evidence would be welcome. Of course I know that depression can be caused by well known medical conditions. Thyroidism for example. But a ‘mental disorder’ I am not so certain about.

    “Are you assuming that ALL mental disorders are caused by life circumstances or do you think that depression is distinctly different from anxiety disorders, schizophrenia, bipolar, etc?”

    There are almost 400 mental disorders (DSM5). Do you think all of these are NOT caused by life’s circumstances? Where is your evidence. Can you cite some scientific research please.

  47. Dirk Stool says:

    # Narad says

    ‘Improvement in personal circumstances will work and will result in less relapse.

    How do you propose implementing this? Take two bootstraps and call me in the morning?’

    I am only suggesting that the best way to effect a cure is first and foremost an understanding of the cause. Maybe our only ‘cure’ is to tranquillise. Do you support this method?

  48. Dirk Stool says:

    Anyway I do not have answers. I just want to get the questions right first.

  49. mousethatroared says:

    (Sigh)

  50. Narad says:

    I am only suggesting that the best way to effect a cure is first and foremost an understanding of the cause. Maybe our only ‘cure’ is to tranquillise. Do you support this method?

    Answer the original question.

  51. James Coyne says:

    Wow, I went down to the coffee shop and this thread really lengthened. Not to slight some of the comments that have been made, but I would like to return to Dr. Khan who so graciously joined the discussion.

    #arkhan

    Thank you Dr. Khan for offering your opinions about African shaman, Indian Sadhu, fake acupuncture, and antidepressants. However, in two blog posts and the accompanying threads, we have been discussing your systematic review that appeared in PLOS One. Because it is labeled a systematic review, we would expect that a systematic methodology been applied to arrive at some evidence-based conclusions. However, once I looked at the review more closely, I see that it concluded with a set of opinions more akin to what you are expressing here then to what the data supported.

    I confess to not being able to follow the journey through the literature with your systematic review as a guide, because it’s lack of transparency as to just what was done and the links between what was done and your conclusions was, to borrow an apt phrase from you, “a journey analogous to the one taken by Alice in her wonderland.”

    Let’s start with the acupuncture literature. When I went to systematic reviews available in the literature, I came to a very different appraisal than the one offered in your review. The impression I got was that the acupuncture literature was largely untrustworthy and that with reference to depression, the low quality studies did not afford any evaluation of its efficacy. One narrative review even raise doubts as to whether acupuncture trials described as randomized really were randomized, in the conventional Western sense.

    It was very difficult to see how you selected five of the many studies of acupuncture for depression available, but three were from a single American group and two were from China. All three of the papers expressed reservations about being accepted as anything but preliminary or exploratory. None of them provided any head-to-head comparisons between acupuncture and antidepressants or acupuncture and psychotherapy. I don’t see where you felt confident in making conclusions about these issues in the absence of such comparisons.

    When it came to extracting, summarizing and integrating data, you applied an approach that I’ve never seen outside of an article co-authored by Irving Kirsch, who happened to be a co-author in this instance. Your approach is novel and I would appreciate you showing me where anyone else has used it besides Kirsch and co-authors. Having done meta-analyses before, you are undoubtedly familiar with the pitfalls and problems in integrating data from diverse studies. There are tools for determining whether the studies are too heterogeneous in designs and results to attempt an integration, but of course those tools are quite fallible. However I see no evidence that you attempted such a check on what you did. As I pointed out in my blog post, and please correct me if I am wrong, the approach that you took destroyed all benefit of the data coming from randomized trials. To get to a more concrete level, what basis do you have for confidence that your comparison of a sham acupuncture group with antidepressants can legitimately be made? Can we even conclude that similar patients are being studied in acupuncture studies, psychotherapy studies, and evaluations of antidepressants?

    I could go on and on about the oddities of your systematic review and its departure from established standards and its conduct and its reporting. But let’s start with these questions.

    Again thank you very much for joining the dialogue.

  52. Dirk Stool says:

    Narad

    ” I am only suggesting that the best way to effect a cure is first and foremost an understanding of the cause. Maybe our only ‘cure’ is to tranquillise. Do you support this method?

    Answer the original question.”

    Sorry. Which was? I did say ”Anyway I do not have answers. I just want to get the questions right first.”

  53. Dirk Stool says:

    @mousethatroared

    Why do you sigh?

  54. mousethatroared says:

    Dirk Stool – I’m glad that you don’t think you have the answers. Neither do I. But if you think that anti-depressants and Cognitive Behavior Therapy (The two most evidence based treatments for depression, I believe) are akin the tranquilizing, then it seems that you have some incorrect preconceived notions.

    Personally, as a patient, I feel somewhat at a loss. Do you expect someone to deliver you neuropsychology 101 on a blog? I’m afraid I don’t understand where you are coming from.

    There probably is some sort of general blog on the science behind psychiatric disorders…somewhere. If you are genuinely interested, not just interested in arguing an agenda, maybe someone will be kind enough to post a good link. I’m not sure that I know one off the top of my head.

  55. Narad says:
    Answer the original question.

    Sorry. Which was?

    Um, the one in what you were replying to? Allow me to jog your memory. You asserted the following: “Improvement in personal circumstances will work and will result in less relapse.” I asked you how you thought this would be implemented.

    I did say ”Anyway I do not have answers. I just want to get the questions right first.”

    No, you directly advanced an “answer.”

  56. Dirk Stool says:

    @mousethatroared

    Dirk Stool “Maybe. I am only asking to see the science behind this. Where is it?”

    I think it’s published in books and journals and stuff…maybe you should start in the library.

    Yeh but yeh but yeh.. I quite like this guy and what he says..

    http://www.youtube.com/watch?v=JKRIoY9aXK8

    Maybe you can suggest the library I need to read. Can you link? Thanks.

  57. mousethatroared says:

    And by the way, I never said that depression isn’t caused by life circumstances. Please review my comment.

  58. Dirk Stool says:

    @Narad

    The question?

    “Improvement in personal circumstances will work and will result in less relapse.” I asked you how you thought this would be implemented.

    I did say ”Anyway I do not have answers. I just want to get the questions right first.”

    No, you directly advanced an “answer.”

    It is my personal belief that improvements in social, political, and personal circumstances can cure the ‘disease’ of depression. I do not know for sure . What is your evidence against this view?

  59. Dirk Stool says:

    I suppose my only question is ‘is depression a medical disease caused by a biochemical/genetic/physiological abnormality. Or is it caused by adverse human conditions. The answer determines the treatment and the cure. I donot know. Do you?

  60. Dirk Stool says:

    @ mousethatroared says

    “And by the way, I never said that depression isn’t caused by life circumstances. Please review my comment.”

    I never said you did.

  61. Dirk Stool says:

    Look. I have been banned by Steven Novella from his blog http://theness.com/neurologicablog/. I expect to be banned from here. Just getting my questions in first. Thanks.

  62. mousethatroared says:

    Dirk Stool “# Dirk Stoolon 30 Jan 2013 at 6:26 pm
    “I suppose my only question is ‘is depression a medical disease caused by a biochemical/genetic/physiological abnormality. Or is it caused by adverse human conditions.”

    Why do you think it’s either/or? Does the answer depend upon the patient or is all depression exactly the same?

  63. Dirk Stool says:

    @mousethatroared

    Dirk Stool “# Dirk Stool
    “I suppose my only question is ‘is depression a medical disease caused by a biochemical/genetic/physiological abnormality. Or is it caused by adverse human conditions.”

    Why do you think it’s either/or? Does the answer depend upon the patient or is all depression exactly the same?

    How do you tell what is what ? Scienctific evidence? Please cite your sources. Thanks.

  64. Dirk Stool says:

    Please someone cite the scientific paper that proves that depression is not caused by personal circumstances. That is all I ask. Thanks.

  65. mousethatroared says:

    Okay then. Thanks Dirk

  66. Dirk Stool says:

    Please someone cite the scientific paper that proves that depression is not caused by personal circumstances. That is all I ask. Thanks.

    well?

  67. Dirk Stool says:

    http://theness.com/neurologicablog/index.php/responding-to-a-szaszian/
    Burning books is a easy answer. What is the question?

  68. Dirk Stool says:

    I am sorry to have ever questioned the diagnosis.

  69. Narad says:

    Look. I have been banned by Steven Novella from his blog

    Color me unsurprised.

  70. mousethatroared says:

    I thought 42 was the answer.

  71. nybgrus says:

    Look. I have been banned by Steven Novella from his blog
    Color me unsurprised.

    Same. It was not immediately obvious the trollery in Dirk but nearly immediately. I’m happy to continue to ignore. Especially considering that Dr. Khan has joined the discussion and that is vastly more interesting and educational.

  72. mousethatroared says:

    Well, you could have clued me in… but luckily Dirk was kind enough to do so himself.

  73. pmoran said,
    “a strong collection of placebo and other non-specific influences could match weakly active pharmaceuticals when caring for patients with certain illnesses.”

    Dr. Khan said,
    “Of the 90 patients left, about 20 to 30 will get better whether they see an African shaman, Indian Sadhu or get fake accupuncture. There are several descriptors that people have used, with of course placebo be
    ing the most misused and abused word. As many of you on this website have stated it is the non-specific therapeutic effect.”

    –And I will revisit my point, and add a point that I guess was not totally obvious:
    There can be two categories of influence working upon depressed enrollees in a depression trial: one is genuine, recognized placebo factors, and the other is nonspecific therapeutic factors.

    I entered a relatively LONG post to illustrate what nonspecific therapeutic factors are. They are actual psychotherapy curative mechanisms. I try to “activate” or “use” these when I am counseling.

    Before I greet you in the waiting room and ask you to come back to the counseling room, I take a moment and put my life’s concerns and worries to the side, and I mentally dedicate my next hor of life to you. I think about how God loves you unconditionally, and that I am instructed to love my enemy since my enemy is actually simply just another of God’s children, and I reflect on the fact that emotions, thoughts, and behaviors all seem reasonable once you can see the world through the client’s point of view.

    I then remind myself of the many people that have gotten better from therapy, and have effusively provided me thanks – then I believe things can get better, and I am in a mental state to impart hope – not pollyanna but realistic, seen-it-happen hope.

    I then take a deep breath and go greet you warmly in the waiting room. I am prepped to maximize the nonspecific therapeutic factors.

    That is my ritual.

    I give you no ritual other than general pro forma. I don’t use lingo, and I challenge you to avoid the lingo that others may have thrown on you. for example, I always claim that I do not know what “bipolar” means, what “narrative” means, and what “hyperactive” or “ADD” means.

    I make you break it down into actual descriptions.

    I never promise or swear something will work. I know that most of my interventions will work about 2/3 of the time, and fail the rest. I am honest. When you hear this, you figure out that I am honest and real. You are getting unconditional positive regard.

    When I express uncertainty, but still seem to be optimistic, you feel hope.

    These are active ingredients of psychotherapy. The unspecific therapeutic factors.

    When I identifiy maladaptive thought patterns and help you develop more realistic ones, that is a specific therapeutic factor.

    When I teach you to practice deep breathing and pratice thinking positive thoughts when in cetain difficult situations, those are specific therapeutic factors.

    Placebo effects are totally different from nonspecific therapeutic factors.

    In placebo-prone situations, you usually have a great set-up for both nonspecific therapeutic factors and placebo factors.

    I believe both happen. I believe both are separate phenomena.

    A placebo effect can clearly be suspected when a dubious person takes a biologically inert intervention and gets relief – it happens. There is something about our brain and physiology that makes this belief-based effect happen. But that is not a “non-specific therapeutic factor.”

    Placebos can do amazing things well beyond psychotherapy. Drop blood pressure, etc. Don’t confuse the two influences.

    A therapist seeking to boost response by a placebo effect is doshonest. A therapist seeking to maximize non-specific therapeutic factors is just an educated and strategic practitioner.

  74. On a different note: Feyerebend and others are big on noting how science cannot exist outside of culture / cannot be conducted outside of culture.

    Scientific standards grew and evolve through culture. That is what we are doing here.

    Part of the prevailing culture is that an article is published, and it stands, nearly always unchanged, with any corrigendum much less disseminated than the original paper.

    There is nothing in science that declares that results must appear in a peer-reviewed journal.

    This is a social convention. A decent but not optimal strategy for sharing findings, which is a tenet of science.

    Coyne brings up a great limit in our prevailing culture of science: how do we provide alternative points of views, and rebuttals, in a platform coensurate with that of the original publication?

    A paper is published, and has some great weight over decent criticisms. That ship cannot turn quickly.

    i have learned so much at SBM and at Bad Medicine (same web-identity). It is sad that this type of learning has not been well-recognized and harnessed by the prevailing science culture. Also, NYT commenters can be amazingly insightful. It is like these college classes we took where the prof wanted to throw a topic out there, and get intellectual work out of a class discussion. Usually these are lame. But on SBM, or Bad Med, or some NYT articles, you get tons of insightful, relevant discussion.

    I have many articles saved and put aside in folders b/c some anonymous fake-name blog commenter posted their two cents.

    The best term for this is crowd-sourcing. You throw an issue or matter out ot a crowd, and let them address it.

    This criticism and comment is an essential component of science. But it happens to an almost negligible degree via the “peer-reviewed journal” mechanism.

    I agree with JC (being a Christian, how could I not disagree with a JC) that a journal article should be both a finished product and a conversation starter.

    I believe that this could be in the future. Once all journals go electric, the ability to post a comment or rebuttal will be there. Comments may need to be in more than one category. Simple voting up-or-down by posting campaigns cannot swamp genuine discourse. Would moderation be OK? sure, except for when the moderator silences relevant points. So, maybe more than one type of response posting would be needed.

    SBM could delete all of the alt-medicine posters, who are usually shown to be plain ol’ cult members. However, if those comments were routinely policed and deleted, a fair bunch of mine would have been deleted, and if wrong I at least can post references to support the reasonableness of my points.

    The alt-med true believers posting here are, in the long run, a strength. They could over-run SBM, however, and a bunch of fools could over-run an online peer-reviewed journal that had a “comments” section for every manuscript posted.

    So, I don’t know how things could be done better, but I agree with JC that the prevailing model of joournal-based peer-reviewed article falls short when measured by scientific ideals of opennness, scrutiny, and debate.

  75. weing says:

    “I suppose my only question is ‘is depression a medical disease caused by a biochemical/genetic/physiological abnormality. Or is it caused by adverse human conditions.”

    Sorry to jump in so late. I was taught that if winning the lotto cured your depression, then you weren’t really depressed.

  76. Narad says:

    On a different note: Feyerebend and others are big on noting how science cannot exist outside of culture / cannot be conducted outside of culture.

    This observation, of course, is utterly devoid of semantic value.

  77. Dirk Stool says:

    “I suppose my only question is ‘is depression a medical disease caused by a biochemical/genetic/physiological abnormality. Or is it caused by adverse human conditions. ”

    Where is the scientific research that backs up the claim that depression is caused by a chemical abnormality and not by circumstance?

    Of course even by asking this question I am accused of a being a troll. It seems to me that in the US Bad Pharma adverts have conditioned thinking to such an extent that my thoughts have become heretical. And of course no-one in the USA understands irony. Is there a pill for this also?

  78. Dirk Stool says:

    So my proposal for advancing science is; take a bunch of depressed people. Give a third of them $50,000 each, give a third prozac, and give a third acupuncture/placebo. After 4 weeks measure their depression using the Ham-D scale. Publish the results.

    What would you expect to find?

  79. Dirk Stool says:

    As the Buddha once asked ‘How do we cure the suffering of the human condition?’ His answer was ‘right thoughts’ Your answer here is ‘right chemicals’ . You lot are following a religious belief and not a scientific quest.

  80. Dirk Stool says:

    When I suffered from MDD I went to my doctor and he prescribed me 4 drams of whisky a day. It really helped and I swear by it! I have already put myself forward for the trials by David Nutt in the UK to see if MDMA or Ketamine will assist me also. I can’t wait….

  81. Dirk Stool says:

    Oh! Did I forget to mention the medical marijuana? It is a buzz man! I immediately forget my depression caused by the fact that I cannot pay the mortgage and that my ex wife is a bitch milking me for all she can get.

  82. Dirk Stool says:

    Did I discuss the fact that my psychiatrist has suggested that I have an overactive amygdala. He suggests ECT as a cure – http://theness.com/neurologicablog/index.php/how-electroconvulsive-therapy-works/. If that does not work we are going to investigate surgery. I am looking forward to it.

  83. Dirk Stool says:

    I am called a troll just because I advance my opinions. You lot make me feel depressed. :-(

    What pill do you recommend?

  84. Dirk Stool says:

    and acupuncture and prozac has not helped not one little bit. Nor did the placebo. :-(

  85. Dirk Stool says:

    When I was in the mental hospital I met a woman being treated for severe depression. She lived in a two bedroom house with her new partner and her three sons. She told me that her psychiatrist had recommended ECT to cure her woes. I laughed. My psychiatrist wrote that this reaction was the result of my having a chemical imbalance that would be cured by neuroleptic drugs. So I took them and they were not bad at all! Apart from I now have this incurable twitch affecting my mouth – tardive dyskinesia – i think he said. . But my psychiatrist has told me that they have a drug that can cure this also! Thank god I have insurance and thus reassurance. That put my mind at ease.

  86. Dirk Stool says:

    One has got to laugh yeh? Only my psychiatrist doesn’t like it when I do. So I have to suppress it. So now I am diagnosed with MDD.

  87. nybgrus says:

    I know we take banning very seriously here, but Dirk has already mentioned being banned at NeuroLogica and linked the post where Dr. Novella discussed him and he is obviously just spamming the comments at this point. I’d much, much, much, rather see a discussion betwwen Dr. Coyne and Dr. Khan if that is even a possibility at the moment.

  88. Dirk Stool says:

    I am trying to make a serious point also. Although my use of humour and irony is ignored. Let the debate begin. I will disappear now.

  89. mousethatroared says:

    I second that motion.

    Spam is only good in a Monty Python skit.

  90. David Gorski says:

    Dirk is not banned—yet. He’ll get one last chance before that happens. However, because of his flooding this comment thread with drivel, thus annoying regulars and drowning out substantive discussion of Dr. Coyne’s post, Dirk’s comments will from this point on all be moderated (until I decide otherwise), with all the attendant delays in appearing. Any attempt at sock puppetry on his part will result in immediate banning with extreme prejudice.

  91. Dirk Stool says:

    Perhaps you may address the issues that I raise? Before you ban me.

  92. Dirk Stool says:

    Of course I know that my views are heretical. And will result in a ban. I have studied myth and reality. I know we can dismiss the old myths as being pathetic. I try to disclose the myths that we currently live by. A taboo subject. Ban me. I am used to it – by your skeptical religion.

  93. Dirk Stool says:

    I am a supporter of the views of Karl Popper. I know that these views are not in fashion now. But I will try to promote them still. Ban me. This action says more about you then it does about me.

  94. Dirk Stool says:

    Oh David Gorski! You recently provided a link to mental illness denial on rationalwiki… http://rationalwiki.org/wiki/Mental_illness_denial

    are you really trying to dismiss the anti-psychiatry movement with this link. I urge everyone to read it and decide. Thanks.

  95. Dirk Stool says:

    Hey David Gorski. Just ban me. Like Steven Novella did. I ask awkward questions which you fail to answer. So ban me.

  96. Dirk Stool says:

    Do the bAD Pharma contribute financially to this site? How much are you willing to sacrifice your ideals?

  97. Dirk Stool says:

    of course I know how much money Steven Novella and David Gorski receive from Pharma to promote their views. A simple google search is enough.

Comments are closed.