Sep 18 2008
Anyone attempting a systematic review of the medical literature on sectarian medical systems (“CAM”) starts with a serious disability; the literature itself. The National Library of Medicine still lists abstracts for over 30 “alternative medicine” journals, but more concerning, is my estimate that half or more of the articles on sectarian systems published in standard medical journals range from the erroneous to the fraudulent. If one is conscientious, honest, and wants to produce a realistic review of sectarian systems that reflects reality, one cannot do it.
The problem spills over and beyond boundaries of medical research and practice. Often neglected is a massive literature of the allied professions – nursing, psychology, social work, and others. There are data bases for these professions as well (CINAL, EMBASE, CISCOM, Psychlit.) So not only do physicians and patients deal with a disabled medical literature, other professions also face the same problem in theirs. Little wonder that the “CAM” – “Integrative” movement has been tolerated instead of rejected in the community of health and allied health.
The editor of Research on Social Work Practice sent me a copy of a review of intercessory prayer published in his journal (Hodge DR, A Systematic Review of the Empirical Literature on Intercessory Prayer. Res Soc Work Pract 2007;17(2):174-186.) Intercessory prayer is the prayer offered by others for an ill person, usually not in the person‘s presence. Prayer is usually performed on a regular daily schedule by groups of prayers of one or more religious denominations.
This review’s author used standard methods for collating pertinent reports for systematic reviews and also reviewed other systematic reviews, including the Astin review published in Annals Internal Medicine in 2000. Selected reports in that review of 23 papers made up a proportion of the reports in the 17 of the Hodge review. Hodge apparently used stricter criteria than did Astin, as 10 studies were published after the Astin review, leaving only 7 or fewer from the 23 of the Astin review.
The Hodge review found its 17 reports, qualified for analysis by entry criteria (randomization, masking, blinding, analyses, etc.) Of the 17 reports, 5 showed no effect of intercessory prayer on disease outcomes; 5 reports showed a “non-significant trend” to positive effects, and 7 reports showed “significant effects.” The author concluded the results to be inconclusive. The previous Astin review concluded its results showed positive effects.
Summary of the Astin Annals review. We had already shown the Astin review to be inaccurate largely because of acceptance of invalid individual reports (Sampson W. Are systematic reviews …sufficient and necessary for assessing the effectiveness of prayer? Sci Rev Altern Med 2000, 4(2):12-15.) Astin recorded 15 of 23 studies to have shown positive statistical results. The conclusion made national news at the time, and Timothy Johnson MD of ABC sang praises to it over national TV. But problems we and others discovered with the individual reports resulted in an inability to form a ratio because so many of the reports had been faked or neutralized by error or misrepresentation. Besides, 8 of the papers were from non-medical journals and 1 or 2 were from magazines. The Hodge review seems to have been more selective and includes a number of recent reports from the spirituality in medicine department at Duke, supported by the Templeton Foundation.
We have learned more about the reports in the Astin review since 2000. Although we suspected that the four reports on prayer and “non-touch healing” by Wirth were defective, we did not understand why that author of several fraudulent papers would claim to have repeated them and found negative results. Perhaps a matter of conscience. Perhaps a move to throw investigators off the track. [Barry and Bruce Beyerstein, with aid of local Bay Area Skeptic members tracked Wirth down in the 1990s, and asked some pertinent questions, and received threats of lawsuts in return - and no data.)
We did not learn with more certainty until 2002 that Wirth had probably faked some 6 studies, 4 of which were in the Astin review. The healing papers were part of Wirth’s master’s thesis, but his advisor admitted to never having seen the original data. The story of Wirth’s frauds and convictions is in preparation.
In 2001 Wirth, Cha, and Lobo published their implausible report claiming that 3 separate prayer groups in 3 countries - Canada, US, Australia - produced a 100 percent increase in pregnancies from in vitro fertilization in South Korea. Later revelations showed that Cha was involved in the fraudulent Korean cloning report a few years ago, Lobo admitted to have met Cha only after the trial data were in hand, and did not review the data. He was relieved of his post as OB-Gyn Chairman at Columbia; and Wirth has just spent 5 years in Federal prison for embezzlement and other frauds. The story of the Cha/ Wirth/ Lobo paper was published by Bruce Flamm of UC Irvine in Sci Rev Altern Med in a 3-part series several years ago. [Blackballed by “CAM” sectarian reviewers, SRAM is still unlisted on the Medline pages, so most readers are not clued to the papers’ existence.] The four prior Wirth studies in the Astin review were excluded from the Hodge review, but the probably fraudulent Cha, Wirth study of 2001 was included despite the revelation of fraud. After these revelations, one of the Astin review authors (E. Ernst) recanted his conclusion and publicly voiced his opinion that there is poor to no convincing evidence for prayer efficacy on the course of disease. Yet the Astin review remains on Medline and is often quoted in other papers. And, the Cha/Wirth paper made it into the Hodge review even though Hodge revealed his knowledge of the controversy (as published in J Repr Med, 2005, 50:71.)
Both reviews included the Sicher/Targ prayer study claiming positive results in AIDS patients. That paper was shown by a writer in the on-line Salon magazine to have had a procedure violation. The original end point for the AIDS patients, treated and controls, was to have been longevity (death.) But after the study began, triple therapy and protease inhibitors appeared, and all patients were still alive. So the authors changed the end points to multiple indirect measures such as transfusions, hospitalizations, and such. They then were able to select end points post hoc. I verified this information with the statistician of record. The authors did not reveal this violation so the report remains in all data bases. This paper was included in the Hodge review. [BTW, I was shown the raw data in confidence and saw that significant weight loss and decreased hemoglobin occurred in many of the patients but those end points were not selected for report.)
Dr. Targ, daughter of Russel Targ (of Targ and Puthoff of ESP note,) died of a brain tumor, and remains an icon of the “CAM” community.
Both the Byrd study of 1989 and the Harris study of 1999 were included in both reviews. Both recorded complications in coronary care patients in prayed-for and control groups. Kim Atwood reviewed problems with both in a letter to Annals Int. Med. as did Kevin Courcey (in: Medical claims for intercessory prayer remain elusive. Sci Rev Altern Med 2000;4(2):9-11.) Both pointed out that the results of the Byrd and Harris studies disconfirmed each other through conflicting results - many of the positive correlations in Harris were negative in Byrd and vice versa. Besides, the Harris study control patients had far more and more serious medical complications at entry to the study. The multiple end points in both allowed for all sorts of chance correlations. Perhaps worse, the Byrd study might not have been blinded as claimed, as Kim Atwood discovered from just reading closely. Neither paper should have qualified for publication had knowledgeable reviewers been employed.
So, without looking any further into the more recent studies, four of the six positive (P<.05) studies in the Hodge review 1) were probably fraudulent (Cha/Wirth) 2) were compromised by protocol - post hoc changed end point - violation (Sicher/Targ,), or 3) contradicted each other, had unbalanced control and treatment groups, or were actually negative by most criteria (Byrd, Harris.) In addition, 2 of the 6 studies showing a positive trend we have some reason to believe had broken blinding - one study by Matthews having been analyzed by Gary Posner in Sci Rev Altern Med, but these are not necessary to neutralize the positive findings of the Hodge review.
Why have these frauds and irregularities passed so many editorial and peer reviews? This evidence shows that most peer reviewers are not familiar with the numerous ways a report’s findings can be altered, they are not familiar with authors we know have cheated or otherwise misinterpreted results, and formal filters for fraud do not exist. Frauds are usually found by chance observation or by someone who had original suspicions.
Beside individual report irregularities, there is a considerable bias in authors and systematic review literature reviewers. We now have evidence that the large preponderance of investigators, report authors, and the reviewers of literature have vested interests in positive outcome. They seem to be the ones with enough interest, or who have enough grant money, to carry out a time-consuming and often unrevealing project. Not only do they depend on grants to investigate these improbable proposals, not only do their academic positions depend on obtaining positive results from their work, but the wording of their reports contains neologisms and language distortions explored in these pages and elsewhere. The failure of the research and editor communities to recognized career and ideological conflicts of interest leaves this large hole in the scientific defensive line.
What causes such distortion in the literature, with its subsequent evidence based belief? Although a majority of the US public believe in formal religions of one kind or another, the Hodge review revealed some statistics I was not aware of:
“A surprisingly high percentage of social workers appear to use intercessory prayer in their work with clients … One national survey of direct practitioners affiliated with the National Association of Social Workers (N=2069) found that 28% of respondents had engaged in verbal prayer with their clients, whereas 57% prayed privately for their clients. (Canda ER, Furman LD. Spiritual diversity in social work practice Free Press, New York, 1999.) Similarly, among a national sample of NASW affiliated gerontological workers (N=299) 43% reported praying with their clients either “sometimes” or “often.” In terms of their private prayer interventions, two thirds of respondents indicated they prayed either sometimes (43%) or often (24%) for their clients.”
Hodge in his review comments: “Qualitative research …suggests that many social workers hold strong beliefs, both in favor and against [ …] using prayer in therapeutic settings. Consequently, it is not surprising that practitioners’ personal metaphysical beliefs rather than established professional protocols may be guiding therapeutic decisions.” It is now apparent that the literature itself leaves a large opening for the play of opinion and belief in “CAM,” and especially in the effects of prayer. It is easy to regard systematic reviews as showing prayer results to have positive effects, thus reinforcing erroneous belief of health professionals.
But if one has the background information on proponents, and one is familiar with their histories of misrepresentation, and the true origins of their proposals, one can have access to openings and chinks in the sectarian propaganda armor not otherwise apparent.
8 Responses to “Trouble in the Library”