Last week I wrote about a study that purported to show that antidepressants have no effect in mild to moderate depression. A careful reading of the paper shows that the authors dramatically overstated their findings, particularly in their public statements to the media. The study has another implication beyond the misleading claims about antidepressants. It is an object lesson in an ongoing and disturbing phenomenon in mainstream journalism, the wholesale reprinting of press releases of scientific papers instead of reading and analyzing the papers themselves.
Pick up any newspaper or magazine and you can read about the latest scientific breakthroughs in cancer, Alzheimer’s or heart disease. Just keep in mind that what you are reading is probably a commercial message direct from the authors, not an accurate representation of the paper itself. Medical journalists are supposed to interpret the findings of recent medical publications and present them to the general public in ways that they can understand. They are supposed to provide context for the discovery, explaining what it might mean for disease treatment or cure. Yet, they rarely do. Instead, they simply copy the press release.
Most people are unaware that scientists issue press releases about their work and they are certainly unaware that medical journalists often copy them word for word. Instead of presenting an accurate representation of medical research, medical journalists have become complicit in transmitting inaccurate or deceptive “puff pieces” designed to hype the supposed discovery and hide any deficiencies in the research.
Imagine if a journalist reviewing the newest Ford cross-over vehicle didn’t bother to drive the car, but simply copied the Ford brochure word for word. Could you rely on the journalist’s evaluation? Of course not. Yet that is precisely what medical journalists are doing each and every day.
Not surprisingly, there is a scientific paper describing recent trends in medical press releases. The paper in the of Annals of Internal Medicine, Press Releases by Academic Medical Centers: Not So Academic?, by Woloshin, and colleagues finds:
Of all 113 releases about human studies … [f]orty percent reported on inherently limited studies (for example, sample size <30, uncontrolled interventions, … or unpublished meeting reports). Fewer than half (42%) provided any relevant caveats…
Among the 87 releases about animal or laboratory studies, most (64 of 87) explicitly claimed relevance to human health, yet 90% lacked caveats about extrapolating results to people…
Twenty-nine percent of releases (58 of 200) were rated as exaggerating the finding’s importance…
Almost all releases (195 of 200) included investigator quotes, 26% of which were judged to overstate research importance…
Although 24% (47 of 200) of releases used the word “significant,” only 1 clearly distinguished statistical from clinical significance. All other cases were ambiguous …
Why is this a problem? The harm extends beyond the obvious point that it is deceptive, and a failure of medical journalists to do their job, which is to interpret the accuracy and relevance of scientific publications when writing about them. Because medical journalists credulously publish press release as if they were true, they are constantly publishing conflicting reports, contributing to the public’s distrust of medical research. Each day seems to bring a new report of a food, or a drug that will prevent or cure cancer. Within a week or a month or a year, the journalists are reporting that that food or drug does not prevent or cure cancer.
To the public, it looks like medical researchers are constantly making mistakes. Today they claim that a food will prevent cancer. Next month, the same food will be found to cause cancer. In reality, medical research never demonstrated either claim, but medical journalists reported preliminary findings or flawed research as if they were definitive even though that was untrue.
The willingness of journalists to pass on the information in press releases without checking is not just a function of laziness. Journalists often lack the knowledge of science and statistics that is needed to analyze the paper. Moreover, journalists appear to suffer from a misunderstanding of the scientific literature. Publication of a scientific paper is not the end of a process confirming the truth of a paper; it is only the beginning. Publication does not mean that the findings should be accepted uncritically; it merely means that the findings are worthy of being included in the ongoing public discussion that characterizes science. The findings of the paper may ultimately be deemed worthless or wrong.
The Annals of Internal Medicine has done an important service in bringing this disturbing practice to light. You can’t believe what you read about medical research in newspapers and magazines because medical journalists are simply copying press releases, not analyzing the research for accuracy or relevance. Therefore, in the interest of accuracy and relevance, I must disclose a caveat to this important scientific paper. In what surely is an unintentional irony, The Annals of Internal Medicine publicly unveiled the paper and its findings by issuing a press release.