The overriding them, the raison d’être if you will, of this blog is science-based medicine. However, it goes beyond that in that we here at SBM believe that science- and evidence-based medicine is the best medicine. It’s more than the best medicine, though; it’s the best strategy for medicine to improve therapy for our patients. We frequently contrast science-based medicine with various forms of “complementary and alternative medicine,” specifically pointing out that SBM changes its practices as new science and new evidence mandates it while CAM tends to rely on ancient, vitalistic, pre-scientific or pre-modern scientific beliefs about how disease occurs as the basis for its therapies. Although it may be painfully slow and frustrating at times and even though there may be major stumbles along the way, the overall course of SBM over the last century has in general been to produce ever more effective therapies and to discard therapies that are either ineffective or whose risk-benefit ratios are insufficiently favorable. The one single most important thing behind the advancement of medicine is good science.
That’s why I really, really hate scientific fraud, and I’m really, really upset, perhaps even more so than Dr. Atwood, over the discovery last week of what is arguably one of the most massive scientific frauds in medical history. It doesn’t matter that Dr. Atwood is an anaesthesiologist and I am not, meaning that the specific scientific fraud unearthed, which was perpetrated by an anesthesiologist studying multimodal anesthesia, as reported in Anesthesiology News, the Wall Street Journal, and the New York Times. I am a surgeon, and the relief of surgical pain in my patients is an important part of my practice. If the scientific basis of what my colleagues in anesthesiology do before, during, and after my operations is called into doubt, I have to wonder if I am giving my patients the best surgical care. Aside from that, there is the intellectual outrage I feel as a result of seeing science and patients betrayed in such a systematic and blatant manner.
One reason for this sense of betrayal is that science as it is practiced today (and, in fact, as it has always been practiced) relies on a fair measure of trust. One part of the reason that science relies on a certain degree of trust is that the culture of science values openness, hypothesis testing, and vigorous debate. The general assumption is that most scientists are honest. Another part of the reason is that it is not possible for other scientists and peer reviewers to oversee the experiments and data collection of other scientists. There has to be some level of trust that scientists reporting their results aren’t making data up out of whole cloth. It’s true that we as scientists know that our colleagues (and, indeed ourselves) usually do try to present our data in the most favorable light possible in order to support our hypotheses, but we also know that we’d better find the mistakes and weaknesses in our experimental design and data analysis before we publish, because if we do not very likely our colleagues will, especially if we are working in a “hot” area where there is a lot of competition. Of course, we are also all human, and none of us is immune to the temptation to leave out our manuscripts and manuscripts that inconvenient bit of data that doesn’t fit with our hypothesis or to cherry pick the absolutely best-looking blots or graphs for use in our grant applications or scientific manuscripts. However, scientists also value their reputation among other scientists, and there’s no quicker way to seriously damage one’s reputation than to engage in dodgy behavior with data, and there’s no quicker way to destroy it utterly than to make up data.
Unfortunately, opposing these forces favoring openness and honesty are the need to “publish or perish” in order to remain funded, to advance academically, and to become tenured, a pressure that can be particularly intense among basic scientists, who will fail to win tenure, lose their jobs, and very likely risk ending their academic careers if they cannot support their labs and support 50% or more of their salaries through grants. As an academic surgeon who has been fantastically fortunate enough to have landed two jobs where I spend at least as much time doing research as I do taking care of patients, I always remember that I’m in a much better position because, even if I failed utterly to renew all my grants and managed to burn through all my startup funds and whatever bridge funds my university provides, I’d be unlikely to be fired, as I could just go back to operating full time. True, it would be a huge career setback (although I would not be the first surgeon that this happened to; many give up trying to do both research and surgery because it is so hard). On the other hand, I’d even be likely to generate more income for my department by doing surgery than I could through research. Clinician-scientists are in general a drag on the finances of an academic department, their salaries and expenses partially subsidized by the clinical income of the high-producing surgeons in the department.
Let’s get to the specifics of the fraud in question:
In what experts are calling one of the largest known cases of academic misconduct, a leading anesthesiology researcher has been accused of falsifying data and other fraud in potentially dozens of published studies.
Scott S. Reuben, MD, of Baystate Medical Center in Springfield, Mass., a pioneer in the area of multimodal analgesia, is said to have fabricated his results in at least 21, and perhaps many more, articles dating back to 1996. The confirmed articles were published in Anesthesiology, Anesthesia and Analgesia, the Journal of Clinical Anesthesia and other titles, which have retracted the papers or will soon do so, according to people familiar with the scandal (see list). The journals stressed that Dr. Reuben’s co-authors on those papers have not been accused of wrongdoing.
In addition to allegedly falsifying data, Dr. Reuben seems to have committed publishing forgery. Evan Ekman, MD, an orthopedic surgeon in Columbia, S.C., said his name appeared as a co-author on at least two of the retracted papers, despite his having had no hand in the manuscripts. “My names were forgeries on the documents,” Dr. Ekman told Anesthesiology News.
Dr. Reuben has been an extremely active and visible figure in multimodal analgesia, particularly as an advocate for its use in minimally invasive orthopedic and spine procedures. His research has provided support for several mainstays of current anesthetic practice, such as the use of nonsteroidal anti-inflammatory drugs and neuropathic agents instead of opioids and preemptive analgesia. Dr. Reuben has also published and presented data suggesting that multimodal analgesia can significantly improve long-term outcomes for patients.
It’s hard to overstate how serious this revelation of scientific fraud is for the field of anesthesiology and medicine. Dr. Reuben was considered a pioneer in his field, and his work is not only widely cited, but serves as the basis for an amount of anesthesia practice that few academic anesthesiologists can lay claim to. Perhaps I would be best served to quote Dr. Atwood, who as an anesthesiologist understands better than I just how influential Dr. Reuben was:
The case hit home in a couple of ways. Although SBM readers may think of me as one of the 10 or so most knowledgeable skeptics of pseudomedicine in the entire world, in my day job I am a practicing anesthesiologist in Massachusetts. A big problem in anesthesia and surgery is how to provide adequate pain relief for patients after especially painful operations, without having to use such high doses of narcotics that many, particularly the elderly, will become somnolent and have depressed respiratory drives. The issue thus involves both comfort and safety. Multimodal regimens such as those reported by Reuben seemed to offer one solution, and have become increasingly popular over the past couple of years.
In fact, the twenty-one papers now retracted represent a body of work that appeared on its surface to be quite impressive. Even knowing about the fraud, I find it impressive in another way–namely how Dr. Reuben could have maintained this fiction for so long, but more on that later.) In addition to Dr. Atwood’s, these two quotes will also give you an idea of just how influential Dr. Reuben was:
All of that is now in question, said Steven L. Shafer, MD, editor-in-chief of Anesthesia and Analgesia, which retracted 10 of Dr. Reuben’s articles. “We are left with a large hole in our understanding of this field. There are substantial tendrils from this body of work that reach throughout the discipline of postoperative pain management,” Dr. Shafer said. “Those tendrils mean that almost every aspect will need to be carefully thought through. What do we still believe to be true? Do the conclusions hold up to scrutiny?”
Dr. Shafer said that although he still believes “philosophically” in multimodal analgesia, he can no longer be absolutely certain of its benefits without confirmation from future studies.
Dr. Shafer brings up another point, namely that multimodal anesthesia might actually be highly beneficial. After all, Dr. Reuben wasn’t the only researcher working on the topic. However, he was probably the preeminent voice advocating this form of anesthesia, which means that the entire scientific and evidentiary basis of multimodal anesthesia has now been called into question:
Jacques Chelly, MD, PhD, MBA, director of the Division of Regional Anesthesia and Acute Interventional Perioperative Pain at the University of Pittsburgh Medical Center (UPMC), said that the Reuben episode has left multimodal analgesia “in shambles concerning many of the drugs we use”–particularly celecoxib and pregabalin. “The big chunk of what people have based their protocol on is gone.”
What this means is that, even though it’s possible that multimodal anesthesia and analgesia might still be an effective modality, there is now considerable doubt about it. Dr. Reuben through his fraud has not only betrayed science, but betrayed his patients as well in one or both of the following two ways. If, for example, multimodal analgesia is no more effective than other methods, then his body of work based on falsified data over well over a decade has betrayed patients with pain, both chronic and acute, by persuading anesthesiologists to use a methodology that is no more effective than others and may be more risky. If, on the other hand, multimodal analgesia is indeed effective and Dr. Reuben just happened to “show” it by falsfying his data, then he has betrayed future patients, who now may not get the benefit of effective multimodal analgesia as anesthesiologists try to work out whether this modality works or not. Either way, it’s not good.
Dr. Reuben’ scientific fraud is also highly unusual in another way. Usually, fraud involves a single paper or, at most, a handful of papers. Even Andrew Wakefield, as far as can be discerned, only fabricated data for one paper. The rest of Wakefield’s results derived from his utter incompetence as a scientist, which led to his unwillingness to worry about little things such as controls for his PCR or keeping stock plasmids (whose presence can result in false positives given just how sensitive PCR is) separate from samples being tested. Dr. Reuben’s fraud appears to eclipse even that of Andrew Wakefield, as hard as that is to believe. Indeed, I’ve never seen an example involving such a massive body of work so important to a field over so many years. It’s truly staggering. A PubMed search shows that Dr. Reuben has over 70 peer-reviewed papers. It’s hard not to wonder how many of his other publications are also based on fraudulent data, and, even if they aren’t fraudulent, Reuben’s other work will be forever tainted by them, regardless of whether the science was good or not. It’s not for nothing that Dr. Reuben has been likened to Jason Blair in journalism.
Whenever I see an example of fraud like this, I wonder: How could Dr. Reuben get away with, in essence, “making it up” for so long? If he were a lone scientist working alone in a laboratory, I could see how he might be clever enough to pull it off. However, these days science, and in particular biomedical science, is a highly collaborative endeavor. Gone are the days of single labs working on problems. These days, such publications, with one or a handful of authors, are increasingly rare, and if they’re rare for basic science they are virtually unheard of in clinical trials.
The reason is that clinical trials have become very complex. Many people are involved, from the clinical trials office, the scientific review board, the institutional review board, and all the clinicians who enroll patients on clinical trials. Also, they inevitably involve statisticians who analyze the data. Most clinical science journals these days will not even consider publishing the results of a clinical trial without a biostatistician listed among the authors, and the higher profile clinical journals include biostatisticians as peer reviewers for any manuscript reporting the results of a clinical trial. Then there is the question of all the other authors on Dr. Reuben’s papers. (As mentioned before, at least one of his co-authors over the years were included on the author list even though he had nothing to do with the studies.)
Although it has been emphasized that none of Dr. Reuben’s co-authors over the years have been accused of scientific fraud, I find it hard to believe that so many people over so many years failed to notice even a whiff of a problem. My guess is that some of them probably at least sensed that something was not entirely kosher but remained silent because they trusted Dr. Reuben and liked being co-authors on papers with him. It was good for their careers. Alternatively, they just signed on without taking much responsibility for the actual manuscript, something that’s far too easy and tempting to do far too much of the time. Again, unfortunately the number of published papers counts when a scientist or clinical investigator is evaluated.
Indeed, if you’ve noticed a bit of a parallel, here, you won’t be alone. The way Dr. Reuben’s co-authors signed on to his papers reminds me of the way physicians have signed on to pharmaceutical company-sponsored seeding trials. For example, in the case of the infamous ADVANTAGE Trial, the first author of the main publication resulting from the trial, Dr. Jeffrey R. Lisse, did not have a role in data collection. Even I haven’t been immune. In fact, I used to wonder if I should have managed to get myself on more papers, regardless of my level of involvement, because my publication record, quite frankly, is probably not as good as it should be at this stage in my career. The temptation is always there. But, looking at this incident, one thing I can say about my publication record is that there are some damned good papers there published in excellent journals, and that I either wrote or heavily participated in the writing of each and every one of them. Even so, it would be nice to have several extras. However, the Reuben affair has cautioned me that I will be very careful to make sure that I am involved in the data analysis and the writing of the paper.
The other thing I wonder in the case of such massive fraud, and, no doubt so do my readers, is how so many fraudulent papers from one author could get past peer reviewers. One reason is that peer reviewers can only look at the data presented in the manuscript. They are not usually equipped to identify fraudulent data unless it’s blatantly obvious. That’s not to say that peer reviewers don’t catch obvious fraud (they do, the most common examples I’ve heard of being autoradiographs cropped or Photoshopped in suspicious ways or duplicated from other publications in different contexts, or what have you), but there is a presumption of honesty in science. Reviewers don’t start with the presumption that the data themselves in a manuscript might be fraudulent. Rather, they are on the lookout for problems in experimental design, analysis and interpretation of the data, and drawing conclusions from that data. Moreover, they only look at one manuscript at a time. Rarely do they have time to read the references cited, and if they do read cited references usually they don’t have time to do more than to pick a couple and then skim them. If an author’s other manuscripts are all fraudulent, it’s unlikely the peer reviewer would detect it, as he or she in essence works in a vacuum for each manuscript reviewed.
What this incident does reveal is that there are almost always indications. If there’s one thing about science, it’s often messy. Results rarely always turn out the way one expects or wants. If they did, then science would be pretty useless. Often the results that lead to new discoveries are the anomalous results, the unexpected result. Dr. Reuben’s work suffered from what is in retrospect a very suspicious degree of consistency:
“Interestingly, when you look at Scott’s output over the last 15 years, he never had a negative study,” said one colleague, who spoke on the condition of anonymity. “In fact, they were all very robust results–where others had failed to show much difference. I just don’t understand why anyone would do this or how anyone could pull this off for so long.”
When you see an outlier, an investigator whose results are always more robust than those of his colleagues, be wary. It may not be scientific fraud. It may not even be mistaken or evidence of a bit of “data cleaning,” if you know what I mean. But it should send up red flags. Indeed, to bring up another example, I recently saw Brian Deer give a talk at the University of Michigan about the revelations of Andrew Wakefield’s fraudulent research that launched the decade-long MMR scare. He started out by saying that, when something looks too good to be true, it usually is too good to be true and that such was the case with Andrew Wakefield’s 1998 Lancet paper. Unfortunately, however, often suspicious consistency–looking “too good to be true”–is all too often only obvious in retrospect.
Perhaps the greatest failure of the scientific community in this case is that Dr. Reuben’s fraud was not discovered by other scientists trying to replicate his work as a basis for extending it. Rather, what brought Dr. Reuben to the attention of his hospital were two abstracts that he submitted for presentation. A hospital bureaucrat charged with keeping the hospital’s human subjects research in compliance with federal and state law noticed that no record of approval for human subjects research could be found for the work presented in these abstracts:
Dr. Reuben’s activities were spotted by Baystate after questions were raised about two study abstracts that he filed last spring, Ms. Albert said. The health system determined that he had not received approval to conduct human research, Ms. Albert said.
Baystate investigators determined that Dr. Reuben had concocted data for 21 studies, and the health system asked the journals in which those studies were published to withdraw them.
Truly, we in the academic medical community seriously dropped the ball on this case. As I’ve heard pointed out elsewhere, that Dr. Reuben’s fraud was finally detected was not a triumph of science policing its own. Rather, it was a triumph of administrative due diligence. Actually, I’m not so sure it was even that, given that it took Baystate over 12 years to detect his fraud. One wonders if Dr. Reuben didn’t start with outright fraud. Perhaps back in the 1990s, he succumbed to the temptation to “clean up” his data by leaving out patients, tweaking values, or otherwise realigning the data to match his hypothesis. Once he started down that path, it became easier to do it more and more until finally he started just “making stuff up” and publishing it as though he had done the studies.
What most angers me about this case is the massive betrayal of trust. For SBM to fulfill its promise of improving medical care for patients, the science underlying it must be at the very minimum trustworthy and free of fraud. That is the absolute lowest bar, the very first requirement that is necessary (although not sufficient), of the science behind any treatment. Moreover, the public quite rightly expects that its scientists, at the very least, will be honest about their results. Too much depends on it, especially in medical science, where it is people’s health that is at stake.
Even worse, much of Dr. Reuben’s work was underwritten by the pharmaceutical companies that manufacture the the very nonsteroidal anti-inflammatory drugs combinations of which he studied. There is no evidence that Pfizer, for instance, which underwrote some of Dr. Reuben’s more recent work, was complicit in the fraud; indeed, I rather suspect that Pfizer was just as snookered as anyone else. More than likely, Pfizer picked Dr. Reuben because of his earlier work, which supported the use of its products. Even so, one can’t help but wonder if, once underwritten by Pfizer, Dr. Reuben felt pressure to produce results that supported the use of Pfizer drugs. Be that as it may, what this link to a pharmaceutical company means for those of us who defend science- and evidence-based medicine is that every crank alt-med site and blog on the Internet is going to be harping on this incident as The Proof That Conventional Medicine Is Hopelessly Corrupt and their favorite woo is being kept down by The Big Bad Pharma Man. The uber-quack-friendly site NaturalNews.com has already run with it. Not only has Reuben put a stain on scientific medicine that will be very hard to erase, if it’s even possible at all, but he has made our work at SBM harder.
But even worse than that, Reuben’s wasted huge amounts of resources and left a huge mess for his colleagues to clean up. It will take years for the studies he’s done to be either redone or for other investigators to find out if the very concept of multimodal analgesia is even valid. He’s screwed every anesthesiologist who, based largely on his work, came to accept this therapeutic modality as the best for their patients, and he’s screwed those patients, too. (As an aside, he’s also screwed a friend of mine who is the Chief of Surgical Oncology at Baystate. The two never worked together, but Dr. Reuben’s fraud has tainted the institution and, by extension, my friend.)
I don’t want to end on such a downer though. As bad as the Reuben fraud is, I can’t help but contrast it to an other recently discovered incident of scientific fraud, namely that committed by a hero of the antivaccination movement, Andrew Wakefield. One huge contrast that I note is that there is no one out there starting up a “We support Dr. Scott Reuben” movement, claiming that he was framed, that it’s all a big conspiracy to get him. Not so Andrew Wakefield, for whom there is a misinformation- and logical fallacy-filled website proclaiming that We Support Dr. Andrew Wakefield. Meanwhile the antivaccine blog Age of Autism, which had given Wakefield its Galileo Award for 2008, has relentlessly tried to defend Andrew Wakefield and slime Brian Deer, the reporter who uncovered his fraud. Indeed, chief antivaccine propagandist, David Kirby, the ex-journalist who now apparently blogs full time for AoA, even bragged about having persuaded Keith Olbermann to feature Brian Deer in one of his “Worst Person in the World” segments, apparently having played Olbermann for a fool by using Olbermann’s hatred of Rupert Murdoch to convince Olbermann that Brian Deer had some sort of horrific conflict of interest that he did not, in fact, have. Meanwhile, the antivaccine movement, led by AoA, continues to beat the drum attacking Brian Deer and The Times for tenuous conflicts of interest that do not in any way detract from Deer’s reporting on Wakefield, which is extensive and spans nearly 10 years. If you want to see evidence of one of the very real differences between science-based medicine and pseudoscience- or faith-based medicine, all you have to do is to compare the difference between the reaction to revelations of Dr. Reuben’s fraud in the anesthesiology community to that of the “autism biomedical” and antivaccine community to Dr. Wakefield’s fraud.
None of this excuses Dr. Reuben or the scientific establishment for not having detected his fraud for so long. Such a failing will hurt us and provide the pro-quackery forces with fodder to attack SBM for years. However, it is still instructive to note that, when a scientist is detected as a fraud, he is immediately discredited and, in essence, shunned by the scientific community. Dr. Reuben’s career as an academic physician is almost certainly over; he is likely to be kicked out of the community of science with extreme prejudice. In marked contrast, in the realm of pseudoscience, such as the antivaccine movement, when scientific fraud is detected by one of their “scientists” promoting the vaccine-autism link, the reaction is to circle the wagons, defend the scientist who committed the fraud, and blame the accusations on a huge conspiracy between big pharma, the government, and various other nefarious forces to “suppress” their hero’s work.
I find that somehow reassuring about scientific medicine, even in the depth of such a research scandal.
Additional commentary:
- Steve Novella, Another Case of Scientific Fraud
- Ben Goldacre, Scumbag
- Rogue Medic, Dr. Reuben’s Fraudulent Science and Patient Care