The continuum of surgical research in science-based medicine

Editor’s note: Three members of the SBM blogging crew had a…very interesting meeting on Friday, one none of us expected, the details of which will be reported later this week–meaning you’d better keep reading this week if you want to find out. (Hint, hint.) However, what that means is that I was away Thursday and Friday; between the trip and the various family gatherings I didn’t have time for one of my usual 4,000 word screeds of fresh material. However, there is something I’ve been meaning to discuss on SBM, and it’s perfect for SBM. Fortunately, I did write something about it elsewhere three years ago. This seems like the perfect time to spiff it up, update it, and republish it. In doing so, I found myself writing far more than I had expected, making it a lot more different from the old post than I had expected, but I guess that’s just me.

In the meantime, the hunt for new bloggers goes on, with some promising results. If we haven’t gotten back to you yet (namely most of you), please be patient. This meeting and the holiday–not to mention my real life job–have interfered with that, too.

The continuum of surgical research in science-based medicine

One of the things about science-based medicine that makes it so fascinating is that it encompasses such a wide variety of modalities that it takes a similarly wide variety of science and scientific techniques to investigate various diseases. Some medical disciplines consist of mainly of problems that are relatively straightforward to study. Don’t get me wrong, though. By “straightforward,” I don’t mean that they’re easy, simply that the experimental design of a clinical trial to test a treatment is fairly easily encompassed by the paradigm of randomized clinical trials. Medical oncology is just one example, where new drugs can be tested in randomized, double-blinded trials against or in addition to the standard of care without having to account for many difficulties that arise from difficulties blinding. We’ve discussed such difficulties before, for instance, in the context of constructing adequate placebos for acupuncture trials. Indeed, this topic is critical to the application of science-based medicine to various “complementary and alternative medicine” modalities, which do not as easily lend themselves to randomized double-blind placebo-controlled trials, although I would hasten to point out that, just because it can be very difficult to do such trials is not an excuse for not doing them. The development of various “sham acupuncture” controls, one of which consisted even of just twirling a toothpick gently poked onto the skin, shows that.

One area of medicine where it is difficult to construct randomized controlled trials is surgery. The reasons are multiple. For one thing, it’s virtually impossible to blind the person doing the surgery to what he or she is doing. One way around that would be to have the surgeons who do the operations not be involved with the postoperative care of the patients at all, while the postoperative team doesn’t know which operation the patient actually got. However, most surgeons would consider this not only undesirable, but downright unethical. At least, I would. Another problem comes when the surgeries are sufficiently different that it is impossible to hide from the patient which operation he got. Moreover, surgery itself has a powerful placebo effect, as has been shown time and time again. Even so, surgical trials are very important and produce important results. For instance, I wrote about two trials for vertebral kyphoplasty for ostoporotic fractures, both of which produced negative results showing kyphoplasty to be no better than placebo. Some surgical trials have been critical to defining a science-based approach to how we treat patients, such as trials showing that survival rates are the same in breast cancer treated with lumpectomy and radiation therapy as they are when the treatment is mastectomy. Still, surgery is a set of disciplines where applying science-based medicine is arguably not as straightforward as it is in many specialties. At times, applying science-based medicine to it can be nearly as difficult as it is to do for various CAM modalities, mainly because of the difficulties in blinding. That’s why I’m always fascinated by strategies by which we as surgeons try to make our discipline more science-based.

Three years ago, I attended a rather fascinating session sponsored by the Society of University Surgeons on surgical research issues involving bioethics and institutional review boards. I realize that there are those out there, particularly on the medical side of things, who with a sneer will refer to the term “surgical research” as an oxymoron, but, trust me on this, there are a lot of surgeons out there trying to do scientifically rigorous studies within the bounds of their discipline to advance the art and science of surgery, all for the good of their patients. (OK, there’s also glory to be had, but the primary motivation still usually boils down to wanting to do good for our patients, with the glory being a secondary motivation.) One other aspect of surgery, besides the ones that I just mentioned, is that surgery is just as much a craft as it is a science. How a surgeon ties knots, where he places the incision and how large he makes it, how he puts together two ends of bowel so that they don’t leak, all of these things are examples where the skill of the individual surgeon matters as much as the scientific validity of what operation the individual surgeon is performing. Surgical research differs from research carried out in medical specialties in that it is much more difficult to standardize operations; placebo controls (i.e., sham operations) are rarely ethically acceptable; and double blind studies are in essence impossible, because the operator will always know what he did and blinding the patient to what was done is similarly impossible without doing sham operations. Moreover, new surgical procedures have a learning curve, such that they may seem less desirable early on in their development but their benefits become apparent the more practice surgeons have at them and as more surgeons learn how to do them well. All of these factors tend to make surgical research, I would argue, in many ways more difficult than medical research, particularly research comparing drug treatments, where double blinding is possible and placebo controls can often be used.

One commonly accepted definition of research is found in federal regulations about human subject research: “Research means a systematic investigation, including research development, testing, and evaluation, designed to develop or contribute to generalized knowledge.” Innovative surgery is often not systematic. Rather, it is designed to benefit individual patients; there is often no intent to publish the surgical series at a later time, or such intent is secondary. In reality, many, if not most, advances in surgery don’t come from careful controlled trials (at least not at first), and the topic of the conference was how to decide when alterations in an established surgical procedure constitute what was referred to as “tinkering” and when they constitute an “innovation,” which is more substantive (more on this later). And when does a surgeon cross the line from tinkering into innovation into actual human subjects research, anyway? These are actually pretty tough questions, with a lot of complexities involved, and the main impetus for the SUS to undertake a project to try to write guidelines about these issues, the preliminary draft of which were presented at the session I atteneded, was as a pre-emptive strike: To do it first before the government steps in and does it in its typically ham-fisted manner. Also, a lot of this tinkering and innovation go on outside the confines of academic medical centers, out in the “real” world of private practice surgeons. The end result was a position statement published in 2008 in the Journal of the American College of Surgeons entitled, appropriately enough, Responsible Development and Application of Surgical Innovations: A Position Statement of the Society of University Surgeons. It begins:

The field of surgery has a unique culture and rich tradition of innovation. Surgeons are trained to perform continuous situational assessment, decision analysis, and improvisation, in preparation for the challenges and creativity required by nearly every clinical case. Indeed, as Riskin and colleagues point out, “most surgeons innovate on a daily basis, tailoring therapies and operations to the intrinsic uniqueness of every patient and their disease.” In addition, unsolved problems and repetitive failures—or at least suboptimal outcomes—stimulate surgeons to strive for a better way to address challenging surgical problems. Consequently, the practice of surgery is steeped in innovation.

In the United States, development of new drugs and medical devices is closely monitored by the federal government. The Food and Drug Administration (FDA) regulates biologics, devices, and drugs by reviewing their safety and efficacy for a given indication before granting approval for marketing and distribution. The labeling (package insert) summarizes what the FDA has deemed the safe and effective use of the product, and “off-label” use of a product, although allowed, may be subject to regulatory scrutiny.

Such oversight does not necessarily extend to surgical innovations. The surgical innovator has historically been allowed to “tinker” with procedures, introducing modifications of varying degrees to the point that a procedure could arguably be called new. In fact, there are numerous examples of such innovations currently in clinical use, most notably, the myriad variations of minimally invasive procedures. Rapidly advancing technology offers the surgeon a steady stream of opportunities to innovate, and this is encouraged by the incentive to publish or present novel approaches, the need to attract new patient referrals, and the innate desire to improve the quality of care delivered by one’s profession. But there are currently no formal regulations that apply to surgical innovations.

Consequently, the gap between regulatory goals and professional reality is widening. With the importance of ensuring patient safety, it is clear that surgical innovation can no longer enjoy its unmonitored status.

Surgical innovation in itself is not problematic. Indeed, a safer, more effective, or less morbid procedure would have a direct and favorable impact on patients. The absence of any organized oversight or mechanism to protect patients from becoming unwitting research subjects is, however, problematic.

Patients may be harmed by innovations that are less effective or more dangerous than expected. More than a decade ago, Roy and colleagues observed, “When large numbers of innovative treatments are being continuously introduced into clinical practice, rigorous testing is mandatory for the protection of individual patients and the just use of limited resources. This holds true with even greater force in light of the evidence that many innovations show no advantage over existing treatments when they are subjected to properly controlled study.” Concerns about inadequate testing of operations have led to increasing calls for vigilance to prevent cavalier innovations and to ensure scientific evaluation of new procedures.

This last paragraph tells what is so problematic about much surgical “innovation.” For example, as I’ve mentioned before, in the late 1980s and early 1990s, laparoscopic cholecystomy rapidly replaced the open cholecystectomy as the standard of care for gallbladder disease requiring surgical intervention. It did so not based on randomized clinical trials showing it to be superior (or at least not inferior) to the prior “gold standard” operation, but rather because the technology was sexy and it was very satistfying (not to mention attractive to patients) to be able to remove a gallbladder while leaving only a few tiny scars and at the same time being able to send the patient home the same day or the next day, compared to a several-day hospital stay. In essence, marketing won out before science, and science has played catch up ever since. Older surgeons had to learn the new technique or watch their bread-and-butter cases be referred to surgeons who could do the new technique. Moreover, in the early days of laparoscopic cholecystectomy, it was documented that the rate of common bile duct injuries was considerably higher than for the old operation. Given that common bile duct injuries can be devastating, even to the point of leading to liver failure, and that they can require huge operations to repair, sometimes with unsatisfactory results, this was not a problem to dismiss lightly. These days, the rate of common bile duct injury for laparoscopic cholecystectomy is very low, but in the “learning curve” phase of the adoption of the procedure it was not and it is arguably still somewhat higher than it was “in the old days” before laparoscopic cholecystectomy. I’ve also written about other examples of surgical procedures that spread like wildfire before they had been adequately proven through clinical trials. The aforementioned kyphoplasty for ostoporotic vertebral fractures is an example of just such a procedure, as well.

Still, there is a gradation between a modification of an old procedure and a whole new procedure. I’ll illustrate with an example. Sid Schwab described a perfect example of tinkering by a private surgeon when he told how, over time, he developed a “mini-cholecystectomy.” (This was before the days of laparoscopic cholecystectomies.) There was no clinical trial to test his hypothesis that he could do a cholecystectomy through a smaller incision. He just did it, confident that he could always make the incision bigger if he was struggling too much. (It’s an axiom in surgery that you can always make the incision bigger, but you can’t make it smaller.) Clearly this qualifies as “tinkering”; i.e., the minor modification of a surgical procedure that does not produce a reasonable expectation of increased risk to the patient. Of course, if Dr. Schwab were too much of a “cowboy” to admit when he couldn’t do the operation using a the smaller incision and started getting into trouble because he couldn’t see or get his instruments in there adequately, you could imagine that this tinkering might have the potential to cause harm. But Dr. Schwab is a prudent and skilled surgeon; it’s reasonable to trust that his judgment is good and that he’ll do the right thing and make the incision bigger if he finds himself struggling too much. Lots of procedures have been improved by such tinkering, and, in fact, often such tinkering is simply a reflection on the skill of the surgeon; for example, one of my partners can do a right colectomy through what to me is an amazingly small incision, while other surgeons use much bigger incisions to do the same operation.

Basically, it was widely agreed that there is generally no need for IRB involvement when tinkering with surgical procedures, as long as informed consent is given or, if the operation was done emergently, the patient is informed afterwards what was done. In practice, though, I can see this as a bit problematic. Consider Sid in the example above. Suppose he went up to his patient and said, “I’m going to do your cholecystectomy, but I’m going to do it a little different. I’m going to use a small incision. It means I might struggle a bit and the operation may take a little longer, but I can always open wider if I have to.” Most patients wouldn’t object, but many would ask: “How much smaller?” A reasonable question. If the incision is only, say, 5% smaller than the usual incision, would it be necessary to tell the patient? After all, only Sid would know for sure that it was smaller than his usual incision. Other surgeons might make that size incision just from their own training, and even Sid, unless he measures all of his incisions to the millimeter, probably varies his incision size by as much as 5% from patient to patient just in the normal scope of his practice.

So, what is innovation? Innovation differs from research in that it is a change in therapy designed to benefit an individual, whereas research is a study according to protocol in which the goal is to gain knowledge, not necessarily to benefit the individual being treated. As the Belmont Report, the basis of all human subjects research regulations in the U.S. states:

By contrast, the term “research’ designates an activity designed to test an hypothesis, permit conclusions to be drawn, and thereby to develop or contribute to generalizable knowledge (expressed, for example, in theories, principles, and statements of relationships). Research is usually described in a formal protocol that sets forth an objective and a set of procedures designed to reach that objective.

When a clinician departs in a significant way from standard or accepted practice, the innovation does not, in and of itself, constitute research. The fact that a procedure is “experimental,” in the sense of new, untested or different, does not automatically place it in the category of research. Radically new procedures of this description should, however, be made the object of formal research at an early stage in order to determine whether they are safe and effective. Thus, it is the responsibility of medical practice committees, for example, to insist that a major innovation be incorporated into a formal research project.

The position statement recommends:

Surgical Innovations Requiring Formal Review

If the innovation is planned, AND:
The surgeon seeks to confirm a hunch or theory about the innovation;


The innovation differs significantly from currently accepted local practice;


Outcomes of the innovation have not been previously described;


The innovation entails potential risks for complications;


Specific or additional patient consent appears appropriate,


a) The described review by a local surgical innovations committee is required, plus
b) submission to the national innovations registry is required, and
c) additional informed consent is required of the patient specific to the nature of the proposed innovation.

The recommendations also state that such innovations must comply with FDA and NIH regulations and that informed consent must be obtained, except in the case of emergency surgery, in which case the patient must be informed afterwards, in the case of “unanticipated innovation.” (Of course, one surgeon’s “unanticipated innovation” is another surgeon’s “surgical misadventure.”) It was also recommended that, if all possible, such innovations should be tried out first in an animal before being performed on humans. A further recommendation that was discussed at the SUS meeting but apparently didn’t make its way into the final draft was the creation of a “national innovations registry,” to which surgeons could report their attempted innovations and how they worked. This would be particularly important because most innovations do not work, and some even cause harm. If a surgeon is thinking of trying a new innovation, it would be useful if that surgeon could consult a database to see if someone had tried it before. After all, what’s the first thing we scientists do when considering embarking on a new research project? We hit the scientific literature, of course, to see if anyone has done anything like what we’re thinking of doing before! Scientists have PubMed; many, if not most, innovations, particularly the ones that don’t work, are never published in the peer-reviewed literature. The final recommendation was that “innovation review committtees” be set up to review attempts at surgical innovations. These would tend to be ad hoc committees based on the specific case and far more informal than an institutional review board.

Of course, the distinction between tinkering and innovation (and, for that matter, research) is artificial, and definitely somewhat arbitrary. It all really exists as a continuum. Consider the example of Dr. Schwab’s “minicholecystectomy” again. It’s really a remarkable story of how a surgical procedure, in the hands of a skilled surgeon, evolved over time. Simply by trying to decrease the size of the incision in a systematic fashion, Dr. Schwab ultimately ended up changing his practice. He started using a headlight, special instruments, and a different surgical technique to get the gallbladder out. Ultimately, his consistent tinkering led to innovation, and, I daresay, had it not been for the sudden advent of laparoscopic cholecystectomy in the late 1980’s and early 1990’s, the mini-cholecystectomy that he and others developed would likely have propagated far and wide among surgeons and become the preferred technique for removing gallbladders. In terms of the scientific method, tinkering could be looked at as the stage of hypothesis generation, while innovation could be considered preliminary testing and refining of the hypothesis. The actual research is then the stage where the refined hypothesis is subjected to rigorous testing in a highly controlled, protocolized manner.

Oddly enough, laparosocopic cholecystectomy (LC) is a great example of an innovation that rather got ahead of itself. The potential benefits of the procedure in terms of decreased hospital stay, decreased postoperative pain, and faster return to work were obvious and quickly recognized. However, this procedure, perhaps more than any other, reveals some of the pitfalls in surgical innovation. For one thing, it revealed the effects of marketing and patient demand. Surgeons who learned how to do LCs in those heady early days advertised it heavily, and patients came to see the new procedure as obviously superior to the old before there was much comparative data on complication rates. They started to demand it, and more and more surgeons felt obligated to learn it even before the data from research (formal definition) was all in, lest their patients go elsewhere and their referring physicians abandon them. Eventually, as the research was finally done, it became apparent that there was a considerably higher rate of common bile duct injuries in LC compared to the old method. Granted, it was still a very small number, but the number was nearly ten times smaller still using the boring, old-fashioned open approach to taking gallbladders out, and common bile duct injuries can be devastating, requiring large operations to repair. In the end, it was fairly clear that the benefits outweighed the larger (albeit still very small) risk of common bile duct injuries with LC, but by the time that information was known the procedure had almost entirely supplanted the older procedure. There was no chance to weigh the risk-benefit ratio of LC in anything approaching a calm, measured fashion before introducing the procedure; the analysis was all done after the “cat was out of the bag,” so to speak. Ironically, Sid’s mini-cholecystectomy produced similar good results in his hand to LC; maybe it would have been a better, albeit less sexy, alternative, although it’s always possible that there were complications that would only become apparent if thousands of patients were studied. That’s why innovation needs to be accompanied by research.

There are many problems in considering innovations and research, not the least of which are issues of liability. As one surgeon at the session three years ago asked: “When does ‘innovation’ plus bad outcome equal departure from the standard of care?” The answer was not that encouraging. Innovation by definition is departure from the standard of care, and in the case of a bad outcome it can be a big problem. Surgical innovation committees can ameliorate this problem somewhat by giving cover to the surgeon, as can rigorous informed consent, but clearly in this litigious age the threat of being sued will have a chilling effect on innovation that even these measures may not forestall. Another issue is the composition of these committees. In academic centers, finding the expertise is usually not that big of a problem, but what about in community settings, where either the expertise is lacking or the potential members of such a committee would often be riddled with conflicts of interest, given that potential members would tend to be either partners of or competitors with the surgeon involved? Finally, there is the issue of intellectual property, particularly in the case of devices, even simple ones. Surgeons will be reluctant to share their ideas in a database if they are thinking of patenting them, although it was pointed out that it’s fairly easy to protect one’s idea beforehand by filing a preliminary patent application. And, of course, we must remember that, as much as we hate to admit it, surgeons are human too. Not all of them can always be counted on to act based primarily on what they perceive to be the best interests of their patients.

The bottom line, regardless, is that academic surgeons are going to have to get used to more oversight. Vigorous self-regulation would be the best option if we can pull it off. Gone are the days of the “Wild West,” where “cowboy” surgeons could try whatever innovative idea that caught their fancy. Although many bemoan this new oversight and worry that it will quash surgical innovation, that need not be so if surgeons, rather than the government, act to codify how we distinguish between “tinkering,” innovation, and research and require the level of oversight appropriate to each: IRBs for research and something formalized, but less rigorous, for innovation. Unfortunately, since the SUS guidelines were published in 2008, I haven’t seen much evidence of their having been widely adopted.

On the other hand, what I have seen that is very encouraging is the American College of Surgeons Oncology Group (ACOSOG). Two weeks ago, I attended the Academy of Surgery of Detroit meeting at which Dr. Mitchell Posner, who ran the training program at the University of Chicago when I was a fellow there, presented data on the latest ACOSOG trials. What was most impressive was the effort to which ACOSOG goes to standardize cancer operations rigorously between surgeons. ACOSOG requires surgeons to present videotapes of their operation in order to make sure that the technique is correct and that a surgically adequate resection was performed. In essence, ACOSOG is trying to make surgical trials as close to a randomized, double-blind trial as possible and to remove as many sources of inter-surgeon variation in skill and technique as possible.

In the end, as much as we try to make it a science, surgery can never escape the fact that, by its very nature, it is also a craft. Surgeon skill matters when it comes to patient outcome, and there are often as many ways of throwing a stitch or tying a knot as there are surgeons. Unfortunately, IRBs, as currently constituted, are not sufficiently flexible to take into account tinkering versus innovation versus research. While it is probably not possible to test various new tinkerings or innovations rigorously, it is possible to be a lot more rigorous about overseeing them, preferably before the fact but at least after the fact. Such oversight is increasingly critical as “natural orifice” surgery now breathes down the neck of laparoscopic surgery as the “latest and greatest” fad in surgery. It is also possible to standardize operations as much as possible for purposes of clinical trials. All of these are important developments in the quest to make “surgical science” cease to sound like an oxymoron.

Posted in: Clinical Trials, Science and Medicine, Surgical Procedures

Leave a Comment (15) ↓

15 thoughts on “The continuum of surgical research in science-based medicine

  1. Kausik Datta says:

    Awesome, awesome post. It answers some of the questions I have had for long about the process of making breakthroughs in surgical techniques. I think a part of the proposed oversight should be dedicated to better use of the existing and future technologies.

    Taking the example of the laparoscopic gall bladder surgery, the fact that it could be done would not have been discovered unless some maverick surgeon actually went ahead and did it. And post facto it was discovered that while it had several key patient benefits, there was a small but significant risk, that of bile duct injuries. This discovery has worked out well, too, for unless one knows what the actual risk is, it is difficult to do a proper risk-benefit analysis.

    The caution comes from the fact that surgeons are dealing with human lives. However, unlike in the early days, now there is the added benefit of technology; innovation need not be undertaken blindly. For instance, advances in ultrasound technologies, fiber optics and computational power have provided a much larger window of observation that was unavailable to surgeon-innovators earlier. Can these not be used for a priori or in situ risk assessment for individual patients, and approach devised accordingly? Surely that will lead to harm reduction, no? This is the part that can be codified IMO without stifling innovation in any way.

    David, am I making sense?

  2. Plonit discussed a system of having all clinical studies be public so that every clinical innovation could be part of a study and advance medical knowledge. “The currently approved treatment for your condition is X, with Y benefits and Z risks. There are RCTs being conducted for possible new treatments A and B, which might have a better risk/benefit profile than current treatment X. Would you like to enroll in it?” (Alternatively, “The currently approved treatment for your condition is X, with Y benefits and Z risks. There is not yet an RCT being conducted for any new treatments which might have a better risk/benefit profile than current treatment X. Would you like to put yourself on a list to enroll if one becomes available?”) (Alternatively again, “Hm. Maybe I should look at this formally.”)

    She proposed this as an alternative to things like off-label prescribing, where the conversation goes more like, “The currently approved treatment for your condition is X, with Y benefits and Z risks. Some people think that A is also a possible treatment, which might have a better risk/benefit profile than current treatment X. Would you like to try it?” In this scenario the patient may or may not benefit, but there is no new information contributed to medical science.

    You are probably all familiar with Plonit’s model of networked clinical research; it probably has a name that I could look up in Wikipedia if I knew it, and it is probably generally considered too idealistic. But could it be usefully applied to the problem of surgical research?

  3. The reference that Plonit was using was Iain Chalmers et al:

    For purposes of this thread, Chapter 8 Blueprint for a Revolution starting on Page 96 would be the one that would be applicable (if it is applicable at all).

  4. qetzal says:

    It seems to me that laparoscopic gall bladder surgery could very easily have been tested through a series of fairly conventional clinical trials.

    First, show that the procedure is feasible in a suitable animal. Then develop a Phase I protocol to randomize sutiable patients to conventional vs. laparoscopic procedures. Get the protocol reviewed & approved by an IRB. Like any new treatment, keep the first study small (10-20/arm). Evaluate the results for any indication of frequent, unexpected, serious complications. Proceed to larger trials with more surgeons/centers to gain further info on outcomes, complication rates, etc.

    In a case like that, where the new procedure is significantly different from accepted procedure, I honestly can’t see a good justification for anything less rigorous. The need for oversight and patient protection is no different than for an investigational drug study. Relying on an informal belief that Dr. So-and-so is a good and prudent surgeon seems obviously inconsistent with the applicable ethical principles.

  5. Plonit says:

    It seems to me this is throwing the baby out with the bathwater.

    Just because trials can’t be double-blinded, placebo trials doesn’t mean that trials of any sort are worthless. Sure, it’s a limitation, but not an insuperable one – especially if you have some objective organic measures which you don’t expect to be affected by lack of blinding or lack of placebo.

    The other area where surgery needs more scrutiny is in comparison with other therapies or ‘do nothing’. Not a lot of point in comparing full and partial hysterectomy if you haven’t compared either to non-surgical treatments.

  6. TimonT says:

    This post is extremely interesting and informative to me as someone unfamiliar with the specific issues in surgery research and also one who has wondered about how best to research problems where RCTs can’t be used.

  7. laproxdoc says:

    @qetzal – Although the original development of laparoscopic cholecystectomy was indeed led by a “cowboy-innovator” surgeon it did proceed initially with animal lab testing for feasibility and safety. It began with somewhat Rube Goldberg adaptations of an operative technique that had been in use by Gynecologists for many years but had rather stagnated. Way back then the first adopters practiced their handiwork in animal labs. Real advances in minimally invasive surgery did not come however until there was an increasing demand from patients and surgeons such that it became worthwhile for industry to invest significantly in developing highly technical innovations. When I look back and compare the instrumentation currently in use to the instruments available when the procedure first appeared it is surprising to me that the complication rates were not significantly worse than the open technique. The early equipment was rude and crude and the operations were long and tedious. With modern equipment the operation has become routine, very safe, quick and slick, the view and control of the surgical field is much better than the open operation, patient recovery is rapid, and there is much greater flexibility to deal with unexpected problems and complications. The pace of development has been fast. Interestingly, there has been little development and adoption of better instrumentation for the open operation; for those cases requiring conversion from laparoscopic to open the instrumentation (except for a few gadgets) is pretty much the same as it was 20 years ago.

  8. qetzal says:


    Thanks for the info. Makes sense that there would be animal studies with the instruments, whether or not there was any formal IRB or FDA requirement for such.

    Even so, I can’t see any reason that human studies wouldn’t have fit into a pretty conventional clinical trial scheme. I also can’t see any good reason not to have done so. At least, if we believe that the clinical trial paradigm is the proper way to protect patients when it comes to new drugs, I can’t see why new surgical techniques would be fundamentally different.

    If you or any other surgeons out there disagree, I’d be very interested to hear why.

  9. edgar says:

    I have an ER doc friend who is confounded by informed consent when it comes to treating head trauma patients. How in the world you you test new developments when they are unconscious and cannot consent to research?

  10. DREads says:

    In the end, as much as we try to make it a science, surgery can never escape the fact that, by its very nature, it is also a craft. Surgeon skill matters when it comes to patient outcome, and there are often as many ways of throwing a stitch or tying a knot as there are surgeons.

    Reminds me of something my British colleagues told me once. In the UK, doctors relinquish their title Dr. and use Mr. once elected to the Royal College of Surgeons. For some time, surgery and medicine followed separate paths in England: surgeons were taught in a master-apprentice fashion, and physicians, in a formal university setting. Prior to the 17th century, neither a formal university education nor a medical degree was required to practice surgery. Nowadays, surgeons must first earn a medical degree. But in keeping with tradition, it is considered an honor to revert back to Mr.

  11. antipodean says:

    It was good to hear your thoughts on this, David. I’m wrestling with precisely the same issue in my field.

    Thanks very much for the SUS statement. It will undoubtably be useful for clarifying my approach

  12. DREads on honorifics:

    “In the UK, doctors relinquish their title Dr. and use Mr. once elected to the Royal College of Surgeons.”

    Would that explain why only 25% of surgical trainees in Britain are women, even though they represent 57% of medical school entrants?

  13. JMB says:

    qetzal wrote,
    “At least, if we believe that the clinical trial paradigm is the proper way to protect patients when it comes to new drugs, I can’t see why new surgical techniques would be fundamentally different.”

    I’m not a surgeon (but I did do a year of surgery internship). I think that the point Dr Gorski was making about the fundamental differences between drugs and surgical techniques are; 1 – there is no easy equivalent to the sugar pill used in drug studies to control for the placebo effect in surgery (sham operations are unethical), and 2 – surgical outcomes are very dependent on the skill of the surgeon, medication preparations are much easier to manufacture in uniform quality than surgeons. A consequence of #2 is that it may be more important for the protection of patients to monitor surgical outcomes, than to perform a randomized clinical trial. A rigorous certification process for surgeons to be credentialed to perform laparoscopic cholecystectomy might have had a greater impact on the initial morbidity and mortality than a RCT.

  14. antipodean says:

    JMB- RCTs do not exist for the purpose of reducing initial morbidity and mortality. They exist for the purpose of showing that there exists a reduction in M&M.

    The fundamental problem still exists that RCTs in surgery and implantable and non-implantable devices are hard to do.


  15. JMB says:

    antipodean- That is what I was arguing. qetzel seemed to be indirectly suggesting that the initial M&M of laparoscopic cholecystectomy could have been avoided if an RCT had been completed. However, such an RCT completed on laparoscopic cholecystectomy would have been performed in a few large centers with well trained surgeons. That would not have prevented the M&M that resulted from the rapid rollout of the technique. Rather than spending the money and effort on an RCT of laparoscopic cholecystectomy, I argue that it would have been better to spend it on trials of how to train and certify practicing general surgeons in the technique of laparoscopic surgery.

    The RCT is based on British empiricism which emphasized identification of causal relationships. An RCT is applicable to the problem of whether performing a cholecystectomy in a patient with gall bladder disease will result in a reduction of M&M from biliary tract disease. It is less suitable for the problem of determining which surgical technique of cholecystectomy will have the lowest M&M. It is not suitable to address the problem of how to train and certify surgeons in the surgical technique of laparoscopic cholecystectomy. Once the causal issue is settled, we should move on to the optimization problem.

    An EBM approach does not address the issue of “production quality”, it only emphasizes RCTs to prove a causal relationship and an acceptible harms vs benefit ratio. SBM can go beyond the emphasis on RCTs, and incorporate more scientific (and engineering) methods to translate the causality emphasis of medical research into the optimization problem of medical practice. SBM can address the production quality issue.

Comments are closed.