Comparative Clinical Effectiveness Research: Good News In Shades Of Gray

When I first heard about the new emphasis on comparative clinical effectiveness research (CCER) in Obama’s economic stimulus bill I thought, “Thank goodness! Maybe now science will truly regain its rightful place and we’ll end the CAM, ‘me-too’ drug, and excessive-use-of-technology madness that is wasting so much money in healthcare.” In fact, I was so excited about the new administration’s apparent interest in objective analysis of medical treatment options, that I intended to write a jubilant blog post about it. However, as with most things that seem black and white at first glance, further analysis reduces them to shades of gray.

What Is Comparative Clinical Effectiveness Research?

The new economic stimulus bill, also known as The American Recovery and Reinvestment Act (ARRA) includes 1.1 billion dollars for clinical comparative effectiveness research. Interestingly, CCER is not defined in the bill though AHRQ describes it this way in their glossary:

“A type of health care research that compares the results of one approach for managing a disease to the results of other approaches. Comparative effectiveness usually compares two or more types of treatment, such as different drugs, for the same disease. Comparative effectiveness also can compare types of surgery or other kinds of medical procedures and tests. The results often are summarized in a systematic review.”

Any mention of “comparative cost effectiveness” or value-based language is notably absent.

How Does It Work?

The government’s new CCER initiative will be administered through a Federal Coordinating Council for clinical comparative effectiveness research. The FCC consists of a group of 15 federal employees, half of whom “must be physicians or other experts with clinical expertise.” [Meaning, none have to be physicians.] Some have suggested that the FCC is the first step toward an organization modeled after Britain’s National Institute of Health and Clinical Excellence (NICE). NICE is regularly tasked with helping the NHS to decide which medical treatments should be available to their beneficiaries, and which should not be covered (based on their efficacy and cost).

The budget for the CCER will be divvied up as follows:

400 million – left to the discretion of the Secretary of HHS with 1.5 million to go to the Institute of Medicine for a report regarding where to focus CCER attention initially
400 million – to the office of the director, NIH
300 million – to AHRQ

Here is a quote from the ARRA bill, discussing the mechanics of CCER:

“The funding appropriated in this paragraph shall be used to accelerate the development and dissemination of research assessing the comparative clinical effectiveness of health care treatments and strategies, including through efforts that: (1) conduct, support, or synthesize research that compares the clinical outcomes, effectiveness, and appropriateness of items, services, and procedures that are used to prevent, diagnose, or treat diseases, disorders, and other health conditions and (2) encourage the development and use of clinical registries, clinical data networks, and other forms of electronic health data that can be used to generate or obtain outcomes data: Provided further, That the Secretary shall enter into a contract with the Institute of Medicine, for which no more than $1,500,000 shall be made available from funds provided in this paragraph, to produce and submit a report to the Congress and the Secretary by not later than June 30, 2009 that includes recommendations on the national priorities for comparative clinical effectiveness research to be conducted or supported with the funds provided in this paragraph…”

A Game-Changer For Pharma, Medical Technology, and Biotech

There is no doubt that CCER stands to radically change how industry does business. I anticipate that industry will develop their own internal CCER teams, and begin the process of comparing their new product (to others currently available) as early as phase 3 clinical trials. By and large, that’s probably a good thing – though there are potential unintended consequences that deserve mention.

While it’s appropriate for drug, device, and equipment manufacturers to consider whether or not their new product actually contributes something new/better to our current cadre of treatment options, there will be clear winners and losers in this game. And when companies lose, we lose companies. That’s generally not good for the economy. Manufacturers without diversified product lines are more likely to go out of business – and it will become more difficult for new players to enter the marketplace.

Although comparative clinical effectiveness research is distinct from comparative cost effectiveness research – it is likely that payers will use CCER to build their formularies. This means that even though the government (at this point in time) is not mandating coverage decisions based on CCER, health insurers are going to be using the information liberally to justify coverage preferences and even potential denials of coverage.

There’s also the question of stifling innovation. Blockbuster drugs are rarely discovered in a vacuum. They are the result of incremental steps in understanding the biology of disease, with an ever improving ability to target the offending pathophysiologic process. The first few therapies may offer marginally improved outcomes, but can lead to discoveries that substantially improve their efficacy. If an early drug is found to be only marginally better than the standard of care, an unfavorable comparative effectiveness rating could kill the drug’s sale. Without sales to recoup the R&D losses and reinvestment in the next generation of the drug, development may cease for financial reasons, and the breakthrough drug that could cure patients would never exist.

As Dr. Rich argues in his excellent blog: healthcare rationing is inevitable – but it’s more ethical to do it overtly than covertly. I would also like to suggest that insofar as physicians can be enlisted to translate CCER for patients (rather than being handed down inflexible rules from on high) and help them make the best decision for them – that would be even better. The inflexibility of national decisions about healthcare rationing does make me nervous.

What’s Good For The Geese Isn’t Necessarily Good For The Goose

First of all, most key healthcare stakeholders would like to be able to compare efficacy of one treatment option over another. Informed decision-making is hard to do when head-to-head studies are simply not available for most treatment options.

However, population-based conclusions do not always provide a clear “best choice” for individual patients. Individual genetic differences, allergy profiles, complicated drug regimens, unique constellations of diseases, socioeconomic factors, and psychological issues all influence clinical decision-making.

Dr. Nancy Nielsen recently voiced concern about CCER at the Medicare Policy Summit. She said that the AMA’s position is that CCER is for information purposes, not for coverage decisions. CCER’s goal is to help patients make informed choices, not limit their choices. I’m afraid the horse may have already left the barn on that one – but I agree with Dr. Nielsen’s sentiment. It would be wonderful if CCER could remain in its supportive role for shared physician-patient informed decision-making. My fervent wish is that in doing the right thing by the geese, we don’t kill off the occasional goose. Physicians need the flexibility to make exceptions when necessary for their patients.

Republican Unrest

A certain degree of hysteria related to CCER has recently wafted up through the hallowed halls of government. Are republicans overreacting to the bill? Maybe – though the bill doesn’t include any provisions for using CCER to mandate coverage decisions or ration care, it seems that Pete Stark has made it clear that he’d like the FCC to “direct medicine” which does kind of send a shiver down my physician spine, and provides some insight into what some democrats are hoping to accomplish with CCER – laying the foundation for future government involvement in the diagnosis and treatment of patients.

Also one particular congressional report is proving helpful in “reading the tea leaves” regarding the democrats’ plan for CCER. In describing the comparative effectiveness provision, the report states that items, procedures, and interventions “that are found to be less effective and in some cases, more expensive, will no longer be prescribed.”

While congressional reports are not binding, they do give an indication of intent.

The bottom line is that though CCER is not supposed to be used for “cost effectiveness” decisions – there’s no policy in place to protect that from happening.


Information about the comparative clinical effectiveness of treatment options is critical for the practice of evidence based medicine. Such information supports informed decision-making, and could be the single most important strategy for reducing the use of wasteful or ineffective therapies in healthcare.

On the other hand, CCER will certainly have some negative consequences, both anticipated and unanticipated. When “cost effectiveness” conclusions are drawn from clinical effectiveness data, rationing ensues, patient choices are limited, people lose their jobs, and some companies go out of business. As a recent article in the New England Journal of Medicine points out, “saying no isn’t nice.” I greet this 1.1 billion dollar initiative with muted enthusiasm.

Post Script

In my research for this blog post I came across some interesting quotes. I thought I’d add them here for your consideration:


And before you tell me we need such bills in order to be more scientific, take a minute and ask yourself just how scientific you think the government will be when it applies cost-cutting measures to medicine.  The congress is certainly a hot-bed of evidence-based legislation, isn’t it?

- Edwin Leap, M.D.

When things go wrong, which of course they will, we reach for 2 tools to try to fix them: rules, and incentives. We see this at work in our response to the current financial crisis – but the truth is that neither rules nor incentives are enough to do the job.

When we turn increasingly to rules and incentives, they may make things better in the short run but they create a downward spiral that makes them worse in the long run. Moral skill is chipped away by an over reliance on rules that deprive us of the opportunity to improvise and learn from our improvisations, and moral will is undermined by an incessant appeal to incentives that destroy our desire to do the right thing.

Without intending it, by appealing to rules and incentives we are engaging in a war on wisdom.

Don’t get me wrong, we need rules. Most Jazz musicians need some notes on the page, and we need more rules for the bankers, God knows. But too many rules prevent jazz musicians from improvising and as a result, they lose their gifts – or worse, they stop playing altogether.

We need incentives – people have to make a living. But over-reliance on incentives demoralizes professional activity. It causes people who engage in that activity to lose morale, and it causes the activity itself to lose morality.

- Barry Schwartz, Ph.D. from his lecture at TED

Posted in: Clinical Trials, Pharmaceuticals, Politics and Regulation, Public Health

Leave a Comment (14) ↓

14 thoughts on “Comparative Clinical Effectiveness Research: Good News In Shades Of Gray

  1. caoimh says:

    Great Article Val!

  2. scottg416 says:

    Nice summary. Very pleased to see this, from a Canadian perspective.

    In managing a public drug formulary, we have often asked pharmaceutical manufacturers for comparative effectiveness data to justify why one drug deserves to be reimbursed at a higher price over a less expensive comparator.

    Invariably we would be told that this research was unlikely to ever emerge, as this type of evidence simply isn’t necessary for market success in the U.S. market.

  3. Harriet Hall says:

    Is this effort directed at things like drug vs drug or drugs vs surgery, or is there any indication that they might do broader studies like comparing homeopathy or naturopathy to scientific medical care?

  4. Wallace Sampson says:

    I think one should evaluate a proposal considering the motivations of the proposers. Behind this one is a simmering resentment of medicine as now practiced freely and independently. That threatens some people. Resentment becomes directled broadly at physicians as well as the “system” – HMOs, insurers, administrators, pharma, etc. I detect the same resentments motivating and engineering the sectarian medicine movement. Jealousy.

    Congressman Stark of (East Bay) Calif. is a well known critic of medicine, with an emotional load and a short fuse (he once accused a war vet of being a faker, chews people out over the phone…etc.) Anything that would control physicians and their incomes would be fine with him.

    Who wrote this proposal? No doubt it had been written months to years before a stimulus or rescue plan was ever conceived. There is an army of critics and system rearrangers at work in the open as well as behind the scene in government back offices and social think tanks. They all have a solution to someone else’s problem.

    Val touched on something as did some of those she quoted. The success of our scientific and economic systems have developed attached at the umbilicus. They appear from outside to be chaotic. That offends many people’s sense of order, and raises supicions of uncontrolled ripoffs.

    Scientific investigation often appears not to have clear goals, and that principle has been debated long and hard, with the winner so far – at least in the US – being independence of scientists to pursue their own thinking and passion without Manhattan Projects or financial puppet masters.

    Out of the crucible of individual thought evolves success that system managers could not predict or even imagine. There are a few exceptions such as the human genome project, but that was launched only when ready, and even then, the privately run system won that race and has gone on more rapidly from there.

    The chaos of freedom does not have an organized advocate. Almost by definition. Conservative politicians come close to voicing the principles … but often get lost in emotional private tangents like religion and gun control, while others, instead of listening, shut out the important parts, individual freedom from central controls.

    It’s a psychological thing. But in practice, we are doing just fine. Practicing medicine is a profession, not a job. Medicine has independently evolved a behavior code and system of ethics that are only partially encoded into law, and all are better for that.

    Medicine costs a lot. But society seems to afford it and want it. So what’s with the control freaks in DC? A governmentally based effectiveness system is dangerous stuff.

  5. Dacks says:

    “Behind this one is a simmering resentment of medicine as now practiced freely and independently.”

    “But in practice, we are doing just fine. Practicing medicine is a profession, not a job. Medicine has independently evolved a behavior code and system of ethics that are only partially encoded into law, and all are better for that.”

    Dr. Sampson,
    As a consumer of medicine, I am not privy to the calculations made by my physician in prescribing a certain drug or treatment. I am reassured by believing that the treatment prescribed for me is based on the best standards of care. These standards do not arise solely from whatever my physician thinks is best, but should, ideally, be backed by clinical studies. From there, it is a short step to expect that medicine is based on comparative effectiveness of treatments.

    Of course, a more regulated system would have its faults, including the insertion of politics into science, but a less regulated system is open to other influences, and can be more confusing to the patient. If I go to three biology classrooms I don’t expect to hear widely differing opinions on the significance of natural selection. If I go to three doctors, I would prefer not to hear three widely varied opinions on the best course of treatment.

  6. Fifi says:

    Wallace Sampson, thanks for you post – mainly because it illustrates just why the “health freedom” meme pushed by SCAM in the US has so much traction. I take it you also support CAM being integrated into medicine and given equal footing to evidence-based medicine?

    I disagree with your argument that regulation is bad and no regulation equals “freedom” (which goes along with the American political belief that democracy=capitalism). As it is, corporations have more say in healthcare decisions in the US than doctors or patients simply because insurance companies decide what is permitted – not based on best treatment but based purely on economics and what they can get away with (such as creating roadblocks and refusing lifesaving treatment to people too sick to be able to object and fight for the services they paid for when healthy).
    Big Pharma makes many arguments about why they need to have exclusivity and make more money. The idea that there would never be any innovation without Big Pharma looking to make a buck makes absolutely no sense to me. The dark side of this bargain with the devil that many don’t see is that research into cheap and easily available substances – or prevention – is never done by Big Pharma because there’s no profit in it. It’s hardly the best thing for medicine to have players that are only interested in developing expensive drugs doled out by one or two corporations with a monopoly who will work together to drive up prices. It’s not at all the same as two scientists competing to make a medical discovery – corporate science is hardly the same thing as academic science.

  7. David Gorski says:


    Reading your comment I’m a bit confused. The reason is that medicine is regulated. The question is not whether it should be regulated but rather how much. Given that some level of government regulation is necessary, don’t you want it to be based on the best science available whenever possible? That’s why I’m really puzzled by your blanket statement that a “governmentally based effectiveness system is dangerous stuff.” The reason I’m puzzled is because such a system, implemented well, would go a long way towards eliminating a lot of the arbitrariness currently inherent in which treatments government insurers cover and which ones they do not. Such a system could go a long way towards making that decision making process more science-based.

    Perhaps a better question would be this: Given that you clearly don’t like government regulation of the medical profession, how much government regulation is enough and how much is too much? In other words, you castigate the government for not protecting us against quacks and for facilitating the acceptance of non-science-based medicine, but how is the government to protect patients otherwise? Let’s look at the two far extremes we could take.

    First, there’s the American Association of Physicians and Surgeons’ position, which is on the libertarian extreme end:

    AAPS members believe this patient-physician relationship must be protected from all forms of third-party intervention.


    AAPS oppose continued and increasing government interference, supervision, and control in the practice of medicine; promote the immediate repeal of all laws, regulations, and policies that allow direct or de facto supervision or control over the practice of medicine by federal officers or employees; and call for a moratorium on any further laws, regulations, or policies that authorize government control over the practice of medicine.

    I trust that you recognize that having no government regulation of medicine is a pathway to chaos and quackery. On the other extreme, there is a government-run system, which has its own shortcomings, but, I would argue, nowhere near the same level that a completely libertarian model of health care would have. So, the question remains: What is the “sweet spot” between no regulation and total government control of health care that would provide the best care at a cost society can afford while protecting patients as much as possible against quackery? You’ve criticized what you perceive as excessive governmental intervention in regulating medicine, but what would you eliminate? What, if anything, would you strengthen? I’m guessing you would like to repeal the DSHEA of 1994, but doing that would automatically expand the regulatory purview of the FDA. More regulation! Certainly (or at least I hope) you don’t want to neuter the FDA any further. Or how about state medical boards? They are toothless enough as it is to stop quackery. However, the only solutions I can think of to rein in quackery would involve strengthening and educating state medical boards.

    Perhaps you have better ideas about how to rein in quackery without increasing government oversight.

  8. Wallace Sampson says:

    This will be short to let you know I have read your post-comment comments. I wrote a long answer to Dacks and Fifi, left to do some yard work, and “submit”ted it only to find the page had signed off (time expired) and the note not recoverable. OOff and drats.

    I will reproduce it as best I can later tonight.

    For now I admit to a degree of frustration that some of you come from an orientation that assumes I state and think things neither thought nor stated. Also, by my using words in their classical sense, some react emotionally, having become too attached to your own meanings and implications. The most important words being “freedom,” and another, “control.” My erased answer goes to the basic meanings and to the natural order of things.

  9. Dr Benway says:

    An interesting debate.

    Trilafon will likely be at the top of some state approved algorithm for first-line treatment of psychosis. However, if the patient is underweight, doesn’t sleep, and poorly tolerates anticholinergic side effects, Zyprexa might be a better fit.

    I’d hate to get so boxed in by abstract, evidence-based cost-benefit guidelines that I’m not allowed to tweak things for an individual patient without UR forms and phone conferences.

    I’d like “evidence-based” to mean, “nothing batshit” at this early stage of the game.

  10. Fifi says:

    Dr Sampson – I look forward to your response to myself and Dacks, and also to Dr Gorski. I would have looked forward to it more if you didn’t feel the need to try to talk down to Dacks and myself by proposing that you are representing “the natural order of things” and “basic meanings” while you assert others are “attached to their own meanings” and “reacting emotionally”. The more you defend your position, the more you sound like advocates of the “health freedom” movement and unaware of why and how “health freedom” has been such useful and marketable doublespeak for “no regulations on sCAM”. You seem irrationally afraid of regulation – so much so that you’re conjuring up nightmare scenarios which aren’t the reality of what happens in other countries (while you seem to be ignoring who is actually controlling and defining what treatments doctors can provide). If this isn’t the case and you didn’t mean to come off sounding that way – and were simply not conscious about how “health freedom” as a concept and terminology is now tied in with sCAM and keeping sCAM deregulated – then I’d suggest that you may want to rethink both the unconscious content (both superficial and meaningful) of what you’re expressing and how it aligns with sCAM’s arguments for deregulation.

    Once again I’m left with the impression that contemporary US culture may simply not be politically or socially mature or collectivist enough in nature – or enough of American society isn’t, particularly those with power – to be able to manage universal healthcare.

  11. Dacks says:

    I’ m not sure we need to learn about the “natural order of things” here – we might want to confine the discussion to whether comparative effectiveness studies help or hinder good medical practice. Or, more to the point, whether the proposal to require them is a useful idea.

  12. weing says:

    It all boils down to who will be the 15 people comprising the FCC. This appears to be the first step in the direction of a National Health Board as described by Ezekiel Emanuel, Rahm’s brother in his book “Healthcare, Guaranteed”. It is supposed to be analogous to the Federal Reserve System, with the members
    “nominated by the president and confirmed by the Senate for fixed, long, and staggered terms, which can be renewed only once.”
    I kind of agree with that model as it would make the board independent of political interference. My stipulation would be, that these 15 all be physicians.

Comments are closed.