US moves to improve health decisions

BMJ 2010; 341 doi: (Published 23 August 2010) Cite this as: BMJ 2010;341:c3615
  1. Sean R Tunis, executive director1,
  2. Steven D Pearson, president2
  1. 1Center for Medical Technology Policy, Baltimore, MD 21210, USA
  2. 2Institute for Clinical and Economic Review, Massachusetts General Hospital, Boston, MA, USA
  1. Correspondence to: S R Tunis sean.tunis{at}
  • Accepted 30 June 2010

A new US institute for comparative effectiveness research will help patients, clinicians, and policy makers make more informed choices. Sean Tunis and Steven Pearson explain why it has been controversial and assess its prospects for success

One of the provisions of the recent US health reform legislation was to create a national Patient-Centered Outcomes Research Institute. The institute, which will be an independent non-profit organisation, will guide the expansion of comparative effectiveness research.1 It will be overseen by a multistakeholder governing board to be appointed by September 2010, with public and private funding rising above $500m (£330m; €400m) a year by 2014.

Expanding the national capacity to generate better evidence to guide decision making seems uncontroversial, but research into comparative effectiveness rapidly became a lightning rod for intense debate. The new institute will soon be launched in a political atmosphere still smouldering with distrust and anger. We describe the central issues and controversies in the debate, explain how they were dealt with in the health reform law, and discuss the critical steps that will be needed to allow the new institute to meet its high expectations.

What is comparative effectiveness research?

Comparing risks and benefits of alternative healthcare strategies has been a longstanding goal of clinical research and health technology assessment, and it is also fundamental to comparative effectiveness research. The discipline uses a wide range of methods including syntheses of existing evidence, analyses of routinely collected data, and the generation of new evidence through prospective registries and clinical trials. The key defining characteristic of comparative effectiveness research is that its explicit purpose is to generate evidence that will help patients, clinicians, and health insurers make more informed clinical and health policy decisions.2 This is the first time that a large research portfolio specifically dedicated to the information needs of decision makers has been funded, and it will significantly affect every facet of research, including priority setting, research methods, peer review procedures, research implementation strategies, and workforce training.

The new institute’s heavy emphasis on primary research distinguishes it from the UK’s National Institute for Health and Clinical Excellence (NICE) and other international agencies that focus on making recommendations based on the synthesis of existing evidence. The scope of topics to be covered is extremely broad, including the evaluation of quality improvement interventions, public health programmes, and organisational and financial interventions as well as studies of specific tests and treatments. Research on healthcare delivery systems was the primary or secondary focus for 50 of the top 100 priority topics for the new institute identified by the US Institute of Medicine.3

Causes for concern

Criticism of comparative effectiveness research during the protracted debate on US healthcare reform centred on three areas. Firstly, critics warned that its methods of research and evaluation would emphasise average effects over individual effects and therefore interfere with the progress of “personalised medicine.”4 Secondly, they predicted that it would hinder innovation by creating new hurdles for reimbursement. The near sighted goal of containing costs could, it was argued, fail to recognise that innovation contributes to both public health and economic growth.5 6

Lastly, many commentators surmised that the real purpose of comparative effectiveness research was to augment the federal government’s power to deny payment for expensive health interventions. This concern is understandable given statements by senior administration officials about the large amount of spending on services with no or unknown benefit, and the role of comparative effectiveness research in determining which services were effective.7 These concerns gained traction in the context of a rising chorus of conservative commentators warning of the dangers of expanded federal powers. In this context, it took just a dash of political alchemy to produce the most memorable epithet of the entire healthcare debate—the claim that a federal comparative effectiveness research institute would inevitably be transformed into a government “death panel.”8


The final version of the legislation creating the comparative effectiveness research institute includes several critical elements that are intended to overcome the above concerns. Perhaps most importantly, the new institute is placed entirely outside of government, providing it with an unusual degree of autonomy and private sector engagement compared with other government funded bodies.

The law also contains extensive specifications limiting the institute’s use of its research. The institute was provided with no authority to mandate coverage or reimbursement decisions or even to make explicit recommendations to guide clinical or health policy decisions. Opposition lawmakers made several attempts to prohibit use of institute reports or findings to guide coverage decisions for the national Medicare programme (which provides health benefits to elderly, disabled, and dialysis patients). These efforts were turned aside in favour of alternative requirements that Medicare use a “transparent and iterative process” for any coverage decision that considers new evidence from comparative effectiveness research.

The use of cost effectiveness was another major flashpoint of controversy. Proponents of cost effectiveness research argued that patients and clinicians, in addition to insurers, had an interest in knowing when treatment options of equivalent effectiveness were more or less costly over the long term. But to critics, any consideration of costs raised the spectre that the federal government would use comparative effectiveness research to ration care, withholding effective but costly services from elderly, disabled, and other vulnerable populations.

The final version of the law tried to balance these viewpoints. The research to be commissioned by the institute was labelled “comparative clinical effectiveness” to emphasise that the studies should not focus on costs or value. The institute is prohibited from developing or using a measure of cost per quality adjusted life year (QALY) as a threshold for guidance. Although these restrictions do not formally prevent the institute from funding studies that compare the costs and benefits of alternative medical interventions, the wording could be interpreted as ruling out funding for the version of cost effectiveness analyses that health economists and policy makers most often use.

Keys to success

Despite these unresolved issues, we believe that the new institute will prove to be enduring and highly influential. Setting research priorities based on what patients, clinicians, and payers most want to know (and spending $500m a year answering those questions) should transform the infrastructure, methods, and data sources for clinical and health services research. But to succeed the institute’s staff and governing board must be mindful of the lessons from the key controversies and legislative debates of the past few years. To counter the arguments that federally funded comparative effectiveness research will be primarily a government tool to save money by denying access to services, the institute will need to show that it is genuinely committed to meeting the information needs of patients and clinicians. That will require procedures to ensure that clinicians and patients have a lead role in setting the institute’s priorities for research, defining specific research questions, and in selecting the outcomes that will be measured when comparing health interventions.

Secondly, the institute will need to address potential doubts about its integrity given that multiple conflicts of interest are embedded in its governance structure.9 With drug companies, private health plans, and medical professional societies all having influential roles, the institute’s policies, procedures, and priorities will need to be deliberated openly and clearly focused on ensuring complete disclosure and effective management of financial and intellectual conflicts of interest.

Thirdly, the institute will need to focus on developing and promoting research methods that balance internal validity with relevance, feasibility, and timeliness. This will require a clearly articulated framework for deciding whether randomised clinical trials or other research methods based on non-experimental data will provide evidence that is both credible and useful.10 Some of the promising comparative effectiveness research methods that attempt to achieve validity, relevance, and feasibility include pragmatic clinical trials, adaptive trials, cluster randomised controlled trials, and instrumental variables and propensity scoring for observational data.11 A methodology committee will be established within the institute to develop methodological guidance for this emerging field.

Ultimately, the most important political task for the institute is to show convincingly— and soon—that it provides information of real value for patients, consumers, and clinicians. This will require innovative approaches to enable action on its research evidence. In the absence of direct decision making authority, the institute will need to collaborate with patients, clinicians, and payers to develop effective ways to link comparative effectiveness research to clinical guidance, patient information aids, and payment policies.12 Critical to the success of these efforts will be allied efforts within health reform to shift overall economic incentives in the healthcare system to reward higher quality and more efficient care. Mechanisms such as bundled payments for clinicians, tiered copayments for patients, and value based pricing strategies for tests and treatments will enhance the incentives to use evidence and will make all stakeholders in the healthcare system more enthusiastic users of accurate and relevant comparative effectiveness research.13 14


The prospects for comparative effectiveness research in the United States depend on a wise reading of the political lessons of the past mixed with a forward looking and creative approach to translating “centralised” evidence into tools that will serve the decentralised and pluralistic US healthcare system. The legislation creating the new institute, although imperfect, provides a good basis from which to start. The next few years will determine whether the institute will be able to gain the trust of the American public and contribute to growing international efforts to turn better evidence into better health.


Cite this as: BMJ 2010;341:c3615


  • We thank Alan Garber, Kalipso Chalkidou, Carolyn Clancy, and Michael Rawlins for feedback on earlier drafts of this article. Merianne Tiglao provided editorial help.

  • Contributors and sources: SRT has a longstanding professional interest in health technology assessment, with an emphasis on strategies to improve the quality of primary research for clinical and health policy decision making. SDP has worked extensively on improving methods to synthesise existing research and translating this information into useful tools for decision makers. Both were heavily involved in the policy discussions of comparative effectiveness research that preceded the establishment of the new federal institute. SRT and SDP both contributed to developing the concepts, writing, and revising all drafts of the manuscript.

  • Competing interests: All authors have completed the unified competing interest form at (available on request from the corresponding author) and declare no financial support for the submitted work from anyone other than their  employer; SRT and SDP are employees of private sector academic (MGH) and non-profit (CMTP) organisations that receive funding from government, private foundations, health plans, life sciences companies, and other sources to support comparative effectiveness research projects; SRT was the chief medical officer of the Centers for Medicare and Medicaid Services, with oversight of clinical policy for the Medicare programme.

  • Provenance and peer review: Not commissioned; externally peer reviewed.