Intended for healthcare professionals

Analysis

Raising the bar for using surrogate endpoints in drug regulation and health technology assessment

BMJ 2021; 374 doi: https://doi.org/10.1136/bmj.n2191 (Published 16 September 2021) Cite this as: BMJ 2021;374:n2191

Linked Feature

Should regulatory authorities approve drugs based on surrogate endpoints?

  1. Dalia Dawoud, senior scientific adviser1,
  2. Huseyin Naci, associate professor of health policy2,
  3. Oriana Ciani, associate professor of practice34,
  4. Sylwia Bujkiewicz, professor of biostatistics5
  1. 1Science, Evidence and Analytics Directorate, Science Policy and Research Programme, National Institute for Health and Care Excellence, London, UK
  2. 2Department of Health Policy, London School of Economics and Political Science, London, UK
  3. 3Centre for Research on Health and Social Care Management, SDA Bocconi, Milan, Italy
  4. 4College of Medicine and Health, University of Exeter, Exeter, UK
  5. 5Biostatistics Research Group, Department of Health Sciences, University of Leicester, Leicester, UK
  1. Correspondence to: H Naci H.Naci{at}lse.ac.uk

Surrogate endpoints provide no guarantee of clinical benefit, and Dalia Dawoud and colleagues argue they should be used only as a last resort in drug trials

In June 2021, the US Food and Drug Administration granted accelerated approval to aducanumab for treating Alzheimer’s disease based on the drug’s amyloid reducing effects. This was despite evidence from several earlier studies that shrinkage of β-amyloid protein plaques does not predictably delay cognitive impairment.1 The controversial decision has drawn attention to the use of surrogate endpoints—laboratory values, radiographic images, or other physical measures that may serve as indicators of clinical outcomes such as symptom control or mortality—in clinical trials of new drugs.2 In fact, the approval of aducanumab is only the latest example of growing regulatory reliance on surrogate endpoints, even though their use can cause problems for patients, clinicians, drug regulators, and health technology assessment bodies.

We argue for more selective use of surrogate endpoints when evaluating new drugs, restricting their use to chronic diseases, especially when collecting data on patient relevant clinical outcomes requires trials with unattainably long follow up.

The problem with surrogates

Using surrogate endpoints to measure whether a new drug works can reduce the duration, cost, and complexity of clinical trials before regulatory assessment and facilitate faster patient access to new therapies, especially for chronic diseases.3 For example, in early stage gastric cancer, clinical outcomes such as overall survival—how long patients live after receiving treatment—are of primary interest to patients, but surrogate endpoints such as disease-free survival can potentially provide earlier indications of a drug’s effect.4 In a recent evaluation, using surrogate endpoints in cancer drug trials reduced clinical development time by about 11 months compared with measuring overall survival.3 However, the use of such endpoints can also have negative implications.

Regulatory reliance on surrogate endpoints makes it challenging for health technology assessment bodies such as the National Institute for Health and Care Excellence (NICE) to make decisions. These bodies typically compare clinical and cost effectiveness of treatments. When new drugs are approved based on surrogate endpoints alone, assessing how well they work in terms of patient relevant clinical outcomes, such as health related quality of life and survival, in the short and long term are fraught with considerable uncertainty.

For patients and clinicians, surrogate endpoints can complicate treatment decisions.5 Clinicians and patients may misinterpret drug effects on surrogate endpoints as clinically meaningful improvements.6 This matters, because drugs approved on the basis of surrogate endpoints may not ultimately influence patient relevant outcomes. In cancer, for example, most approved drugs with effects on surrogate endpoints such as response rates and progression-free survival (that were imagined to predict patient relevant benefit) do not, in fact, improve quality of life or prolong survival.789

There is a long history of drugs that were originally approved on the basis of surrogate endpoints and for which later studies failed to show evidence of clinical benefit.10 A commonly cited example is bevacizumab for metastatic breast cancer.11 In 2008, the FDA granted the drug accelerated approval based on its early effects on a surrogate endpoint, progression-free survival. This approval was revoked in 2011 when clinical trials failed to show that patients receiving bevacizumab lived longer than those receiving control treatment.

Other examples include olaratumab, which extended progression-free survival but did not prolong survival for patients with soft tissue sarcoma12; hydroxyprogesterone caproate, which reduced the risk of recurrent preterm births but did not improve neonatal outcomes13; and atezolizumab, which achieved a higher response rate than control treatment but did not extend overall survival in patients with urothelial carcinoma.14 In some cases, drugs approved on the basis of surrogate endpoints were later found to be harmful. For example, patients with multiple myeloma who received venetoclax had shorter survival than those who received a control treatment, despite evidence that the drug improved progression-free survival.15

Regulatory enthusiasm

Over the past three decades, the proportion of clinical studies measuring the efficacy of new drugs using surrogate endpoints alone has increased, rising from fewer than 50% in the mid-90s to roughly 60% in 2015-17.16 In some therapeutic areas such as cancer, surrogate endpoints account for almost 80% of all clinical studies supporting regulatory approvals.17 This means that in some therapeutic areas, only a minority of new drugs are now approved on the basis of evidence that they improve how patients feel or function, or how long they live.

The recent proliferation of surrogate endpoints is partly the result of an increase in the use of “expedited” regulatory pathways that are aimed at speeding up the development, review, and approval of drugs.18 Over the past quarter century, lobbying by drug companies has put pressure on policy makers to establish several expedited pathways in Europe and the United States.19 These pathways also meet perceived patient demand for faster access to potentially effective therapies for conditions with substantial unmet needs.

In the US, the FDA “accelerated approval” pathway was established at the height of the HIV/AIDS crisis in the early 1990s. Other examples in the US include the “breakthrough therapy,” “priority review,” and “fast track” designations. Programmes in Europe include the European Medicines Agency’s “accelerated assessment” and “priority medicines” schemes.20

The use of surrogate endpoints in expedited regulatory pathways may result in “conditional” approvals, in which drug manufacturers are legally mandated to conduct additional trials to prove the clinical benefit of their products. However, clinical efficacy of drugs initially approved on the basis of surrogate endpoints is often subsequently “confirmed” on the basis of other surrogate endpoints.2122 For example, both pre-approval and mandated post-approval studies supporting FDA’s accelerated approval of crizotinib for metastatic non-small cell lung cancer used surrogate endpoints.21 This practice may meet regulators’ expectations but falls far short of reliable evidence of patient benefit.

Limited guidance from regulators and assessment bodies

There is little consensus for defining a “valid” surrogate, as it is difficult to set specific thresholds to grade the strength of association with the final clinical outcome. A few organisations, such as the German Institute for Quality and Efficiency in Healthcare (IQWiG), have prescriptive criteria for accepting surrogate endpoints. IQWiG sets a threshold for the lower bound of the confidence interval on the correlation coefficient (R≥0.85) to conclude a high correlation exists between the surrogate and final clinical outcome.23 But most agencies have no similar cut-offs for accepting surrogate endpoints.

Methodological efforts for evaluating surrogate endpoints have a long history. In 2009, Taylor and Elston recommended a three step framework, based on biological plausibility alone, an observed association between the surrogate and the clinical endpoint at the individual patient level, and evidence from multiple randomised trials showing that drugs improving the effect on the surrogate also improve the final clinical outcome.24 This framework was further extended to quantify the expected treatment effect on the final clinical outcome based on the surrogate.25

However, regulatory agencies rarely use this framework. In 2018, the FDA published a table listing all surrogate endpoints that it has used in its assessments without disclosing any information about their usefulness in predicting clinical benefit.26 Academic researchers are increasingly filling this evidence gap and examining the strength of the association between surrogate endpoints that are commonly used by regulators and patient relevant clinical outcomes.2728 In a recent study, researchers found only weak or missing correlations between surrogate endpoints and survival in breast cancer using the Taylor and Elston framework.29 In another analysis, researchers found that none of the surrogate endpoints used in EMA expedited approvals had been independently evaluated.30

Health technology assessment bodies also rarely use this framework to evaluate surrogate endpoints.31 Indeed, their guidance on the use of surrogate endpoints has been highly variable.32 In a recent survey of methodological guidance by 73 organisations, only 40% specifically considered surrogates.33 Such variation yields heterogenous conclusions about the relevance of the same putative surrogate endpoints across different settings.34

Evaluating surrogate endpoints

Methodologists stress that evidence at the individual patient level is insufficient to evaluate surrogate endpoints, especially when such evidence is obtained from a single trial.35 This is because the observed relationship between a surrogate and a clinical outcome for one drug may not hold for another, as it depends on the treatment’s mechanism of action.35 For example, progression-free survival was previously shown to be a good surrogate for overall survival in advanced colorectal cancer based on evidence from trials of traditional chemotherapy.36 However, the relationship is weaker between these endpoints for modern therapies with different mechanisms of action.37

Meta-analysis, which combines data from several randomised trials, is more appropriate for evaluating the association between the treatment effects on the candidate surrogate endpoint and on the final patient relevant outcome.38 Methodological consensus is growing for using bivariate meta-analysis methods to evaluate these relationships.394041424344 These methods take into account not only the correlation between the treatment effects (quantifying the surrogate relationship), but also uncertainty around this relationship, which is crucial for decision making.4445

Table 1 gives some examples of candidate surrogate endpoints evaluated using meta-analysis methods with authors’ conclusions regarding the strength of the surrogate relationship. It is perhaps unsurprising that bevacizumab’s effect on progression-free survival never translated to prolonged survival for patients with metastatic breast cancer, as an earlier meta-analysis concluded that progression-free survival was not a good surrogate for overall survival in this setting.49

Table 1

Examples of candidate surrogate endpoints evaluated using meta-analysis and authors’ conclusions regarding strength of surrogate relationship with the clinical outcome

View this table:

A potential problem when evaluating surrogate endpoints is the limited amount of randomised trial data in some areas—for example, for drugs targeting genetic biomarkers in small patient populations. In such cases, novel bivariate network meta-analysis methods,55 or hierarchical models,56 allow use of available data on similar drugs or drug classes.4445

Way forward

Regulators should be more selective in their use of surrogate endpoints. Surrogate endpoints should not be used when a drug’s effect on the final clinical outcome can be observed within a relatively short time frame, as in acute conditions.57 Hence, their use should be reserved for chronic diseases when they can provide early and accurate measurement of a drug’s effect, especially when long follow-up is required before patient relevant clinical outcomes can be assessed.58 Even in such cases, regulators can use other tools to ensure patients who have exhausted all available treatment options can receive investigational treatments before regulatory approval.59 Such “expanded access” programmes can bridge the access gap while evidence on patient relevant endpoints accrues.

When using surrogate endpoints is justified, regulators should consider the strength of available evidence on how well surrogates predict clinical benefit. The recent US accelerated approval of aducanumab for the treatment of Alzheimer’s disease shows why this is essential. The FDA’s decision was controversial partly because an earlier meta-analysis of randomised controlled trials showed that changes in amyloid level had little to no effect on cognition.1 Thus, it is debatable whether a reduction in amyloid levels is an acceptable surrogate for cognition. This has also been reflected in a report released by the Institute for Clinical and Economic Review, an independent health technology assessment body in the US.60

In the absence of regulatory guidance, there are promising signs that assessment bodies are increasingly raising the bar for using surrogate endpoints. For example, NICE has recently proposed changes to strengthen the evidence requirements for the use of surrogate endpoints, while still allowing flexibility when desired evidence is not available.6162 Involving assessment bodies in early regulatory interactions with manufacturers may help align evidence requirements on surrogate endpoints. The UK Innovative Licensing and Access Pathway managed by the Medicines and Healthcare Products Regulatory Agency, NICE, and the Scottish Medicines Consortium is aimed at facilitating such alignment.63

Ultimately, regulatory and health technology assessment bodies need to weigh the strength of available evidence on the validity of surrogates alongside other considerations such as unmet therapeutic need. When making such trade-offs, quantifying how well a candidate surrogate predicts the final clinical outcome can provide valuable information.4455 If recommended meta-analysis methods are used, the strength (or weakness) of the surrogate will be reflected in the uncertainty around the predicted treatment effect on the final outcome. A weaker surrogate will yield a larger interval and hence greater uncertainty.

Raising the bar for using surrogate endpoints may increase the cost and duration of drug development. However, this need not hamper pharmaceutical innovation. In the past, regulatory guidance encouraging manufacturers to evaluate the cardiovascular outcomes of diabetes treatments incentivised the generation of patient centred evidence without adversely affecting research and development.6465

Greater involvement of patients (and organisations representing patients) in regulatory and health technology assessment processes is also essential to ensure that the conditions for accepting surrogate endpoints for decision making are adequately met. When using such endpoints is justified, patients can help ensure that uncertainty related to surrogates is explicitly presented and taken into account. Patient input can also help guide decisions regarding the appropriate use of surrogate endpoints.

Key messages

• Surrogate endpoints are widely used by regulators to expedite the approval of new drugs, but most are not reliable predictors of outcomes that matter most to patients

• Regulators should only accept surrogate endpoints when generating data on clinical outcomes is not attainable

• When measuring clinical outcomes would require very long trials, the appropriateness of surrogate endpoints should be evaluated using meta-analysis

Footnotes

  • Contributors and sources: DD is an expert on health technology assessment methods research and has been involved in the ongoing update of NICE’s health technology evaluation methods. HN’s research examines the evidence supporting regulatory decisions on drugs in the US and Europe. OC has written extensively on the role of surrogate endpoints in healthcare policy and cost-effectiveness models. She previously contributed to the development of surrogate validation frameworks. SB has developed novel methods for modelling surrogate endpoints, which are proposed to be included in NICE’s update of its methods guide. HN devised the idea for this article. All authors contributed to developing the first draft and writing of subsequent versions. DD is the guarantor.

  • Competing interests: DD is an employee of NICE, funded by grants from the European Union’s IMI and Horizon 2020 research and innovation programme. SB has served as a paid consultant providing methodological advice to NICE, Roche and RTI Health Solutions. SB was supported by the Medical Research Council through the methodology research panel grants and has previously received research funding from European Federation of Pharmaceutical Industries & Associations (EFPIA) as part of unrelated European Union IMI GetReal project. HN previously received funding from the Pharmaceutical Group of the European Union for an unrelated systematic review on community pharmacists. HN currently receives funding from the Health Foundation on an unrelated project on pharmaceutical policy. OC received funding from the European Union’s Horizon 2020 research and innovation programme. The views expressed in this article are those of the authors and do not necessarily reflect those of their employers or funders.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

References