Editorials

Interpreting composite outcomes in trials

BMJ 2010; 341 doi: http://dx.doi.org/10.1136/bmj.c3529 (Published 18 August 2010) Cite this as: BMJ 2010;341:c3529
  1. Nick Freemantle, professor of clinical epidemiology and biostatistics,
  2. Melanie J Calvert, senior lecturer
  1. 1School of Health and Population Sciences, University of Birmingham, Birmingham B15 2SP
  1. n.freemantle{at}bham.ac.uk

    Difficulty lies in poor reporting, not the measure itself

    The potential problems with the use of composite outcomes in clinical trials are well known.1 The linked systematic review by Cordoba and colleagues (doi:10.1136/bmj.c3920) indicates that many of these challenges are long lasting.2 This review of 40 trials published in 2008 that reported a composite primary outcome found little justification for the choice of outcome, imbalance in the importance of different components in 70% of trials, and problems in the definition and reporting of composite outcomes in 40% of trials. Before we abandon composite outcomes because of the authors’ assertion that their use leads to much confusion and bias,2 we should consider carefully the opportunity cost and tackle some commonly held misconceptions on composites.

    A key advantage of using composite outcomes is that they can increase statistical efficiency and enable us to answer questions that could not otherwise be tackled. Increased efficiency means that trials may be smaller or of shorter duration, allowing effective treatments to be available to patients in a timely manner.1

    Cordoba and colleagues also question the rationale for the choice of components of the composite.2 Consistency in the use of composites for a particular disease undoubtedly aids interpretation. Thus, in the development of new treatments in heart failure and subsequently in cardiovascular disease more generally, the composite outcome of unplanned hospital admission for cardiovascular reasons and all cause mortality has become the standard primary outcome for regulatory trials. Consensus on appropriate composites could be adopted in other clinical areas.

    The concern that composite outcomes can lead to the incorrect perception that end points such as mortality are reduced by treatments that only show significant benefits on other less important components is also not new,1 and the Food and Drug Administration (FDA) in the United States seems to have dealt with this. The ATHENA trial, which recently examined the effects of dronedarone in elderly people with atrial fibrillation and additional risk factors for death, adopted the primary outcome of all cause mortality and unplanned hospital admission for cardiovascular disease.3 Although dronedarone had a strong effect on the composite and on the hospital admission component, its effect on all cause mortality alone was not significant. The FDA duly excluded mortality from the indication (which governs marketing activity for the product), stating that dronedarone had been shown to “reduce the risk of cardiovascular hospitalisations.”4 Furthermore, although a secondary outcome of death from cardiovascular disease was significant, this was not included in the US indication, presumably because the two components of the composite primary outcome were considered first in the alpha spending approach, which preserves the study wide type 1 error of 2.5% for benefit.5 In other words, the trial was deemed to have used up all its available statistical information to answer questions before reaching the apparently nominally significant outcome of death from cardiovascular disease.

    So if the regulators, at least in the US, have raised their game, why then are journals still doing such a poor job? Some may feel that Cordoba and colleagues are making too much of the data they present: 87% of primary composite outcomes they assessed were presented consistently in papers, although the wording was ambiguous in 20%; 78% provided data for the components of the composites.2

    Cherry picking in outcome measures is not restricted to the components of composite outcomes6 and is always wrong. Journals should examine the original trial protocols and ensure that outcomes are reported fully, accurately, and honestly. They should also ensure that data on each component of a composite are reported, even when a composite has many components (such a table could be included as additional web based material if too large to be included in the standard publication). This not only helps interpretation but ensures that data from trials with composite outcomes can be included in meta-analyses.

    Cordoba and colleagues advance the idea that all components of a composite outcome should be similarly important.2 Although superficially attractive, this premise can be challenged. Packer has argued that mortality is not the only adverse clinical event worth considering in a clinical trial, and it is an incomplete measure of disease progression. He makes the point that many patients with heart failure deteriorate symptomatically, are repeatedly admitted to hospital for worsening symptoms, and require intensive therapeutic interventions yet they are still alive at the end of the study. If the primary outcome measure is all cause mortality, “such patients would be considered (inappropriately) to have fared as well as patients who remained clinically stable and minimally symptomatic.”7

    It is a common misconception that the primary outcome in a clinical trial should be the most important outcome. Instead, the primary outcome “should be the variable capable of providing the most clinically relevant and convincing evidence directly related to the primary objective of the trial.”8 For example, it would be wrong to have all cause mortality as the primary outcome of a trial that would have insufficient statistical power to provide a reliable answer on that outcome. Thus the end point must be important, but not necessarily the most important, and must take into account estimation issues such as the number of available patients and predicted event rates; too few patients or events may preclude the examination of some rarer end points in a trial.8 In trials of heart failure, investigators have increasingly included only important hospital admissions (such as those that are unplanned, involve an overnight stay, and are for cardiovascular reasons) to avoid including relatively unimportant events.7

    Composite outcomes attract surprisingly violent reactions from some authors.9 10 11 12 But we do not argue that randomised trials should be abandoned because investigators sometimes cherry pick the results.6 We should proscribe cherry picking rather than composites. As Cordoba and colleagues have pointed out, composite outcomes are sometimes poorly reported in journals. We have the collective opportunity to correct this limitation and raise our game in the same way that the FDA has done. If we are to benefit from the potential advantages of composite outcomes, we must not be frightened by the hard work that this opportunity affords us.

    Notes

    Cite this as: BMJ 2010;341:c3529

    Footnotes

    • Research, doi:10.1136/bmj.c3920
    • Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that: (1) They have received no support for the submitted work; (2) NF has received support for consultancy and travel from Sanofi Aventis, the manufacturer of dronedarone, in the previous three years. MJC has no financial conflicts of interest; (3) Their spouses, partners, or children have no financial relationships that may be relevant to the submitted work; and (4) They have no non-financial interests that may be relevant to the submitted work.

    • Provenance and peer review: Commissioned; not externally peer reviewed.

    References