- Keith A A Fox, British Heart Foundation professor of cardiology1,
- Omar H Dabbous, statistician2,
- Robert J Goldberg, epidemiologist2,
- Karen S Pieper, statistician3,
- Kim A Eagle, cardiologist4,
- Frans Van de Werf, cardiologist5,
- Álvaro Avezum, cardiologist6,
- Shaun G Goodman, cardiologist7,
- Marcus D Flather, cardiologist8,
- Frederick A Anderson Jr, research professor of surgery2,
- Christopher B Granger, cardiologist3
- 1Royal Infirmary of Edinburgh, University of Edinburgh, Edinburgh EH16 4SB
- 2University of Massachusetts Medical School, Worcester, MA 01655, USA
- 3Duke Clinical Research Institute, Durham, NC 27705, USA
- 4University of Michigan Cardiovascular Center, Ann Arbor, MI 48109-0477, USA
- 5Universitair Ziekenhuis Gasthuisberg, Leuven, Belgium 3000
- 6Dante Pazzanese Institute of Cardiology, 04012-909 Sao Paulo, Brazil
- 7Terrence Donnelly Heart Centre, Division of Cardiology, St Michael's Hospital, Toronto, ON, Canada M5B 1W8
- 8Royal Brompton and Harefield NHS Trust, Royal Brompton Hospital, London SW3 6NP
- Correspondence to: K A A Fox
- Accepted 12 September 2006
Objective To develop a clinical risk prediction tool for estimating the cumulative six month risk of death and death or myocardial infarction to facilitate triage and management of patients with acute coronary syndrome.
Design Prospective multinational observational study in which we used multivariable regression to develop a final predictive model, with prospective and external validation.
Setting Ninety four hospitals in 14 countries in Europe, North and South America, Australia, and New Zealand.
Population 43 810 patients (21 688 in derivation set; 22 122 in validation set) presenting with acute coronary syndrome with or without ST segment elevation enrolled in the global registry of acute coronary events (GRACE) study between April 1999 and September 2005.
Main outcome measures Death and myocardial infarction.
Results 1989 patients died in hospital, 1466 died between discharge and six month follow-up, and 2793 sustained a new non-fatal myocardial infarction. Nine factors independently predicted death and the combined end point of death or myocardial infarction in the period from admission to six months after discharge: age, development (or history) of heart failure, peripheral vascular disease, systolic blood pressure, Killip class, initial serum creatinine concentration, elevated initial cardiac markers, cardiac arrest on admission, and ST segment deviation. The simplified model was robust, with prospectively validated C-statistics of 0.81 for predicting death and 0.73 for death or myocardial infarction from admission to six months after discharge. The external applicability of the model was validated in the dataset from GUSTO IIb (global use of strategies to open occluded coronary arteries).
Conclusions This risk prediction tool uses readily identifiable variables to provide robust prediction of the cumulative six month risk of death or myocardial infarction. It is a rapid and widely applicable method for assessing cardiovascular risk to complement clinical assessment and can guide patient triage and management across the spectrum of patients with acute coronary syndrome.
Although patients with acute coronary syndrome share key pathophysiological mechanisms, they present with diverse clinical, electrocardiographic, and enzyme or marker characteristics and experience a wide range of serious cardiovascular outcomes.1 2 Estimated risk, based on clinical characteristics, is challenging and imprecise, yet risk assessment is needed to guide triage and key management decisions. Regulatory authorities such as the National Institute for Health and Clinical Excellence (NICE) and guideline groups recommend treatments according to specific clinical and risk groupings, and trials show that certain benefits may be predominantly or exclusively restricted to higher risk patients with coronary syndrome.2 3 4 Binary methods of stratifying risk (for example, normal or raised troponin concentration or abnormal or normal findings on electrocardiography) lack sufficient precision.5 6 7 8 9 10 11 To provide more accurate prognostic information, and to target treatment more appropriately, more precise yet user friendly risk stratification is required. To ensure general applicability, risk stratification methods should be derived from unrestricted populations that are representative of patients with acute coronary syndrome in the real world12 and should use widely available clinical variables.
The large multinational observational global registry of acute coronary events (GRACE) has been used to derive regression models to predict death in hospital13 and death after discharge14 in patients with acute coronary syndrome. However, a comprehensive risk model is required to predict the cumulative risk of death and death or myocardial infarction during the high risk first six months after initial presentation with acute coronary syndrome, the period when most complications occur.15 16 Because triage and management decisions are required within the first hours or days after initial presentation we derived a risk tool from characteristics of patients with acute coronary syndrome at initial presentation.
GRACE methods and design
Full details of the GRACE rationale and methods have been published elsewhere.17 18 The registry was designed to reflect an unbiased population of patients with acute coronary syndrome in 94 hospitals in 14 countries. All cases were assigned to one of the following categories: ST segment elevation myocardial infarction, non-ST elevation myocardial infarction, or unstable angina (see appendix on bmj.com for inclusion criteria and standard definitions). Trained coordinators collected data using standardised case report forms.
We used two primary end points: all cause death or the composite measure of death or non-fatal myocardial infarction during admission to hospital or after discharge (presentation to six months).
We have summarised the distributions of continuous variables with medians and 25th and 75th centiles and reported the categorical variables as frequencies and percentages. Events that occurred after six months were censored. Table 1⇓shows the variables included in the analysis from hospital admission to six month follow-up. We used a Cox regression model to compute crude hazard ratios and 95% confidence intervals to examine the individual relation between each predictor and death and death or myocardial infarction during follow-up (0 to 6 months).
We entered all demographic and clinical variables identified by the crude regression analysis into the stepwise multiple Cox regression (backward) analysis to produce final models for predicting death and death or myocardial infarction. Only those variables associated with an α≤0.05 were retained; all variables in the final model met the assumptions for proportional hazards. No imputation was performed in these final models. Imputation was tested but did not influence the identification of multivariable predictors or the discriminative power of the model for predicting death.13 The discriminative power of the final models was assessed by the mean of the area under the receiver operating characteristic (ROC) curve (C-statistic). The curve is a measure of the discriminating ability of the risk model and is a plot of sensitivity versus 1−specificity. Accuracy of calibration was evaluated by plotting the predicted versus the observed mortality according to population tenths of predicted risk. The model was tested prospectively in a separate dataset in GRACE (n=22 122) and also in an independent external dataset, the GUSTO IIb (global use of strategies to open occluded coronary arteries IIb) dataset,19 comprising the entire spectrum of patients with acute coronary syndrome (12 142 patients, 4131 with ST elevation myocardial infarction, 8011 with non-ST elevation myocardial infarction). The analysis was performed with SAS software package (version 8.2, SAS Institute, Cary, NC) and S-Plus (MathSort, Seattle, WA).
The derivation population comprised 26 267 patients with suspected acute coronary syndrome enrolled between 1 April 1999 and 30 September 2002. We excluded patients found to have a non-cardiac or non-acute coronary cardiac diagnosis (fig 1).⇓ We also excluded patients transferred into a study hospital because they lacked some baseline information and their inclusion may also have led to bias because of morbidity associated with the indications for transfer. The study population therefore comprised 21 688 patients of whom 19 931 were alive at six month follow-up.
A total of 1757 (9.1%) deaths occurred, 1046/21 573 in hospital (4.9% among patients with a diagnosis of acute coronary syndrome on admission) and 711/15 265 during the period after discharge (4.7%). We had no information on mortality (in hospital or after discharge) for 51 patients. In the derivation set, 3110 (15.8%) patients died (n=1757) or experienced a non-fatal myocardial infarction (n=1353) between presentation and six month follow-up.
Early risks were highest for patients with ST segment elevation myocardial infarction but by six months the risk of death was similar to those with non-ST segment elevation myocardial infarction (fig 2)⇓. Of those who survived to six months after discharge, 36.2% (258/711) presented with ST segment elevation myocardial infarction compared with 50.0% (880/1757) of those who died during admission or follow-up. Raised cardiac markers were detected in 35.0% (6883/19688) of those who survived compared with 53.2% (905/1701) of those who died.
The validation set comprised 22 122 patients enrolled in this multinational registry between 1 October 2003 and 30 September 2005. A total of 1730 (9.0%) patients died between hospital admission and six month follow-up, 948 in hospital (4.3% among patients with an admission diagnosis of acute coronary syndrome) and 782 (5.4%) after discharge. No information on mortality was available for 38 patients. In total, 2720 patients died (n=1730) or experienced a non-fatal myocardial infarction (n=990) between presentation and six month follow-up.
Predictors of mortality
From admission to six month follow-up, Killip class20 and advanced age were the most powerful predictors of death in the univariable analysis (table 1).⇑ Table 1 also shows the other baseline characteristics and clinical parameters that predicted death or death or myocardial infarction.
After multivariable analysis, the highest hazard ratios for death were cardiac arrest on admission and increasing age. These two key prognostic factors were closely followed by raised cardiac markers or enzyme activity and ST segment deviation (table 2)⇓.
Risk models predicting death and death or myocardial infarction
The risk model comprises 14 predictors of death and 12 predictors of death or myocardial infarction. The predictive accuracy of the model was good, with C-statistics of 0.82 for death in hospital and 0.70 for death or myocardial infarction in hospital (table 3)⇓. Nine factors independently predicted death and the combined end point in the period from admission to six months after discharge: age, congestive heart failure, peripheral vascular disease, systolic blood pressure, Killip class, initial serum creatinine concentration, positive initial cardiac markers, cardiac arrest on admission, and number of leads with ST deviation. The highest hazard ratio for adverse outcome was for cardiac arrest (tables 1 and 2).⇑ ⇑
Prospective and external validation of the GRACE risk score
When we tested the risk model in the prospective validation set, it had excellent predictive accuracy for death (C-statistic=0.81, simplified model) and death or myocardial infarction (C-statistic=0.73).The predictive accuracy was maintained across the acute coronary syndrome subgroups (table 3).⇑
We validated the model externally using the GUSTO IIb dataset of 12 142 patients with acute coronary syndrome. There was excellent discrimination despite the fact that one of the key parameters was not recorded in GUSTO IIb (cardiac arrest). The C-statistic for the death model in all patients was 0.82 (C-statistics=0.80 for ST segment elevation myocardial infarction and 0.76 for non-ST segment elevation myocardial infarction).
Development of a simplified nomogram for clinical application
We reduced the overall models to include the most important variables that contained most (>90%) of the predictive information. This nomogram retained excellent discriminant characteristics based on eight variables and was used for the calculation of risk (fig 3)⇓.
The GRACE risk prediction tool (simplified nomogram) includes variables that are readily available to clinicians even in smaller community hospitals. It provides a novel and widely applicable method of assessing the cumulative six month risk of death and death or myocardial infarction across the spectrum of patients admitted to hospital with acute coronary syndrome. Accurate longer term assessment of risk is important because most cardiac ischaemic events occur within the first few weeks after initial presentation with acute coronary syndrome.15 16 Our findings, based on 48 389 patients, support the validity of the GRACE models for mortality in hospital and after discharge,14 which were derived from data from about 11 000 and 15 000 patients, respectively.
The need for risk prediction in patients with acute coronary syndrome
In clinical practice, initial stratification of patients aims to identify those suitable for reperfusion therapy (on the basis of a clinical syndrome and ST segment elevation or other electrocardiographic markers of acute infarction). Binary approaches are commonly applied among others with acute coronary syndrome, but separating patients based on one or two characteristics may substantially overestimate or underestimate the risk of death or myocardial infarction. There is therefore a need for one predictive instrument that performs well in all patients with acute coronary syndrome.
Robust evidence and practice guidelines (including NICE) suggest that interventional and pharmacological therapies predominantly benefit patients at higher risk.2 3 21 Despite the availability of such guidelines, identification of patients at high risk of cardiac ischaemic events remains challenging.22 23 In addition, the triage of patients into high intensity care units (cardiac care units) is based predominantly on the criteria for reperfusion therapy rather than risk in the patient. For example, a 55 year old woman (blood pressure 142/80 mm Hg; heart rate 88 per minute) who presents with ST elevation and raised troponin concentration but without complications of a myocardial infarction (normal creatinine concentration, no heart failure) has a probability of death of only 3% in the next six months. However, a 55 year old woman with non-ST segment elevation myocardial infarction (blood pressure 118/68; heart rate 92 per min) with mild heart failure and raised creatinine concentration has a six month risk of death of 16%. Without formal risk stratification, the second patient would probably be managed in a low intensity ward area and the management on discharge may not reflect the risk in the patient.
Resolving intermediate risk
Despite similarities in key pathophysiological mechanisms, the characteristics on presentation of patients with acute coronary syndrome depend on the extent of the ischaemic territory (influenced by acute thrombotic risk) and previous risk features (such as older age, heart failure, and renal insufficiency). Whereas patients with high risk features, including cardiogenic shock and heart failure, are relatively straightforward to identify, most patients lie in the intermediate range and risk is less obvious (table 4)⇓. This intermediate range encompasses up to 10-fold differences in the risk of death. Binary approaches, including those that require separation of patients into high or low risk, are not accurate enough for most patients in the middle range.2 3 We propose that an appropriate instrument for risk prediction needs to be applicable across the spectrum of acute coronary syndrome, should be derived from a representative and broadly based population, and needs to use variables that are readily available to most clinicians shortly after the patient arrives at hospital.
How does the present model differ from previous methods of risk stratification?
Several other multivariable prognostic models have been developed,5 6 7 8 9 10 24 25 26 27 28 most of which were derived from clinical trial databases or specific subgroups of patients with acute coronary syndrome. Patients with complications and comorbidity tend to be excluded from such trials, thus limiting applicability in clinical practice. Models developed from large claims databases are potentially subject to bias.8 11 In contrast, the GRACE registry spans the spectrum of acute coronary syndrome and is based on an unselected contemporary population.
A C-statistic of less than 0.70 has been suggested to be of limited clinical value.12 The TIMI (thrombolysis in myocardial infarction) model performs well in patients who are eligible for reperfusion therapy but is less effective in more general patients, including those who are ineligible for reperfusion (C-statistic=0.65).24 An independent study suggests that the unselected GRACE mortality model is superior to either the TIMI or the PURSUIT (platelet glycoprotein IIb/IIIa in unstable angina: receptor suppression with eptifibatide) models.29 We have shown that the cumulative (0 to six month) GRACE risk model performs well across the spectrum of acute coronary syndrome and has prospective and external validity. External validation with the GUSTO IIb dataset confirms the discriminant characteristics of the model when applied to patients with ST segment elevation myocardial infarction and those with non-ST segment elevation myocardial infarction. Although we excluded transferred patients from the derivation of this model (because such patients may lack data for several baseline characteristics), testing the model in the transfer dataset confirmed its applicability to such patients (C-statistic=0.83 for predicting death and 0.70 for predicting myocardial infarction, simplified model).
Simplified risk calculation for clinical application
The simplified model includes most the predictive information: >92% of the total model χ2 for death and >90% for death or myocardial infarction (fig 3).⇑ The GRACE risk calculator (fig 3) (available at www.outcomes.org/grace) can be used to derive a prognostic score and to estimate the risk of clinically important end points—death or the combined risk of death or myocardial infarction—in individual patients. For ease of use, this nomogram can be installed into a handheld device or personal computer (data entry takes about 30 seconds) and is also available as a score card.14
GRACE is designed to enrol an unselected and generalisable population of patients, though some participating centres are required to obtain informed consent from patients before enrolment. Therefore some patients who died early or who experienced major clinical complications immediately on arrival in hospital may be under-represented. The model may not be appropriate for stratifying low risk patients with non-specific chest pain without acute coronary syndrome, but such patients do not require the same therapeutic and management decisions as those with acute coronary syndrome.
What is already known on this topic
Specific treatments are indicated in higher or lower risk patients with acute coronary syndrome
Conventional clinical assessment and binary methods for predicting risk based on results of electrocardiography and markers of injury are not sufficiently accurate
Previous risk models were based on subgroups of patients with acute coronary syndrome and were derived from large clinical trials or healthcare claims databases
What this study adds
The GRACE risk tool can be used to predict the cumulative risk of death and death or myocardial infarction in the period from admission to hospital to six months after discharge
The tool is simple to apply, robust, externally validated, and applicable to patients across the complete spectrum of acute coronary syndrome
Full details of inclusion criteria and standard definitions can be found on bmj.com.
We thank the physicians and nurses who participated in GRACE. The risk calculator is available together with further information about the project and the complete list of participants from www.outcomes.org/grace. We thank Sophie Rushton-Smith for editorial services.
Contributors: KAAF, RJG, KAE, FVdeW, ÁA, SGG, FAA, and CBG were responsible for study concept and design. KAAF, KAE, FVdeW, ÁA, SGG, and CBG acquired the data. KAAF drafted the manuscript and is guarantor. All authors critically revised the manuscript for important intellectual content and approved the final version. OHD and KSP carried out statistical analyses.
Funding: The GRACE Registry is supported by an unrestricted educational grant from Sanofi-Aventis to the Center for Outcomes Research, University of Massachusetts Medical School. Sophie Rushton-Smith was funded by Sanofi-Aventis.
Competing interests: KAAF has received grant funding from the British Heart Foundation and his department is supported by the British Heart Foundation, Medical Research Council, Wellcome Trust, Sanofi-Aventis, Bristol-Myers Squibb, and MSD. KAE has received grants from Biosite, Bristol-Myers Squibb, Cardiac Sciences, Blue Cross Blue Shield of Michigan, Hewlett Foundation, Mardigian Fund, Sanofi-Aventis, Varbedian Fund, National Heart, Lung and Blood NIH, and Pfizer. FVdeW has received research grants from Boehringer Ingelheim, Sanofi-Aventis, Proctor and Gamble, Servier, Novartis, MSD, and Schering Plough. ÁA has received funding from Sanofi-Aventis, Population Health Research Institute, and Boehringer Ingelheim. SGG has received funding from AstraZeneca, Sanofi-Aventis, Boehringer Ingelheim, Bristol-Myers Squibb/Sanofi Pharmaceuticals Partnership, Hoffmann-LaRoche Pharmaceuticals, Merck, Novartis, Pfizer, Sanofi-Synthelabo, Schering Corp, and Millennium Pharmaceuticals. MDF, FAA, CBG, and BK have all received funding from Sanofi-Aventis.
Ethical approval: Approval was obtained from local institutional review boards.