Predicting early death in patients with traumatic bleeding: development and validation of prognostic model
BMJ 2012; 345 doi: https://doi.org/10.1136/bmj.e5166 (Published 15 August 2012) Cite this as: BMJ 2012;345:e5166
- Pablo Perel, senior clinical lecturer1,
- David Prieto-Merino, lecturer, medical statistics2,
- Haleema Shakur, senior lecturer1,
- Tim Clayton, senior lecturer, medical2,
- Fiona Lecky, clinical professor3, honorary professor4, honorary consultant5,
- Omar Bouamra, medical statistician6,
- Rob Russell, senior lecturer7,
- Mark Faulkner, paramedic advisor8,
- Ewout W Steyerberg, professor9,
- Ian Roberts, professor1
- 1Clinical Trials Unit, London School of Hygiene and Tropical Medicine, London WC1E 7HT, UK
- 2Medical Statistics Department, Epidemiology and Population Health Faculty, London School of Hygiene and Tropical Medicine
- 3University of Sheffield, Sheffield, UK
- 4University of Manchester, Manchester, UK
- 5Salford Royal Hospitals NHS Foundation Trust Trauma Audit and Research Network, Salford, UK
- 6Trauma Audit and Research Network, Health Science Research Group, School of Community Based Medicine, University of Manchester, Salford Royal Hospital, Salford, UK
- 7UK Royal Centre for Defence Medicine, Birmingham, UK
- 8London Ambulance Service and London Trauma Office
- 9Department of Public Health, Center for Medical Decision Making, Erasmus MC, Rotterdam, Netherlands
- Correspondence to: P Perel London School of Hygiene & Tropical Medicine, Keppel Street, pablo.perel{at}lshtm.ac.uk
- Accepted 11 July 2012
Abstract
Objective To develop and validate a prognostic model for early death in patients with traumatic bleeding.
Design Multivariable logistic regression of a large international cohort of trauma patients.
Setting 274 hospitals in 40 high, medium, and low income countries
Participants Prognostic model development: 20 127 trauma patients with, or at risk of, significant bleeding, within 8 hours of injury in the Clinical Randomisation of an Antifibrinolytic in Significant Haemorrhage (CRASH-2) trial. External validation: 14 220 selected trauma patients from the Trauma Audit and Research Network (TARN), which included mainly patients from the UK.
Outcomes In-hospital death within 4 weeks of injury.
Results 3076 (15%) patients died in the CRASH-2 trial and 1765 (12%) in the TARN dataset. Glasgow coma score, age, and systolic blood pressure were the strongest predictors of mortality. Other predictors included in the final model were geographical region (low, middle, or high income country), heart rate, time since injury, and type of injury. Discrimination and calibration were satisfactory, with C statistics above 0.80 in both CRASH-2 and TARN. A simple chart was constructed to readily provide the probability of death at the point of care, and a web based calculator is available for a more detailed risk assessment (http://crash2.lshtm.ac.uk).
Conclusions This prognostic model can be used to obtain valid predictions of mortality in patients with traumatic bleeding, assisting in triage and potentially shortening the time to diagnostic and lifesaving procedures (such as imaging, surgery, and tranexamic acid). Age is an important prognostic factor, and this is of particular relevance in high income countries with an aging trauma population.
Introduction
Each year around 4 million people die worldwide from unintentional injury and violence, and tens of millions are left permanently disabled. Most of the victims are from low and middle income countries.1 Although many of these deaths occur at the scene of the injury, 44% are estimated to occur after admission to hospital.2
Severe bleeding accounts for about one third of in-hospital deaths due to trauma and is an important contributory factor for other causes of death, particularly head injury and multi-organ failure.3 Failure to start appropriate early management in bleeding trauma patients is a leading cause of preventable death from trauma.4 Triage criteria that allow the rapid identification of patients at high risk have the potential to reduce mortality from trauma. Recent evidence that the early administration of tranexamic acid substantially reduces mortality in bleeding trauma patients further underscores the clinical importance of the timely identification of life threatening bleeding.5 However, any such early prediction would have to be based on variables that can be readily measured soon after injury. Several clinical variables related to the physiological response to reduced intravascular volume predict the risk of death in bleeding trauma patients. These include blood pressure, capillary refill time, level of consciousness (Glasgow coma score), heart rate, and respiratory rate.6 Because all of these variables are of limited predictive value when considered in isolation, prognostic models that combine variables are needed for better predictive accuracy.7 8 9 An accurate and user friendly prognostic model to predict mortality in patients with traumatic bleeding could assist doctors and paramedics in pre-hospital triage, whether in civilian or battlefield settings; its use could shorten the time to diagnostic and lifesaving procedures (such as surgery and tranexamic acid). We have previously published a prognostic model for patients with traumatic brain injury, which was accurate, user friendly, and clinically useful for supporting physicians’ decision making.10 11
Existing prognostic models for bleeding trauma patients are limited.12 Most were developed using data collected many decades ago and have methodological limitations. Models based on contemporary data are needed, as treatment practices have changed and the age of trauma patients has increased in high income countries. Furthermore, although most deaths due to trauma occur in low and middle income countries, most prognostic models are based on data from high income countries.12 We aimed to develop a simple prognostic model that could be used at the point of care to estimate risk of death in patients with traumatic bleeding.
Methods
Model development
For the development of the prognostic model, we involved potential users from three settings: pre-hospital, battlefield, and emergency departments. We held meetings with paramedics, military doctors, and consultants in emergency medicine to identify variables and interactions that they considered important and convenient for their settings and to obtain information on how to present the prognostic model in a user friendly format.
We included patients from the Clinical Randomisation of an Antifibrinolytic in Significant Haemorrhage (CRASH-2) trial.13 The trial included 20 127 trauma patients with, or at risk of, significant bleeding, within eight hours of injury, and took place in 274 hospitals in 40 countries. The primary outcome was all cause mortality. Patients’ outcomes were recorded at discharge, at death in hospital, or 28 days after injury, whichever occurred first.
Predictors
We took variables to be analysed as potential predictors from the patients’ entry forms completed before randomisation. Variables included in the entry form for the CRASH-2 trial can be divided into patients’ demographic characteristics (age and sex), characteristics of the injury (type of injury and time from injury to randomisation), and physiological variables (Glasgow coma score, systolic blood pressure, heart rate, respiratory rate, and central capillary refill time).
Age was recorded as a continuous variable measured in years. Type of injury had three categories—penetrating, blunt, or blunt and penetrating—but we analysed it as “penetrating” or “blunt and penetrating.” Time from injury was recorded as a continuous variable measured in hours. The five physiological variables were recorded according to usual clinical practice. For each of these variables, the value given on the entry form was the first measurement taken at hospital admission.
Multivariable analysis
We did complete case analysis, as the amount of missing data was very low in CRASH-2. We initially included all candidate predictors in the multivariable logistic regression. We adjusted analyses for treatment by including treatment allocation as a covariate in the models. We also included a variable for economic region (that is, low, middle, or high income country, as defined by the World Bank).14 We used logistic regression models with random intercepts by country. We initially analysed continuous variables as linear terms. We assessed departure from linearity graphically and by adding quadratic and cubic terms into the model. We specifically explored interactions by age and by type of injury. We dichotomised time since injury into less than or more than three hours, as the effect of this variable was reasonably well captured by treating it as binary.
We used a backward stepwise approach. Firstly, we included all potential prognostic factors and interaction terms that users considered plausible. These interactions included all potential predictors with type of injury, time since injury, and age. We then removed, one at a time, terms for which we found no strong evidence of an association, judged according to the P values (<0.05) from the Wald test. Each time, we calculated a log likelihood ratio test to check that the term removed did not have a big effect in the model. Eventually, we reached a model in which all terms were statistically significant. We used the R software environment (version 2.13.1; R Foundation for Statistical Computing, Vienna, Austria).
Performance
We assessed the predictive ability of the prognostic model in terms of calibration and discrimination. Calibration indicates whether observed risks agree with predicted risks; we assessed this graphically by plotting the observed outcomes versus the predicted probabilities of the outcomes. Discrimination indicates whether patients at low risk can be separated from those at high risk; we assessed this by using a concordance (C) statistic.15 We assessed optimism in the performance by bootstrap re-sampling. We drew 200 samples with replacement from the original data, with the same size as the original derivation data. In each bootstrap sample, we repeated the entire modelling process, including variable selection. We averaged the C statistics of those 200 models in the bootstrap samples. We then estimated the average C statistic when each of the 200 models was applied in the original sample. The difference between the two average C statistics indicated the “optimism” of the C statistic in our prognostic model.15
External validation
For the external validation, we used the data from the Trauma Audit and Research Network (TARN). Membership is voluntary and includes 60% of hospitals receiving trauma patients in England and Wales and some hospitals in Europe. Data are collected on patients who arrive at hospital alive and meet any of the following criteria: death from injury at any point during admission, stay in hospital of longer than three days, need for intensive or high dependency care, need for inter-hospital transfer for specialist care.
We excluded patients with isolated closed limb injuries and those over 65 years old with isolated fractured neck of femur or pubic ramus fracture. The physiological data available in TARN are identical to those in CRASH-2, in that for every patient the heart rate, systolic blood pressure, Glasgow coma score, respiratory rate, and capillary refill time on arrival are entered by the hospital data coordinators. For each patient, the volume of blood loss is estimated. This is done by allocating an estimated percentage of total volume of blood lost to each injury code in the abbreviated injury scale dictionary by blinded, then consensus, agreement from two emergency physicians. This estimation is based on previous work on blood loss in specific injuries.16
We selected adult (age over 15 years at the time of injury) patients presenting between 2000 and 2008 to hospitals participating in TARN. The definition of significant haemorrhage used in the CRASH-2 trial was not available, so we selected only patients with an estimated blood loss of at least 20%, whom we considered would be clinically comparable to the CRASH-2 patients.
For the validation in the TARN dataset, we did multiple imputations to substitute the missing values of the predictors included in the prognostic model by using the procedure of imputation by chained equations in Stata Release 11. We applied the coefficients of the model developed in CRASH-2 with the estimated UK intercept to the five imputed datasets of TARN, obtaining five predictions of mortality for each patient in TARN. We then averaged over these five predictions to calculate calibration and discrimination.15
Simple prognostic model
For ease of use at the point of care, we developed a simple prognostic model. For this model, we included the strongest predictors with the same quadratic and cubic terms as used in the full model, adjusting for tranexamic acid.
We presented the prognostic model as a chart that cross tabulates these predictors with each of them recoded in several categories. We made the categories by considering clinical and statistical criteria. In each cell of the chart, we estimated the risk for a person with values of each predictor at the mid-point of the predictor’s range for that cell. We then coloured the cells of the chart in four groups according to ranges of the probability of death: <6%, 6-20%, 21-50%, and >50%. We decided these cut-offs by considering feedback from the potential users of the simple prognostic model and by looking at previous publications.17 18
Results
Tables 1⇓ and 2⇓ show the characteristics of the patients in the CRASH-2 and TARN datasets. In the CRASH-2 trial, most patients were men and the median age was 30 years. Patients who died were on average older and had lower systolic blood pressure, higher heart rate, and lower Glasgow coma score than those who did not. Few data were missing for all the variables. In comparison, the patients from TARN were older (median age 39 years) and had a higher systolic blood pressure. In all, 3076 (15%) deaths occurred among the 20 127 CRASH-2 patients and 1765 (12%) in the 14 220 TARN patients.
Characteristics of CRASH-2 patients
Characteristics of TARN patients
Age showed a positive and increasing association with risk of death; systolic blood pressure, heart rate, and respiratory rate showed U shaped relations; and Glasgow coma score had a negative association with risk of death (fig 1⇓). Table 3⇓ shows that in the CRASH-2 trial, age was positively associated with mortality for each of the described causes of death; the highest relative increase was for vascular occlusive death.
Fig 1 Association between continuous predictors and death among CRASH-2 patients
Cause of death according to age in CRASH-2 patients. Values are numbers (percentages)
We included quadratic or cubic transformations in the prediction model to accommodate for the departures from linearity. In the multivariable analysis, Glasgow coma score, systolic blood pressure, and age were the three strongest predictors. Heart rate, respiratory rate, and hours since injury were associated with mortality and were also included in the final model. Users considered all of these variables to be important. Patients in low and middle income countries were more likely to die in comparison with those in high income countries. Although capillary refill time was weakly associated with mortality, we did not include it in the prognostic model because in situations with poor visibility, such as in the battlefield, it is difficult to measure. In addition, capillary refill time was missing in more than 80% of the TARN patients. We found some evidence of a statistical interaction between Glasgow coma score and type of injury. Low Glasgow coma score was associated with worse prognosis for blunt injuries (see web appendix for details of the multivariable analysis).
Validation
The model showed a good internal validity, with a C statistic of 0.84 (fig 2⇓) and good calibration, except in patients at very high risk for whom the model over-predicted risk (fig 3⇓). Internal validation using bootstrapping showed a minimal decrease in the C statistic from 0.836 to 0.835. This indicates that very low over-optimism existed in the development of the model.
Fig 2 Internal and external discrimination displayed by receiver operating characteristics curves. AUC=area under curve; PV+=positive predictive value; PV–=negative predictive value
Fig 3 Internal and external calibration of prognostic model by levels of predicted risk
For the external validation, we used the same variables as were included in the derivation model except hours since injury, as this variable had a very large number of patients with missing data. Discrimination was good (C statistic 0.88), and calibration was satisfactory (figures 2⇑ and 3⇑).
Model presentation
The prognostic model is available at http://crash2.lshtm.ac.uk/, so the risk of death can be obtained for individual patients. Entering the values of the predictors results in display of the expected risk of death at 28 days. For example, a 70 year old patient from a low income country, with a Glasgow coma score of 14, systolic blood pressure of 100 mm Hg, heart rate of 110 beats per minute, and respiratory rate of 35 breaths per minute, has a 32% probability of death at 28 days.
Users also highlighted the importance of a simple prognostic model that could be used at the bedside. The simple prognostic model includes the three strongest prognostic variables: Glasgow coma score, systolic blood pressure, and age (see appendix). We developed different prognostic models for patients in low, middle, and high income countries and presented them as charts (fig 4⇓). These simple charts also showed good internal and external calibration (fig 5⇓).
Fig 4 Chart to predict death in trauma patients. GCS=Glasgow coma score
Fig 5 Internal and external calibration of simple chart
Discussion
We have developed and validated a prognostic model for trauma patients by using clinical parameters that are easy to measure. The model is available as a web calculator and can be used at the point of care in its simplified form. Separate models are available for patients from low, middle, and high income countries. This simple prognostic model could inform doctors about the risk of death and guide them in the early assessment and management of trauma patients.
Strengths and limitations
Our study has several strengths. Our models were based on a prospective cohort of patients with traumatic bleeding, with standardised collection of data on prognostic factors, very little missing data, and low loss to follow-up. Unlike previous prognostic models, we explored more complex relations between continuous predictors and mortality and captured non-linear relations. All of these factors provide reassurance about the internal validity of our models. The large sample size in relation to the number of prognostic variables is also an important strength. Whereas most previous models were derived from single centre studies in high income countries, we developed separate models for low, middle, and high income countries. Unlike most previous models, we did an external validation in a large cohort of trauma patients. This confirmed the discriminatory ability of the model in patients from high income countries and showed good calibration.
Another methodological strength was our use of imputation to replace missing data, which is rarely done in model validation studies. To the best of our knowledge, this is the only prognostic model for this population that is available in a web based calculator and a simplified chart that can be used at point of care. Importantly, we obtained advice from the potential users throughout its development.
The study also has some limitations. The data from which the models were developed come from a clinical trial, and this could limit external validity. For example, patients were recruited within eight hours of injury, and we cannot estimate the accuracy of the models for patients evaluated beyond this time. Nevertheless, the CRASH-2 trial was a pragmatic trial that did not require any additional tests and therefore included a diversity of “real life” patients. In addition, the relation between predictors and outcome could be different in patients included in a clinical trial and in routine practice. However, the model’s good performance in a trauma registry population provides reassurance that any potential bias (if present) was small.
Another limitation was that for the validation we used a cohort of trauma patients that were not equally defined, and we included them by using an estimation of the blood loss. In any case, this weakness could have led to underestimation of the accuracy of the model. Other potentially important variables such as pre-existing medical conditions, previous drugs, and laboratory measurements were not collected in the CRASH-2 trial and, therefore, not available for inclusion in the model. However, these are variables that are usually unavailable in the acute care trauma setting in which the model is intended to be used. The prognostic model predicts overall death rather than death due to bleeding, as death due bleeding was not available in the TARN dataset. However, bleeding would be expected to contribute to the other main causes of death in trauma patients. In addition, some deaths classified as “non-bleeding” could in fact have been due to bleeding. Finally, we observed some miscalibration; in particular, we observed overestimation for patients with predicted high risk in the internal validation. This finding might be related to the imprecision due to the low number of patients in the very high risk group. Only 100 patients (84 events) had a predicted risk of death above 90% in the CRASH-2 dataset. However, miscalibration at this high risk end of the spectrum (that is, 80% v 90% probability of death) is very unlikely to change clinical decision making.
Implications of study
Many trauma protocols use blood pressure as the main criterion for determining who should receive urgent intervention. However, according to our model, a 75 year old with blunt trauma and a systolic blood pressure of 110 mm Hg, heart rate of 80 beats per minute, respiratory rate of 15 breaths per minute, and Glasgow coma score of 15 has a similar risk of death to a 45 year old patient with exactly the same parameters but a systolic blood pressure of 60 mm Hg. These findings have important practical implications. According to many trauma protocols, only the younger patient would receive urgent interventions such as tranexamic acid, and the older one would be denied this lifesaving intervention. The effect of age is particularly important, bearing in mind that in high income countries the average age of trauma patients is increasing. Data from TARN show that one quarter of the deaths due to trauma in England and Wales are in patients older than 70 years. The effect of age is likely to reflect the increased incidence of coexisting diseases, particularly cardiovascular diseases. Older patients are more likely to have coronary heart disease, and the decrease in oxygen supply associated with traumatic bleeding can increase the risk of myocardial ischaemia.19 Another potential explanation for the increased risk of death from vascular occlusive disease is related to the trigger of the inflammation process after trauma. After trauma, a potent inflammatory response involves increased serum concentrations of interleukin-1, interleukin-2, tumour necrosis factor-α, interleukin-6, interleukin-12, and interferon-γ.20 In patients with traumatic bleeding, activation of plasmin occurs and plays a key role in the fibrinolytic response in the early hours after injury. Plasmin also has pro-inflammatory effects through the activation of cytokines, monocytes, neutrophils, platelets, and endothelial cells.21 Vascular risk may rise in short time periods of inflammatory responses to exposures such as infections or major surgery.22 Some of the observed prognostic role of age in trauma patients may be due to the inflammatory response to acute trauma, which might trigger acute vascular events, particularly in older patients who have a more widespread atherosclerotic condition. Furthermore, the prognostic role of age could be explained partially by a “self fulfilling prophecy” phenomenon, as age has been shown to be positively associated with “do not resuscitate” orders.23
We acknowledge that estimating the risk of death in a trauma patient with bleeding is challenging. It is an ongoing process that uses not only physiological variables but other variables such as laboratory measurements and response to treatments. A prognostic model would never replace clinical judgment, but it can support it.
We found that trauma patients in low and middle income countries were at higher risk of death compared with those from high income countries. We emphasise that the income classification refers to the country and not to individual patients. Some of the effect of classification of income might be the consequence of the differences in healthcare settings. Other studies have shown similar results, but to our knowledge this is the first one to include a large number of low and middle income countries.24 Although we did not have enough information to explore the causes of these differences, the rapid increase in the number of trauma patients combined with the lack of resources in poorer countries is probably among the most important reasons. Scaling up cost effective interventions in these settings could save hundreds of thousands of lives every year.
Future research
The relation between age and mortality needs further exploration. A better understanding of the mechanism by which age is associated with increasing mortality could lead to effective interventions to improve the outcome in this vulnerable population. As we were able to validate the model only in patients from high income regions, future studies should also explore its performance in low and middle income countries. Finally, future research should evaluate whether the use of this prognostic model in clinical practice has an effect on the management and outcomes of trauma patients.25
What is already known on this topic
Failure to start appropriate early management in patients with traumatic bleeding is a leading cause of preventable death from trauma
An accurate and user friendly prognostic model to predict mortality could assist the appropriate early management in bleeding trauma patients
The methodological quality of published prognostic models is generally poor, sample sizes are small, and only a few models have included patients from low-middle income countries, where most deaths from trauma occur
What this study adds
An accurate and user friendly prognostic model to predict mortality in trauma patients with bleeding has been developed and validated
The prognostic model is available as a web based calculator, and a simplified model is available as a chart to be used at the bedside
This prognostic model can assist in triage and can shorten the time to diagnostic and lifesaving procedures such as imaging, surgery, or tranexamic acid
Notes
Cite this as: BMJ 2012;345:e5166
Footnotes
This study will be published in full in the Health Technology Assessment journal series. We thank the CRASH-2 Trial Collaborators and the TARN Executive for making their data available. We also acknowledge the ambulance crew, military personnel, and emergency doctors who gave feedback in the different stages of development and validation of the prognostic model. PP and IR are members of the Medical Research Council Prognosis Research Strategy (PROGRESS) Partnership (G0902393/99558).
Contributors: PP, HS, and IR designed the study. DP-M and OB analysed the data. PP and IR wrote the first draft of the paper. FL, RR, and MF gave feedback about the potential clinical use and format of the prognostic model. PP, DP-M, HS, TC, FL, OB, RR, MF, EWS, and IR contributed to writing the paper. PP, HS, IR, FL, and OB participated in the collection of data from which this manuscript was developed.
Funding: This study was funded by the UK Health Technology Assessment programme (09/22/165). The views and opinions expressed are those of the authors and do not necessarily reflect those of the Department of Health.
Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisation that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: The London School of Hygiene and Tropical gave medicine ethics approval for this study and the use of the CRASH-2 trial data. TARN already has ethical approval (PIAG section 60) for research on the anonymised data that are stored securely on the University of Manchester server.
Data sharing: Full information on accessing the data from the CRASH-2 trial is available via freeBIRD (free bank of injury and emergency research data), a data repository hosted by the Clinical Trials Unit, London School of Hygiene and Tropical Medicine, at http://ctu2.lshtm.ac.uk/freebird.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.