- Kathleen Daly, research nursea,
- R Beale, consultant intensive care physiciana,
- R W S Chang, consultant transplant surgeon ()b
- Correspondence to: R W S Chang
Objective: To develop a predictive model to triage patients for discharge from intensive care units to reduce mortality after discharge.
Design: Logistic regression analyses and modelling of data from patients who were discharged from intensive care units.
Setting: Guy's hospital intensive care unit and 19 other UK intensive care units from 1989 to 1998.
Participants: 5475 patients for the development of the model and 8449 for validation.
Main outcome measures: Mortality after discharge and power of triage model.
Results: Mortality after discharge from intensive care was up to 12.4%. The triage model identified patients at risk from death on the ward with a sensitivity of 65.5% and specificity of 87.6%, and an area under the receiver operating curve of 0.86. Variables in the model were age, end stage disease, length of stay in unit, cardiothoracic surgery, and physiology. In the validation dataset the 34% of the patients identified as at risk had a discharge mortality of 25% compared with a 4% mortality among those not at risk.
Conclusions: The discharge mortality of at risk patients may be reduced by 39% if they remain in intensive care units for another 48 hours. The discharge triage model to identify patients at risk from too early and inappropriate discharge from intensive care may help doctors to make the difficult clinical decision of whom to discharge to make room for a patient requiring urgent admission to the unit. If confirmed, this study has implications on the provision of resources.
What is already known on this topic
What is already known on this topic In the United Kingdom, the mortality of patients who die on the ward after discharge from intensive care is unacceptably high (9% to 27%)
Indirect evidence has shown that this is due to too early and inappropriate discharge from intensive care that has increased over the past 10 years
What this study adds
What this study adds A triage model identifies patients at risk from inappropriate discharge from intensive care
Mortality after discharge from intensive care may be reduced by 39% if these patients were to stay in intensive care for another 48 hours
An estimated 16% more beds are required if mortality after discharge from intensive care is to be reduced
The winter of 1999 highlighted the acute shortage of intensive care beds in the United Kingdom. A consequence of shortage is that patients are often discharged early and perhaps inappropriately to make room for more severely ill patients. A study in 1993 reported mortality after discharge from intensive care from 6.1% to 16.3 %. 1 2 The causes of death after such discharge may be due to factors occurring before 3 4 or after discharge.5–7 Goldfrad and Rowan, who used discharges at night as a proxy measure of inappropriate early discharge from intensive care, reported a 1.4-fold increase in ultimate hospital mortality among patients discharged at night.8 Patients who died after discharge had significantly higher severity of illness scores or therapeutic intervention scores on the day of discharge than those who survived. 9 10
We report on the development of a predictive triage model for discharge to identify patients at risk of dying after discharge from intensive care. We also explored the implications of its use.
We included in the study all patients discharged from the 13 bed intensive care unit at Guy's hospital between 1 June 1990 and 31 December 1998 and from 19 UK units (Riyadh ICU program users group) between June 1989 and September 1996. We analysed daily physiological and treatment data collected prospectively through the Riyadh ICU program (Medical Associated Software House, London) to identify candidate variables for the model. We measured severity of illness and intensity of treatment with the acute physiology and chronic health evaluation (APACHE II) system,11 the organ failure score,12 and the therapeutic intervention scoring system.13 These data, together with demographic data including the presence of chronic ill health (as defined with APACHE II criteria) and patients' hospital outcome, were entered daily on to the computer by a team of specifically trained nurses and doctors.
In general, patients were considered for admission to the participating intensive units if the combined assessment of the referring clinician and the doctor in charge of the unit were that the patient would benefit from intensive care. Clinical judgment on the basis of physiological variables, concurrent treatment, and clinical assessment was used to discharge patients from the intensive care unit. When there is pressure on beds, the least ill patient who can be managed outside an intensive care unit (for example, without mechanical ventilation) would be considered for discharge from the unit. None of the 20 units had a high dependency unit during this study.
There were 6319 patients admitted to the 13 bed general (medical, surgical, and cardiothoracic) adult intensive care unit at Guy's hospital between 30 June 1990 and 31 December 1996. We excluded from the analysis the 844 (13.4%) patients who died on the unit. Of the 5475 (87.0%) survivors, 200 (3.7%) patients died on the wards and 5275 (96.3%) survived to leave hospital. Twenty five (12.5%) patients who died on the ward and 117 (2.2%) hospital survivors were readmitted to intensive care during the same hospital stay. Only data from the patient's last day (the day of discharge or the day immediately preceding discharge; the last day with at least 8 hours of data) in the unit during their first admission to intensive care were used to develop the predictive model. There were 3133 (57.2%) patients who were admitted to intensive care after cardiothoracic surgery (97% after elective surgery)—a relatively low risk group. We created a variable denoting whether or not the patient had undergone cardiothoracic surgery (code 1 and 0, respectively).
We used univariate analysis to identify candidate variables for the model. Variables with a significant influence on survival (P<0.05) after discharge from intensive care were subjected to multivariate logistic modelling as linear effects with the binary logistic regression model (SPSS, Woking). A stepwise forward logistic regression procedure was used to derive the model. The initial attempts with model building were not successful, possibly because of loss of discriminatory power of the logistic regression function because of the large disparity in the number of survivors compared with non-survivors. To correct for this, we merged a random selection of 250 cases from the 5275 hospital survivors with the 200 ward deaths to obtain a dataset for model development. (This was a rather unconventional approach borrowed from the training of neural networks that commonly uses data equalisation.14) We repeated this 20 times to produce 20 modelling datasets; the 250 randomly selected cases were replaced before we selected another 250 cases (table 1). We then selected the “best” model (largest area under the receiver operator characteristic curve) and cut off (best trade off between sensitivity and specificity). Calibration of the model was assessed by the Hosmer- Lemeshow “goodness of fit statistic”15 for significance (P>0.05). We assessed discrimination with receiver operating curve analysis.16
We evaluated the triage model by applying it to a different dataset, derived from 1136 survivors (84.3% of admissions) from the intensive care unit at Guy's hospital who had been admitted between 1 January 1997 and 31 December 1998 and 7313 survivors (76.6% of admissions) from 19 other UK units (Riyadh ICU program users group) who had been admitted between June 1989 and September 1996. We used a new dataset for validation of the model to avoid any overoptimistic findings that may have occurred had we used the development dataset.17 Furthermore, as the development dataset contained many patients who had undergone cardiac surgery we considered it important to evaluate the model's validity among units in which this was not the case.
Use of model to alter outcome
For the model to be of any use we must be able to affect the outcome of patients identified as at risk. To test this, we selected patients who had stayed in intensive care for more than three days and had been at risk of death at any time within the 48 hours before discharge from the unit. We excluded from analysis those patients who died on the ward and who had been classified as “not for resuscitation” at discharge from intensive care. The patients were classified into four subgroups: group 0 comprised patients predicted to be at risk on the day of discharge; group 1 comprised patients predicted to be at risk in the 24 hours before discharge; group 2 comprised those predicted to be at risk in the 48 hours before discharge; and group 3 comprised patients who were not at risk in the 48 hours before discharge.
Data analysis was performed with the statistical software package SPSS version 9.0. Categorical data were analysed with χ2 tests. Non-normally distributed continuous data were evaluated with the Mann-Whitney test. Logistic regression analysis was used to develop the predictive model. P<0.05 was considered significant.
This study was approved by the local ethics committee of Guy's Hospital.
Table 2 gives demographic data and details of clinical features, severity of illness, and candidate variables for the model. The following variables were considered in the models: acute physiology points, length of stay in intensive care, therapeutic intervention score, duration (days) on mechanical ventilation, dialysis, age, presence of chronic ill health, number of failing organs, and whether or not the patient had had cardiothoracic surgery. Acute physiology points was used in preference to APACHE II score as the latter is derived from the acute physiology points, age points, chronic ill health points, and presence or absence of emergency surgery.
Forward stepwise multivariate analyses on each of the 20 modelling datasets (table 1) selected the following five variables: patient's age, chronic health points, acute physiology points at discharge from unit, length of stay in unit, and whether or not the patient had had cardiothoracic surgery for inclusion in the model in 15 instances. A cut off of 0.6 gave the best sensitivity and specificity (65.5% and 87.6%, respectively, in model 18). Table 3 gives details of the final model, and figure 1 shows its receiver operating curve.
As the results of the two validation datasets were similar we merged the data (table 4). The sensitivity and specificity were 74.3% and 71.1%, respectively; and the area under the receiver operator characteristic curve (fig 2) was 0.80 (95% confidence interval 0.79 to 0.81). The area under the curve ranged from 0.68 to 0.87 for the 20 individual intensive care units. Mortality in patients identified as at risk was 25% while the mortality in those not at risk was 4%, giving a relative risk of 5.61 (4.89 to 6.44). In the development dataset the figure for relative risk was 9.44 (7.12 to 21.51).
There were significant differences in mortality after discharge from intensive care between groups 0, 1, and 2 (table 5). In the development dataset, 14% of at risk patients died on the ward. In at risk patients who stayed an additional 48 hours in intensive care, during which time the probability of dying fell below 0.6, mortality after discharge from intensive care was only 4% (P=0.034). The relative risk of mortality for groups 1 and 2 versus group 0 (discharged on the day risk was predicted) was 0.385 (0.18 to 0.826). In the validation dataset there was a reduction in mortality from 28% in group 0 to 17% among those who stayed another 48 hours (P=0.011)—that is, their relative risk was reduced from 6.76 (4.87 to 9.56) in group 0 versus group 3 to 3.46 (2.21 to 5.41) in group 1 and 2 versus group 3. The relative risk of mortality for those who stayed an additional 24 and 48 hours compared with group 0 was 0.512 (0.373 to 0.706).
Potential impact on the provision of intensive care beds
We used the validation dataset to estimate the impact on the provision of intensive care resources. There were 8449 patients who stayed in intensive care for a total of 34 588 days, with an overall mortality after discharge from intensive care of 11.3%. We identified 2875 patients (34% of total) as at risk, with a mortality after discharge of 25%. If we assume that our model is valid, mortality after discharge from intensive care could be reduced by nearly 39% if these patients stayed another two days before discharge. We estimated that this would required 5750 additional intensive care bed days or the provision of fully staffed intensive care bed days would have to be increased by 16%.
A considerable number of patients die on the wards after discharge from intensive care units. Mortality after discharge from intensive care ranges from 9% to 27%. 18 19 Our discharge triage model used objective data (age, end stage disease, physiology, length of stay, and cardiac surgery) in a logistic regression equation to identify patients at risk from inappropriate early discharge. We were able to do this because the database of the Riyadh ICU program captures daily data throughout a patient's stay in intensive care.
Among patients in the dataset we used to develop the model, those identified as at risk had a mortality of 14% compared with a mortality of only 1.5% among those not at risk. This is despite the low mortality after discharge from intensive care at Guy's of only 3.7%. The model was applicable to the validation dataset despite a large difference in its mortality after discharge: 11.3% compared with 3.7% in the development dataset. The main difference between the two datasets was that the development dataset contained more patients who had undergone cardiac surgery. Eighteen out of the 19 other intensive care units did not treat patients who had undergone cardiac surgery. This disparity was accounted for by the cardiac surgery variable in the model.
By modelling a “what if” situation, whereby patients at risk and discharged on the same day were compared with patients who stayed for another 24 to 48 hours, we showed a reduction in relative risk from 6.76 to 3.46. Acute physiological points is the only variable in the model for which a reduction will lead to fall in the probability of dying on the ward. This variable is an aggregate of the weights of 12 physiological variables; normalisation of physiology will lead to a reduction in the variable and therefore a reduction in the probability of dying after discharge from intensive care.
UK resources for intensive care
The United Kingdom has limited resources allocated for the provision of intensive care facilities compared with many of its European counterparts,20 and regional differences in the number of available intensive care beds have been shown.21 Although the overall number of intensive care and high dependency beds has increased over the past 10 years, there has been a concurrent rise in hospital activity. 22 23 Our modelling exercise suggests that up to 34% of patients are at risk and an increase of 16% in the number of intensive care beds is required to avoid deaths from inappropriate early discharges. Although this finding needs confirmation by a prospective study, it is consistent with the finding in the report by the Audit Commission in 1999 that up to 25% (with a median value of 5%) of patients were still being discharged prematurely to allow more seriously ill patients to be admitted.24 Neither our discharge triage model nor discharge guidelines published by the Department of Health,25 which deal with the process of care, will have much impact until and unless the shortfall in provision of intensive care beds is corrected.
In summary, our model can provide additional information to help the doctor with the difficult problem of who to discharge from an intensive care unit to make room for a patient who needs urgent admission. The modelling exercise supports the findings of others that the provision of intensive care resources in the United Kingdom needs to be increased.
We thank members of the Riyadh ICU program users group for access to their database.
Contributors: KD (as part of her PhD) and RWSC (PhD supervisor) collected data and were responsible for data pre-processing, development of the triage model, data analysis, and literature search. KD, RWSC, and RB wrote the paper jointly. RC will act as guarantor.
Funding The Special Trustees of St Thomas's Hospital contributed to the funding of this project.
Competing interests R W S Chang designed and developed the Riyadh ICU program and is a director of Medical Associated Software House, which markets the software.