Outcome data and scoring systemsBMJ 1999; 319 doi: https://doi.org/10.1136/bmj.319.7204.241 (Published 24 July 1999) Cite this as: BMJ 1999;319:241
- Kevin Gunning,
- Kathy Rowan
Intensive care has developed over the past 30 years with little rigorous scientific evidence about what is, or is not, clinically effective Without these data, doctors delivering intensive care often have to decide which patients can benefit most. Scoring systems have been developed in response to an increasing emphasis on the evaluation and monitoring of health services. These systems enable comparative audit and evaluative research of intensive care.
Why are scoring systems needed?
Although rigorous experiments or large randomised controlled trials are the gold standard for evaluating existing or new interventions, these are not always possible in intensive care. For example, it is unethical to randomly allocate severely ill patients to receive intensive care or general ward care. The alternative is to use observational methods that study the outcome of care patients receive as part of their natural treatment. However, before inferences can be drawn about outcomes of treatment in such studies the characteristics of the patients admitted to intensive care have to be taken into account This process is known as adjusting for case mix.
The death rate of patients admitted to intensive care units is much higher than that of other hospital patients. Data for 1995-8 on 22 057 patients admitted to 62 units in the case mix programme, the national comparative audit of patient outcome, showed an intensive care mortality of 20.6% and total hospital mortality of 30.9%. However, mortality across units varied more than threefold. Clearly, it is important to account for this variation.
Given the relatively high mortality among intensive care patients, death is a sensitive, appropriate, and meaningful measure of outcome. However, death can result from many factors other than ineffective care Outcome depends not only on the input (equipment, staff) and the processes of care (type, skill, and timing of care) but also on the case mix of the patients. The patient population of an intensive care unit in a large tertiary centre may be very different from that of a unit based in a district general hospital. Patients are admitted to intensive care for a wide range of clinical indications; both the nature of the current crisis and any underlying disease must be considered. Intensive care units admitting greater proportions of high risk patients would be expected to have a higher mortality. For example, the risk of death would be higher for a 76 year old with chronic obstructive airways disease admitted with faecal peritonitis than for a 23 year old in diabetic coma.
Various characteristics such as age have been recognised as important in increasing the risk of death before discharge from hospital after intensive care. It is essential to account for such patient characteristics before comparing outcome.
Factors increasing risk of death after intensive care
Greater severity of acute illness
History of severe clinical conditions
Emergency surgery immediately before admission
Clinical condition necessitating admission
Scoring systems are aimed at quantifying case mix and using the resulting score to estimate outcome. Outcome has usually been measured as death before discharge from hospital after intensive care. In the mid-1970s William Knaus developed the APACHE (acute physiology and chronic health evaluation) scoring system, which scored patients according to the acute severity of illness by weighting physiological derangement.
Initially, 34 physiological variables which were thought to have an effect on outcome were selected by a small panel of clinicians. These were then reduced to 12 more commonly measured variables for the APACHE II scoring system published in 1985. Up to four points are assigned to each physiological variable according to its most abnormal value during the first 24 hours in intensive care Points are also assigned for age, history of severe clinical conditions, and surgical status. The total number of points gives a score ranging from 0-71, with an increasing score representing a greater severity of illness.
The reason for admission to intensive care has also been shown to affect outcome. As most intensive care units do not see a sufficient number of patients with the same condition, mathematical equations were developed to estimate probabilities of outcome derived from databases containing several thousand patients from many intensive care units APACHE II allows the probability of death before discharge from hospital to be estimated. The probability of death for each patient admitted to intensive care can be summed to give the expected hospital death rate for the whole group. The expected hospital death rate can then be compared with the actual hospital death rate. This is often expressed as the standardised mortality ratio, the ratio of actual to expected deaths.
Acute physiology and chronic health evaluation (APACHE II) scoring system
Glasgow coma scale: Score is subtracted from 15 to obtain points.
Age <45=0 points, 45-54=2, 55-64=3, 65-75=5, ≥= 75=6.
Chronic health points (must be present before hospital admission): chronic liver disease with hypertension or previous hepatic failure, encephalopathy, or coma; chronic heart failure (New York Heart Association grade 4); chronic respiratory disease with severe exercise limitation, secondary polycythaemia, or pulmonary hypertension; dialysis dependent renal disease; immunosuppression—for example, radiation, chemotherapy, recent or long term high dose steroid therapy, leukaemia, AIDS. 5 points for emergency surgery or non-surgical patient, 2 points for elective surgical patient.
Proposed roles for scoring systems
Comparisons of actual and expected outcomes for groups of patients can be used to compare different providers. It is assumed that a standardised mortality ratio greater than 1.0 may reflect poor care and, conversely, a ratio below 1.0 may reflect good care. The reasons for any unexpected results can then be investigated locally. Review of deaths among patients estimated to be at lower risk of death may show that a particular group of patients or those discharged at a particular time of day have a poorer prognosis.
Proposed roles for scoring system
Clinical management of patients
Estimation of probability of death in hospital by applying APACHE II for 71 year old man admitted to intensive care from the hospital's accident and emergency department with (a) abdominal aortic aneurysm and (b) asthma attack
When non-randomised or observational methods are used to evaluate interventions a valid means of adjusting for differences in case mix is needed. Accurate estimates of expected hospital death rates for groups of patients can be used in research studies to identify those components of intensive care structure and process that are linked to improved outcome.
Scoring systems have also been proposed to aid stratification in randomised controlled trials. Given the considerable heterogeneity of patients in intensive care stratification based on an accurate, objective estimate of the probability of death before hospital discharge should create a more homogeneous subset of patients and improve isolation of the effects of an intervention.
Clinical management of individual patients
Scores obtained from scoring systems have been proposed as a clinical shorthand—that is, a common, standard terminology to rapidly convey information about a patient. They have also been proposed for use in triage to classify patients according to severity of illness.
Although early scoring systems were designed only for comparing observed and expected outcomes, some of the second and third generation scoring systems are promoted as methods to guide clinical care and treatment. Such decisions might include when to withdraw treatment or when to discharge a patient. This proposal has generated considerable debate, even though scoring systems have been shown to be as good as clinicians in predicting survival. Some of the more recent methods have incorporated trend analysis to try to improve the ability to predict outcome for individual patients. However, current scoring systems provide only probabilities and do not accurately predict whether an individual will survive. They therefore should not be used alone to determine decisions about intensive care.
Types of scoring systems
Scoring systems in intensive care can be either specific or generic. Specific scoring systems are used for certain types of patient whereas generic systems can be used to assess all, or nearly all, types of patient The scoring system may be either anatomical or physiological Anatomical scoring systems assess the extent of injury whereas physiological systems assess the impact of injury on function. Scores from anatomical scoring systems, once assessed, are fixed whereas physiological scores may change as the physiological response to the injury or disease varies.
The first scoring systems were developed for trauma patients and were either specific anatomical methods (abbreviated injury score, 1969; burns score, 1971; injury severity score, 1974) or specific physiological methods (trauma index, 1971; Glasgow coma scale, 1974; trauma score, 1981; sepsis score, 1983).
The total score is the sum of the three variables.
The Glasgow coma scale is still in general use in intensive care. The scale avoids having to describe a patient's level of neurological function in words and the assumption that colleagues understand the same meaning from those words.
The later scoring systems developed for intensive care have been generic. Two main approaches have been used; one is aimed at measuring severity by treatment intensity and the second at measuring severity by patient characteristics and physiological measurements.
Measuring severity by treatment
The therapeutic intervention scoring system (TISS) published in 1974 was developed to quantify severity of illness among intensive care patients based on the type and amount of treatment received. The underlying philosophy was that the sicker the patient, the greater the number and complexity of treatments given. By quantifying this, a proxy measure of the severity of illness for a patient could be obtained. The system scored 76 common therapeutic activities and was last updated in 1983. A simplified version based on 28 therapeutic activities (TISS 28) has been published and a version for patients in high dependency units has been proposed.
Another approach is to assess the severity of organ dysfunction based on the type and amount of treatment received. These organ failure scoring systems are used to give a probability of hospital death which takes into account the severity of dysfunction in each organ system and the effect on prognosis of dysfunction in several organ systems.
Measuring severity by patient characteristics and physiological measurements
The first generic physiological scoring system developed to quantify severity of illness by patient characteristics was the APACHE method, described above. The original system was too complex and time consuming to collect routinely, so two derivations were developed—the simplified acute physiology score (SAPS) and the APACHE II system. These were both subsequently updated to APACHE III in 1991 and SAPS II in 1993. An alternative system is the mortality prediction model (MPM II).
Selecting a scoring system
The scoring system chosen depends on the proposed use. The main criteria for selection should be the accuracy (validity and reliability) of the score and the goodness of fit (calibration and discrimination) of the mathematical equation used to estimate outcome. Rigorous comparison of the accuracy and goodness of fit of most scoring systems has not been done in the United Kingdom. APACHE II has been tested and is the most widely used.
Criteria for selecting a scoring system
Validity of score
Reliability of score
Discrimination of scoring system
Calibration of scoring system
Outcome from intensive care
Although death before discharge from hospital is the usual measure of outcome, disability, functional health, and quality of life should not be ignored. A study published in 1994 showed that in the first year after discharge from intensive care the risk of patients dying was 3.4 times greater than that of a matched population; the excess risk did not disappear until the fourth year after discharge.
Quality of life after a critical illness has been measured by various methods. The results differ according to the method used and the types of patient studied. Age and pre-existing severe clinical conditions seem to greatly affect quality of life after intensive care. In one study, 62% of young trauma victims who survived intensive care reported significant severe social disability and modest to severe impairment at work 10 months after discharge. In contrast, another study of a mixed group of patients found that those with pre-existing severe clinical conditions showed some improvement in their quality of life 6 months after admission to intensive care. A systematic review of the literature is underway.
Kevin Gunning is consultant in anaesthesia and intensive care, John Farman Intensive Care Unit, Addenbrooke's Hospital, Cambridge, and Kathy Rowan is scientific director, Intensive Care National Audit Research Centre, London.
The ABC of intensive care is edited by Mervyn Singer, reader in intensive care medicine, Bloomsbury Institute of Intensive Care Medicine, University College London, and Ian Grant, director of intensive care, Western General Hospital, Edinburgh. The series was conceived and planned by the Intensive Care Society's council and research subcommittee.