BMJ 1998;316:1697-1700 ( 6 June )

Papers

Cumulative risk adjusted mortality chart for detecting changes in death rate: observational study of heart surgery

See Editor's choice

J Poloniecki, lecturer in statistics, ^a O Valencia, research fellow, ^b P Littlejohns, director, Health Care Evaluation Unit. ^a

^a Public Health Sciences, St George's Hospital Medical School, London SW17 0RE, ^b Regional Cardiothoracic Unit, St George's Hospital, London SW17 0RE

Correspondence to: Dr Poloniecki j.poloniecki{at}sghms.ac.uk

Objective: To detect changes in mortality after surgery, with allowance being made for variations in case mix.
Design: Observational study of postoperative mortality from January 1992 to August 1995. 
Setting: Regional cardiothoracic unit.
Subjects: 3983 patients aged 16 and over who had open heart operations.
Main outcome measures: Preoperative risk factors and postoperative mortality in hospital within 30 days were recorded for all surgical heart operations. Mortality was adjusted for case mix using a preoperative estimate of risk based on additive Parsonnet factors. The number of operations required for statistical power to detect a doubling of mortality was examined, and control limits at a nominal significance level of P=0.01 for detection of an adverse trend were determined.
Results: Total mortality of 7.0% was 26% below the Parsonnet predictor (P<0.0001). There was a highly significant variation in annual case mix (Parsonnet scores 8.7-10.6, P<0.0001). There was no significant variation in mortality after adjustment for case mix (odds ratio 1-1.5, P=0.18) with monitoring by calendar year. With continuous monitoring, however, nominal 99% control limits based on 16 expected deaths were crossed on two occasions.
Conclusions: Hospital league tables for mortality from heart surgery will be of limited value because year to year differences in death rate can be large (odds ratio 1.5) even when the underlying risk or case mix does not change. Statistical quality control of a single series with adjustment for case mix is the only way to take into account recent performance when informing a patient of the risk of surgery at a particular hospital. If there is an increase in the number of deaths the chances of the next patient surviving surgery can be calculated from the last 16 deaths.

Key messages Changes in the patient population affect a hospital's annual death rate Year to year differences in death rate can be large even when there is no change in the underlying risk or case mix It takes surprisingly many operations before an increase in death rate can be distinguished from random fluctuation A formal inquiry should take place in a hospital if the death rate rises above control limits The chances of the next patient surviving surgery should be calculated using the surgeon's most recent results

---

© BMJ 1998
CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Detection of changes in mortality after heart surgery

Steve Gallivan, Jocelyn Lovegrove, Christopher Sherlaw-Johnson, and J D Poloniecki
BMJ 1998 317: 1453. [Extract] [Full Text]

The dark side of medicine
BMJ 1998 316: 0. [Full Text]

CRAM chart detects increase in number of surgical deaths
BMJ 1998 316: 0. [Full Text]

Lessons from the Bristol case

Tom Treasure
BMJ 1998 316: 1685-1686. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• Holt, P. J E, Poloniecki, J. D, Thompson, M. M (2008). How to improve surgical outcomes. BMJ 336: 900-901 [Full text]  
• Coory, M., Duckett, S., Sketcher-Baker, K. (2008). Using control charts to monitor quality of hospital care with administrative data. Int J Qual Health Care 20: 31-39 [Abstract] [Full text]  
• Ferraris, V. A., Edwards, F. H., Shahian, D. M., Ferraris, S. P. (2008). Risk Stratification and Comorbidity. Card Surg Adult 3: 199-246 [Full text]  
• Biau, D. J, Resche-Rigon, M., Godiris-Petit, G., Nizard, R. S, Porcher, R. (2007). Quality control of surgical and interventional procedures: a review of the CUSUM. Qual Saf Health Care 16: 203-207 [Abstract] [Full text]  
• Murphy, G. J., Rogers, C. A., Caputo, M., Angelini, G. D. (2005). Acquiring Proficiency in Off-Pump Surgery: Traversing the Learning Curve, Reproducibility, and Quality Control. Ann. Thorac. Surg. 80: 1965-1970 [Abstract] [Full text]  
• Rogers, C. A., Reeves, B. C., Caputo, M., Ganesh, J. S., Bonser, R. S., Angelini, G. D. (2004). Control chart methods for monitoring cardiac surgical performance and their interpretation. J. Thorac. Cardiovasc. Surg. 128: 811-819 [Full text]  
• Ng, C.-Y., Ramli, M. F., Awang, Y. (2004). Coronary Bypass Surgery in Patients Aged 70 Years and Over: Mortality, Morbidity, Length of Stay and Hospital Cost. Asian Cardiovasc. Thorac. Ann. 12: 218-223 [Abstract] [Full text]  
• Albert, A.A., Walter, J.A., Arnrich, B., Hassanein, W., Rosendahl, U.P., Bauer, S., Ennker, J. (2004). On-line variable live-adjusted displays with internal and external risk-adjusted mortalities. A valuable method for benchmarking and early detection of unfavourable trends in cardiac surgery. Eur. J. Cardiothorac. Surg. 25: 312-319 [Abstract] [Full text]  
• Poloniecki, J., Sismanidis, C., Bland, M., Jones, P. (2004). Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups. BMJ 328: 375- [Abstract] [Full text]  
• Grunkemeier, G. L., Wu, Y. X., Furnary, A. P. (2003). Cumulative sum techniques for assessing surgical results. Ann. Thorac. Surg. 76: 663-667 [Full text]  
• Sismanidis, C., Bland, M., Poloniecki, J. (2003). Properties of the Cumulative Risk-Adjusted Mortality (CRAM) Chart, Including the Number of Deaths Before a Doubling of the Death Rate is Detected. Med Decis Making 23: 242-251 [Abstract]  
• Tekkis, P. P, McCulloch, P., Steger, A. C, Benjamin, I. S, Poloniecki, J. D (2003). Mortality control charts for comparing performance of surgical units: validation study using hospital mortality data. BMJ 326: 786-788 [Abstract] [Full text]  
• Agabiti, N., Ancona, C., Forastiere, F., Arca, M., Perucci, C. A. (2003). Evaluating outcomes of hospital care following coronary artery bypass surgery in Rome, Italy. Eur. J. Cardiothorac. Surg. 23: 599-606 [Abstract] [Full text]  
• Grigg, O A, Farewell, V T, Spiegelhalter, D J (2003). Use of risk-adjusted CUSUM and RSPRTcharts for monitoring in medical contexts. Stat Methods Med Res 12: 147-170 [Abstract]  
• SPIEGELHALTER, D., GRIGG, O., KINSMAN, R., TREASURE, T. (2003). Risk-adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. Int J Qual Health Care 15: 7-13 [Abstract] [Full text]  
• Miles, H, Litton, E, Curran, A, Goldsworthy, L, Sharples, P, Henderson, A J (2002). The PATRIARCH Study. Using outcome measures for league tables: Can a North American prediction of admission score be used in a United Kingdom children's emergency department?. Emerg. Med. J. 19: 536-538 [Abstract] [Full text]  
• McCulloch, P., Taylor, I., Sasako, M., Lovett, B., Griffin, D. (2002). Randomised trials in surgery: problems and possible solutions. BMJ 324: 1448-1451 [Full text]  
• Walsh, K, Gompertz, P H, Rudd, A G (2002). Stroke care: how do we measure quality?. Postgrad. Med. J. 78: 322-326 [Abstract] [Full text]  
• Sergeant, P., de Worm, E., Meyns, B. (2001). Single centre, single domain validation of the EuroSCORE on a consecutive sample of primary and repeat CABG. Eur. J. Cardiothorac. Surg. 20: 1176-1182 [Abstract] [Full text]  
• Sergeant, P., de Worm, E., Meyns, B., Wouters, P. (2001). The challenge of departmental quality control in the reengineering towards off-pump coronary artery bypass grafting. Eur. J. Cardiothorac. Surg. 20: 538-543 [Abstract] [Full text]  
• Lawrance, R. A, Dorsch, M. F, Sapsford, R. J, Mackintosh, A. F, Greenwood, D. C, Jackson, B. M, Morrell, C., Robinson, M. B, Hall, A. S (2001). Use of cumulative mortality data in patients with acute myocardial infarction for early detection of variation in clinical practice: observational study. BMJ 323: 324-327 [Abstract] [Full text]  
• Kang, N., Edwards, M., Larbalestier, R. (2001). Preoperative intraaortic balloon pumps in high-risk patients undergoing open heart surgery. Ann. Thorac. Surg. 72: 54-57 [Abstract] [Full text]  
• Cook, D. A. (2000). Performance of APACHE III Models in an Australian ICU. Chest 118: 1732-1738 [Abstract] [Full text]  
• Wynne-Jones, K, Jackson, M, Grotte, G, Bridgewater, B, North West Regional Cardiac Surgery Audit Steering, (2000). Limitations of the Parsonnet score for measuring risk stratified mortality in the north west of England. Heart 84: 71-78 [Abstract] [Full text]  
• Sherlaw-Johnson, C, Lovegrove, J, Treasure, T, Gallivan, S (2000). Likely variations in perioperative mortality associated with cardiac surgery: when does high mortality reflect bad practice?. Heart 84: 79-82 [Abstract] [Full text]  
• Darzi, A., Smith, S., Taffinder, N. (1999). Assessing operative skill. BMJ 318: 887-888 [Full text]  
• Gallivan, S., Lovegrove, J., Sherlaw-Johnson, C., Poloniecki, J D (1998). Detection of changes in mortality after heart surgery. BMJ 317: 1453-1453 [Full text]