BMJ 2003;326:786 ( 12 April )

Papers

Mortality control charts for comparing performance of surgical units: validation study using hospital mortality data

Paris P Tekkis, research fellow of the Royal College of Surgeons of England, ^a Peter McCulloch, senior lecturer in surgery, ^b Adrian C Steger, consultant surgeon, ^c Irving S Benjamin, professor of surgery, ^a Jan D Poloniecki, senior lecturer in biostatistics. ^d

^a Academic Department of Surgery, King's College Hospital, London SE5 9RS, ^b Academic Unit of Surgery, University of Liverpool, University Hospital Aintree, Liverpool L9 7AL, ^c Department of Surgery, University Hospital Lewisham, London SE13 6LH, ^d Department of Public Health Sciences, St George's Hospital, London SW17 0QT

Correspondence to: P P Tekkis ptekkis{at}blueyonder.co.uk

Objective: To design and validate a statistical method for evaluating the performance of surgical units that adjusts for case volume and case mix.
Design: Validation study using routinely collected data on in-hospital mortality.
Data sources: Two UK databases, the ASCOT prospective database and the risk scoring collaborative (RISC) database, covering 1042 patients undergoing surgery in 29 hospitals for gastro-oesophageal cancer between 1995 and 2000.
Statistical analysis: A two level hierarchical logistic regression model was used to adjust each unit's operative mortality for case mix. Crude or adjusted operative mortality was plotted on mortality control charts (a graphical representation of surgical performance) as a function of number of operations. Control limits defined as 90%, 95%, and 99% confidence intervals identified units whose performance diverged significantly from the mean.
Results: The mean in-hospital mortality was 12% (range 0% to 50%). The case volume of the units ranged from one to 55 cases a year. When crude figures were plotted on the mortality control chart, four units lay outside the 90% control limit, including two outside the 95% limit. When operative mortality was adjusted for risk, three units lay outside the 90% limit and one outside the 95% limit. The model fitted the data well and had adequate discrimination (area under the receiver operating characteristics curve 0.78).
Conclusions: The mortality control chart is an accurate, risk adjusted means of identifying units whose surgical performance, in terms of operative mortality, diverges significantly from the population mean. It gives an early warning of divergent performance. It could be adapted to monitor performance across various specialties.

What is already known on this topic League tables are an established technique for ranking the performance of organisations such as healthcare providers Mortality control charts are another way to compare the performance of healthcare providers, particularly for outcomes of surgery What this study adds Mortality control charts can be adjusted for case mix and case volume and are better than league tables for monitoring surgical performance Mortality control charts have a "buffer zone" for indicating divergence from the mean mortality and are particularly useful for specialties with a low volume of surgery

---

© 2003 BMJ Publishing Group Ltd
CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Relevant Articles

Mortality control charts: Both components of POSSUM ratio require critical analysis

George A Khoury
BMJ 2003 326: 1397. [Extract] [Full Text]

Mortality control charts: Assessment of outcome is complex

Frank A Frizelle and John Frye
BMJ 2003 326: 1397. [Extract] [Full Text]

Mortality control charts improve monitoring of surgical performance
BMJ 2003 326: 0. [Full Text] [PDF]

This article has been cited by other articles:

• Mohammed, M A, Worthington, P, Woodall, W H (2008). Plotting basic control charts: tutorial notes for healthcare practitioners. Qual Saf Health Care 17: 137-145 [Abstract] [Full text]  
• Moloney, E. D, Bennett, K., Silke, B. (2007). Effect of an acute medical admission unit on key quality indicators assessed by funnel plots. Postgrad. Med. J. 83: 659-663 [Abstract] [Full text]  
• Spiegelhalter, D J (2005). Handling over-dispersion of performance indicators. Qual Saf Health Care 14: 347-351 [Abstract] [Full text]  
• Leandro, G, Rolando, N, Gallus, G, Rolles, K, Burroughs, A K (2005). Monitoring surgical and medical outcomes: the Bernoulli cumulative SUM chart. A novel application to assess clinical interventions. Postgrad. Med. J. 81: 647-652 [Abstract] [Full text]  
• Rogers, C. A., Ganesh, J. S., Banner, N. R., Bonser, R. S., On behalf of the steering Group, (2005). Cumulative risk adjusted monitoring of 30-day mortality after cardiothoracic transplantation: UK experience. Eur. J. Cardiothorac. Surg. 27: 1022-1029 [Abstract] [Full text]  
• Blackstone, E. H. (2004). Monitoring surgical performance. J. Thorac. Cardiovasc. Surg. 128: 807-810 [Full text]  
• Rogers, C. A., Reeves, B. C., Caputo, M., Ganesh, J. S., Bonser, R. S., Angelini, G. D. (2004). Control chart methods for monitoring cardiac surgical performance and their interpretation. J. Thorac. Cardiovasc. Surg. 128: 811-819 [Full text]  
• Mohammed, M A (2004). Using statistical process control to improve the quality of health care. Qual Saf Health Care 13: 243-245 [Full text]  
• Marshall, T., Mohammed, M. A., Rouse, A. (2004). A randomized controlled trial of league tables and control charts as aids to health service decision-making. Int J Qual Health Care 16: 309-315 [Abstract] [Full text]  
• Treasure, T. (2004). Congenital heart disease. BMJ 328: 594-595 [Full text]  
• Frizelle, F. A, Frye, J. (2003). Mortality control charts: Assessment of outcome is complex. BMJ 326: 1397-1397 [Full text]  
• Khoury, G. A (2003). Mortality control charts: Both components of POSSUM ratio require critical analysis. BMJ 326: 1397-1397 [Full text]  

Rapid Responses:

Read all Rapid Responses

What determines the standard?

John Robson
bmj.com, 14 Apr 2003 [Full text]

Only one standard: 0% morbidity and mortality

Richard G Fiddian-Green
bmj.com, 16 Apr 2003 [Full text]

Assessment of outcome is complex

Frank A. Frizelle, et al.
bmj.com, 17 Apr 2003 [Full text]

Why Not to Judge Performance with Statistical Process Control

Anthony P Morton
bmj.com, 26 Apr 2003 [Full text]

Control charts could mislead

Chris Sherlaw-Johnson
bmj.com, 2 May 2003 [Full text]

Data validation and interpretation in POSSUM

George A Khoury
bmj.com, 13 May 2003 [Full text]

Online poll

Find out more >>