Intended for healthcare professionals

Papers

Mortality control charts for comparing performance of surgical units: validation study using hospital mortality data

BMJ 2003; 326 doi: https://doi.org/10.1136/bmj.326.7393.786 (Published 12 April 2003) Cite this as: BMJ 2003;326:786
  1. Paris P Tekkis, research fellow of the Royal College of Surgeons of England (ptekkis{at}blueyonder.co.uk)a,
  2. Peter McCulloch, senior lecturer in surgeryb,
  3. Adrian C Steger, consultant surgeonc,
  4. Irving S Benjamin, professor of surgerya,
  5. Jan D Poloniecki, senior lecturer in biostatisticsd
  1. a Academic Department of Surgery, King's College Hospital, London SE5 9RS
  2. b Academic Unit of Surgery, University of Liverpool, University Hospital Aintree, Liverpool L9 7AL
  3. c Department of Surgery, University Hospital Lewisham, London SE13 6LH
  4. d Department of Public Health Sciences, St George's Hospital, London SW17 0QT
  1. Correspondence to: P P Tekkis
  • Accepted 6 February 2002

Abstract

Objective: To design and validate a statistical method for evaluating the performance of surgical units that adjusts for case volume and case mix.

Design: Validation study using routinely collected data on in-hospital mortality.

Data sources: Two UK databases, the ASCOT prospective database and the risk scoring collaborative (RISC) database, covering 1042 patients undergoing surgery in 29 hospitals for gastro-oesophageal cancer between 1995 and 2000.

Statistical analysis: A two level hierarchical logistic regression model was used to adjust each unit's operative mortality for case mix. Crude or adjusted operative mortality was plotted on mortality control charts (a graphical representation of surgical performance) as a function of number of operations. Control limits defined as 90%, 95%, and 99% confidence intervals identified units whose performance diverged significantly from the mean.

Results: The mean in-hospital mortality was 12% (range 0% to 50%). The case volume of the units ranged from one to 55 cases a year. When crude figures were plotted on the mortality control chart, four units lay outside the 90% control limit, including two outside the 95% limit. When operative mortality was adjusted for risk, three units lay outside the 90% limit and one outside the 95% limit. The model fitted the data well and had adequate discrimination (area under the receiver operating characteristics curve 0.78).

Conclusions: The mortality control chart is an accurate, risk adjusted means of identifying units whose surgical performance, in terms of operative mortality, diverges significantly from the population mean. It gives an early warning of divergent performance. It could be adapted to monitor performance across various specialties.

What is already known on this topic

What is already known on this topic League tables are an established technique for ranking the performance of organisations such as healthcare providers

Mortality control charts are another way to compare the performance of healthcare providers, particularly for outcomes of surgery

What this study adds

What this study adds Mortality control charts can be adjusted for case mix and case volume and are better than league tables for monitoring surgical performance

Mortality control charts have a “buffer zone” for indicating divergence from the mean mortality and are particularly useful for specialties with a low volume of surgery

Footnotes

  • Funding The Hue Falwasser Fellowship of the Royal College of Surgeons of England. The guarantors accept full responsibility for the conduct of the study, had access to the data, and controlled the decision to publish.

  • Competing interests None declared.

  • Ethical approval: The multicentre research ethics committee for Wales.

  • Accepted 6 February 2002
View Full Text