Intended for healthcare professionals

Papers

Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups

BMJ 2004; 328 doi: https://doi.org/10.1136/bmj.37956.520567.44 (Published 12 February 2004) Cite this as: BMJ 2004;328:375
  1. Jan Poloniecki, senior lecturer (j.poloniecki{at}sghms.ac.uk)1,
  2. Charalambos Sismanidis, research assistant1,
  3. Martin Bland, professor1,
  4. Paul Jones, medical director2
  1. Community Health Sciences, St George's Hospital Medical School, London SW17 0RE,
  2. St George's Healthcare NHS Trust, London SW17 0QT
  1. Correspondence to: J Poloniecki
  • Accepted 13 November 2003

Abstract

Objective To examine the efficacy of different methods of detecting a high death rate and determining whether an increase in deaths after heart transplantation could be explained by chance.

Design Retrospective analysis of deaths after heart transplantation. Seven methods were used: mortality above national average, mortality excessively above national average, test of moving average mortality, test of number of consecutive deaths, sequential probability ratio test (SPRT), cusum graph with v-mask, and CRAM chart. The national average mortality was not available and a rate of 15% was used instead as the benchmark.

Setting Regional cardiothoracic unit.

Participants All 371 patients who received a heart transplant in the programme, 1986-2000.

Main outcome measures 30 day survival after transplantation.

Results All methods provided evidence that the 30 day mortality had been high at some stage. The probability that the finding was a false positive depended on which test was used. At the end of the series the average mortality, sequential probability ratio, and cusum tests indicated a level of deaths higher than the benchmark while the remaining four tests yielded negative results.

Conclusions If the decision to test for outlying mortality is made retrospectively, in the light of the data, it is not possible to determine the false positive rate. Prospective on-site mortality monitoring with the CRAM chart is recommended as this method can quantify the death rate and identify periods when an audit of cases is indicated, even when data from other institutions are not available. A hospital mortality monitoring group can routinely monitor all deaths in the hospital, by specialty, using hospital episode statistics (HES) data and appropriate statistical methods.

Footnotes

  • Contributors Andrew Murday provided all the data and encouraged us to analyse them. He commented on an early version of the paper. JP did the analyses, wrote the paper, and is guarantor. CS managed the data and provided the graphs. MB reviewed the analyses. PJ wrote the paper

  • Funding NHS Executive research and development project grant number SPGS738.

  • Competing interests JP is a trustee of Constructive Dialogue for Clinical Accountability (CDCA) and a member of St George's Mortality Monitoring Group and is funded by St George's Healthcare NHS Trust. PJ is medical director of St George's Healthcare NHS Trust and chairs the St George's Mortality Monitoring Group.

  • Ethical approval Not required.

  • Accepted 13 November 2003
View Full Text