Intended for healthcare professionals

Research

Cumulative funnel plots for the early detection of interoperator variation: retrospective database analysis of observed versus predicted results of percutaneous coronary intervention

BMJ 2008; 336 doi: https://doi.org/10.1136/bmj.39512.529120.BE (Published 24 April 2008) Cite this as: BMJ 2008;336:931
  1. Babu Kunadian, research fellow,
  2. Joel Dunning, specialist registrar in cardiothoracic surgery,
  3. Anthony P Roberts, clinical effectiveness specialist adviser,
  4. Robert Morley, clinical audit lead,
  5. Darragh Twomey, clinical teaching fellow,
  6. James A Hall, consultant cardiologist,
  7. Andrew G C Sutton, consultant cardiologist,
  8. Robert A Wright, consultant cardiologist,
  9. Douglas F Muir, consultant cardiologist,
  10. Mark A de Belder, consultant cardiologist
  1. 1Department of Cardiology, James Cook University Hospital, Middlesbrough TS4 3BW
  1. Correspondence to: M A de Belder mark.debelder{at}stees.nhs.uk
  • Accepted 24 February 2008

Abstract

Objective To use funnel plots and cumulative funnel plots to compare in-hospital outcome data for operators undertaking percutaneous coronary interventions with predicted results derived from a validated risk score to allow for early detection of variation in performance.

Design Analysis of prospectively collected data.

Setting Tertiary centre NHS hospital in the north east of England.

Participants Five cardiologists carrying out percutaneous coronary interventions between January 2003 and December 2006.

Main outcome measures In-hospital major adverse cardiovascular and cerebrovascular events (in-hospital death, Q wave myocardial infarction, emergency coronary artery bypass graft surgery, and cerebrovascular accident) analysed against the logistic north west quality improvement programme predicted risk, for each operator. Results are displayed as funnel plots summarising overall performance for each operator and cumulative funnel plots for an individual operator’s performance on a case series basis.

Results The funnel plots for 5198 patients undergoing percutaneous coronary interventions showed an average observed rate for major adverse cardiovascular and cerebrovascular events of 1.96% overall. This was below the predicted risk of 2.06% by the logistic north west quality improvement programme risk score. Rates of in-hospital major adverse cardiovascular and cerebrovascular events for all operators were within the 3σ upper control limit of 2.75% and 2σ upper warning limit of 2.49%.

Conclusion The overall in-hospital major adverse cardiovascular and cerebrovascular events rates were under the predicted event rate. In-hospital rates after percutaneous coronary intervention procedure can be monitored successfully using funnel and cumulative funnel plots with 3σ control limits to display and publish each operator’s outcomes. The upper warning limit (2σ control limit) could be used for internal monitoring. The main advantage of these charts is their transparency, as they show observed and predicted events separately. By this approach individual operators can monitor their own performance, using the predicted risk for their patients but in a way that is compatible with benchmarking to colleagues, encapsulated by the funnel plot. This methodology is applicable regardless of variations in individual operator case volume and case mix.

Footnotes

  • Contributors: BK, JD, MdeB, and APR designed the study. JAH, AGCS, RAW, DFM, and MdeB carried out the procedures and collected the in-lab data. BK, DT, JD, and RM ensured completeness of the database and did the analysis with support from PR. BK, MdeB, and PR led on the writing of the manuscript but all authors contributed to the submitted versions of the manuscript. MdeB is the guarantor for the study.

  • Funding: None.

  • Competing interests: BK, JAH, AGCS, RAW, DFM, and MdeB have received travel grants from manufacturers of coronary stents and percutaneous coronary intervention related pharmaceutical companies. MdeB has sat on advisory boards for stent manufacturers and percutaneous coronary intervention related pharmaceutical companies and has received research grants from a few stent manufacturers. As members or fellows of the Royal College of Physicians we have an interest in methods of revalidation and may be involved in standard setting. As, respectively, president of the British Cardiovascular Intervention Society and council member of the British Cardiovascular Society, MdeB and JAH will have a role in informing the debate about these issues. The authors do not believe that any of these declarations constitute a conflict of interest as regards this study.

  • Ethical approval: Not required.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

View Full Text