BMJ  2005;330:1139-1142 (14 May), doi:10.1136/bmj.330.7500.1139

Education and debate

Building a framework for trust: critical event analysis of deaths in surgical care

¹ Department of Surgery and Molecular Oncology, University of Dundee, Dundee DD1 9SY, ² Institute for Cardiovascular Research, Ninewells Hospital and Medical School, Dundee DD1 9SY

The British public's confidence in doctors and hospitals has been dented in recent years. Use of an independent review of deaths before, during, or after surgery reflects an attempt to improve care in this area and may also help to restore the public's trust in their health service

Introduction

Events over recent years have undermined patients' and society's trust in clinicians and healthcare institutions. For clinicians to restore and retain public confidence, they need to show that effective mechanisms exist for assessing events such as death and to justify patients' faith in the delivery of care.¹

In the high profile world of cardiac surgery, the audited results of key elective procedures may be sufficient to restore confidence.² In wider surgical practice, simple death rates are unlikely to be sufficient; case-mix (particularly for emergency admissions), institutional, and national issues can jeopardise the interests of high risk patients.³

Any critical incident review by peers working in the same speciality or subspecialty of patients who die under surgical care should take into account the nature of the patient and the circumstances of admission. In Scotland, the Scottish Audit of Surgical Mortality (SASM) is a national system of peer review of deaths that occur under surgical care that has been in place since 1994.⁴ Full details of how SASM works, and the annual reports, are on its website (www.sasm.org.uk).

Assessment process

SASM uses the consultant surgeon responsible for the care of the deceased patient as a proxy for the secondary care system and includes all deaths under surgical care in Scotland (including the private sector) except for cardiac surgery and obstetric deaths, which are currently audited separately. SASM assesses the patient's last journey of care, not simply the role of a single clinician or clinical team.

The method of case assessment and the review process (fig 1) are now well established In brief, deaths occurring within 30 days of an operation or during the patient's last admission are identified by using multiple sources of case ascertainment (including mortuaries and ward secretaries), which compare favourably with the central register of deaths. Data are collected via validated general, orthopaedic, neurosurgical, and anaesthetic proformas. The proformas are anonymously assessed by a surgical assessor (and by an anaesthetic assessor if an anaesthetic has been given). Assessors work in the same specialty but a different region; guidance notes for assessors, assessors' forms, and hypothetical feedback letters are on SASM's website.

View larger version (36K): [in this window] [in a new window]   Fig 1 The pathway of data gathering and assessment starts with a surgical death and proceeds through the critical event pathway. Where there is a cause for consideration or concern, an adverse event may be identified and a case-note review requested. For 10% of the deaths where no adverse event is identified, the case is reassessed. The assessments and review are fed back to the individual clinician, who has the right to reply or request a further review, leading to a final feedback to the clinicians involved in the care of the patient before death

 
Each surgeon receives individual feedback on every death assessed (including a detailed case-note based assessment if adverse events have occurred in the management of the patient) within 79 days (mean time in 2003). In addition, collated patient histories with a message for the wider anaesthetic and surgical community are mailed at regular intervals to all surgeons, anaesthetists, and trainees in Scotland. As a quality assurance measure, 1 in 10 cases (selected at random) are reviewed by a second independent assessor; in 2004, there was 98% concordance with the first assessor. The peer reviewers also individually assess a spectrum of real cases from the previous year and meet annually for group review of this portfolio to confirm the reliability of the process.

The requirement for clinical ownership and leadership to facilitate change⁵ is satisfied in SASM by the use of a tiered structure, including senior members of the relevant surgical colleges, professional staff, and lay representation (box). This process is to ensure that the quality of data collected to monitor clinical care is of high standard⁶. Compared with other surgical audits, SASM's methods seem to be more robust than most.⁷ However, changes in computer database systems, financial sustainability, the development of the adverse events coding system, and the new UK Freedom of Information Act continue to present challenges. The Central Legal Office in Scotland has assessed the SASM's process as sound (see SASM's website).

High compliance

SASM was started by surgeons and anaesthetists, is supported by professional bodies, is clinically led, and remains a voluntary process. This bottom-up culture of reflection rather than blame has led to a belief by the profession that the process is fair and open, with peer pressure to participate. Surgeons and anaesthetists participating in SASM to the required level (85%) of compliance are listed in the published annual report. Those who fall below the compliance threshold receive a letter from the chairman of the board to encourage them to participate more actively with SASM, an obligation commensurate with good clinical practice.⁸

Although in 2003 some 4084 out of 4478 deaths (91%) were audited, only 14 surgeons out of over 1200 surgeons and anaesthetists failed to comply to the required level. Neither an analysis of adverse events versus the number of completed proformas, nor additional peer review of patients selected at random, has identified selective reporting of events. Indeed, failure to retrieve case notes seems to be a common reason for incomplete compliance. With a substantially higher completion rate than most large clinical audits,⁹ there remains little evidence that making SASM compulsory would increase compliance further,¹⁰ but such a move would alter the basis of the relationship between the audit and the clinicians.

Extracted data analysis

The audit has reviewed some 45 000 deaths under surgical care in the past 10 years at a current cost per death of £59 ($113; 86). A national annual report collates the data in summary form. The reports consistently show the predominance of emergency cases (about 90%)—and in only half of these cases does the patient have surgery—with general and vascular surgery, followed by orthopaedics, accounting for most deaths. In recent years, some 10% of cases have gone on to case-note review; and despite 60% of case-note reviews failing to identify adverse aspects of care, this review rate has recently increased.¹¹

View larger version (39K): [in this window] [in a new window]   Fig 2 Percentage of operations at which consultants have an input in various ways, 1994 to 2003

 

Structure of the Scottish Audit of Surgical Mortality (SASM) SASM's board is chaired in rotation by one of the presidents of the surgical royal colleges in Scotland, with professional and lay representation A management team, elected from the participants, runs the day to day business of SASM The Liaison Group comprises specialist society, trainee, and non-consultant grade representatives An Assessors' Forum (of 128 assessors) comprises surgeons and anaesthetists who practise in the same field with the same problems and issues

Analyses of individual areas of surgery have included mortality from colorectal cancer,¹² deaths in patients with a hernia,¹³ deaths after endoscopy¹⁴ and deep venous thrombosis prophylaxis.¹⁵ These analyses provide insight into surgical practice in the NHS in Scotland.

Trends over a decade

Over the 10 years since SASM was started, the annual reports show an elderly population (median age at death has risen from 75 to 76), with a stable overall annual mortality of 4500 deaths. The percentage of deaths that occur after elective inpatient surgical admission has remained static (at 0.27%) in 2003; the proportion of deaths occurring after emergency surgical admission also remained static (at 2.2%).

Consultant input from both surgeons and anaesthetists in the decision making process and in the operating theatre (fig 2) has increased compared with historical series from the same¹² and other^{9 16} populations. Alongside this increase in consultant input, there was a decrease in the percentage of deaths in which adverse events in management contributed to or caused the death (figs 3 and 4). Whether the two are causally linked is unknown. Delay in transfer to surgical care, diagnosis missed by surgeons, and failure to use deep venous thrombosis prophylaxis have all declined as adverse events in the care of patients over the decade. Although the assessors' criticism in the choice of operation has changed little, criticism of the seniority of the surgeon operating and of delay in recognising postoperative complications has also fallen.

View larger version (18K): [in this window] [in a new window]   Fig 3 Percentage of deaths in surgical care for which adverse events in management were identified as contributing to death, 1994 to 2003

 

View larger version (18K): [in this window] [in a new window]   Fig 4 Percentage of deaths in surgical care for which adverse events in management were identified as causing death, 1994 to 2003

 
Although adverse events contributing to or causing death have declined over the past 10 years, and despite the fact that for the vast majority (94%) of patients who died in 2003 adverse events did not contribute to death—a figure comparable with other healthcare systems¹⁷—the media choose to focus on the 1 in 20 deaths in which such events contributed to death. Many of the adverse events identified by SASM relate to the hospital systems and the process of care. This contrasts with a perception that technical or intraoperative errors are common.^{18 19} The increase in "failure to use high dependency unit/intensive therapy unit" as an adverse event from 1994 to 1998 was highlighted in the annual reports and subsequently by the media. A working party recommended additional resources for improving existing high dependency and intensive therapy units in the NHS in Scotland and for commissioning such units in hospitals that did not already have them. As a result, this category of adverse event has declined markedly to 1% of cases since that time (fig 5).

View larger version (18K): [in this window] [in a new window]   Fig 5 Percentage of deaths for which "failure to use a high dependency unit/intensive care unit" was cited as an adverse event, 1994 to 2003

 

Failure to use deep venous thrombosis prophylaxis has also been highlighted by the SASM feedback. Together with the publication and regular updating of the Scottish Intercollegiate Guideline Network (SIGN) on deep venous thrombosis prophylaxis,²⁰ omission of deep venous thrombosis prophylaxis is now rarely identified (12/3907 (0.3%) deaths in 2003) as an adverse event for patients who die under surgical care. More recently, persistent concerns raised by SASM over postoperative care have resulted in SIGN producing consensus guidelines.²¹

What else might have accounted for the changes?
When SASM has heightened awareness and publicised concerns about areas of patient care, changes in practice have ensued and criticisms of adverse events have subsequently declined. However, other factors, such as SIGN guidelines, have undoubtedly contributed. Less rigorous peer review than a decade ago seems unlikely given that the proportion of cases undergoing case-note review has increased and the number of adverse events recorded as a consequence has not.¹¹

What next?

In the first half of 2005, surgeons in Scotland will receive a pilot "individual annual report" that contains a range of individual and comparative data. This document may be used for voluntary submission in the appraisal process¹¹ as it contains institutional and national data alongside the clinician data. Although this individual report is not designed to detect malpractice, clinicians with complete, peer reviewed individual reports are unlikely to be able to conceal avoidable deaths. The report should also empower clinicians during the two way appraisal process, which will highlight institutional responsibilities. As a clinical team, rather than a single doctor, may be responsible for a patient's care, the individual annual report may in time become a team document.

For the first 10 years of SASM, the consultant surgeon in charge of the patient was the point of contact but a consultant anaesthetist not involved in the case was not necessarily identifiable. Pressure from the consultant anaesthetists led to guidance from the Royal College of Anaesthetists in Scotland and the Association of Anaesthetists that from 2004 a named consultant anaesthetist will be recorded for all operations, thus allowing an individual annual report to be produced for anaesthetists.

The nursing profession will soon pilot a similar scheme to SASM in Tayside, Scotland, assessing nursing issues around the death of any patient on a surgical ward. Other areas of development include identifying patients who die outwith hospital but within 30 days of surgical care, which is currently undergoing a pilot study of SMR01 linkage.

Although the audit of non-fatal critical events is currently beyond the resources of SASM, a web based system of self reporting critical incidents, analogous to that used by pilots, may soon be available.

International perspectives

Healthcare systems in other countries have already adopted an audit process based on SASM. The West of Australia Audit of Surgical Mortality's first annual report was for 2002 (see www.waasm.uwa.edu.au). The report compared that region's data with those from SASM, and for many adverse events it identified a similar starting point to SASM's a decade ago. The SASM approach looks set to be rolled out elsewhere in the antipodes and will complement root cause analysis already in use.^{22 23} SASM, designed for review and reflection, but subject to the caveats of all peer review systems, may in future be used as a preliminary review process, with root cause analysis for further review of selected cases in both Australia and the United Kingdom. For future meaningful comparison between different healthcare systems, issues of coding taxonomy and adverse event codes will need to be resolved, and the same core questions will need to be asked.

Summary points Participation in a peer review critical event analysis of surgical mortality is a standard part of clinical practice in Scotland Voluntary mortality audit has been associated with important changes in practice, such as more consultant participation in decision making and care, a reduction in adverse events, and better care Highlighting adverse events in the process of care may have an influence in the future on all patients in hospital care

Balanced governance

The adverse events identified relate to the process of care, rather than individual clinicians making errors. Examining the process of care at hospital and health board level should have a greater impact in the medium to long term²⁴ than simply feeding the data back to individual surgeons and anaesthetists. Institutions should also be accountable for clinical governance and outcome, and the SASM process looks set to become involved in the concept of "balanced governance"—in which both clinicians and institutions are responsible for the care of patients. Until recently, hospital trusts and health boards seem to have shown little interest in individual hospital reports, which compare a given institution with others in Scotland. As many adverse events are concerned with process, managers are beginning to acknowledge the benefits of using high quality SASM data in balanced clinical governance. Perhaps it is also time for a similar process to start in other specialties, such as general medicine.

---

Editorial by Baxter, and p 1144

Both authors also hold positions on the management committee of the Scottish Audit of Surgical Mortality. We thank all participants in and contributors to the Scottish Audit of Surgical Mortality process; Z Ashaf, G McPhillips, and C Sproul for statistical analysis; H Burton for managing the data collection and storage; and Wendy Ritchie for secretarial support.

Contributors: AMT and PAS initiated the analysis and wrote the paper.

Funding: No special funding.

Competing interests: None declared.

References

1. Smith J. Demonstrating accountability, fostering trust [Editor's Choice]. BMJ 2004; 329. 21 August.
2. Keogh B, Spiegelhalter D, Bailey A, Roxburgh J, Magee P, Hilton C. The legacy of Bristol: public disclosure of individual surgeons' results. BMJ 2004;329: 450-4.[Free Full Text]
3. Schneider E, Epstein A. Influence of cardiac surgery performance reports on referral practices and access to care. New Engl J Med 1996;335: 251-6.[Abstract/Free Full Text]
4. Stonebridge PA, Thompson AM, Nixon SJ. Completion of the journey of care: Scottish Audit of Surgical Mortality (SASM). J R Coll Surg Edinb 1999;44: 185-6.[Medline]
5. Smith R. Is the NHS getting better or worse? BMJ 2003;327: 1239-41.[Free Full Text]
6. Donaldson LJ. Clinical governance: a statutory duty for quality improvement. J Epidemiol Community Health 1998;52: 73-4.[Medline]
7. Russell EM, Bruce J, Krukowski ZH. Systematic review of the quality of surgical mortality monitoring. Br J Surg 2003;90: 527-32.[Medline]
8. General Medical Council. Good medical practice. London: General Medical Council, 1998.
9. Gray CJG, Hoile RW, Ingram GS, Sherry KM. The report of the national confidential enquiry into perioperative deaths 1996/1997. London: National Confidential Enquiry into Perioperative Deaths, 1998.
10. Cohen MR. Why error reporting systems should be voluntary. BMJ 2000;320: 728-9.[Free Full Text]
11. Thompson AM, Ritchie W, Stonebridge PA. Could sequential, individual, peer reviewed mortality audit data be used for appraisal and revalidation? Br J Surg 2004;91(suppl 1): 146.[ISI][Medline]
12. Macarthur DC, Nixon SJ, Aitken RJ. Avoidable deaths still occur after large bowel surgery. Scottish Audit of Surgical Mortality, Royal College of Surgeons of Edinburgh. Br J Surg 1998;85: 80-3.[CrossRef][Medline]
13. McGugan E, Burton H, Nixon SJ, Thompson AM. Deaths following hernia: room for improvement. J R Coll Surg Edinb 2000;45: 183-6.[Medline]
14. Thompson AM, Wright DJ, Murray W, Ritchie GL, Burton HD, Stonebridge PA. Analysis of 153 deaths following upper gastrointestinal endoscopy: room for improvement? Surg Endosc 2004;18: 1257-62.[Medline]
15. Gillies TE, Ruckley CV, Nixon SJ. Still missing the boat with fatal pulmonary embolism. Br J Surg 1996;83: 1394-5.[ISI][Medline]
16. Inbasegaran K, Kandasami P, Sivalingam N. A 2 year audit of perioperative mortality in Malaysian hospitals. Med J Malaysia 1998;53: 321-4.[Medline]
17. Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The quality in Australian health care study. Med J Aust 1995;163: 458-71.[ISI][Medline]
18. Sarker SK. Courses, cadavers and counsellors: reducing errors in the operating theatre. BMJ 2003;327: S109.
19. Carter D. The surgeon as a risk factor. BMJ 2003;326: 832-3.[Free Full Text]
20. Scottish Intercollegiate Guidelines Network. SIGN guideline 62: prophylaxis of venous thromboembolism. A national clinical guideline. 2002. www.sign.ac.uk/pdf/sign62.pdf (accessed 15 March 2005).
21. Scottish Intercollegiate Guidelines Network. SIGN guideline 77: postoperative management in adults. A practical guide to postoperative care for clinical staff. 2004. www.sign.ac.uk/pdf/sign77.pdf (accessed 15 March 2005).
22. Vincent CA, Adams S, Stanhope N. A framework for the analysis of risk and safety in medicine. BMJ 1998;316: 1154-7.[Free Full Text]
23. Vincent C, Taylor-Adams S, Chapman EJ, Hewett D, Prior S, Strange P, et al. How to investigate and analyse clinical incidents: Clinical Risk Unit and Association of Litigation and Risk Management protocol. BMJ 2000;320: 777-81.[Free Full Text]
24. Reason J. Human error: models and management. BMJ 2000;320: 768-70.[Free Full Text]

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Scottish model for surgical mortality used in Australasia

Robert James Aitken
BMJ 2005 330: 1389-1390. [Extract] [Full Text]

Monitoring surgical mortality

Nancy N Baxter
BMJ 2005 330: 1098-1099. [Extract] [Full Text] [PDF]

Excellent review scheme for critical incidents but insufficient for revalidation

Mayur Lakhani
BMJ 2005 330: 1143. [Extract] [Full Text] [PDF]

Failure to act on good intentions

Aneez Esmail
BMJ 2005 330: 1144-1147. [Extract] [Full Text] [PDF]

The legacy of Bristol: public disclosure of individual surgeons' results

Bruce Keogh, David Spiegelhalter, Alan Bailey, James Roxburgh, Patrick Magee, and Colin Hilton
BMJ 2004 329: 450-454. [Extract] [Full Text] [PDF]

Is the NHS getting better or worse?

Richard Smith
BMJ 2003 327: 1239-1241. [Extract] [Full Text] [PDF]

The surgeon as a risk factor

David Carter
BMJ 2003 326: 832-833. [Extract] [Full Text] [PDF]

Why error reporting systems should be voluntary

Michael R Cohen
BMJ 2000 320: 728-729. [Extract] [Full Text] [PDF]

Human error: models and management

James Reason
BMJ 2000 320: 768-770. [Extract] [Full Text] [PDF]

How to investigate and analyse clinical incidents: Clinical Risk Unit and Association of Litigation and Risk Management protocol

Charles Vincent, Sally Taylor-Adams, E Jane Chapman, David Hewett, Sue Prior, Pam Strange, and Ann Tizzard
BMJ 2000 320: 777-781. [Extract] [Full Text] [PDF]

Framework for analysing risk and safety in clinical medicine

Charles Vincent, Sally Taylor-Adams, and Nicola Stanhope
BMJ 1998 316: 1154-1157. [Extract] [Full Text]

This article has been cited by other articles:

• Guru, V., Tu, J. V., Etchells, E., Anderson, G. M., Naylor, C. D., Novick, R. J., Feindel, C. M., Rubens, F. D., Teoh, K., Mathur, A., Hamilton, A., Bonneau, D., Cutrara, C., Austin, P. C., Fremes, S. E. (2008). Relationship Between Preventability of Death After Coronary Artery Bypass Graft Surgery and All-Cause Risk-Adjusted Mortality Rates. Circulation 117: 2969-2976 [Abstract] [Full text]  
• Aitken, R. J. (2005). Scottish model for surgical mortality used in Australasia. BMJ 330: 1389-1390 [Full text]  
• Baxter, N. N (2005). Monitoring surgical mortality. BMJ 330: 1098-1099 [Full text]  
• Lakhani, M. (2005). Excellent review scheme for critical incidents but insufficient for revalidation. BMJ 330: 1143-1143 [Full text]  

Rapid Responses:

Read all Rapid Responses

Western Australian Audit of Surgical Mortality

Robert James Aitken
bmj.com, 16 May 2005 [Full text]

Good work, but wrong goal

Neville W Goodman
bmj.com, 20 May 2005 [Full text]

SASM unsuitable for revalidation

Ian G Kestin
bmj.com, 28 May 2005 [Full text]