Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Western Australian Audit of Surgical Mortality
- Robert James Aitken (16 May 2005)
-
Good work, but wrong goal
- Neville W Goodman (20 May 2005)
-
SASM unsuitable for revalidation
- Ian G Kestin (28 May 2005)
|
|
|||
|
Robert James Aitken, Consultant, Sir Charles Gairdner Hospital Western Australia
Send response to journal:
|
Two days before the BMJ published Baxter's editorial expressing doubts as to whether the Scottish Audit of Surgical Mortality (SASM) was applicable elsewhere the Western Australian Audit of Surgical Mortality (WAASM) published its second annual report <http://www.surgeons.org/cgi-bin/redirector.cgi>. WAASM, which is based on SASM, commenced in 2001. It has has already shown clear changes in local practice. This despite a very different surgical environment. For example, unlike the NHS a majority of surgical operations in Australia are undertaken in the private sector. Current participation (96% of surgeons submitted 60% of all deaths) is not as complete as Scotland. This needs to be considered against the very different background of a county that, unlike Scotland, did not have an established culture of regional surgical audit. A particular problem at the outset was the very aggressive medical legal environment in Australia. Qualified privilege was an essential prerequisite. The principal area of public and media interest was the level of surgeons participation. The clear expectation is that surgeons participate in the audit process. Little attention was directed to the adverse events themselves, The Royal Australasian College of Surgeons has announced its intention of establishing the Australian and New Zealand Audit of Surgical Mortality. This will be based on WAASM methodology, which although now modified for local practice, retains the concepts of SASM at it core. RJ Aitken Chairman, WA Audit of Surgical Mortality Competing interests: None declared |
|||
|
|
|||
|
Neville W Goodman, Consultant Anaesthetist Southmead Hospital, Bristol, BS10 5NB
Send response to journal:
|
Thompson and Stonebridge, or perhaps the sub-editors at the BMJ, headlined this article: 'The British public's confidence in doctors and hospitals has been dented in recent years. Use of an independent review of deaths before, during, or after surgery reflects an attempt to improve care in this area and may also help to restore the public's trust in their health service.' While not denying the current attempts to develop robust ways of measuring medical outcomes, imagining they will improve the 'public's trust' is misguided. The public trust the health service; it is the politicians and media who misuse this perceived distrust for their own purposes: the politicians are jealous of the trust placed in doctors; the media need to sell their product, which makes bad news good news. We don't need surrogate reasons. We need to develop ways of measuring medical outcome because otherwise we don't know whether what we are doing is worth doing, or is being done properly. However, in much of medicine, the measurement may be too difficult. You can be sure that if we sort out the easier outomes, such as deaths after surgery, the politicians and media will taunt us with the more difficult ones. Competing interests: None declared |
|||
|
|
|||
|
Ian G Kestin, Consultant anaesthetist Department of Anaesthesia, 30 Shelley Court, Gartnavel Hospital, Glasgow G12 0XD
Send response to journal:
|
The Scottish Audit of Surgical Mortality has many serious flaws that make it quite unsuitable for any use in revalidation (1). Correctly labelled in the accompanying commentary as a survey of critical incidents (2), large national projects like SASM and the National Confidential Enquiry into Post-operative Deaths (NCEPOD) have had considerable value in determining institutional factors associated with avoidable deaths, like deficiencies in Recovery, intensive care and high dependency facilities. It is quite another matter to extend these projects into areas for which they are not designed or suited. Even as a peer reviewed series of surgical deaths, SASM has serious flaws. The review process is inadequate. Each operative case is reviewed by a surgeon and an anaesthetist, which means there is essentially only one reviewer for each specialty. The reviewers are encouraged but not obliged to support their conclusions with published evidence (3). Without supporting evidence, the reviewer’s conclusions are no more than personal opinion. A random 10% of reviews are subject to a second independent assessment as a quality assurance audit, and there is a high level of agreement of these two reviews. As has been pointed out before in this context (4), good inter-rater agreement is no guarantee that these judgements are correct. Lagasse provides a good discussion of how to set up a well structured and reliable peer review process (5). The forms completed by the anaesthetist seek the judgements and opinions of the responsible anaesthetist, and the other information collected would not enable the reviewer to make an independent judgement if there were any anaesthetic deficiencies. To rely on the insight of the responsible clinician to identify deficiencies in the care means that there must be an unknown number of deaths that have areas of concern and are not detected by SASM. The forms should capture data only, not the opinions of those who performed the procedure. It is the responsibility of the reviewers to determine the areas of concern, not the clinicians to point them out. Normal doctors all make errors and misjudgements, and in medicine, we have not begun to accept the implications of this, If tomorrow, during a difficult case, I am distracted and connect the epidural infusion of local anaesthetics to the central venous catheter instead of the epidural catheter, the patient may well die. Critical comment from SASM will be the least of my problems as I will probably be suspended, referred to the GMC and may well be charged with manslaughter. These slip-lapse errors are routine, and although it may be the only error I make in my entire career, it will be treated essentially as a criminal act at all levels within medicine and the law. We persist with the paradigm that good doctors do not make mistakes and only bad doctors make errors, and projects like SASM essentially perpetuate this concept. Because we do not know ‘normal’ error and misjudgement rates, using SASM process for revalidation is quite wrong. Measuring the quality of an individual’s practice is difficult, and only the cardiac surgeons have developed any sort of reliable methods. Robust statistical measures of process control like cusum analysis (6) or Mahalanobis distance (7) are needed. These are not well understood but the only valid way to measure an individuals performance, and separate the good practitioner who has had a bad run from a poor practitioner who has had a good run. One of the lessons from the statistical enquiry at the Bristol Inquiry was that there were episodes of good performance within a background of overall poor performance. For this reason alone, a ‘good’ report from SASM can never be taken as evidence of good clinical performance, and a ‘bad’ report cannot be taken as evidence of poor performance of an individual. The limitations of the SASM type approach can be seen in comparison with the recent Dutch study using multivariate analyses of the factors influencing anaesthetic morbidity and mortality (8). This study indicates appropriate data that could be used to estimate the quality of an individuals' practice, and SASM should be developing this type of monitoring rather than trying to justify using existing methods just because they exist. 1 Thompson A M, Stonebridge P A. Building a framework for trust: critical event analysis of deaths in surgical care. BMJ 2005;330:1139- 1142. 2 Lakhani M. Commentary: Excellent review scheme for critical incidents but insufficient for revalidation. BMJ 2005:330:1143. 3 Scottish Audit of Surgical Mortality. Guidelines for assessors. http://www.sasm.org.uk/Process/Assessors%20guidelines.htm (accessed 24th May 2005). 4 Keats AS : Mortality in perspective. Anesth Analg 1990 ;71: 113–9. 5 Lagasse RS - Anesthesia safety: model or myth? A review of the published literature and analysis of current original data Anesthesiology 2002;97:1609-17. 6 Poloniecki, J, Sismanidis C, Bland M, Jones P. Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups BMJ 2004;328:361-2. 7 Rousseeuw PJ, Leroy AM. Robust regression and outlier detection. New York: Wiley, 1987. 8 Arbous MS, Meursing AEE, van Kleef JW, de Lange JJ , Spoormans HHAJM, Touw P et al. Impact of anesthesia management characteristics on severe morbidity and mortality. Anesthesiology 2005;102:257-268. Competing interests: None declared |
|||