Is WHO’s surgical safety checklist being hyped?
BMJ 2019; 366 doi: https://doi.org/10.1136/bmj.l4700 (Published 05 August 2019) Cite this as: BMJ 2019;366:l4700
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Dear Sir/Madam
The WHO surgical list has at least changed team behaviour in the UK NHS hospitals and empowered the team members to speak. But a significant risk of information governance failure has cropped up because of the change in culture in the way we do the check list.
In the past individual patients were discussed before the start of the procedure and it was easy to do the WHO Checklist as clinical discussions could be done while referring to the anaesthetic charts and patient notes. But there has been a gradual shift in almost most places to discuss about all the patients for that session. This has brought a cultural change again amongst the anaesthetists who still rely on a paper based system and rushing from one of the hospital to other end they are scribbling patient related information on their theatre list so that multiple patients can be discussed at the WHO. Although we all act responsibly but it is only time when a list with information about all the patients is misplaced or finds its way in the hands of wrong person.
Keeping this in mind, we may find that doing a WHO for individual patient rather than all the patients,minimises the risk faced by the nhs trusts to a potential information governance failure.
Yours Sincerely
Dr B Saha
Competing interests: No competing interests
One may argue about the finer points of the WHO surgical safety checklist. It may have some faults, it may be improved. But it allows all of the theatre staff to be prepared. Because the alternative is that poor preparation promotes poor performance.
There may be no clear evidence to support the WHO surgical safety checklist. But consider this. In 1908 Lieut General Baden Powell CB came up with "Be Prepared" as the motto for the Boy Scout movement. It has served the movement valiantly for the last 111 years. With no need to change it, despite being in an ever changing world.
Surgical safety checklists allow the theatre team to be prepared. Surgical safety checklists are here to stay. And here's why. The idea that in the 21st Century a surgeon can turn up unprepared to an operating session is simply inconceivable. And reckless.
Competing interests: No competing interests
Shamelessly quoting from my own paper that addresses the frustrating oversimplification of the "checklist" narrative.
"There is no question that the right checklist, in the right place, with the right design and implementation, can be used enthusiastically by the right people with the right skills and can be highly effective. Yet, in translating checklists from aviation and other industries to healthcare, we may have misunderstood their strengths, failed to design them based on well established principles and failed to engineer them as a component of a wider socio-technical system. We have made assumptions about their use, effectiveness and ‘evidence base’ that are readily and easily challenged, and have defined compliance criteria and penalties based on assumptions that may not reflect how they contribute to better outcomes. The superimposition of teamwork and communication— without specifically providing training for those skills, or indeed the sociocultural support for them— further contributes to the difficulties in successful implementation.
A checklist is a complex socio-technical intervention that requires careful attention to design, implementation and basic skills required for the task. Understanding and specifying these mechanisms of effect with greater precision would enable us to move beyond the moot ‘checklists do/don’t work’ commentaries."
- Catchpole K, Russ S. BMJ Qual Saf 2015;24: 545–549.
Competing interests: No competing interests
The real benefit of the checklists should have been specifically emphasized by WHO as improving the culture of safety. Perhaps it was taken for granted and projected as a significant reducer of mortality (for stunning effects) and morbidity.
There are more videos on the web about 'How not to use the Checklist' than the real purpose or accruable benefits.
The before and after studies were inappropriately taken as such and in some centres were discontinued after the study!!!
The authors, while scholarly, demolish the end point mortality metrics, but strongly advocate its usage for the right purpose of using the tool to improve the culture - a prerequisite for safer healthcare. The same theme is visible in the repudiating responses from authors of quoted studies.
I wish to reiterate and request the editor as well as all enlightened authors / authorities to emphasize it as a helpful physical tool to improve the culture - a seemingly nebulous but a necessary prerequisite. The tool can help improve Communication, Collaboration and Coordination - the pillars of culture. The greatest benefit would be to rebuild TRUST in healthcare - perhaps the greatest need today worldwide.
Competing interests: No competing interests
Re: Is WHO’s surgical safety checklist being hyped?
Arvid Steinar Haugen1, Stig Harthug2,3, Nick Sevdalis4, Eirik Søfteland1,5
1Department of Anaesthesia and Intensive Care, Haukeland University Hospital, Bergen, Norway; 2Department of Research and Development, Haukeland University Hospital, Bergen, Norway; 3Department of Clinical Science, University of Bergen, Bergen, Norway; 4Centre for Implementation Science, Health Service & Population Research Department, King’s College London, United Kingdom; 5Department of Clinical Medicine, University of Bergen, Bergen, Norway.
It’s not miraculous, it’s a no-brainer
The scientific evidence of the World Health Organisation’s surgical safety checklist is not about pseudoscientific or paranormal phenomena as Urbach & Dimik imply. In all types of research, it is important not to over or under interpret your findings, one needs to be critical and sceptic when reviewing, but balanced and based on the best available methodologies possible to address the problem. In their scepticism and reasoning against studies that report positive checklist effects, there are however few traces of such a balance [1].
First, Urbach & Dimick refer to the one published randomized study of the checklists; a stepped wedge cluster randomized controlled trial that was carried out in our hospitals [2]. It is correct that we did not find an overall reduction of mortality, only a significant reduction of mortality in one of the hospitals. We also observed a significant reduction of morbidity. Urbach & Dimick’s statement that our study did not report the intention-to-treat group in the ‘control-arm’ is directly incorrect. The numbers are laid out in Table 2, with an absolute risk reduction of complications in the intention to treat group at 7.5 (95% confidence interval, 5.5 to 9.5), and even higher absolute risk reduction when using all three parts of the checklist 8.4 (95% confidence interval, 6.3 to 10.5) [2]. Their statement seems not balanced on this point.
Second, they state that there was a lack of blinding in our study. Well, it is indeed true that the healthcare staff was not blinded for using the checklist, but no checklist study could possibly be designed for that. Using a cluster RCT design is more robust than a pre- and post-study [3], but labelling this as pseudoscience by Urbach & Dimick presents itself more as ‘click bate’ reasoning than a scientific balanced viewpoint. In fact, as far as possible, healthcare workers performing the checklist were in our study blinded for the study outcomes, data assessors were blinded for the checklist use and post-operative staff was blinded for the use of checklist and outcome [2].
Third, why did we find such a large effect on complications when the checklist had been used? We followed Avedis Donabedian’s hypothesis for quality improvement: structure improves process, and improved processes (and structure) provide better outcome. In a causal analysis of the checklist impact on care processes and patient outcomes, we found that the checklist improved patient care with more use of warm fluids, forced air warming blankets, and better timing of antibiotics. These improved care processes were again associated directly with improved patient outcome [4]. The checklist effect on complications is not caused by magic or a miraculous phenomena, it’s a no-brainer. The effect is caused by the actions the checklist ask for. If the checklist does not induce change in actions or improvement of care, better outcomes could not be obtained [5].
We support the statement from Urbach & Dimick that checklists have real and meaningful benefits and strongly encourage the use of them, which requires engagement from perioperative nurses, surgeons, and anaesthetic personnel, and not at least managers. In line with Haynes & Gawande's points, we endorse having focus on implementation approaches and team communication.
References
1. Urbach DR, Dimick JB, Haynes AB, et al. Is WHO’s surgical safety checklist being hyped? British Medical Journal 2019;366:l4700. doi: 10.1136/bmj.l4700
2. Haugen AS, Søfteland E, Almeland SK, et al. Effect of the World Health Organization Checklist on Patient Outcomes: A Stepped Wedge Cluster Randomized Controlled Trial. Annals of Surgery 2015;261(5):821-28. doi: 10.1097/sla.0000000000000716
3. Hemming K, Haines TP, Chilton PJ, et al. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. British Medical Journal 2015;350:h391. doi: 10.1136/bmj.h391
4. Haugen AS, Wæhle HV, Almeland SK, et al. Causal Analysis of World Health Organization's Surgical Safety Checklist Implementation Quality and Impact on Care Processes and Patient Outcomes: Secondary Analysis From a Large Stepped Wedge Cluster Randomized Controlled Trial in Norway. 2019;269(2):283-90. doi: 10.1097/sla.0000000000002584
5. Leape LL. The Checklist Conundrum. N Engl J Med 2014;370(11):1063-64. doi: doi:10.1056/NEJMe1315851
Competing interests: Arvid Steinar Haugen received postdoctoral grant from the Western Norwegian Regional Health Authority with grant number: HV1172. He represent the International Federation of Nurse Anaesthetists in the European Society of Anaesthesiologists' Patient Safety & Quality Committee. Nick Sevdalis research is funded by the NIHR via the ‘Collaboration for Leadership in Applied Health Research and Care South London’ at King's College Hospital NHS Foundation Trust, London, UK. NS is also a member of King’s Improvement Science, which is part of the NIHR CLAHRC South London and comprises a specialist team of improvement scientists and senior researchers based at King’s College London. Its work is funded by King’s Health Partners (Guy’s and St Thomas’ NHS Foundation Trust, King’s College Hospital NHS Foundation Trust, King’s College London and South London and Maudsley NHS Foundation Trust), Guy’s and St Thomas’ Charity, the Maudsley Charity and the Health Foundation. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. SH had a role as lead of the Scientific Advisory Board appointed by the Norwegian Directorate of Health 2011-2013. The Directorate had no role in planning the design, conduct, or analysis of this study. The funders had no role in the design, conduct, or analysis of this study. NS is also the Director of London Safety and Training Solutions Ltd, which provides quality and safety training and advisory services on a consultancy basis to healthcare organization globally. The other authors report no conflicts of interest.
Safety checklists, believing in hype and lessons from the Tour de France
Urbach and Dimick question the miraculous results the WHO's surgical safety checklist have been credited with. I enjoyed the reference to the famed astronomer, Carl Sagan, that extraordinary claims require extraordinary evidence. I wonder if the authors are aware this quote was often used by Lance Armstrong, in the last days of his empire before his doping was exposed, to repel brave investigative journalists who dared question his incredible dominance of the Tour de France. The supposed mechanism of the associated reduction in mortality, ensuring that the team members, patient, and procedure are properly identified and confirming that the team has contemplated several processes of care should not be taken for granted.
Urbach and Dimick are correct in challenging the causal relationship and quality of some of the evidence. They question the plausibility of the effect considering that most of these pre-existing quality improvement processes had not been proven to reduce surgical mortality when applied individually. This brings us back to the Tour de France. The checklist has galvanised all the individual processes and provided a tangible focal point for all staff and like the philosophy behind the recent dominance of the British Tour Winners Sir Bradley Wiggins, Chris Froome and Geraint Thomas, the principle of the aggregation of marginal gains has been demonstrated with spectacular effect for surgical safety.
I agree with Haynes and Gawande that the checklist is a powerful tool and furthermore it serves as a useful surrogate marker of quality in surgical teams and institutions. A recent study continues the trend, where use of the checklist was associated with reduced peri-operative mortality, especially in low and middle Human Developmental Index countries. [1] Sceptisicm for causal relationships that seem to good to be true are always welcome in surgical outcomes research. However, I encourage the Yes side to embrace the wealth of extraordinary evidence behind the checklist. To paraphrase what Mr Armstrong boldly said to his growing doubters after his 7th and final Tour victory [2] ' the last thing I'll say for the people that don't believe in the checklist, the cycnics and the sceptics, I'm sorry you can't dream big and I'm sorry you don't believe in miracles.' Unlike the doping years of the Tour de France, let's hope the legacy of the checklist endures.
[1] Pooled analysis of WHO Surgical Safety Checklist use and mortality after emergency laparotomy.Br J Surg. 2019 Jan;106(2)
[2] https://www.theguardian.com/sport/2012/jun/16/lance-armstrong-drugs-tour...
Competing interests: I was a collaborator and data validator on GlobalSurg projects which included use of the WHO checklist. I also came near to last place in the 2019 Etape du Tour.
Re: Is WHO’s surgical safety checklist being hyped?
Dear Editor
When I read the article on surgical checklists, the first question that springs to mind is this: does anyone seriously suggest, on the available data, that checklists are working to increase mortality and complication rates (i.e. that they are doing harm)?
The same question arose when I saw criticism of our compulsory cycle helmet laws, and of our aggressive advertising campaigns against smoking/tobacco.
There is no doubt that checklists could be improved, and that a "one size fits all" approach will work poorly in some contexts. It's also true (in that context) that an ill-conceived application (ill-conceived in a given context) could easily see the whole approach treated with derision, which is not what we want to see.
It's simply unreasonable to suggest that we can answer such a sweeping question (Are checklists useful?) in a manner that deals with all operations, in all clinical scenarios, for all conditions, and performed by a variety of different surgical teams. If there is one major criticism of the EBM movement, it would be that it reinforces the dichotomous thinking that everything will either be proven, or proven wrong, which grows from the way that so much of the medical/scientific literature is published.
Far more intelligent and useful to examine various circumstances separately, and to ask what is working and what is not working in each one, and then to go on to tailor a more and more effective intervention for each circumstance.
A final comment, and one that we should have heard before in discussing the role of cancer MDTs (the "evidence base" isn't strong, although I declare that I didn't re-review the whole literature before making that comment) is that the teams and hospitals most likely to implement these interventions early and then to do research on them and their implementation are (intuitively) the teams and hospitals less likely to need that intervention. Their results may not generalise to the whole world!
Kind Regards
Sean Mackay MBBS MD FRACS
Competing interests: No competing interests