Multiple component patient safety intervention in English hospitals: controlled evaluation of second phaseBMJ 2011; 342 doi: https://doi.org/10.1136/bmj.d199 (Published 03 February 2011) Cite this as: BMJ 2011;342:d199
- Amirta Benning, programme manager1,
- Mary Dixon-Woods, professor of medical sociology2,
- Ugochi Nwulu, senior research associate/coordinator3,
- Maisoon Ghaleb, lecturer in pharmacy practice/patient safety45,
- Jeremy Dawson, research fellow6,
- Nick Barber, professor of the practice of pharmacy4,
- Bryony Dean Franklin, professor of medication safety and director, centre for medication safety and service quality47,
- Alan Girling, senior research fellow1,
- Karla Hemming, senior research fellow1,
- Martin Carmalt, consultant physician8,
- Gavin Rudge, data scientist1,
- Thirumalai Naicker, honorary research associate1,
- Amit Kotecha, registrar8,
- M Clare Derrington, independent contractor, public health physician9,
- Richard Lilford, professor of clinical epidemiology1
- 1School of Health and Population Sciences, University of Birmingham, Edgbaston, West Midlands B15 2TT, UK
- 2Department of Health Sciences, University of Leicester, Leicester LE1 7RH
- 3Clinical Investigation Unit, University Hospitals Birmingham NHS Foundation Trust, Queen Elizabeth Hospital, Birmingham B15 2TH
- 4Department of Practice and Policy, School of Pharmacy, University of London, London WC1N 1AX
- 5School of Pharmacy, University of Hertfordshire, Hatfield AL10 9AB
- 6Work and Organisational Psychology Group, Aston Business School, Aston University, Birmingham B4 7ET
- 7Imperial College Healthcare NHS Trust, St Mary’s Hospital, London W2 1NY
- 8Royal Orthopaedic Hospital, Northfield, Birmingham B31 2AP
- 945 Weston Road, Edith Weston, Rutland LE15 8HQ
- Correspondence to: R J Lilford
- Accepted 12 October 2010
Objective To independently evaluate the impact of the second phase of the Health Foundation’s Safer Patients Initiative (SPI2) on a range of patient safety measures.
Design A controlled before and after design. Five substudies: survey of staff attitudes; review of case notes from high risk (respiratory) patients in medical wards; review of case notes from surgical patients; indirect evaluation of hand hygiene by measuring hospital use of handwashing materials; measurement of outcomes (adverse events, mortality among high risk patients admitted to medical wards, patients’ satisfaction, mortality in intensive care, rates of hospital acquired infection).
Setting NHS hospitals in England.
Participants Nine hospitals participating in SPI2 and nine matched control hospitals.
Intervention The SPI2 intervention was similar to the SPI1, with somewhat modified goals, a slightly longer intervention period, and a smaller budget per hospital.
Results One of the scores (organisational climate) showed a significant (P=0.009) difference in rate of change over time, which favoured the control hospitals, though the difference was only 0.07 points on a five point scale. Results of the explicit case note reviews of high risk medical patients showed that certain practices improved over time in both control and SPI2 hospitals (and none deteriorated), but there were no significant differences between control and SPI2 hospitals. Monitoring of vital signs improved across control and SPI2 sites. This temporal effect was significant for monitoring the respiratory rate at both the six hour (adjusted odds ratio 2.1, 99% confidence interval 1.0 to 4.3; P=0.010) and 12 hour (2.4, 1.1 to 5.0; P=0.002) periods after admission. There was no significant effect of SPI for any of the measures of vital signs. Use of a recommended system for scoring the severity of pneumonia improved from 1.9% (1/52) to 21.4% (12/56) of control and from 2.0% (1/50) to 41.7% (25/60) of SPI2 patients. This temporal change was significant (7.3, 1.4 to 37.7; P=0.002), but the difference in difference was not significant (2.1, 0.4 to 11.1; P=0.236). There were no notable or significant changes in the pattern of prescribing errors, either over time or between control and SPI2 hospitals. Two items of medical history taking (exercise tolerance and occupation) showed significant improvement over time, across both control and SPI2 hospitals, but no additional SPI2 effect. The holistic review showed no significant changes in error rates either over time or between control and SPI2 hospitals. The explicit case note review of perioperative care showed that adherence rates for two of the four perioperative standards targeted by SPI2 were already good at baseline, exceeding 94% for antibiotic prophylaxis and 98% for deep vein thrombosis prophylaxis. Intraoperative monitoring of temperature improved over time in both groups, but this was not significant (1.8, 0.4 to 7.6; P=0.279), and there were no additional effects of SPI2. A dramatic rise in consumption of soap and alcohol hand rub was similar in control and SPI2 hospitals (P=0.760 and P=0.889, respectively), as was the corresponding decrease in rates of Clostridium difficile and meticillin resistant Staphylococcus aureus infection (P=0.652 and P=0.693, respectively). Mortality rates of medical patients included in the case note reviews in control hospitals increased from 17.3% (42/243) to 21.4% (24/112), while in SPI2 hospitals they fell from 10.3% (24/233) to 6.1% (7/114) (P=0.043). Fewer than 8% of deaths were classed as avoidable; changes in proportions could not explain the divergence of overall death rates between control and SPI2 hospitals. There was no significant difference in the rate of change in mortality in intensive care. Patients’ satisfaction improved in both control and SPI2 hospitals on all dimensions, but again there were no significant changes between the two groups of hospitals.
Conclusions Many aspects of care are already good or improving across the NHS in England, suggesting considerable improvements in quality across the board. These improvements are probably due to contemporaneous policy activities relating to patient safety, including those with features similar to the SPI, and the emergence of professional consensus on some clinical processes. This phenomenon might have attenuated the incremental effect of the SPI, making it difficult to detect. Alternatively, the full impact of the SPI might be observable only in the longer term. The conclusion of this study could have been different if concurrent controls had not been used.
We thank Michael D L Morgan, Martyn R Partridge, and Philip W Ind for their expertise and contribution in the development of the forms for the explicit case note review for respiratory care; Dion Morton and David Thomas for their expertise in the development of the forms for the surgical explicit case note review; Sheldon Stone and Chris Fuller for access to data collected as part of the NOSEC study; David Harrison and Kathy Rowan from ICNARC for access to data collected as part of the Case Mix Programme; Dale Webb, Louise Thomas, and Simona Arena for their help in describing the SPI intervention; Steven Thornton, chief executive of the Health Foundation, for providing a superb role model in sponsoring formal, summative evaluations of service level interventions; Peter Chilton for his assistance in the preparation of this manuscript; and Frank Davidoff, Laura Morlock, and Tim Hofer for excellent comments on the manuscript.
Contributors: AB, MD-W, JD, NB, and RL designed the study and submitted the grant proposal. RL was chief investigator. AB, NB, RL, MG, and BDF designed the forms for the explicit case note review and methods for the explicit case note review. AB, RL, and UN designed the semistructured forms for the holistic case note review and methods for data extraction. AB and UN were responsible for collecting the case note reviews. MG and BDF conducted the review of acute medicine case notes. MC and TN conducted the holistic review of case notes. MC and CD carried out a separate review of deaths. UN and MG designed the database for acute medicine case note review. GR and AB created the queries for data extraction. UN, AB, and RJL designed the forms for the perioperative case note review. AB and UN designed the database for the perioperative case note review. UN and AK conducted the review of case notes. AG analysed all the data from explicit reviews of case note. GR and AB designed and wrote database queries for final analysis and to assess the learning effect on the case reviewers. GR captured processed raw mortality data and calculated hospital standardised mortality rates for hospitals in both arms and undertook analysis of the socioeconomic composition of the admitted patient populations of hospitals in the study. KH analysed the data from the holistic review of case notes. KH carried out analysis of the infection related data, intensive care mortality data, and hand hygiene related data. JD was responsible for all aspects of the staff and patient surveys. RJL and MDW led on writing of the paper and interpretation of the findings. All authors contributed to the final manuscript. RL is guarantor.
Funding: This study was funded by the Health Foundation and the National Patient Safety Agency. KH was funded by the National Institute for Health Research Collaborations for Leadership in Applied Health Research and Care for Birmingham and Black Country, and AG by the Engineering and Physical Sciences Research Council, Multidisciplinary Assessment of Technology Centre for Healthcare programme. The Centre for Medication Safety and Service Quality is affiliated with the Centre for Patient Safety and Service Quality at Imperial College Healthcare NHS Trust, which is funded by the National Institute of Health Research. The evaluation was sponsored for research governance purposes by the University of Birmingham. The study was designed independently by the researchers. The researchers acted independently but worked collaboratively with the funder. The researchers independently collected, analysed and interpreted the data. The researchers wrote this article independently. The funders were given the opportunity to provide comments before submission. All researchers (apart from CD) had the opportunity to access participant anonymised data.
Competing interest: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: financial support as specified elsewhere; no financial relationships with commercial entities that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Each substudy had its own ethical approval. The staff and patient surveys were approved by the North West multi-centre research ethics committee and each site granted access to their data. The National Research Ethics Service deemed the case note review as audit/service evaluation and no further ethical approval was required. Permission was also granted from each site to access ICNARC, NOSEC and healthcare associated infection data. Local research governance was followed at each site.
Data sharing: No additional data available, but see full report.2
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.