Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluationBMJ 2011; 342 doi: https://doi.org/10.1136/bmj.d195 (Published 03 February 2011) Cite this as: BMJ 2011;342:d195
All rapid responses
A Safer NHS, but why? Large scale organisational intervention to improve patient safety in four UK hospitals: measuring the changes over six years.
Benning et. al.'s evaluations of Safer Patient Initiative
effectiveness at SPI1 and SPI2 hospitals was looking for sustainability of
improvement across whole organisations. The Institute for Health
Improvement faculty encouraged us to start measuring harm events through
the use of a set of triggers (called the Global Trigger Tool) that
identified potential harm through short case note reviews. At Luton &
Dunstable we have continued this for a six-year continuous period now, and
I would like to disseminate information that I collated for an open
session at the Quality & Safety Forum in Amsterdam this week. We have
examined 1470 case notes for this review.
2005 2008 General Triggers G1 Lack of Early Warning Score response. 148 93 G2 Any Patient fall. 22 46 G3 Decubiti. 7 20 G4 Readmission to hospital within 30 days. 173 207 G5 Shock or cardiac arrest. 20 18 G6 DVT/PE following admission (imaging or D-dimers). 20 24 G7 Complication of procedure or treatment. 88 144 G8 Transfer to higher level of care. 24 31 Surgical S1 Return to Theatre. 24 12 S2 Change in planned procedure 34 78 S3 Removal/injury or repair of organ 17 9 Critical Care (ITU, HDU, CCU or Outreach) I1 Readmission to ICU or HDU 3 3 I2 Unplanned transfer to ICU or HDU 13 5 Medicines M1 Vitamin K 12 6 M2 Naloxone 4 1 M3 Flumazenil 2 0 M4 Glucagon or 50% Glucose 3 12 M5 Abrupt Medication Stop 88 153 Laboratory L1 High INR>5. 14 8 L2 Transfusion of blood products 45 45 L3 Abrupt drop in Hb or Hct >25%. 81 69 L4 Rising urea or creatinine >2x baseline. 26 40 L5 Na+: <120, >160. 7 8 L6 K+: <2.5, >6.5. 6 4 L7 Hypoglycaemia < 3mmol/L. 13 8 L8 Raised Troponin >0.04ng/ml. 23 23 L9 MRSA bacteraemia. 9 1 L10 Clostridium difficile. 12 1 L11 Vancomycin Resistant Enterococci. 2 0 L12 Wound infection. 18 9 L13 Nosocomial pneumonia. 5 1 L14 Positive Blood Culture. 22 16
Summary of Harm measures.
NCC- MERP Severity Scores of Adverse Events from Random Audits.
E requires intervention, F results in increased length of stay, G
indicates permanent physical harm, H shows critical care areas were
I-level harm is where harm caused death.
January 2005 - December 2007. January 2008- December 2010.
20 per month amounting to 1470 case note reviews to date.
2005 - 2007 2008 - 2010 E 30/750 = 4.0% 58/720 = 8.0% F 76/750 = 10.0% 127/720 = 17.6% G 9/750 = 1.2% 3/720 = 0.4% H 6/750 = 0.8% 6/720 = 0.8% I 22/750 = 2.9% 7/720 = 1.0% Total 143/750 = 19.1% of the first triennium have some adverse event. Total 201/720 = 27.9% of the subsequent triennium have adverse events.
The more major levels of harm have diminished, but the more minor harm
categories have increased. Our interventions for deteriorating patients
are more effective, our medicine reconciliation has improved, and adverse
events with warfarin are less. The infection control team have made
significant inroads into hospital acquired infections along with the
activity of our critical care doctors and widespread hand hygiene
training. The leadership team are addressing issues around the admission
processes for emergency patients, and this may be demonstrated later by
falling F and E categories of harm.
These measurements of harm have been provided by an anaesthetist and
Patient Safety Manager, and the PSM has led and sustained many of the
As an involved senior member of staff in this organisation, I am grateful
to the Health Foundation for the sustainable quality improvements that the
Institute for Healthcare Improvement faculty have enabled this district
general hospital to achieve. The Global Trigger Tool has a robust design,
and its evaluation is continuing. It has been developed and used by
practicing clinicians and pilot-tested in many nations. Patient Safety has
been prioritised by leaders such as the Chief Medical Officer and agencies
such as the National Patient Safety Agency, so care in the UK has improved
in the last decade. As a full-time clinician, I have participated in
meetings with the human factors and systems engineers, the social
scientists and the health service researchers. I believe our patient care
improves with this open sharing of the benefits and pitfalls of our
Dr Michael Carter
(disclosure of interest: Since April 2005 I have been paid for an
extra session per week for this audit work. The work itself takes longer,
but there are greater motivations than financial reward).
Benning A. et. al. BMJ 2011; 342:d195 and d199
Editorial "Is quality of care improving in the UK?" P J Provonost, S M
Berenholtz, L L Morlock; BMJ 2011; 342:c6646
I have been paid one session per week since April 2005 to do the case note reviews using the Global Trigger Tools.
Competing interests: 2005 2008 General TriggersG1 Lack of Early Warning Score response. 148 93 G2 Any Patient fall. 22 46 G3 Decubiti. 7 20 G4 Readmission to hospital within 30 days. 173 207 G5 Shock or cardiac arrest. 20 18 G6 DVT/PE following admission (imaging or D-dimers). 20 24G7 Complication of procedure or treatment. 88 144 G8 Transfer to higher level of care. 24 31 SurgicalS1 Return to Theatre. 24 12 S2 Change in planned procedure 34 78 S3 Removal/injury or repair of organ 17 9 Critical Care (ITU, HDU, CCU or Outreach)I1 Readmission to ICU or HDU 3 3 I2 Unplanned transfer to ICU or HDU 13 5 MedicinesM1 Vitamin K 12 6 M2 Naloxone 4 1 M3 Flumazenil 2 0 M4 Glucagon or 50% Glucose 3 12 M5 Abrupt Medication Stop 88 153 Laboratory L1 High INR>5. 14 8 L2 Transfusion of blood products 45 45 L3 Abrupt drop in Hb or Hct >25%. 81 69 L4 Rising urea or creatinine >2x baseline. 26 40 L5 Na+: <120, >160. 7 8 L6 K+: <2.5, >6.5. 6 4 L7 Hypoglycaemia < 3mmol/L. 13 8 L8 Raised Troponin >0.04ng/ml. 23 23 L9 MRSA bacteraemia. 9 1 L10 Clostridium difficile. 12 1 L11 Vancomycin Resistant Enterococci. 2 0 L12 Wound infection. 18 9 L13 Nosocomial pneumonia. 5 1 L14 Positive Blood Culture. 22 16
We thank the chief executive officers of SPI(1) sites who contributed
letters in response to our articles, and comment as follows:
1. They seem to imply that the results of SPI(1) were entirely
negative, when we found some changes favouring SPI(1) and none going the
2. A sharp improvement in the vigilance of monitoring was noted
across the NHS in both phases of SPI, so knock-on effects on the need for
cardio-pulmonary resuscitation are indeed plausible. It would be
worthwhile to try to find a way to make measurements of cardiac arrests
that are consistent across organisations and over time.
3. Our data are based on a method that is explicitly described and
pre-specified; measured by the same observers across hospitals; episode of
care is blinded; and data are controlled in space as well as time - the
figures quoted in the letters do not meet any of these criteria.
4. We think there are many fine NHS institutions that did not take
part in SPI where the chief executives' families would be well cared for.
Noting that improvement has taken place is not tantamount to saying that
things are fine: we are all fully in agreement about the need to avoid
5. We agree that improvements may have occurred beyond the
observation period, and said so in the paper.
The chief executive officers and the authors share a desire to
improve care and to measure improvement. So we hope they would support us
in a further assessment to test the last point above.
1. Benning A, Ghaleb M, Suokas A, Dixon-Wood M, Dawson J, Barber N,
Franklin BD, Girling A, Hemming K, Carmalt M, Rudge G, Naicker T, Nwulu U,
Choudhury S, Lilford R. Large scale organisational intervention to improve
patient safety in four UK hospitals: mixed method evaluation. BMJ. 2011;
2. Benning A, Dixon-Woods M, Nwulu U, Ghaleb M, Dawson J, Barber N,
Franklin BD, Girling A, Hemming K, Carmalt M, Rudge G, Naicker T, Kotecha
A, Derrington MC, Lilford R. Multiple component patient safety
intervention in English hospitals: controlled evaluation of second phase.
BMJ. 2011; 342:d199.
Competing interests: An author of the above papers.
Safer Patients Initiative Evaluation
As a senior leader in the NHS for over 30 years I am disappointed in the
evaluation of the the Safer Patients Initiative (SPI). In the past, the
NHS has been slow to research and implement change, and variation in
practice has persisted harming patients. The SPI program introduced the
staff in NHS Tayside to the science of improvement, enabling individuals
to use innovation to make care safer. Simple improvement techniques
designed by frontline clinicians were used to deliver the correct
treatments to every patient every time, increasing the reliability of our
practice; demonstrated through improved results. Our local data clearly
confirms, using the Global trigger tool produced a 63% reduction in
All of our results were data driven. A significant change occurred when
frontline staff developed the ability to reliably measure what happens to
patients as they go through the care process. We knew the evidence base
said patients should receive certain care elements, but we had no way of
knowing whether they did or not. The initiative taught us how to measure
that, and we started to make sure that our practices and how they needed
to change were driven by data. Each work stream produced remarkable
Working on early rescue of the deteriorating patient (recording vital
signs and calling for assistance) produced high reliability in practice
and a 30% reduction in the crash call rates overall. The units involved
included 12 wards and over 300 staff: a high level of staff engagement
The interventions in this work stream concentrated on medicine
reconciliation- patients receiving the right medication at the right time
in the right dose. NHS Tayside's work focused on the most difficult area
of acute medical admissions. The majority of patients admitted to the unit
come in, on average, with 14 different medications. The pharmacists,
junior doctors, nurses and improvement support staff redesigned the system
to achieve 90% improvement in getting the list correct on admission.
Peri-operative care and Surgical site infection
Years before the WHO, a surgical checklist was introduced, resulting in
daily safety briefings in the peri-operative areas. Reliable
implementation of the surgical site infection bundle (in three hospitals),
produced a 40% reduction in infection by the end of the program and have
sustained this to date achieving an additional 30% by December 2010. For
patients, this is great news: the likelihood of getting a surgical site
infection has reduced by over 70 %.
In ICU in the first year of the initiative, 7 patients had central line
infections. Once the staff had learned how to reliably apply safer
practices only one patient had an infection in the next year. The overall
gaol of 30% reduction in ventilated associated pneumonia was achieved.
Patient safety leadership Walkrounds Began in February 2005 and continue
weekly six years later. The walkrounds involve senior leadership, patients
and non executive staff. The discussions about patient safety revealed new
safety information leading to: a national review of the charge nurses role
system: changes to the emergency call process and hundreds of
environmental changes improving the flow for patients. All units
clinical and non clinical in three hospitals were visited involving over
1000 staff and patients. Today in 2011, Patient safety leadership
Walkrounds now take place in many different healthcare environments
including community hospitals and mental health units.
Our achievements led not only to great results but recognition for
our staff within NHS Scotland at the NHS Awards receiving Top Team 2006.
But improving safety is not about accolades, it's about people and their
families as an Accountable Officer in healthcare it is unacceptable to
tolerate harm in our systems. The SPI initiative changed everything in our
hospitals with regard to accountability and the measurement of improvement
in frontline clinical practice. One a weekly basis, I engaged with staff
and supported and monitored their progress. SPI was the catalyst for
change creating a seismic movement in the quality of healthcare laying the
foundations for the Scottish patient Safety Program and the Scottish
Governments Healthcare Quality Strategy (2010). As a result of this
program and its outcome, I feel safer as a patient in NHS Scotland.
Mr Gerard Marr
Deputy Chief Executive
Quality Improvement Scotland
Patient Safety Development Manager
4 February 2011
Competing interests: Senior and Clincial leaders in NHS Tayside during the SPI program
The Safer Patients Initiative
As a lead clinician within the NHS Tayside Safer Patients Initiative
team and a faculty member in SPI 2, I find it difficult to reconcile the
results in this evaluation to actual experience and local data. In the
first 2 years of the program. NHS Tayside achieved a 63% reduction in
adverse events using the Global Tool methodology. This method was new to
the organisation and enables clinicians to look for harm within the
patient case record. In our hospitals systems, there is no doubt that
almost all of the trigger events and actual harm caused would not have
been identified by other methods.
Part of my role in the SPI program was to engage and support
frontline staff. On a weekly basis, we would meet, discuss and share ideas
regarding the interventions and processes that were changing to increase
the reliability of evidence into practice. I personally met with over 200
staff each week within the system to talk about their local data and how
closer they were to goals emphasising the local engagement and commitment
to the program. As a result of this engagement, NHS Tayside achieved and
exceeded the goals set by the program.
Some of the findings of the evaluation may be directly related to the
research teams gathering data in an area of the hospital that had not yet
began to participate in the patient safety interventions. Each hospital
had a dedicated spread plan to systematically roll out the interventions
when highly reliable practices were sustained in the pilot units (using
SPC rules). Therefore, it is not surprising in a non-pilot unit staff
were unaware of the patient safety program.
The results achieved in SPI, in NHS Tayside, were a catalyst for
change influencing the Scottish Government to launch a patient safety
program in all acute hospitals. In Scotland, in January 2008, the Scottish
Patient Safety Program began and some 3 years later we have remarkable
aggregate results including over 90% reduction in Central Line Infections
in ICU, 50% reduction in Ventilator Associated Pneumonia, 70% reducing in
ICU C Diff infection. This is great news for patients.
There is no doubt that the Safer Patients Initiative began a movement
for change in the delivery of safer, high quality, more effective care
within NHS in Scotland and throughout the UK. This is evident in the
confidence of the results achieved and the continued emphasis in National
healthcare policy and practice. As a citizen and a potential patient, I am
certain the public are assured by the high priority patient safety has as
a result of the humble beginnings of SPI.
Previously (Head of Safety Governance and Risk )
NHS Tayside Headquarters,
Competing interests: No competing interests
Evaluation of SPI
I was the CEO of Luton & Dunstable Hospital , the England pilot site
for the Safer Patients Initiative 2004-8. The main objective of SPI1 was
to reduce adverse events (as measured by the Global Trigger Tool) by 50%
over the first 2 years of the Initiative. This was achieved by Luton &
Dunstable Hospital. In addition, we achieved a 55% reduction in cardiac
arrests, and believe this to be the main reason for our HSMR reducing from
being regularly 110 to being regularly in the 90s. In our ITU we saw an
elimination of central line infections and a virtual elimination of
ventilator associated pneumonia, both previously seen as inevitable
"complications " of patient care in an ITU. SPI1 did not aim to improve
patient satisfaction nor staff attitude , two indicators measured by the
Experience of SPI1 at Luton & Dunstable Hospital encouraged the
Board to make patient safety its highest priority, strategically, and this
was demonstrated and reinforced continuously by the actions taken over
many years. The new approach we took to transforming patient safety was
largely led by enthusiastic clinicians , indeed in my 34 years as an NHS
manager, nothing has resonated so much with Doctors as this.
The Evaluation authors make much of safety and quality improvements
occurring across the whole of the NHS during this period, therefore SPI
not contributing more, over and above . This observation is at odds with
my own experience in both the leadership of the national patient safety
first campaign and since then as an independent consultant. There are
still too many hospitals not implementing best practice improvements to
avoid patients deteriorating nor recognising the possibilities to avoid
harm in ITU care. Many hospitals still do not even count Cardiac arrests
nor see them as failures in care. We still accept unacceptable practice
going on in our hospitals every day, and need a complete mindset change to
convert " first do no harm " into pro active action at every level.
The authors compare " some despair at an apparent lack of progress" in
the USA " with a more encouraging story on patient safety in the NHS " , a
rather complacent opinion in my view.
Competing interests: former CEO Luton & Dunstable Hospital - SPI1 site