Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review
BMJ 2007; 334 doi: https://doi.org/10.1136/bmj.39031.507153.AE (Published 11 January 2007) Cite this as: BMJ 2007;334:79
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
The work undertaken by Sari et al provide a useful empirical
contribution to this field.
They are right to point out that routine hospital reporting systems
can miss incidents reporting in harm. Reporting from primary care should
also be an important method of identifying system failure. But despite the
estimate that 19% of inpatients may experience a critical indicident at
discharge [Forster AJ et al Ann Intern Med 2003;138:161-7], reporting is
low.
We reviewed 20 months (April 2004 to December 2005) of safety
indicidents from primary care practitioners in our Primary Care Trust. We
identified 35 reports in the study period. At this time there were 32,000
inpatient spells each year at our local hospital provider. This is
equivalent to 0.07% of discharges.
More needs to be done to improve reporting, and use of this
information to improve patient safety.
Competing interests:
None declared
Competing interests: No competing interests
Courageous Lisa Norris - the Glasgow teenage brain-tumour patient who
blew the whistle on radiotherapy safety - was not monitored by in vivo
dosimetry.
These cheap and simple diodes are the seat-belts of radiotherapy ...
recommended by the International Atomic Energy Agency for use even in the
'less developed countries' where resources are scarce. The diodes are
quick to use and protect patients and staff from the majority of
accidental overexposures.
Although standard in several European countries, routine in vivo
dosimetry is not yet compulsory in NHS radiotherapy departments, despite
years of conscientious campaigning by eminent UK physicists, clinical
oncologists and radiographers.
In May 2006 the Royal College of Radiologists convened its 'Near
Misses, Incidents and Errors in Radiotherapy' working party which will
"seek to identify procedures which will minimise the risk or error. The
problem of disseminating learning from incidents and near misses that do
occur is not in the remit of that working party, but the College will be
undertaking further work in that area." The report is due for publication
in autumn 2007.
We are two former radiotherapy patients agitating for the DH to ring-
fence funding for in vivo dosimetry in all NHS radiotherapy centres NOW!
Thank you Ben-Tovim and Matthews for your useful advice about Lean
thinking.
Competing interests:
None declared
Competing interests: No competing interests
I was impressed to read the article entitled “Sensitivity of routine
system for reporting patient safety incidents in an NHS hospital: a
retrospective patient case note review”, and felt that the authors have
touched on a point of risk management which perhaps fill junior medical
staff with worry about blame and reprisals from their more senior
colleagues.
I work in an area of medicine where structured case notes review on a
day to day basis is already in operation (no doubt stimulated because of
requirements by CNST maternity standards). Whilst I don’t claim that all
the admissions to our department have a case notes review, all the
previous days deliveries are reviewed and in a multidisciplinary meeting
which encourages members of the whole team to learn from previous mistakes
and to see where improvements to the service can be made. This method of
case notes review means that all patients, both those who have perhaps
just come to the hospital for elective surgery, and acute admissions are
seen. If a patient has been an inpatient for a while and then delivers,
their notes are also included in the review.
Although a relatively informal meeting, it does do some important
jobs. Firstly it enables questions to be asked and answered with regards
to care, it institutes a “hopefully” non blame and non judgemental enquiry
into patient management if necessary. It enables juniors to find out how
risk could have been reduced or management changed in the non acute
environment and allows discussion with both senior medical and non medical
staff. It also and I think very importantly with regards to the article,
stimulates the filling in of hospital incident forms.
I don’t think that incident reporting should be replaced by case note
review – that would seem to be a very laborious task, which would
potentially need a team specially employed to do just this action, and in
a NHS which seems to be struggling to keep health care employees in post,
I think that that would be perhaps unnecessary expenditure. However, if in
my department where we can manage to spend an hour each day to review the
previous day’s notes and find learning points and potentially fill out the
dreaded incident forms. Then across the NHS as a whole this could be a
manageable option, and help to reduce the fear and blame that some seem to
associate with incident form filling!
Competing interests:
None declared
Competing interests: No competing interests
Taken together, the articles and reports in the current edition of
the British
Medical Journal paint a clear picture of the key issues in contemporary
health
care. But the viewer needs to stand back from the picture before the
individual brush-strokes resolve into a clear image.
First, there is the important article by Sari and colleagues. The
authors
concern is to demonstrate that routine reporting systems have a very low
sensitivity for identifying hospital adverse events in comparison to case-
note
review. They draw less attention to the observation that some kind of
patient
safety incident occurred in 22% of hospital admissions, and that at least
one
safety event resulting in patient harm occurred in 11% of admissions.
Then there is the truly breathtaking news report by Caroline White
(doi:
10.1136/bmj.39091.494213.DB) about the decision not to publish a report
about the standards of care provided for heart surgery patients in the St
Mary’s Hospital NHS trust. The report quotes a spokesperson from the
Healthcare Commission thus ‘information about clinical risk is quite
complex
and can be subject to misinterpretation. This is not hiding things from
patients; but it’s also about not causing undue anxiety to existing and
prospective patients.’
There are two further snippets of note. One a shortcut about a meta-
analysis
demonstrating that the impact of audit and feedback on improving practice
is
small (doi: 10.1136/bmj.334.7584.68-c), and the other a news report by
Michael Day (doi: 10.1136/bmj.39090.709803.4E) about mismatch in NHS
staffing levels, that states that many NHS staff will lose their jobs as
trusts
seek to contain spiralling deficits.
Finally there is the excellent editorial by Charles Vincent (Vincent
C. BMJ
2007; 334; 51-2) that argues that of itself, reporting does not change
behaviour. What is needed is action. In my view, Vincent does not go far
enough in simply calling for ‘improvement programs’.
Let us take the Healthcare Commission first. I don’t think there is
anything
complex about the statistics for risk in healthcare. Any industry that
harms
one in ten of its customers is unsafe. Full stop. The fact that some
places are
relatively worse than others pales into insignificance against the
underlying
rate of harm. Prospective patients have every right to be anxious, and no
change is likely without that anxiety being voiced often, and
persistently.
Furthermore, it is inherently unlikely that detectable safety
incidents
stand alone. It is much more likely that they are the manifest outcomes of
care processes that are error-prone at every step, with many potential
errors
fortunately being detected prior to a safety incident actually occurring,
but
requiring considerable re-work in the process.
We have to take hospital safety out of the safety and quality ghetto
and
beyond strategies such as clinical audit and feedback that embed existing
levels of error into baseline best-practice outcomes. And are then
relatively
ineffective in getting practice even to function at existing error prone
best-
practice levels.
For the last three years we have been experimenting with the
application of
Lean thinking1 to care processes across our teaching general hospital.
Lean
thinking is an approach to improving the sequential processes involved in
production of manufactured goods and services of all kinds. Whilst the
approach was first described in relation to manufacturing, it has since
been
applied to many industry sectors, and interest in Lean thinking is
beginning
to emerge in healthcare also.
To the Lean thinker, error in execution of a process is an absolute
waste. No
one benefits from it. Once it is acknowledged that errors resulting in an
overt
patient safety incident occur in one in five hospital admissions, further
retrospective error analysis is of limited value. No existing care system
can
be satisfactory if it generates the level of error found in studies such
as those
by Sari and colleagues. Simply adding another incident report to the
existing
pile will not change anything. Instead, we prospectively examine and
redesign
care processes of all kinds to make doing the right thing easier than
making
errors. A prospective examination of existing processes, to identify
potential
weaknesses and opportunities to improve, encourages the improvement team
to work at the system level rather than the level of individual blame. It
also
avoids sterile debate about relative risk.
Over a three year period, we have halved the number of serious safety
events
that have had to be reported to our insurers, despite a substantial
increase in
the numbers of patients seeking care in our hospital. There are many
other
benefits from a systematic process of redesign using Lean thinking
principles,
some of which have been described elsewhere2. It is notworthy that at the
beginning of our Lean thinking journey, our hospital was struggling to
contain a deficit. In the last financial year, we were several millions
dollars in
the black in relation to our activity. Without extreme measures such as
shedding staff.
Poorly designed clinical processes are dangerous for patients, and
frustrating
for staff, They are also enormously wasteful. Improving the processes of
care
by the systematic application of comprehensive methodologies such as Lean
thinking is not easy. But it improves outcomes for patients and directs
precious resources to the provision of care, rather than making good the
effect of error. It can be done. All that is needed is the will to do it.
1. Womack JP, Jones DT. Lean thinking. Banish waste and create wealth
in
your corporation. London. Simon & Schuster 1996.
2. King DL. Ben-Tovim DI. Bassham J. Redesigning emergency department
patient flows: application of Lean Thinking to health care. Emerg Med
Australas. 2006;18(4):391-7.
Competing interests:
None declared
Competing interests: No competing interests
The authors may wish to be aware that such case note review systems
have been used in the NHS. In our Trust we started a Clinical Outcomes
Committee four years ago that reports into the Governance system and is an
established case review system on a regular basis. We published our work
[Analysing clinical incidents by clinical mini Root Cause Analysis. Ince
C. McILwain J.C. Health Care Risk Report. 2004.vol 10 no 9 September
issue. Pages 12 - 15] and are still maintaining detailed senior
professional analysis of reported concerns and random sampling of case
records Trustwide.
We have uncovered many issues aside from traditionally
reported incidents and brought about effective changes in clinical
practice in patient safety issues. We use case record reviews to a
structured causal analysis based upon care-service timeline review. To
undertake record review requires both a system and people who understand
how to assess and analyse discovered information. We usually get through
three case records a month in great detail. Each case is reviewed by three
senior clinicians (doctors and nurses), two as assessors and one a case
lead in conference. Results and findings are fed back to both the
clinicians and Trust Governance Board to ensure all lessons to be learned
are communicated. A singular persistent issue is communication between
health professionals – something that any incident reporting system will
never uncover.
There is much to learn from adversity and as described by
the authors there is no single methodology. In this Trusts we have both an
integrated incident reporting system linked to complaints and litigation
as well as to the National reporting database and our case review system
running independently. I would have to say that perhaps we have learned
more from case analysis (random and referred) than incident statistics. In
the coming year it is our hope to extend the case review / incident
reporting analysis tools down deeper into the organisation into clinical
departments akin to the methodologies used for CNST maternity assessments
– the risk forum.
Competing interests:
None declared
Competing interests: No competing interests
Thank you to Drs Ali Baba-Akbari Sari, Trevor A Sheldon, Alison
Cracknell and Alastair Turnbull, for their useful study.
Their findings show, once again, that no single source of data for
safety improvements in healthcare will suffice for learning what may go
wrong and how to prevent it.
Many different data sources (e.g. medical records, audits, complaints,
coroners' reports, medical publications, closed claims studies, incident
reports and even some media material) are required to obtain a balanced
and correct prespective of all healthcare risks.
That said, incident reporting, although only capable of providing
"numerator data", remains a powerful contributor to safety improvements.
In this context its adoption by and within healthcare teams also helps to
diminish any "balme culture" and to improve workplace morale, team trust
and 'esprit de corps'.
Competing interests:
Part-time employee of the Australian Patient Safety Foundation, a not-for-profit, recognised Australian qualtiy assurance activity organisation.
Competing interests: No competing interests
Interesting article.
I would like to know how many of these adverse incidents occurred in
acutely unwell patients, and how many occurred in elective patients.
"Adverse incidents" in acutely unwell patients might just be part of the
natural history of their illness, or might be unavoidable to some extent
e.g. someone with multiorgan failure on multiple drugs and infusions is
more likely to subject to a drug error (but need to be exposed to that
risk) than someone who just attends for a routine out-patients.
I think investigating and doing something about adverse incidents in
elective patients might be a more efficient use of precious time.
Competing interests:
None declared
Competing interests: No competing interests
I would like to congratulate Ali Baba-Akbari Sari and co-workers for
the cumbersome task of reviewing a large number of medical files to
provide estimates for the sensitivity of incident reporting. The results
are as shocking as expected. The conclusion of the paper ("Structured case
note review may have a useful role in surveillance of routine incident
reporting and associated quality improvement programmes") is however not
supported by data.
It has been known for some time that large numbers of patients come to
harm in hospitals in different health service settings [1,2]. The reason
that this continues to be a problem is only partly to be blamed on poor
reporting. A greater problem is the inability of organisations and their
members to act on them.
The efficiency of the response to incidents determines at the end of the
day the impact of a quality assurance system not the completeness of its
documentation of failure.
Let's learn from the faults that we know about and then look for more.
1. Vincent C, Neale G, Woloshynowych M.Adverse events in British
hospitals: preliminary retrospective record review.
BMJ. 2001 Mar 3;322(7285):517-9.
2. Brown P, McArthur C, Newby L, Lay-Yee R, Davis P, Briant R. Cost of
medical injury in New Zealand: a retrospective cohort study. J Health Serv
Res Policy. 2002 Jul;7 Suppl 1:S29-34.
Competing interests:
None declared
Competing interests: No competing interests
Dear Editor,
Your Research article entitled “Sensitivity of routine system for
reporting patient safety incidents in an NHS hospital: a retrospective
patient case note review” confirms what I and others have suspected all
along. Existing attempts to introduce a culture of incident reporting are,
and will continue to be, unworkable while we continue to run our
organisations in the traditional ‘command and control’ way.
No matter how much well intentioned clinicians try to foster a ‘no-
blame’ culture in order to encourage transparency and learning, it will
never work while the rest of the organisation continues to focus on people
rather than processes in their attempts to remedy situations.
Evidence these past few years has clearly shown that only by studying
and improving processes that result in medical error can we ensure patient
safety. Regrettably this simple truth has never really been widely
understood or it would have been applied to everything else that we do in
our various organisations.
Nothing much has changed in the wider NHS or in my own field of
operations – the ambulance service - nor is there much sign that it will.
Unfortunately our managers remain locked into traditional management
culture and ignorant about or resistant to new organisational thinking.
Where effective approaches have or are being tried (lean thinking,
for example) attempts are being made to use the methodology within
existing, traditional systems. In other words, attempts are being made to
graft successful methods onto dysfunctional systems.
Moreover, these dysfunctional systems are governed by targets which
are arbitrary measures that reflect desire rather than useful knowledge
about our system’s capabilities – a prerequisite to improvement.
Targets have had a number of drawbacks. One drawback is their effect
of leading NHS organisations to concentrate on areas needing improvement
at the expense of overall system harmony and, often, patient well-being.
Another is to deflect focus away from appropriate method towards attaining
results by fair means of fowl – often the latter in the absence of
organisational knowledge about how to work differently and better. They
are the reason we are in such a moral, structural and financial mess
today.
Management and government often complain about staff being resistant
to change, but here we have an example of the reverse being the case.
Managers are very keen to urge operational staff to use ‘best evidence’
for clinical practice, but have shown little or no inclination to question
the very basis of their own actions. They are constrained by their limited
training and experience, and are almost entirely focused on doing the
government’s bidding in order to qualify for their funding. The
government’s obsession with arbitrary targets and ‘carrot and stick’
approach ensures compliance with the worst kind of management thinking and
behaviour.
There is much evidence about the drawbacks of our traditional
approach (recommended reading: “Hard Facts, Dangerous Half Truths &
Total Nonsense”, J. Pfeffer & R. I. Sutton;). There are much better
measures we could use and a wealth of knowledge and wisdom about
organisational performance which, if we do not avail ourselves of them
soon will lead to the dismantling of our NHS.
The Government appears to have run out of ideas and instead seems to
be relying more and more on using the private sector in order to attain
some kind of illusory solution. Anyone tempted to believe as they do
should read “Good To Great” by J. Collins in order to understand once and
for all that most of the private sector have similar or worse problems
than we do. The only difference is that they reward failure probably
better than we do!
We have all – government, managers, staff and patients - become
victims of an ineffective and damaging organisational system. If we are
sincere about improving patient safety, then we must have the courage to
question the very basis of our organisational theory and practices.
Anything less will guarantee failure and ever deteriorating quality of
patient care and outcomes, with only occasional, short lasting and
illusory advances.
John Matthews
Paramedic, Ambulance Service.
Competing interests:
None declared
Competing interests: No competing interests
Giving incident reporting systems a chance
Dear Editor,
I read with much interest the article by Sari et al 1 on the
sensitivity of routine reporting systems for patient safety incidents
measurement. The authors compare a routine incident reporting system to
screened case note review to demonstrate that many incidents occurring
during patient care are voluntarily not reported.
This comparison is often used for assessing the sensitivity of
incident reporting systems.2,3 By doing so, however, the authors compare
two fundamentally different measures. As nicely illustrated in C.
Vincent’s editorial,incidents largely reflect individual perceptions of
events by hospital staff members.4 Unless incidents are rigorously
defined, this will lead to a lot of variability in the type of events
reported and of what may be defined as an incident.
On the other hand, events identified through casenote reviews largely
reflect the choice of the screening and analysis methodology.5 Only cases
identified through pre-defined criteria will be selected. As a
consequence, it is not surprising that Sari et al find only 17% of their
pre-defined incident types reported in a routine incident reporting
system. Had they used other pre-defined screens, they may have reached
different conclusions.
Another limitation relates to the failure of the authors to report
how they took into account the technical characteristics of the incident
reporting system they assessed. All systems are not similar. Some are
paper-based, others are electronic. Some include pre-defined categories of
events while others have an exclusively narrative content. Technical
characteristics and design of reporting systems can significantly impact
on the level of reporting. Computer-based incident reporting systems tend
to be more often used than their paper-based counterparts. When the
electronic reporting systems is standardised and incorporated into an
electronic patient record the use of the incident reporting system can
increase to 85%.6
Although it is rarely done, the technical characteristics of an incident
reporting and their impact on the level of reporting should be
systematically considered when assessing reporting systemsperformance.
If such limitations were more often considered and carefully
addressed, incident reporting systems would probably have a better chance.
References
1. Sari AB, Sheldon TA, Cracknell A, Turnbull A. Sensitivity of
routine system for reporting patient safety incidents in an NHS hospital:
retrospective patient case note review.BMJ 2007;334:79-81.
2. Cullen DJ, Bates DW, Small SD, Cooper JB, Nemeskal AR, Leape LL:
The incident reporting system does not detect adverse drug events: a
problem for quality improvement. Jt Comm J Qual Improv 1995; 21: 541-8.
3. Stanhope N, Crowley-Murphy M, Vincent C, O'Connor AM, Taylor-Adams
SE: An evaluation of adverse incident reporting. J Eval Clin Pract 1999;
5: 5-12.
4. Vincent C. Incident reporting and patient safety.BMJ. 2007;334:51.
5. Bates DW, O'Neil AC, Petersen LA, Lee TH, Brennan TA: Evaluation
of screening criteria for adverse events in medical patients. Med Care
1995; 33: 452-62.
6.Haller G, Myles PS, Stoelwinder J, Langley M, Anderson H, McNeil J.
Integrating incident reporting into an electronic patient record system. J
Am Med Inform Assoc. 2007 Jan 9 (in press).
Competing interests:
None declared
Competing interests: No competing interests