Problems in assessing rates of infection with methicillin resistant Staphylococcus aureus
BMJ 2005; 331 doi: https://doi.org/10.1136/bmj.331.7523.1013 (Published 27 October 2005) Cite this as: BMJ 2005;331:1013All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
David J Spiegelhalter draws attention to the statistical difficulties
is assessing rates of infection with methicillin resistant Staphylococcus
aureus (MRSA) and demonstrates the large, apparently random, variation in
these rates[1].
The Government data he analyses[2] are presented in a way that seeks
to explain some of the variation by categorising the NHS Trusts that
operate hospitals into “single specialty” Trusts with low rates of
infection, “specialist” Trusts which include these with the highest rates
and “general acute” Trusts with intermediate levels. The specialist
Trusts have high rates because, it is suggested, they receive patients by
transfer who import infection. This is unlikely to be a complete
explanation; single specialty Trusts also receive such patients and the
Trusts described as “specialist” are usually major teaching hospitals
whose tertiary specialist functions are carried out alongside general
acute functions.
There is in fact a clear relationship with the size of the Trust and
its location in London or the Provinces. Using the 2003-2004 data and
excluding the single specialty Trusts, for provincial Trusts the
correlation coefficient is 0.243532465 for MRSA bacteraemia rate against
bed days (n=128, p<0.01) and for London Trusts it is 0.42544808 (n=27,
p<0.05)
These correlations may reflect the diseconomies of scale associated
with large hospitals[3]. However, the unit of assessment of the rate of
MRSA bacteraemia is the NHS Trust and the largest Trusts were formed by
the merger of smaller ones. They therefore operate several hospitals. It
is most likely that the circumstances, largely concerned with finance and
staffing, that led to the meger of Trusts are the circumstances that lead
to higher rates of MRSA bacteraemia.
1. Spiegelhalter DJ, Problems in assessing rates of infection with
methicillin resistant Staphylococcus aureus BMJ 2005;331:1013-1015
2. Department of Health. Mandatory bacteraemia surveillance
scheme—MRSA bacteraemia by NHS trust: April 2001-March 2004.
www.dh.gov.uk/assetRoot/04/08/58/93/04085893.pdf
3. Hospital volume and health care outcomes, costs and patient
access. Effective Health Care, University of York, 2 (8) 1996
Competing interests:
None declared
Competing interests: No competing interests
We could meet targets for MRSA very easily. All we need do is set up
a nationwide randomised controlled trial of treatments for MRSA. Patients
with MRSA would melt away like medical students before a lecture on
regression towards the mean.
Competing interests:
None declared
Competing interests: No competing interests
Targets look good; they tend to be met. People put extra resources
into
targeted areas (that are therefore not available for non-targeted areas).
Also,
targeted areas are often newer and more high tech; learning curves will
therefore ensure improvement no matter what. When these mechanisms fail,
various subtle gaming methods can be employed.1 None of these
mechanisms has anything to do with improving hospital safety and quality.
Some people even strive to improve the systems that produce the targeted
outcomes but targets are immediate and fixing systems is a difficult long
term ongoing business. My infectious diseases physician colleagues attest
to
the irrational nature of targets just as Deming2 did many years ago.
Spiegelhalter3 shows what happens when hospital acquired infections
are
made targets. In addition, he mentions but does not include difficulties
with
numerators and denominators in his calculations. With MRSA colonisation,
the
harder you look the more you find. Occupied bed-days are frequently used
denominators. They are at best a crude approximation to the true
unmeasurable denominator for susceptible patients.
Brennan and his colleagues4 point out that our efforts to date to
improve
hospital safety and quality have basically been without much success. They
advocate an approach based on best practice. This is concordant with a
huge
body of knowledge about hospital systems.5,6,7,8 First, we must get our
hospital systems working in a stable, reproducible and predictable way and
when we have done this, sequential analysis of surveillance data is
required
to ensure maintenance and further progress. There must be vigorous
feedback for learning how to continue to do better. Within hospital data
vary
very much less than between hospital data and many of the difficulties
identified by Spiegelhalter3 vanish when we stick to using our own data
sequentially. The mania for comparing hospitals has to be replaced with
relentless implementation of systems based on best practice and sequential
within institution monitoring.
If central offices wish to improve hospital safety and quality they
must help
hospital staff implement systems based on best practice. This must include
providing essential resources and setting up surveillance systems that
ensure
that hospitals apply those systems. Quality and safety come from the
unrelenting implementation of systems based on best practice, not on
targets
and deeply flawed comparisons.
Duckworth and Charlett9 point out that knowledge about best practice
is
seriously deficient in some areas. Setting targets when little is known
about
mechanisms is doubly absurd. Clearly, extra effort is needed to attempt to
fill
the gaps. A potentially valuable tool is stochastic modelling as it
appears to
be capable of identifying the system factors that have the greatest
influence.
Central offices could do more to promote research in this area.
References
1. Pitches D, Burls A and Fry-Smith A “How to make a silk purse from a
sow’s ear-a comprehensive review of strategies to optimise data for
corrupt
managers and incompetent clinicians” BMJ 2003;327:1436-9.
2. Deming W “Out of the crisis” Cambridge University Press 1982, page 65.
3. Spiegelhalter D “Problems in assessing rates of infection with
methicillin
resistant Staphylococcus aureus” BMJ 2005;331:1013-5.
4. Brennan T, Gawande A, Thomas E and Studdert D “Accidental deaths,
Saved Lives, and Improved Quality” N Engl J Med 2005;353:1405-9.
5. Batalden P and Stoltz P “A Framework for the Continual Improvement of
Health Care” The Joint Commission Journal on Quality Improvement
1993;19:424-452.
6. Berwick D “A primer on leading the improvement of systems” BMJ
1996;312:619-622.
7. Nolan T “Understanding Medical Systems” Ann Intern Med
1998;128:293-298.
8. Vincent C “Understanding and Responding to Adverse Events” N Engl J
Med 2003;348:1056-56.
9. Duckworth G and Charlett A “Improving surveillance of MRSA
bacteraemia” BMJ 2005;331:976-7.
Anthony Morton MD, MS, MScAppl.
Infection Management Services, Princess Alexandra Hospital, Brisbane.
Competing interests:
None declared
Competing interests: No competing interests
Using measures of MRSA rates to drive improvement
Dr Spiegelhalter's article (BMJ volume 331 29 October 2005 p1013 -
1015) gives an excellent analysis of the problems involved in assessing
the performance of NHS trusts in achieving the Department of Health target
for reducing bloodstream infection rates from methicillin resistant
Staphylococcus aureus (MRSA). The Healthcare Commission has the role in
making these assessments as part of its annual health check. The numbers
of MRSA bloodstream infections could (and should) be reduced, but the
technical difficulties of assessment stem from the fact that the numbers
involved are actually low.
The purpose of assessing trusts using measures of MRSA rates is to
help drive improvement in Healthcare Associated Infection (HAI), and the
Healthcare Commission is working closely with the Department of Health and
the Health Protection Agency on this matter. We do, however, recognise
the complexities of assessing performance in this area, and it is for this
reason that we have commissioned Dr Spiegelhalter to advise us on the most
viable measurement technique.
The approach that we have adopted will use plans produced by
individual trusts to reduce their infection rates over a 3-year period.
The plans were instigated by the Department of Health and must produce
either a 60% reduction by 2007/2008, or reduce and maintain the number of
infections to a maximum of 12 per year. Counts of 12 infections or less
will be considered as meeting the target. For higher levels, statistical
analysis will be employed to determine if a trust’s number of infections
is significantly higher than that expected from its plan.
We believe that with Dr Spiegelhalter’s assistance we have developed
the optimum performance measurement technique for assessing the MRSA
target for 2005/06, and we will continue to work with him to improve on
the sensitivity of the measure.
Such work will also feed into our ‘national study’ on HAI that aims
to determine which aspects of performance are the most successful in
driving improvement in HAI, with particular regard to reducing MRSA rates.
The ‘national study’, which is currently being designed, will be published
next year.
Competing interests:
None declared
Competing interests: No competing interests