Peter J Pronovost, Christine A Goeschel, Elizabeth Colantuoni, Sam Watson, Lisa H Lubomski, Sean M Berenholtz et al
Pronovost P J, Goeschel C A, Colantuoni E, Watson S, Lubomski L H, Berenholtz S M et al.
Sustaining reductions in catheter related bloodstream infections in Michigan intensive care units: observational study
BMJ 2010; 340 :c309
doi:10.1136/bmj.c309
Published reductions in infection rates: Valid data or optimistic thinking?
In 2006 Pronovost et al.1 authored an important study that evaluated
the effectiveness of five infection-control interventions practiced in the
intensive care units (ICUs) of dozens of hospitals predominantly in
Michigan. With significant public-health implications, the results of this
study indicate that during an 18-month period, known as the post-
implementation evaluation period, these evidence-based interventions,
which included hand washing and the use of barrier precautions, were
effective and associated with a significant reduction from baseline in the
rate of catheter-related bloodstream infections (i.e., greater than 60%).
This rate was measured quarterly (i.e., every three months) as the number
of infections reported by each participating ICU for every one thousand
(1000) catheter days (that is, days catheters were used and in place).
Published this year in this journal, Pronovost et al.2 found that
this marked reduction in this rate (of catheter-related bloodstream
infections) published in 2006 was maintained and sustained for an
additional 18 months, known as the sustainability period. Many of the same
hospitals in Michigan that submitted data for Pronovost et al.’s first
study participated in this second study, once again reporting their
infections quarterly and as the number of infections per every one
thousand catheter days. This second study confirmed the researchers’
hypothesis that the significant reduction in the rate of infections
achieved initially during the 18-month post-implementation period would
remain low relative to baseline and be sustained during this subsequent 18
-month sustainability period.2
It is important to note, however, that the respective conclusions
advanced by Pronovost et al. in both of their studies discussing the
reductions in the rates of bloodstream infections associated with specific
infection-control interventions – like the conclusions of other studies
and reports, including one recently published in Consumer Reports
discussing “poorly performing” hospitals with disclosed infection rates
that are significantly higher than the average3 – are based on data
submitted by the participating hospitals themselves.1,2 That an ICU’s
reported infection rate is not ordinarily validated for accuracy by state
agencies or health officials, or by independent researchers, is a
potentially significant, though not always discussed, limitation of
virtually every study comparing as a key indicator of quality and
performance the infection rates publicly disclosed by hospitals.1,4,5
Acknowledging this salient limitation, Pronovost et al. discuss in
their first study the potential for “measurement bias”; the
“underreporting” of infections disclosed by the ICUs; and the exaggeration
of reductions in infection rates reported by participating hospitals.1
Pronovost et al.’s second study published in this journal, however,
focuses primarily on other limitations of their study, and does not
directly discuss or clarifying that the significantly reduced and
sustained infection rates determined from the data the participating
hospitals submitted were not independently validated and, therefore, are
not necessarily accurate and may be partial, overly optimistic, and more
academically useful and insightful than necessarily scientifically sound
and meaningful.2
Though intended by laws in several states to improve the quality of
health care through transparency and public accountability,3 the public
disclosure of infection rates by hospitals can be associated with
unwelcomed consequences, financial, legal, or otherwise. For example, if a
hospital’s disclosed rate is reported to be higher than average (a
reflection, possibly, as much as the hospital’s honest reporting as its
poor performance), it might be subjected to accusations of inferior
quality and unsterile conditions and to unwanted scrutiny. Such a
disclosure could also be associated with costly malpractice proceedings
and claims against the hospital, as well as providing an understandable
impetus for patients to seek medical care at another “competing”4 hospital
reporting (though not necessarily having) a lower infection rate.
That the public’s decisions about health care, and which hospitals to
(and not to) seek treatment, cannot be based on data that are inaccurate
or misleading is indisputable. In truth, however, the data used to monitor
and compare infection rates ordinarily have not been independently
validated, and therefore the soundness of what may be an important
rationale for these decisions by the public, is questioned. Due to a
number of considerations, including competition between hospitals, the
U.S. Government Accountability Office (GAO) concludes that hospitals may
have an incentive to underreport their infection rate.4 Similarly, the
South Carolina Department of Health has expressed concerns that publicly
disclosed infection rates might be manipulated, as well as misinterpreted
by the public.5 Providing another incentive to report low (but not
necessarily accurate) infection rates, Pronovost et al.’s second study
published in this journal acknowledges that participating hospitals were
given an “incentive payment” to meet “performance thresholds.”2 The extent
to which such incentives may have biased the impressive results presented
in Pronovost et al.’s second study is unclear.
Moreover, possibly hinting in their second study to the potential for
such bias and a significant difference between the reported and actual
infection rates submitted by participating hospitals, Pronovost et al.
note that having not “blinded” staff members in a hospital’s ICU to its
infection rate might have influenced their results demonstrating
sustainability.2 These concerns about the potential for bias raise fair
questions about the accuracy and validity of the infection-control data
used by Pronovost et al. in both of their studies. Indeed, the conclusions
of virtually every study (not just Pronovost et al.’s two studies) that is
based, at least in part, on infection rates submitted by participating
(and often not “blinded”2) hospitals can only be as valid as the reported
data are accurate, valid, and standardized. Additionally, that the masking
of infections with antibiotic therapy; inadequate surveillance; staff
discretion; and inconsistent definitions of a specific type of infection,
such as “catheter-related bloodstream infection,” could also contribute to
overly optimistic and misleading results and conclusions, too, warrant
discussion.
Conclusive data indicate that catheter-related bloodstream infections
have been associated with significant patient morbidity and mortality.1,2
Not as clear, however, is the precise amount by which specific
interventions reduce the risk of these infections. Highlighting a
distinction with a potentially significant difference, it is recommended
that studies whose conclusions about infection rates are based on data
that were submitted by hospitals themselves, and which have not been
independently validated by, for example, state agencies, disclose that, as
published by the GAO and others,4,5 a hospital’s publicly reported
infection rate may not be the same as, and may be lower than, its actual
infection rate. In truth, evaluations of the effectiveness of
interventions designed to reduce infection rates would seemingly benefit
from, if independent validation of these rates is not feasible, not only
the use of a standardized definition of an infection, but also the
calculation of a correction factor to account and compensate for the
possibility of measurement bias and to minimize potential disparity
between reported and actual infection rates. Otherwise, cited reductions
in a hospitals’ reported infection rates, while not necessarily without
significance, could be questioned as potentially inaccurate and overly
optimistic, if not invalid and unrealistic.
References:
1. Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove
S, Sexton B, Hyzy R, Welsh R, Roth G, Bander J, Kepros J, Goeschel C. An
intervention to decrease catheter-related bloodstream infections in the
ICU. N Engl J Med. 2006 Dec 28;355(26):2725-32.
2. Pronovost PJ, Goeschel CA, Colantuoni E, Watson S, Lubomski LH,
Berenholtz SM, Thompson DA, Sinopoli DJ, Cosgrove S, Sexton JB, Marsteller
JA, Hyzy RC, Welsh R, Posa P, Schumacher K, Needham D. Sustaining
reductions in catheter related bloodstream infections in Michigan
intensive care units: observational study. BMJ. 2010 Feb 4;340:c309. doi:
10.1136/bmj.c309.
3. Consumers Union. Deadly Infections. Consumer Reports. March 2010.
Pages 16-21.
4. United States Government Accountability Office (GAO). Healthcare-
associated infection in
hospitals. An Overview of State Reporting Programs and Individual Hospital
Initiatives to
Reduce Certain Infections. September 2008. GAO-08-808. p. 1-49.
5. South Carolina Department of Health and Environmental Control
Hospital Acquired
Infections (HAIs) in South Carolina. Shining a Light on Hospital
Infections.
Competing interests:
I have no competing interests. I am, however, employed by a manufacturer of automated equipment used to clean and disinfect flexible endoscopes.
Competing interests: No competing interests