Papers

Collaborative quality improvement to promote evidence based surfactant for preterm infants: a cluster randomised trial

BMJ 2004; 329 doi: http://dx.doi.org/10.1136/bmj.329.7473.1004 (Published 28 October 2004) Cite this as: BMJ 2004;329:1004
  1. Jeffrey D Horbar, chief executive and scientific officer (horbar{at}vtoxford.org)1,
  2. Joseph H Carpenter, director of technical operations1,
  3. Jeffrey Buzas, associate professor of mathematics and statistics2,
  4. Roger F Soll, director of clinical trials1,
  5. Gautham Suresh, assistant professor of paediatrics3,
  6. Michael B Bracken, professor of epidemiology4,
  7. Laura C Leviton, senior program officer5,
  8. Paul E Plsek, quality improvement consultant6,
  9. John C Sinclair, professor emeritus7
  1. 1 Vermont Oxford Network, 33 Kilburn Street, Burlington, VT 05401, USA
  2. 2 University of Vermont, Burlington, VT
  3. 3 Medical University of South Carolina, Charleston, SC, USA
  4. 4 Center for Perinatal, Pediatric and Environmental Epidemiology, Yale University, New Haven, CT, USA
  5. 5 Robert Wood Johnson Foundation, Princeton, NJ, USA
  6. 6 Paul E Plsek and Associates, Atlanta, GA, USA
  7. 7 Department of Pediatrics, McMaster University, Hamilton, Ontario, Canada
  1. Correspondence to: J D Horbar
  • Accepted 13 July 2004

Abstract

Objective To test a multifaceted collaborative quality improvement intervention designed to promote evidence based surfactant treatment for preterm infants of 23-29 weeks' gestation.

Design Cluster randomised controlled trial

Setting and participants 114 neonatal intensive care units (which treated 6039 infants of 23-29 weeks gestation born in 2001).

Main outcome measures Process of care measures: proportion of infants receiving first surfactant in the delivery room, proportion receiving first surfactant more than two hours after birth, and median time from birth to first dose of surfactant. Clinical outcomes: death before discharge home, and pneumothorax.

Intervention Multifaceted collaborative quality improvement advice including audit and feedback, evidence reviews, an interactive training workshop, and ongoing faculty support via conference calls and email.

Results Compared with those in control hospitals, infants in intervention hospitals were more likely to receive surfactant in the delivery room (adjusted odds ratio 5.38 (95% confidence interval 2.84 to 10.20)), were less likely to receive the first dose more than two hours after birth (adjusted odds ratio 0.35 (0.24 to 0.53)), and received the first dose of surfactant sooner after birth (median of 21 minutes v 78 minutes, P < 0.001). The intervention effect on timing of surfactant was larger for infants born in the participating hospitals than for infants transferred to a participating hospital after birth. There were no significant differences in mortality or pneumothorax.

Conclusion A multifaceted intervention including audit and feedback, evidence reviews, quality improvement training, and follow up support changed the behaviour of health professionals and promoted evidence based practice.

Introduction

Health services continue to show major gaps between routine practice and what the research evidence suggests is optimal patient care.1 In neonatology, systematic reviews indicate that prophylactic surfactant treatment of high risk preterm infants reduces risk of death and pneumothorax by 40%, and that earlier treatment is more effective than later treatment.2 3 Despite this evidence, few such infants routinely receive prophylactic surfactant treatment, and many infants, particularly those born at outlying hospitals, receive delayed treatment.4

Various strategies for promoting behaviour change and evidence based practice have been proposed.58 Experience from the Vermont Oxford Network suggests that multidisciplinary collaborative quality improvement based on four key “habits” (change, evidence based practice, systems thinking, and collaborative learning) modifies practice in neonatal intensive care units, improves clinical outcomes, and reduces costs.9 10

We therefore conducted a cluster randomised controlled trial11 to test whether teams in neonatal intensive care units exposed to a multifaceted collaborative quality improvement intervention based on the four key habits would administer the first dose of surfactant sooner after birth, and achieve improved patient outcomes for preterm infants of 23-29 weeks' gestation.

Methods

Network Vermont Oxford

The Vermont Oxford Network is a voluntary collaboration of health professionals whose mission is to improve the quality and safety of medical care for newborn infants and their families (http://www.vtoxford.org/). The network maintains a database for very low birthweight infants at member hospitals.12

Sample size

According to network data for 1998, 16.4% of very low birthweight infants at member hospitals received surfactant in the delivery room (standard deviation of individual hospital proportions 21%), the median time to surfactant administration was 105 minutes, and the infant mortality was 16.8% (standard deviation of individual hospital proportions 7.5%). We set a sample size of 55 hospitals in each of the two study arms before starting the trial to give 90% power (two sided α = 0.05) to detect an increase to 29.4% in the proportion of infants receiving surfactant in the delivery room (13% difference). This sample size also had 90% power to detect a 48 minute difference in median time to surfactant administration and 80% power to detect a decrease in mortality to 12.8% (4% difference). We expected a mortality difference of 4% if delivery room surfactant treatment increased by 60% and if relative risk of death for infants receiving this treatment was 0.62.

To conform to the cluster randomisation, power calculations accounted for both intrahospital and interhospital variability.13 14

Eligibility, enrolment, and randomisation

Of the 300 North American hospitals in the Vermont Oxford Network, 178 were eligible to enter the trial and 114 enrolled (fig 1). A secure computer program assigned enrolled hospitals to one of two study arms using a completely randomised design. Assignments were concealed from investigators except for JDH, who notified hospitals in writing on 24 May 2000. Other investigators learnt the identity of the intervention hospitals at the workshop (see below).

Fig 1
Fig 1

Flow of participants through study. Of 300 North American hospitals in the Vermont Oxford Network, 178 had neonatal units eligible to participate and 114 of these enrolled in the study

Components of the multifaceted intervention

Audit and feedback—In July 2000 intervention hospitals received confidential, individualised feedback from the Vermont Oxford Network including site-specific information and peer comparisons related to the administration and timing of surfactant, and delivery room practice for infants of 23-29 weeks' gestation born in 1998 and 1999.

Workshop—Intervention hospitals were invited to attend a workshop held from 16 to 18 September 2000 (see details of workshop on bmj.com). Of the 57 teams invited, 56 attended (53 physicians, 46 nurses, and 21 respiratory therapists). The workshop included didactic sessions, facilitated site team exercises, and multi-institutional group exercises designed to promote four key “habits” (change, evidence based practice, systems thinking, and collaborative learning).9 15

Ongoing support—Collaboration among intervention arm teams was fostered through quarterly conference calls and an email discussion list.

Control hospitals received centre-specific, confidential reports routinely prepared for members of the Vermont Oxford Network.

Outcome measures

Primary outcome measures consisted of process of care measures (surfactant treatment in the delivery room, first surfactant treatment more than two hours after birth (among those receiving surfactant), and time after birth when first surfactant dose was administered) and infant outcome measures (death before discharge from hospital and pneumothorax).

Secondary outcome measures, identified before the trial started, included common morbidities2 3 and co-interventions for preterm infants (see table 5 for details).

Table 5

Secondary study outcomes for 114 hospitals that participated in trial of quality improvement intervention for surfactant treatment of preterm infants, in the assessment year, 2001.* Values are percentages unless stated otherwise

View this table:

Statistical methods

The baseline year for comparing the two study arms began 25 May 1999. We assessed outcomes for infants born in 2001 (1 January to 31 December) with gestational ages of 23 weeks 0 days to 29 weeks 6 days, with birth weights of 401 to 1500 g, and without major birth defects. We pre-specified conducting primary analyses among all infants, and separately among infants born in the participating hospitals (inborn) and among infants transferred to a participating hospital after birth (outborn).

Our analyses were on an intention to treat basis. We analysed primary and secondary dichotomous outcomes using logistic regression. Generalised estimating equations accounted for within-hospital correlation.16 We adjusted analyses for infant covariates (gestational age, birth location, ethnicity) and hospital covariates (type of neonatal intensive care unit, teaching status, and annual volume of infant patients).

We used a log rank test with adjustment for clustering to test differences in time to administration of surfactant. We used proportional hazards models, accounting for clustering, adjusted for infant and hospital covariates to calculate adjusted hazard ratios (ratios > 1 indicate earlier surfactant).16 We used weighted linear mixed models to analyse changes over time in surfactant treatment and timing of treatment.

We calculated intraclass correlation coefficients17 for binary outcomes using SAS Proc Genmod and for time to first dose using Proc Mixed. The only covariate was trial arm, and complete trial data were used.

The statistician was masked to study assignments for analysis of primary outcomes. SAS version 8.2 (SAS Institute, Cary, NC) and STATA version 7.0 (College Station, TX) were used. Study inferences were based on the adjusted analyses.

Results

Baseline comparability

Table 1 shows the baseline characteristics of the participating hospitals. The hospitals in the two study arms were generally similar in types of neonatal intensive care unit, teaching status, and annual volume of preterm infant admissions, although more of the neonatal intensive care units in the control arm were type A (restricted assisted ventilation or no major surgery available).

Table 1

Baseline characteristics of 114 hospitals in the Vermont Oxford Network that participated in trial of quality improvement intervention for surfactant treatment of preterm infants. Values are medians (interquartile ranges) unless stated otherwise*

View this table:

Infant characteristics in the assessment year

Tables 2 and 3 show the characteristics of the infants in the 2001 assessment year. The proportion of white infants was higher at intervention hospitals.

Table 2

Median (interquartile range) characteristics of infants treated in 114 hospitals that participated in trial of quality improvement intervention for surfactant treatment of preterm infants, in the assessment year, 2001*

View this table:
Table 3

Characteristics of infants treated in 114 hospitals that participated in trial of quality improvement intervention for surfactant treatment of preterm infants, in the assessment year, 2001*

View this table:

Primary outcomes

Delivery room surfactant treatment was significantly higher in the intervention than in the control arm for all infants (adjusted odds ratio 5.38 (95% confidence interval 2.84 to 10.20)), for the infants born at participating hospitals (inborn) (adjusted odds ratio 6.16 (3.03 to 12.54)), and for the infants transferred to participating hospitals after birth (outborn) (adjusted odds ratio 1.99 (1.12 to 3.53)) (table 4). First surfactant treatment more than two hours after birth was significantly lower in the intervention than in the control arm for all infants (0.35 (0.24 to 0.53)), inborn infants (0.29 (0.18 to 0.47)), and outborn infants (0.57 (0.37 to 0.89)). Infants in the intervention arm received surfactant significantly sooner after birth than did infants in the control arm (median 21 minutes (interquartile range 10-128) v 78 minutes (29-410), adjusted hazard ratio 1.57 (95% confidence interval 1.42 to 2.07)). For inborn infants, median times were 18 minutes and 75 minutes in the intervention and control arms (adjusted hazard ratio 1.83 (1.50 to 2.23); for outborn infants the median times were 74 and 103 minutes (adjusted hazard ratio 1.30 (1.00 to 1.69)). When restricted to infants who received surfactant, the median times to first dose for all infants were 15 and 52.5 minutes in the intervention and control arms (adjusted hazard ratio 1.75 (1.47 to 2.09)).

Table 4

Dichotomous primary study outcomes for 114 hospitals that participated in trial of quality improvement intervention for surfactant treatment of preterm infants, in the assessment year, 2001.* Values are percentages unless stated otherwise

View this table:

There were no significant differences in mortality or pneumothorax. There was a trend towards decreased pneumothorax for outborn infants in the intervention arm (adjusted odds ratio 0.58 (0.33 to 1.03)).

There were significant interactions between treatment arm and location of birth for surfactant administration in the delivery room (P < 0.003) and for first surfactant treatment more than two hours after birth (P < 0.001) resulting from larger effect sizes for inborn infants than for outborn infants (table 4). There was a significant interaction between treatment arm and gestation for first surfactant treatment more than two hours after birth among inborn infants (P = 0.01) due to changes in the size, but not the direction, of the effect across gestational age. The differences between treatment arms for this measure were greatest at 25 weeks and smallest at 29 weeks.

The intraclass correlation coefficients were 0.31 for surfactant administration in the delivery room, 0.09 for first surfactant treatment more than two hours after birth, 0.25 for time to first surfactant dose, and 0.01 for both mortality and pneumothorax.

Secondary outcomes

Intervention arm infants were significantly more likely to be intubated in the delivery room (adjusted odds ratio 1.65 (1.19 to 2.29)) and to receive surfactant at any time (1.55 (1.08 to 2.23)) (table 5). The overall proportions of infants who were intubated and received conventional ventilation or high frequency ventilation were similar in the two arms (adjusted odds ratios 1.10 (0.78 to 1.56) and 1.08 (0.65 to 1.80) respectively). Severe intraventricular haemorrhage (grades 3 or 418) was significantly lower in the intervention arm (0.70 (0.56 to 0.87)); there was a trend towards a decrease for any intraventricular haemorrhage (grades 1 to 418) (0.80 (0.63 to 1.00)). There was a trend towards increased risk of patent ductus arteriosus in the intervention arm (1.27 (0.96 to 1.67)).

Changes in 2000 and 2001

We saw persistent reductions in the median time of the first surfactant dose and in interhospital variability for intervention hospitals starting in the last quarter of 2000, after the workshop (fig 2). The intervention hospitals showed significant changes in 2000 and 2001 for the proportion of all infants who received surfactant and time of first dose (both P < 0.001), but not the control hospitals (P = 0.81 and P = 0.18 respectively). The changes were significantly different between the two arms (time by treatment arm interaction, both P < 0.001).

Fig 2
Fig 2

Median (interquartile range) time after birth at which first dose of surfactant was administered to preterm infants in neonatal intensive care units by calendar quarter. Units in intervention group were notified of their status in May 2000, were given individualised feedback in July 2000, and were invited to a quality improvement workshop in September 2000

Discussion

The multifaceted intervention tested in this trial produced significant improvement in the timing of surfactant use for very low birthweight infants in neonatal intensive care units. We observed improvements both for infants born at participating hospitals (inborn) and for those born outside and transferred after birth (outborn), although the effect was larger for the infants born inside (unsurprising given delays between birth and transfer and the involvement of non-study teams in providing delivery room care and initial stabilisation for outborn infants).

Comparison with other studies

Systematic reviews suggest that audit and feedback,5 interactive workshops,6 and multidisciplinary collaboration7 can modify professional practice and improve patient outcomes. A systematic review of 117 studies assessing implementation of practice guidelines, including 46 cluster randomised trials of multifaceted interventions, showed small to modest improvements in the processes of patient care.8 Eight studies that evaluated combinations of educational materials, educational meetings, and audit and feedback showed a median 3% improvement in target outcome (range 2.6% to 9.0%). The 37% increase we observed in delivery room surfactant treatment is large in comparison.

Patient outcomes

On the basis of results from systematic reviews,2 3 we expected that if our intervention resulted in earlier surfactant treatment it would also lead to reductions in death and pneumothorax. However, we observed no significant reductions in either death or pneumothorax. There are several possible explanations. Firstly, our wide confidence intervals for death and pneumothorax are compatible with the effects observed in the systematic reviews. Secondly, our trial was not designed to test the efficacy of prophylactic or early surfactant treatment, since infants were not randomly assigned to these treatment options. Thirdly, our study may have had insufficient power to detect an effect on mortality. We had 80% power to detect a 4% difference in mortality, assuming a 60% increase in surfactant administration in the delivery room and a relative risk for death of 0.6. However, we observed only a 37% increase in delivery room surfactant administration. Furthermore, a relative risk of death of 0.6 for infants receiving delivery room surfactant, although based on the systematic reviews, may have been too large an expected effect: antenatal corticosteroid therapy, which reduces the risk of death of preterm infants, has increased and time after birth to first dose of surfactant has decreased since the original trials were performed.4 19 These changes may reduce the additional benefit expected from prophylactic surfactant.

The intervention resulted in significantly higher rates of endotracheal intubation in the delivery room. This was an anticipated trade off for earlier surfactant treatment, since infants must be intubated to receive surfactant. Importantly, the intervention did not increase the overall intubation rate since the rates of conventional and high frequency ventilation were similar in the two study arms.

Earlier surfactant treatment has not been shown to reduce the risk for intraventricular haemorrhage or to increase the risk of patent ductus arteriosus.2 3 The decreased risk of severe intraventricular haemorrhage in the intervention arm in our trial might have resulted from changes in the composition and performance of delivery room resuscitation and stabilisation teams or other unmeasured changes in care, suggesting subjects for future study. The trends towards decreased risk of pneumothorax in infants transferred to the hospital after birth and increased risk for patent ductus arteriosus should be interpreted with caution.

A test of collaborative improvement

Collaborative improvement initiatives have been applied in a variety of healthcare settings.15 20 Typical elements of the initiatives include audit and feedback, self assessment, multidisciplinary teams, quality improvement training, evidence review, setting aims, and establishing social networks for information sharing and peer accountability.

Systematic reviews show that audit and feedback can improve professional practice, but effects are generally modest.5 8 The minimal feedback in routine reports provided by the Vermont Oxford Network to our control hospitals did not result in changes of timing for surfactant treatment, suggesting that feedback alone is insufficient to result in the large practice changes we observed.

We designed the two day workshop for the intervention hospitals based on four key habits for clinical improvement to be multidisciplinary and highly interactive (see bmj.com). Hospital teams learnt the habit for change involving the PDSA (plan, do, study, act) improvement model and left the workshop having identified their own measurable improvement aims and having designed several initial PDSA improvement cycles.21

During the workshop, the teams applied the habit for evidence based practice,22 reviewing evidence from systematic reviews, reflecting on their own practices, and assessing trade-offs for different treatment strategies. The intervention did not promote a specific practice guideline. After the workshop, the teams reviewed and modified their aims involving the staff back home. The literature on innovation suggests that personal reflection by clinical leaders, resulting in self imposed goals, is essential in obtaining the commitment to change.23 The workshop encouraged participants to apply systems thinking and look beyond the clinical aims to underlying structures, patterns, and processes within their hospitals. Social networking is an important contributor to the success of collaborative initiatives.20 23 Workshop exercises, discussion periods, the opening dinner, conference calls, and the email discussion list were designed to promote collaborative learning within and among teams.

Limitations of this study

Our study was not designed to determine the effects of individual components of the multifaceted intervention, or its cost. Although only 114 of the 178 eligible hospitals enrolled in the study, enrolled and non-enrolled hospitals had similar characteristics, and our findings are probably generalisable to other neonatal intensive care units. However, we cannot predict the generalisability of our findings to other clinical settings involving different target practices and evidence, and we cannot predict whether a different workshop faculty would achieve similar results. Our trial is unlikely to suffer from the problems reported in a recent review of cluster trials since randomisation was secure and blinded, and there was no attrition of clusters.24

Conclusion

A multifaceted collaborative improvement intervention—including audit and feedback, evidence reviews, quality improvement training based on four key habits, and follow up support—changed the behaviour of neonatologists and promoted evidence based practice.

What is already known on this topic

The quality improvement collaborative is a promising intervention to encourage evidence based practices and has been used in a variety of healthcare settings

What this study adds

A multifaceted collaborative quality improvement intervention based on four key habits was used to promote evidence based surfactant therapy for preterm infants

It produced significant changes in the timing of surfactant use and a 37% increase in surfactant treatment in the delivery room

It did not result in significant changes in mortality or pneumothorax, however, but the study's power to detect these changes may have been low

Footnotes

  • Embedded Image Participants in the Vermont Oxford Network, and details of the intervention workshop appear on bmj.com

  • Contributors All authors contributed to the planning, conduct, and reporting of this study. JDH is guarantor for the study.

  • Funding Funded by a grant from the Agency for Healthcare Research and Quality (R01 HS10528, Horbar JD, principal investigator). The authors' work was independent of the funding agency.

  • Competing interests JDH is chief executive and scientific officer, RFS is a director, and JHC is an employee of the Vermont Oxford Network. JB received funding for research from the Vermont Oxford Network. RFS has acted as a consultant and spoken at meetings sponsored by various manufacturers of surfactant products, including Ross Laboratory Division of Abbott Laboratories and Chiesi Pharmaceuticals. RFS has also received grant funding from Ross Laboratories to conduct research regarding surfactant therapy. PEP has served as a paid consultant to the Vermont Oxford Network and other clients who sponsor improvement collaboratives. GS has served as a paid consultant to Dey Laboratories, a manufacturer of surfactant.

  • Ethical approval The study was approved by the University of Vermont Institutional Review Board.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
View Abstract