Published 11 November 2009, doi:10.1136/bmj.b4353
Cite this as: BMJ 2009;339:b4353

Research

Evaluation of modernisation of adult critical care services in England: time series and cost effectiveness analysis

¹ Health Services Research Unit, London School of Hygiene and Tropical Medicine, London WC1E 7HT, ² Intensive Care National Audit and Research Centre, Tavistock House, London WC1H 9HR

Abstract

Objective To evaluate the impact and cost effectiveness of a programme to transform adult critical care throughout England initiated in late 2000.

Design Evaluation of trends in inputs, processes, and outcomes during 1998-2000 compared with last quarter of 2000-6.

Setting 96 critical care units in England.

Participants 349 817 admissions to critical care units.

Interventions Adoption of key elements of modernisation and increases in capacity. Units were categorised according to when they adopted key elements of modernisation and increases in capacity.

Main outcome measures Trends in inputs (beds, costs), processes (transfers between units, discharge practices, length of stay, readmissions), and outcomes (unit and hospital mortality), with adjustment for case mix. Differences in annual costs and quality adjusted life years (QALYs) adjusted for case mix were used to calculate net monetary benefits (valuing a QALY gain at £20 000 ($33 170, 22 100)). The incremental net monetary benefits were reported as the difference in net monetary benefits after versus before 2000.

Results In the six years after 2000, the risk of unit mortality adjusted for case mix fell by 11.3% and hospital mortality by 13.4% compared with the steady state in the three preceding years. This was accompanied by substantial reductions both in transfers between units and in unplanned night discharges. The mean annual net monetary benefit increased significantly after 2000 (from £402 ($667, 445) to £1096 ($1810, 1210)), indicating that the changes were relatively cost effective. The relative contribution of the different initiatives to these improvements is unclear.

Conclusion Substantial improvements in NHS critical care have occurred in England since 2000. While it is unclear which factors were responsible, collectively the interventions represented a highly cost effective use of NHS resources.

Introduction

Politicians, managers, clinicians, and the public have strongly held views as to the costs and benefits (or lack of benefits) of the additional investment and the attempts to "modernise" the English National Health Service (NHS) since 2000. There is little rigorous evidence to inform such debates.

In 2000 the Department of Health in England advocated the "modernisation" of adult critical care¹ and, as a parallel initiative, funded a 35% increase in the number of beds.² Responsibility for leading the transformation of services lay with the newly created NHS Modernisation Agency and focused on integrating critical care with other acute services in the hospital, thus creating "comprehensive critical care."

Key elements were to be the creation of 29 clinical networks, covering the whole country and sharing agreed protocols, and, within hospitals, the establishment of outreach services and the adoption of care bundles. Care bundles are groups of clinical guidelines that, when implemented and monitored together, are believed to produce better outcomes than would be the case with individual implementation.^{3 4} Implementation of one—the ventilator care bundle, which has four components that aim to prevent ventilator acquired pneumonia—was supported by the Modernisation Agency. Others include the central venous line bundle to prevent infection⁵ and the sepsis, resuscitation, and management bundles.⁶ In addition, hospitals were encouraged to undertake local improvement projects, which resulted in many diverse activities.

Although there has been no previous attempt to evaluate the impact of the changes instituted since 2000, some constituent elements have been investigated. There is mixed evidence about the impact of outreach services on mortality, unplanned admissions, and length of stay.^{7 8} Four studies of the ventilator care bundle reported reductions in the rate of ventilator associated pneumonia,^{4 9 10 11} another reported a reduction in mortality,¹² and one reported shorter lengths of stay and duration of ventilation but no impact on crude mortality.¹³

To evaluate the impact of these initiatives it was necessary to define a point in time when transformation started. The Modernisation Agency’s critical care programme got under way at the end of 2000, which coincided with the substantial increase in investment (fig 1). We therefore compared the periods before and after the final quarter of 2000.

View larger version (24K): [in this window] [in a new window] [PowerPoint Slide for Teaching]   Fig 1 Peak periods of implementation of modernisation activities and capacity expansion

 
We compared trends in inputs, organisation, processes, and outcomes of critical care in a representative sample of English hospitals before and after the final quarter of 2000; explored associations of trends with each component intervention; and estimated the cost effectiveness of the interventions overall.

Methods

Data
Inputs
The Department of Health provided data on the number of available beds since 1999,¹⁴ the NHS reference costs provided the number of bed days and unit costs,¹⁵ and the case mix programme database (CMPD) from the Intensive Care National Audit and Research Centre (ICNARC) provided estimates of the number of occupied beds since 1998.¹⁶ Units were categorised by their increase in capacity from 2000 to 2004: <20% growth (40% of units), 20-50% (25% of units), or >50% (35% of units).

Classification of critical care networks
The Modernisation Agency categorised networks on five occasions during 2002-3 as excellent, ready to receive, developing, at risk, or at significant risk. We classified seven networks (24%) that achieved "ready to receive" or "excellent" status at the first assessment as "earlier adopters" and 16 (55%) as "later adopters"—those that did not achieve this status until later assessments. Six networks that had not achieved the "ready to receive" threshold by the final assessment were classified as "other."

Classification of critical care units
A previous survey in 2005 showed that by the end of 2001 half the units had established outreach services (table 1).¹⁷ We conducted a second survey in November 2007 to identify whether and when a unit had implemented the ventilator care bundle, with the level of compliance for each year categorised as >90%, 75-90%, or <75% of eligible patients. By 2004 half of the units reported they had implemented the bundle (table 2). Three quarters of units that implemented the bundle reported compliance levels above 90%, with only seven (6%) below 75%.

View this table: [in this window] [in a new window]   Table 1  Implementation of critical care outreach services by year in case mix programme units (response rate 80%)

 

View this table: [in this window] [in a new window]   Table 2  Survey of implementation of ventilator care bundle in case mix programme units (response rate 82%)

 
Of the 96 English units with case mix programme data, 31 (32%) were classified as "later adopters"—they had not implemented an outreach service by the end of 2003 or had not implemented the ventilator care bundle by the end of 2005, or both—and 58 (60%) were classified as "earlier adopters"—they had implemented both by these dates.

There was no strong association between earlier adopter units and earlier adopter networks. In earlier adopter networks, 19 (73%) units were earlier adopters whereas in later adopter networks 30 (59%) were so classified (P=0.12). There was also no strong association between a unit’s classification and its change in bed capacity between 2000 and 2004.

Processes and outcomes
The case mix programme provided data on case mix, transfers, discharges, length of stay, readmissions, and unit and hospital mortality for patients admitted to critical care units. Data for 1998-2006 were available for 349 817 admissions in 96 English units.

Analysis
Comparison of trends
We compared data for 1998-2000 with data for 2000-6. We used regression analysis to compare trends in processes: Poisson regression for dichotomous dependent variables to estimate risk ratios for the average annual change in each period and linear regression for continuous dependent variables to estimate mean change per year and its 95% confidence intervals. The two time periods were fitted as an interaction to test if the trends differed between them. The time periods were based on quarterly data adjusted for quarterly seasonality with the final quarter of 2000 representing the end of the first time period and the beginning of the second. We used predicted mortality from the ICNARC model to adjust for case mix.¹⁸ Units were included as random effects to allow for clustering of admissions within units. For the Poisson regression analysis we estimated bias corrected 95% confidence intervals using bootstrap resampling with 1000 replications.¹⁹

Cost effectiveness analysis
We adopted a hospital perspective that included all costs for hospital admissions during an episode, including the index admission and any readmissions. Length of stay was valued using NHS reference costs (1998-2005) to give an episode cost for each patient. For 1998-2003, a single unit cost for intensive care unit bed days was available for each trust, whereas for 2004 and 2005 the unit costs were taken as the mean across three levels of care weighted by the relative number of bed days in each category. The unit cost for general wards was the weighted mean for all healthcare resource groups for non-elective bed days from NHS reference costs. All unit costs were inflated to 2006-7 prices.²⁰

Mortality data were extrapolated to calculate lifetime QALYs. We assumed that the life expectancy for survivors was 80% that of the general population matched for age and sex²¹ with their quality of life downweighted by 20%. Those patients who died in hospital were assigned a QALY of zero. Each individual’s lifetime QALYs (discounted at 3.5%) were valued at £20 000, (22 100, $33 170) and their costs subtracted to give a net monetary benefit.²² Similar regression models (to those described above) were used with annual costs, QALYs, and net monetary benefit as dependent variables and separate components of the ICNARC risk prediction model as independent variables.

Association with organisational changes
We used similar regression models to evaluate the impact of three organisational changes after 2000: increase in capacity, adoption of clinical networks, and adoption of outreach and ventilator care bundle. Interactions were fitted between each type of change to test whether trends in processes and outcomes were associated with any particular variable.

Results

Trends in inputs
The annual expenditure on critical care increased in real terms from £700m (1999-2000) to £1bn (2005-6). This was associated with a 35% increase in the number of staffed beds in general intensive care units (fig 2), with more of the increase in high dependency (106%) than in intensive care beds (23%). The main increase occurred during winter 2000-1, when high dependency beds increased by 57.5% and intensive care beds by 7.2%, after which there was an average 9.0% rise per year for high dependency and 1.4% rise per year for intensive care beds. Over the period 1999-2006, the mean cost of an intensive care bed day rose slightly from £1551 to £1647 (2006-7 prices).

View larger version (39K): [in this window] [in a new window] [PowerPoint Slide for Teaching]   Fig 2 Number of critical care beds in England located in general units providing intensive care 1999-2006

 
Trends in processes
Case mix of admissions
From 1998 to 2006 the proportion of women in the case mix increased, as did the mean age of those admitted (from 59.6 to 60.5) (table 3). Although there was no consistent change either in the proportion with at least one chronic condition or in the mean physiology score, the mean predicted risk of mortality rose from 30.5% in 1998 to 32.1% in 2000 but subsequently fell to 31.4% in 2006, indicating that less severe cases were being admitted. Analysis by 10ths of predicted risk of mortality in England showed no widening in the distribution of cases.

View this table: [in this window] [in a new window]   Table 3  Characteristics of patients admitted to case mix programme units in England (1998-2006)

 
Transfers and discharges
Transfers out of units to another unit to receive the same level of care declined by 11.0% a year after 2000 (table 4) compared with annual increases of 2.6% before 2000 (P<0.001). Similarly, transfers into units declined 8.7% a year after 2000 compared with an annual 3.8% increase beforehand (P<0.001) (fig 3).

View larger version (28K): [in this window] [in a new window] [PowerPoint Slide for Teaching]   Fig 3 Critical care transfers as proportion of admissions (transfers in) and discharges (transfers out) 1998-2006

 

View this table: [in this window] [in a new window]   Table 4  Comparison of average annual changes (adjusted for case mix) in transfers and discharges between 1998-2000 and 2000-6 (96 units)

 
Early discharges because of a shortage of critical care beds declined from 7.1% in 1998 to 3.3% in 2006 (fig 4). Although the rate of night discharges (midnight to 4:59 am) steadily increased from 2.8% in 1998 to 4.2% in 2006 (fig 5), the proportion reported as being because of a shortage of critical care beds declined (44.5% to 21.8%), suggesting the proportion deemed "unplanned" fell from 1.2% to 0.9%. Before 2000, unplanned night discharges were increasing by 3.1% a year whereas afterwards they decreased by 7.7% a year (P=0.008) (table 4). A similar pattern emerged when we considered discharges between 10 pm and 6:59 am.

View larger version (35K): [in this window] [in a new window] [PowerPoint Slide for Teaching]   Fig 4 Early discharges, reported delayed discharges, and discharges directly to normal place of residence as proportion of all discharges 1998-2006

 

View larger version (32K): [in this window] [in a new window] [PowerPoint Slide for Teaching]   Fig 5 Night discharges and unplanned night discharges as proportion of all discharges 1998-2006

 
In contrast, the number of discharges delayed due to a shortage of ward beds rose steadily (fig 4) from 2.7% in 1998 to 14.2% by 2006, though the average annual increase after 2000 (16.2%) was lower than before (38.0%) (P<0.001) (table 4). There was an increase in discharges directly to patients’ normal place of residence (from 0.6% to 2.1%). The trend was not significantly greater after 2000 than before.

Length of stay and readmissions
After adjustment for differences in case mix, the mean length of stay increased before 2000 by 0.243 days a year but by only 0.036 days a year afterwards (P<0.001) (table 5). There was no significant difference in the decline in readmissions within 24 hours before and after 2000 (table 6). There was, however, a faster decline in readmissions within 48 hours after 2000 (P=0.006) (fig 6).

View larger version (28K): [in this window] [in a new window] [PowerPoint Slide for Teaching]   Fig 6 Readmissions within 24 and 48 hours as proportion of all discharges 1998-2006

 

View this table: [in this window] [in a new window]   Table 5  Comparison of average annual changes in mean length of stay (days) before and after 2000 (n=96 units)

 

View this table: [in this window] [in a new window]   Table 6  Comparison of average annual changes in readmission rates before and after 2000 (n=96 units)

 
Trends in outcomes
While unit mortality adjusted for case mix did not change between 1998 and 2000, subsequently it fell by an average of 2.0% a year (P<0.001) (table 7), indicating a total fall of 11.3% in the six years after the final quarter of 2000. A similar pattern was observed for hospital mortality adjusted for case mix, with an average annual decline after 2000 of 2.4%, indicating a total fall of 13.4% over six years. Annual unit and hospital mortality adjusted for case mix also show a similar pattern (fig 7).

View larger version (21K): [in this window] [in a new window] [PowerPoint Slide for Teaching]   Fig 7 Relative risk (95% confidence interval) of hospital mortality and unit mortality adjusted for case mix, 1998-2006

 

View this table: [in this window] [in a new window]   Table 7  Relative risk for annual change in unit and hospital mortality adjusted for case mix before and after 2000 (n=96 units)

 
Cost effectiveness of changes in critical care
The decrease in the annual change in the mean length of stay, both in critical care (by 0.18 days, P<0.001) and for the subsequent ward stay (by 0.35 days, P<0.001), meant the mean annual cost increases after 2000 (£196) were smaller than before (£391). The mean incremental cost (adjusted for case mix) therefore fell by £195 (P<0.001).

Annual improvements in the mean lifetime QALYs (adjusted for case mix) were slightly greater after 2000 and so the mean incremental QALYs were positive (0.025), though not significant (P=0.06) (table 8). Valuing a QALY at £20 000, coupled with the decline in incremental costs (–£195), resulted in a positive incremental net monetary benefit of £692 (P=0.008). Hence, if the differences in costs and QALYs can be attributed to the interventions, then they were relatively cost effective.

View this table: [in this window] [in a new window]   Table 8  Incremental cost effectiveness (mean predicted costs*, QALYs, and net monetary benefit) for 2000-6 compared with 1998-2000

 
Sensitivity analysis had little effect on the main findings. Firstly, the effect of taking mean unit costs from all English NHS trusts, rather than just participants from the case mix programme, led to a relative decrease in costs after 2000, lower incremental costs, and a higher incremental net monetary benefit (table 9). Secondly, re-weighting mean unit costs to include a 20% high dependency component led to a relative reduction in mean costs after 2000 and a higher incremental net monetary benefit. Re-weighting to reflect the gradual increase in the size of the high dependency component resulted in similar results to the base case. Thirdly, when the summary baseline probability of death was used for case mix adjustment rather than the separate components of the ICNARC model, this led to greater QALY gains, smaller incremental costs, and a larger incremental net monetary benefit. Fourthly, assuming the long term survival and the quality of life of patients was the same as for the general population²³ yielded a higher incremental net monetary benefit. Finally, extending the analysis from 96 to all 159 units in the case mix programme also led to a higher incremental net monetary benefit as did valuing a QALY at £30 000.

View this table: [in this window] [in a new window]   Table 9  Sensitivity analyses reporting incremental costs (£), incremental QALYs, and incremental net monetary benefit (NMB) (£20 000 per QALY) for different scenarios

 
Impact of organisational changes
The benefit of being in a "later adopter" network was greater than being in a "earlier adopter" network: faster declines in the proportion of admissions that were transfers in (10.0% v 6.2% a year), transfers out (13.4% v 4.4% a year), early discharges (12.6% v 4.5% a year), and unplanned night discharges (8.6% v 1.4% a year); smaller increases in delayed discharges (12.2% v 23.6% a year); less increase in mean length of stay (0.015 v 0.089 days a year); and a faster decline in unit (2.5% v 1.3% a year) and hospital mortality (2.9% v 1.9% a year).

The effects of the time at which units adopted outreach and the ventilator care bundle were less clear. Later adopter units were associated with some benefits: faster declines in transfers in (11.8% v 8.2% a year) and unplanned night discharges (8.9% v 3.0% a year); slower increase in delayed discharges (8.1% v 18.4% a year); less increase in length of stay (0.024 v 0.072 days a year); faster decline in hospital mortality (2.8% v 2.2% a year). These units, however, experienced a slower decline in transfers out (9.6% v 13.0% a year) and in early discharges (7.7% v 11.8% a year).

Units with the largest increases in capacity were associated with faster declines in transfers in (13.6% v 5.8% a year), transfers out (17.9% v 8.5% a year), early discharges (17.5% v 6.5% a year), and unplanned night discharges (12.1% v 6.1% a year); and slower increases in delayed discharges (14.4% v 17.7% a year). There were, however, smaller declines in unit mortality (1.8% v 2.7%), though no significant differences for hospital mortality.

Discussion

Main findings
After 2000, unit mortality adjusted for case mix in England fell dramatically by 2.0% a year and hospital mortality by 2.4% a year (compared with no change between 1998 and 2000). This was accompanied by a decrease of 11.0% a year in transfers out (for the same level of care) to other units and a fall of 8.7% a year in transfers in, whereas previously both proportions had been rising. In addition, the proportion of unplanned night discharges declined by 7.7% a year. Despite small increases in average unit costs, the cost effectiveness of critical care increased after 2000, partly as a result of the improvements in outcome and partly because of smaller increases in the mean length of stay.

If such changes can be attributed to the initiatives to transform critical care (increased capacity, clinical networks, outreach services, and the ventilator care bundle) this package can be regarded as a more cost effective intervention than many other healthcare interventions. Improvements were associated with later, rather than earlier, adoption of organisational changes. There are two possible explanations for what seems to be a counterintuitive finding: later adopters might have benefited from the experiences of earlier adopters and also might have spent longer preparing for change and thus established greater commitment from staff. While increased capacity was not directly associated with improvements in outcomes, it was associated with declines in the rate of transfers and early discharges and might have contributed indirectly via unmeasured effects, such as improvements in staff morale.

Despite striking improvements in processes, outcomes, and cost effectiveness in England, the contribution of the explicit programme promoted by the Modernisation Agency is unclear. It is possible that concurrent changes—such as the introduction of "hospital at night,"²⁴ new staff contracts,²⁵ and nurse consultants²⁶—which were also being implemented during this period contributed to the improvements observed.

Methodological limitations
Defining and measuring interventions
Inclusion of interventions had to be restricted to those that were feasible to measure. While this included the principal elements, it was not comprehensive. Creating simple dichotomies (earlier and later adopters) for networks and units might have masked some associations, and data on when interventions were adopted might have been subject to recall bias. Both limitations, however, would have reduced the likelihood of identifying associations rather than created spurious ones.

Measuring inputs
Given the increasing proportion of lower cost high dependency beds over time, use of overall bed numbers would have overestimated costs in the later period and thus underestimated cost effectiveness. The use of average NHS reference costs is justified by their similarity to deriving average unit costs for critical care with micro-costing techniques.²⁷ Changes in the reporting of unit costs in 2004 made little difference given that the main determinants of the incremental net monetary benefit were the relative reduction in length of stay and the gain in QALYs.

Analysis
While the final quarter of 2000 represented the optimum time point to mark the start of the interventions, adoption did not occur until later in many units. This will have tended to reduce the observed impact. Our approach also assumes that any changes in processes and outcomes observed after 2000 are linear.

There is some uncertainty as to the longevity and quality of life of survivors of critical care. Recent research suggests that the assumption of a 20% deficit might overstate the relative decrement,²⁸ and other analyses have assumed the same survival and quality of life as the general population.²⁹ The sensitivity analysis, however, showed that our findings were robust.

It was important to recognise the hierarchical nature of the data in the analysis.³⁰ The main analyses allowed for clustering within critical care units by fitting units as random effects. Networks were not treated as a separate level because of potential biases in those with a lower participation in the case mix programme. Some caution is therefore required in interpreting the findings for network adoption.

Conclusions
This attempt at a nationwide evaluation of policies pursued since the publication of the NHS Plan in 2000, albeit limited to one specific part of health care, suggests that the interventions represent a highly cost effective use of resources. In any time series analysis the attribution of causality is a challenge. This is particularly so for evaluations of complex interventions at a national level. The ability to make causal inferences about the benefits of "modernisation" is limited by the lack of a comparable control group of critical care units in which there were no modernisation interventions. Attribution is also challenged by the presence of other concurrent interventions, such as the introduction of hospital at night and a myriad local modernisation projects. As a consequence, conclusions must be restricted to the observation that significant improvements in outcomes and processes occurred in 2000-6 without being able to link such changes to specific interventions.

Whatever the reasons for the improvements, considerable additional expenditure on critical care combined with an explicit centrally driven programme of modernisation has resulted in dramatic improvements in outcomes.

What is already known on this topic Since 2000, there have been considerable increases in funding for adult critical care in the NHS in England A specific attempt has been made to "modernise" services by establishing critical care networks, introducing outreach services in hospitals, and adopting clinical guidelines in the form of care bundles There have been conflicting claims as to the impact of these developments on the outcome of care (intensive care unit and hospital mortality) and the processes of care (transfers, early discharges, delayed discharges) What this study adds In the six years since 2000 there have been major improvements in outcome, with a 13.4% fall in hospital mortality and in processes of care (fewer transfers, fewer early discharges) Collectively the interventions associated with the modernisation of critical care represent a highly cost effective use of NHS resources

Cite this as: BMJ 2009;339:b4353

---

We thank all patients and staff in the participating units in the ICNARC case mix programme, members of the adult critical care stakeholders’ forum, and members of the study reference group: Deborah Dawson, Lisa Hinton, Sue MacFarlane, Mick Nielsen, Mike Pepperman, Sue Shepherd, and Keith Young. We are grateful for administrative support and assistance provided by Carolyn Knipe, Emma North, Keryn Vella, Cathy Welch, and Emma Wood.

Contributors: All authors took part in planning the study. AH and RG analysed the data. All authors took part in interpreting the results and reporting the research. AH is guarantor.

Funding: The study was funded by the NIHR SDO research and development programme (grant SDO/133/2006). AH was supported by a MRC special training fellowship in health services research. The study funders had no role in study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication. The researchers were independent from the funders

Competing interests: None declared.

Ethical approval: Not required

Data sharing: No additional data available.

References

1. Department of Health. Comprehensive critical care: a review of adult critical care services. DH, 2000.
2. Department of Health. The NHS plan. DH, 2000.
3. NHS Modernisation Agency. 10 high impact changes for service improvement and delivery: a guide for NHS leaders. NHS Modernisation Agency, 2004.
4. Resar R, Pronovost P, Haraden C, Simmonds T, Rainey T, Nolan T. Using a bundle approach to improve ventilator care processes and reduce ventilator-associated pneumonia. J Comm J Qual Patient Saf 2005;31:243-8.
5. Robson W. The saving lives and 100,000 lives programmes: good news for critical care nurses. Intensive Crit Care Nurs 2006;22:1-3.[CrossRef][Medline]
6. Gao F, Melody T, Daniels DF, Giles S, Fox S. The impact of compliance with 6-hour and 24-hour sepsis bundles on hospital mortality in patients with severe sepsis: a prospective observational study. Crit Care 2005;9:R764-70.[CrossRef][Web of Science][Medline]
7. Esmonde L, McDonell A, Ball C, Waskett C, Morgan R, Rashidian A, et al. Investigating the effectiveness of critical care outreach services: a systematic review. Intensive Care Med 2006;32:1713-21.[CrossRef][Web of Science][Medline]
8. Gao H, Harrison DA, Parry GJ, Daly K, Subbe CP, Rowan K. The impact of the introduction of critical care outreach services in England: a multicentre interrupted time-series analysis. Crit Care 2007;11:R113.[CrossRef][Medline]
9. Jain M, Miller L, Belt D, King D, Berwick DM. Decline in ICU adverse events, nosocomial infections and cost through a quality improvement initiative focusing on teamwork and culture change. Qual Saf Health Care 2006;15:235-9.[Abstract/Free Full Text]
10. Youngquist P, Carroll M, Farber M, Macy D, Madrid P, Ronning J, et al. Implementing a ventilator bundle in a community hospital. Jt Comm J Qual Patient Saf 2007;33:219-25.[Medline]
11. Cocanour CS, Peninger M, Domonoske BD, Li T, Wright B, Valdivia A, et al. Decreasing ventilator-associated pneumonia in a trauma ICU. J Trauma 2006;61:122-9.[Web of Science][Medline]
12. Hampton DC, Griffith D, Howard A. Evidence-based clinical improvement for mechanically ventilated patients. Rehab Nurs 2005;30:160-5.[Medline]
13. Crunden E, Boyce C, Woodman H, Bray B. An evaluation of the impact of the ventilator care bundle. Nurs Crit Care 2005;10:242-6.[CrossRef][Medline]
14. Department of Health. Hospital activity statistics. 2000. www.performance.doh.gov.uk/hospitalactivity.
15. Department of Health. NHS reference costs 2007-08. 2009. www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_098945.
16. Harrison DA, Brady AR, Rowan K. Case mix, outcome and length of stay for admissions to adult, general critical care units in England, Wales and Northern Ireland: the intensive care national audit and research centre case mix programme database. Crit Care 2004;8:R99-111.[CrossRef][Web of Science][Medline]
17. McDonnell A, Esmonde L, Morgan R, Brown R, Bray K, Parry G, et al. The provision of critical care outreach services in England: findings from a national survey. J Crit Care 2007;22:212-8.[CrossRef][Web of Science][Medline]
18. Harrison DA, Parry GJ, Carpenter JR, Short A, Rowan K. A new risk prediction model for critical care: the intensive care national audit & research centre (ICNARC) model. Crit Care Med 2007;35:1091-8.[CrossRef][Web of Science][Medline]
19. Carpenter J, Bithell J. Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Stat Med 2000;19:1141-64.[CrossRef][Web of Science][Medline]
20. NHS Executive. Hospital and community health services (HCHS) pay and price index. NHS Executive, 2007.
21. Office for National Statistics. Interim life tables 2000-2002. Annual online edition. 2008. www.statistics.gov.uk/StatBase/Product.asp?vlnk=14459&Pos=1&ColRank=1&Rank=272.
22. Hoch JS, Briggs AH, Willan AR. Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ 2002;11:415-30.[CrossRef][Web of Science][Medline]
23. Keenan SP, Dodek P, Chan K, Hogg RS, Craib KJ, Anis AH, et al. Intensive care unit admission has minimal impact on long-term mortality. Crit Care Med 2002;30:501-7.[CrossRef][Web of Science][Medline]
24. Department of Health. The implementation and impact of hospital at night pilot projects: an evaluation report.DH, 2005.
25. Department of Health. Agenda for change: final agreement. DH, 2004.
26. Dawson D, McEwen A. The influence of outreach in the development of the nurse consultant role in critical care: cause or effect? Intensive Crit Care Nurs 2006;22:4-11.[CrossRef][Medline]
27. Wordsworth S, Ludbrook A, Caskey F, Macleod A. Collecting unit cost data in multicentre studies: creating comparable methods. Eur J Health Econ 2005;6:38-44.[CrossRef][Medline]
28. Williams TA, Dobb GJ, Finn JC, Knuiman MW, Geelhoed E, Lee KY, et al. Determinants of long-term survival after intensive care. Crit Care Med 2008;36:1523-30.[CrossRef][Web of Science][Medline]
29. Stevens K, McCabe C, Jones C, Ashcroft J, Harvey S, Rowan K. The incremental cost effectiveness of withdrawing pulmonary artery catheters from routine use in critical care. Applied Health Econ Health Policy 2005;4:257-64.[CrossRef]
30. Calhoun AW, Guyatt GH, Cabana MD, Downing L, Turner DA, Valentine S, et al. Addressing the unit of analysis in medical care studies: a systematic review. Med Care 2008;46:635-43.[CrossRef][Web of Science][Medline]

(Accepted 26 June 2009)

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

Relevant Article

Encouraging improvement

Fiona Godlee
BMJ 2009 339: b4696. [Extract] [Full Text]

Related external webpages:

Listen to a podcast interview with Andrew Hutchings and Nick Black.

Rapid Responses:

Read all Rapid Responses

Improved results but why?

Christian P Subbe
bmj.com, 16 Nov 2009 [Full text]

Online poll

Find out more here>> and here>>