Papers

Randomised controlled trial and economic evaluation of a chest pain observation unit compared with routine care

BMJ 2004; 328 doi: http://dx.doi.org/10.1136/bmj.37956.664236.EE (Published 29 January 2004) Cite this as: BMJ 2004;328:254
  1. Steve Goodacre, senior lecturer in health service research and emergency medicine (s.goodacre{at}sheffield.ac.uk)1,
  2. Jon Nicholl, director, Medical Care Research Unit1,
  3. Simon Dixon, senior lecturer in health economics1,
  4. Elizabeth Cross, research associate1,
  5. Karen Angelini, chest pain nurse2,
  6. Jane Arnold, chest pain nurse2,
  7. Sue Revill, chest pain nurse2,
  8. Tom Locker, specialist registrar2,
  9. Simon J Capewell, chair of clinical epidemiology4,
  10. Deborah Quinney, research fellow4,
  11. Stephen Campbell, consultant3,
  12. Francis Morris, consultant2
  1. 1School of Health and Related Research, University of Sheffield, Sheffield S1 4DA
  2. 2Emergency Department, Northern General Hospital, Sheffield S5 7AU
  3. 3Department of Cardiology, Northern General Hospital
  4. 4Department of Public Health, University of Liverpool, Liverpool L69 3GB
  1. Correspondence to: S Goodacre
  • Accepted 13 November 2003

Abstract

Objectives To measure the effectiveness and cost effectiveness of providing care in a chest pain observation unit compared with routine care for patients with acute, undifferentiated chest pain.

Design Cluster randomised controlled trial, with 442 days randomised to the chest pain observation unit or routine care, and cost effectiveness analysis from a health service costing perspective.

Setting The emergency department at the Northern General Hospital, Sheffield, United Kingdom.

Participants 972 patients with acute, undifferentiated chest pain (479 attending on days when care was delivered in the chest pain observation unit, 493 on days of routine care) followed up until six months after initial attendance.

Main outcome measures The proportion of participants admitted to hospital, the proportion with acute coronary syndrome sent home inappropriately, major adverse cardiac events over six months, health utility, hospital reattendance and readmission, and costs per patient to the health service.

Results Use of a chest pain observation unit reduced the proportion of patients admitted from 54% to 37% (difference 17%, odds ratio 0.50, 95% confidence interval 0.39 to 0.65, P < 0.001) and the proportion discharged with acute coronary syndrome from 14% to 6% (8%, −7% to 23%, P = 0.264). Rates of cardiac event were unchanged. Care in the chest pain observation unit was associated with improved health utility during follow up (0.0137 quality adjusted life years gained, 95% confidence interval 0.0030 to 0.0254, P = 0.022) and a saving of £78 per patient (−£56 to £210, P = 0.252).

Conclusions Care in a chest pain observation unit can improve outcomes and may reduce costs to the health service. It seems to be more effective and more cost effective than routine care.

Introduction

Patients with acute chest pain present a common challenge to acute medical services. Rapid and accurate assessment for an acute coronary syndrome is essential,1 yet clinical assessment, electrocardiography, and chest radiography have well recognised limitations.2 Many patients are therefore admitted to hospital for observation.3 Despite this, inadvertent discharge of patients with possible acute coronary syndrome remains common.4 5

The concept of the chest pain observation unit has been developed to address these problems.6 7 Patients with chest pain for which no definite diagnosis has been made after clinical assessment, electrocardiogram, and chest radiograph, receive several hours of observation, electrocardiography, and measurement of biochemical cardiac markers, followed by provocative cardiac testing where appropriate. Patients with positive tests are admitted to hospital, whereas those with negative tests may safely be discharged home. This approach aims to improve care by providing a more rigorous diagnostic assessment while reducing costs to the health service by avoiding unnecessary admissions to hospital.

We aimed to evaluate the effectiveness and cost effectiveness of care in a chest pain observation unit. The specific objectives were to measure the effect of care in the chest pain observation unit on admission to hospital for undifferentiated chest pain, inappropriate discharge home with acute coronary syndrome, health utility, major adverse cardiac events, and reattendance at and readmission to hospital; and to measure the cost effectiveness of care in the chest pain observation unit compared with routine care.

Methods

We undertook a randomised controlled trial comparing care in a chest pain observation unit with routine care in the emergency department of the Northern General Hospital in Sheffield, United Kingdom. The department provides emergency services for adults to the 530 000 population of Sheffield and is attended by some 90 000 adults per year. From 5 February 2001 to 5 May 2002 we randomised days to either care in the chest pain observation unit or routine care, according to a block randomisation schedule. Research staff and chest pain nurses had access to this schedule, but other staff, general practitioners, and patients did not. Patients presenting on days when the chest pain observation unit was open were managed according to the unit's protocols; patients presenting on other days received routine care.

Intervention: chest pain observation unit

The chest pain observation unit was based in the emergency department and consisted of two monitored bays and an adjacent area for an unlimited number of unmonitored patients. It was staffed by three experienced (G grade) chest pain nurses with a background in either coronary care or emergency medicine, who had received additional training in supervising exercise treadmill tests.8 Together they covered the chest pain observation unit from 9 00 am to 9 00 pm Monday to Friday, and 10 00 am to 6 00 pm at weekends. Patients attending outside these hours could be admitted and complete their assessment the following day. Middle grade emergency department staff provided medical cover. We selected patients according to validated clinical predictors9 10 and offered assessment consisting of two to six hours of ST segment monitoring11 and hourly electrocardiography, measurement of biochemical cardiac markers (CKMB(mass) on arrival and at least two hours later12 and of troponin T at least six hours after onset of symptoms13) and exercise treadmill testing.14 Patients with positive tests were admitted; those with negative tests were discharged.

Control: routine care

Patients were managed at the discretion of the emergency department medical staff, without use of the chest pain observation unit or assistance from the chest pain nurses. The emergency department has access to CK-MB(mass) and troponin T assays with a turnaround time of one hour. If patients required observation or further testing they were admitted to hospital.

Reception staff (who were unaware of the randomisation schedule) recorded the presenting complaint of all patients attending the emergency department. The chest pain nurses then identified all patients presenting with a specific list of chest pain related complaints (prospectively when they were on duty and retrospectively, from a computer generated list, at other times), and then excluded patients with changes to their electrocardiogram that were diagnostic for acute coronary syndrome, clinically diagnosed unstable angina, comorbidity, or a serious alternative cause for chest pain; and patients who were younger than 25, with a negligible risk of coronary heart disease, or who were unable to take part in the trial or provide consent. Eligible patients were asked to provide written, informed consent to subsequent follow up. Those who declined to consent were still managed according to availability of the chest pain observation unit but were not followed up.

Follow up consisted of a clinic run by the chest pain nurses some 48 hours after initial attendance, at which an electrocardiogram was recorded and troponin T concentration measured. Postal questionnaires for completion by each patient were provided at 48 hours and one month, for return to the medical care research unit. At six months we searched the hospital computer system for evidence of attendance at the emergency department or admission to hospital. We retrieved and reviewed relevant case notes. Finally, a research assistant telephoned each participant to collect further follow up information.

The primary outcome was the proportion of participants admitted to hospital. A key secondary outcome was the proportion of patients with acute coronary syndrome who were discharged home inappropriately. We defined acute coronary syndrome as a raised concentration of troponin T (> 0.03 ng/ml) at follow up. Patients with raised troponin T concentrations are at increased risk of adverse events15 and benefit from treatment that is currently provided for inpatients.16 Other outcome measures were the major adverse cardiac event rate (cardiac death, life threatening arrhythmia, heart failure requiring admission to hospital, non-fatal myocardial infarction, or revascularisation procedure); health utility, as measured by the EQ-5D questionnaire at two days, one month, and six months; and reattendance at or readmission to hospital.

Cost effectiveness analysis

We identified and measured costs from a health service perspective. Over the following six months we measured use of resources related to the initial attendance: care in the emergency department, initial admission to hospital, reviews of outpatients, investigations related to chest pain, reattendances, readmissions, and cardiac procedures. We used direct observation of a subgroup of patients to measure in detail the use of resources during the initial six hours of care. We measured subsequent use of resources at aggregate level.

To value resources we used UK national unit costs where available17 18 and local costs elsewhere. We used data from the finance department of Sheffield's teaching hospitals to value staff time, with methods outlined by the personal social services research unit.18 We used local unit costs to value overheads, inpatient hospital stays, blood tests, and radiographs. We used national estimates to value other diagnostic tests (such as echocardiography), outpatient reviews, attendances at the emergency department, and cardiac procedures.17 18

We used EQ-5D and six month survival data to measure outcomes. We calculated the area under the curve for health utility to generate estimates of quality adjusted life years (QALYs) accrued over six months. We used regression to impute missing data and undertook a sensitivity analysis to explore the effect of using the following methods for handling missing data: exclusion of patients with missing data, simple mean imputation, and random value imputation from patients with data. We compared the costs and outcomes in the chest pain observation unit with routine care and estimated the incremental cost effectiveness of care in the chest pain observation unit.

Statistical analysis

A sample size of 988 was required to detect a difference of 7.5% in the proportion admitted (α = 0.05, β = 0.80). We did not adjust this estimate for clustering because the small cluster size and large number of clusters indicated important design effect, we had no strong theoretical reason to suspect substantial clustering, and pilot data showed no measurable evidence of clustering. The principal analysis used multilevel random effects modelling (Stata, version 7.0) to adjust for clustering by day of week, but not confounding. Secondary analysis adjusted for potential confounding by age, sex, and past history of coronary heart disease (determined a priori to be potentially important prognostic factors) and any other characteristic that either showed significant baseline imbalance (P < 0.05) or predicted outcome. We derived cost effectiveness estimates from 1000 bootstrap estimates of the difference between the mean costs and QALYs accrued after care in the chest pain observation unit and routine care.

Results

We randomised 442 days in equal numbers (221 each) to care in the chest pain observation unit and routine care. Screening identified 6957 attendances with chest pain or a related complaint (6% of all attendances to the emergency department). Of these 1631 (23.5%) were potentially eligible for care in the chest pain observation unit, 1118 (16.1%) were asked to participate, and 972 (14%) agreed. Figure 1 shows the CONSORT diagram, and table 1 shows the application of exclusion criteria. The study groups are outlined in table 2. Source of referral, smoking status, and electrocardiogram at presentation showed significant baseline imbalance between the study groups. Hence adjusted analyses included these covariates, along with age, sex, and past history of coronary heart disease.

Fig 1
Fig 1

Flow of participants through the trial

Table 1

Application of each exclusion criterion by group. Values are numbers (percentages) unless otherwise indicated

View this table:
Table 2

Baseline characteristics of the study groups. Values are numbers (percentages) unless otherwise indicated

View this table:

The proportion admitted was 36.7% (176/479) of patients receiving care in the chest pain observation unit and 53.8% (265/493) receiving routine care (odds ratio 0.5, 95% confidence interval 0.39 to 0.65, P < 0.001, intraclass correlation coefficient = 0.001). Adjustment for confounding did not alter this result (adjusted odds ratio 0.49, 0.36 to 0.65, P < 0.001). The 146 patients who were eligible for the study but declined to consent were also less likely to be admitted if they attended when the chest pain observation unit was open (22.1% v 40.0%, odds ratio 0.43, P = 0.02).

Seventy patients had a raised concentration of troponin T at follow up (mean concentration 0.65 ng/ml, median concentration 0.29 ng/ml, 57/70 (81.4%) > 0.1 ng/ml). Seven (10%) had been discharged home: 2/34 (5.9%) who received care in the chest pain observation unit and 5/36 (13.9%) who received routine care (difference 8.0%, −7.3% to 23.4%, P = 0.264).

Two participants in each group died during follow up, two from cardiac causes and two from non-cardiac causes. Nine other patients had serious, but non-fatal, cardiac events during follow up (all non-fatal myocardial infarction), four from the group receiving care in the chest pain observation unit and five from the group receiving routine care. Another 24 participants underwent a revascularisation procedure during follow up, which makes a total of 35 patients categorised as having a major adverse cardiac event during follow up (3.6%): 18 (3.8%) from the group receiving care in the chest pain observation unit and 17 (3.4%) from the group receiving routine care (difference 0.4%, −2.0 to 2.7, P = 0.796).

Table 3 shows the EQ-5D scores. The mean area under the curve for health utility was 0.3936 QALYs after care in the chest pain observation unit and 0.3799 QALYs after routine care (difference 0.0137 QALYs, 95% confidence interval 0.0030 to 0.0254, P = 0.022; adjusted difference 0.0143 QALYs, 0.0031 to 0.0255, P = 0.012). The intraclass correlation coefficient was zero for all health utility analyses. The results were robust to sensitivity analysis according to the method of handling missing data.

Table 3

Health utility over six months for groups receiving care in the chest pain observation unit and routine care

View this table:

Over the six month follow up period, 61 participants (12.7%) in the group receiving care in the chest pain observation unit and 85 (17.2%) in the group receiving routine care reattended the emergency department (odds ratio for reattendance after chest pain observation unit care 0.65, P = 0.05; adjusted odds ratio 0.66, P = 0.061). Over the same time, 37 participants (7.7%) in the chest pain observation unit group and 52 (10.5%) in the routine care group were readmitted to hospital (odds ratio 0.67, P = 0.122; adjusted odds ratio 0.65, P = 0.126).

The mean cost per patient for chest pain related care over six months was £478 for the chest pain observation unit group and £556 for the routine care group (difference £78 per patient, -£56 to £210, P = 0.252; intraclass correlation coefficient = 0.046; adjusted difference £53 per patient, -£88 to +£194, P = 0.462). Table 4 shows the breakdown of resource use for each group and table 5 shows the breakdown of costs. Higher initial costs for care in the chest pain observation unit were offset by lower costs for hospital admission and follow up.

Table 4

Breakdown of average resource use per patient. Values are numbers (percentages) of patients receiving the intervention unless otherwise indicated

View this table:
Table 5

Breakdown of costs over six months for groups receiving care in the chest pain observation unit and routine care

View this table:

Figure 2 shows the cost effect pairs plotted on the cost effectiveness plane.19 Each represents a bootstrap estimate of the difference in mean cost (Y axis) and mean effect (X axis) of care in the chest pain observation unit compared with routine care (positive values indicate that the chest pain observation unit is more effective and more expensive). This shows that, although care in the chest pain observation unit is very likely to be more effective than routine care, considerable uncertainty prevails regarding comparative costs, with the 95% confidence interval including the possibility of the chest pain observation unit being more expensive. However, if we are willing to pay £2750 per QALY gained by care in the chest pain observation unit then the probability that the chest pain observation unit will be considered cost effective is 95%.20

Fig 2
Fig 2

Cost effectiveness plane for care in the chest pain observation unit compared with routine care based on 1000 bootstrap estimates of the difference in costs and QALYs. The vertical and horizontal bars indicate the 95% confidence intervals for cost and effect differences. The ellipse indicates the 95% confidence interval for cost effectiveness

Discussion

Main findings

Use of a chest pain observation unit reduced the number of admissions to hospital and reattendance at the emergency department, and improved health utility over the following six months. This difference in health utility represents a 2.7% improvement in quality of life over the six month follow up. Although we detected no significant differences in inappropriate discharges with acute coronary syndrome, readmissions to hospital, or costs to the health service, in each case the point estimate favoured the chest pain observation unit. Rates of major adverse cardiac events were almost identical in the two groups.

Cost effectiveness

From an economic viewpoint, mean estimates show that the chest pain observation unit dominates routine care by providing improved outcomes at lower cost. Yet substantial uncertainty surrounds the cost estimate, and the chest pain observation unit may be more expensive. If it is more expensive it is still likely to be considered cost effective as there is a 95% probability that the chest pain observation unit will be cost effective, given willingness to pay £2750 per QALY gained. This is much lower than the value of £30 000 per QALY that seems to guide decision making by the National Institute for Clinical Excellence.20

Comparison with other studies

Previous studies of cost effectiveness have compared care in a chest pain observation unit with inpatient treatment.2123 Our study shows that such a comparison is inappropriate because 46% of the patients receiving routine care were not admitted to hospital. Consequently, the estimated cost savings generated by our study were lower and subject to greater uncertainty than previous estimates. However, by measuring the patients' self reported health we showed noteworthy improvements in health utility associated with the chest pain observation unit. These data cannot explain why health utility is improved, but the chest pain observation unit may provide greater reassurance through a more rigorous diagnostic work-up. If this is so the chest pain observation unit may be justified by improving outcome, rather than simply reducing costs.

Limitations

The use of cluster randomisation and outcome measures assessed by patients are valuable in the evaluation of changes to the organisation of services. In this study cluster randomisation provided a pragmatic economic comparison of the chest pain observation and routine care, whereas individual randomisation would have measured the cost effectiveness of the chest pain observation unit when managing only half the available patients. However, these methods have some limitations. Firstly, since randomisation takes place before recruitment and consent to participate, it is possible for selection bias to influence results. We attempted to reduce this possibility by rigorous recording of selection criteria and by adjusting for known confounders in secondary analyses. This cannot, however, completely rule out the potential influence of selection bias. Secondly, since it is impossible to blind participants to the fact that they are receiving the chest pain observation unit or routine care, it is possible that a measure reported by patients, such as the EQ-5D, may be influenced by the patients' awareness that they are receiving “new” or routine care.

Implications for future research

Finally, further research is required before we can generalise the results of this study to other hospitals. The Northern General Hospital's chest pain observation unit is currently unique in the United Kingdom,24 and we do not yet know whether it can be reproduced successfully elsewhere. Despite these limitations this study provides the best current available evidence for the effectiveness and cost effectiveness of the chest pain observation unit. It shows that the chest pain observation unit has the potential to improve care and may reduce costs to the health service. Future research should evaluate the implementation of the chest pain observation unit and explore the development and refinement of the chest pain observation unit protocol.

What is already known on this subject

Chest pain observation units have the potential to improve care for patients presenting with acute, undifferentiated chest pain and reduce costs to the health service

Care in the chest pain observation unit is safe and practical, but reliable evidence of effectiveness and cost effectiveness is lacking

What this study adds

Care in the chest pain observation unit reduces hospital admissions without increasing inappropriate discharges with an acute coronary syndrome

Health utility is improved while costs to the health service are reduced

Care in the chest pain observation unit is therefore more effective and more cost effective than routine care

Acknowledgments

Acknowledgements: We thank the staff of the Northern General Hospital departments of emergency medicine, acute medicine, and cardiology for their help with this project. We thank Brian Morris for support with chemical pathology services and Stephen Walters for additional statistical advice.

Footnotes

  • Contributors SG conceived the idea for the study, and JN and SJC developed it. All authors contributed to the study design. SG, EC, KA, JA, SR, and DQ collected and managed data. KA, JA, SR, TL, FM, and SC provided care in the chest pain observation unit. SG, JN, and SD performed statistical and economic analysis. SG wrote the first draft of the paper, and all authors contributed to the final draft. SG is the guarantor.

  • Funding The Northern General Hospital received a grant of £94 000 from the Department of Health, to establish the chest pain observation unit. The study was supported by a £30 000 grant from NHS Trent Policy and Practice research funds. SG was funded by a NHS Trent Health Services research fellowship. DQ was funded by Merseyside Health Action Zone.

  • Competing interests SG, FM, SC, KA, JA, SR, and TL were involved in establishing and developing the Northern General Hospital chest pain observation unit. KA, JA, and SR are currently employed as chest pain nurses running this chest pain observation unit. JN, SD, EC, SJC, and DQ have no competing interests to declare.

  • Ethical approval North Sheffield Research Ethics Committee.

References

View Abstract