- Astrid Guttmann, senior scientist1234,
- Michael J Schull, senior scientist and 2010-11 Commonwealth Fund Harkness fellow14567,
- Marian J Vermeulen, epidemiologist16,
- Therese A Stukel, senior scientist146
- 1Institute for Clinical Evaluative Sciences, Toronto, ON, Canada
- 2Divisions of Paediatric and Emergency Medicine, Hospital for Sick Children, Toronto
- 3Department of Paediatrics, University of Toronto, Toronto
- 4Department of Health Policy, Management and Evaluation, University of Toronto, Toronto
- 5Department of Emergency Services, Sunnybrook Health Sciences Centre, Toronto
- 6Clinical Epidemiology Unit, Sunnybrook Health Sciences Centre, Toronto
- 7Department of Medicine, University of Toronto, Toronto
- Correspondence to: A Guttmann, Institute for Clinical Evaluative Sciences, G-106, 2075 Bayview Ave, Toronto, ON, Canada M4N 3M5
- Accepted 28 March 2011
Objective To determine whether patients who are not admitted to hospital after attending an emergency department during shifts with long waiting times are at risk for adverse events.
Design Population based retrospective cohort study using health administrative databases.
Setting High volume emergency departments in Ontario, Canada, fiscal years 2003-7.
Participants All emergency department patients who were not admitted (seen and discharged; left without being seen).
Outcome measures Risk of adverse events (admission to hospital or death within seven days) adjusted for important characteristics of patients, shift, and hospital.
Results 13 934 542 patients were seen and discharged and 617 011 left without being seen. The risk of adverse events increased with the mean length of stay of similar patients in the same shift in the emergency department. For mean length of stay ≥6 v <1 hour the adjusted odds ratio (95% confidence interval) was 1.79 (1.24 to 2.59) for death and 1.95 (1.79 to 2.13) for admission in high acuity patients and 1.71 (1.25 to 2.35) for death and 1.66 (1.56 to 1.76) for admission in low acuity patients). Leaving without being seen was not associated with an increase in adverse events at the level of the patient or by annual rates of the hospital.
Conclusions Presenting to an emergency department during shifts with longer waiting times, reflected in longer mean length of stay, is associated with a greater risk in the short term of death and admission to hospital in patients who are well enough to leave the department. Patients who leave without being seen are not at higher risk of short term adverse events.
Waiting times in emergency departments are a considerable problem in many countries.1 2 3 Lengthening waiting times and associated crowding have led the Institute of Medicine to describe American emergency departments as “nearing the breaking point.”4 To deal with the problem of waiting times, policymakers in Canada, Australia, and England5 6 7 have instituted health reforms that include setting targets for the time patients spend in the department. Recently, however, the British government has rescinded National Health Service (NHS) emergency department targets for lack of “clinical justification.”8
The evidence on the impact of waiting times has focused on high acuity patients. Long waiting times are associated with delays in time sensitive treatments for serious conditions typically requiring admission to hospital.9 10 About 85% of patients attending an emergency department, however, go home after their visit,11 and whether waiting times adversely affect their outcomes is unknown. Long waiting times can delay every stage of the visit, from initial assessment to treatment to final decision making to admission or discharge (patient disposition) and can alter clinicians’ routines and decision making.12 For patients, the frustration with long waits can cause up to 10% to leave without being seen (that is, before seeing a physician, without a diagnosis or treatment).12 Hence, long waiting times can alter the behaviour of both staff and patients and lead to potentially adverse consequences.
The National Quality Forum in the United States has endorsed the use of annual rates of patients who leave hospital without being seen and lengths of stay in emergency departments as measures of quality and efficiency.13 We examined whether these factors are associated with an increased risk of death and admission to hospital within seven days in patients who are not admitted. We focused on patients who were not admitted to hospital because they form the large majority of patients and are thought to be less seriously ill and therefore rarely studied and subsequent outcomes are more directly associated with care in the emergency department compared with admitted patients. We determined the relative risk of adverse events when patients present to departments with longer waiting times or with higher rates of patients who leave without being seen; in addition we compared outcomes for patients who leave without being seen with those who are seen by a physician and discharged. We hypothesised that patients in departments with longer shift waiting times are at increased risk of adverse outcomes, with higher acuity patients being at particular risk; patients who leave without being seen are not at increased risk of adverse events as they tend to be younger, less sick, and might seek care at another time; and patients in departments with higher annual rates of patient leaving without being seen are at greater risk of adverse events as these rates might reflect departmental inefficiency and dysfunction.
Setting and participants
We conducted a retrospective cohort study of all emergency department patients in fiscal years 2003-7 (April 2003-March 2008) in Ontario, Canada (population 12 million). We ended enrolment on 28 February 2008 so we could assess seven day outcomes to 7 March 2008. Records on hospital admissions are found in the fiscal year of the date of discharge so we allowed a three week period to include all admissions that would have been initiated in the first week of March 2008. We excluded emergency departments with a mean annual patient volume below the 25th centile (<13 324) as they tend to have relatively few problems with waiting times14: in our data, the median length of stay was one hour and the rate of leaving without being seen was less than 1%. We included patients whose emergency department visit resulted in being either “seen and discharged” or “left without being seen.” Left without being seen includes all patients who left before or after triage by a nurse (which typically happens before registration) but before further assessment by a physician; it excludes those who left after seeing a physician either without treatment or against medical advice. We included multiple visits by the same patient; when a patient had two visits within 30 days, however, we selected the earlier visit as these were potentially related.
The exposures of interest were mean emergency department level length of stay by shift, emergency department level annual rate of patients leaving without being seen, and patient level rate of leaving without being seen. The annual rate of patients who leave without being seen is a currently recommended national performance measure.11 How long patients spend in the emergency department is a good proxy for departmental crowding.15 16 It has face validity given that it is an important determinant of patients’ satisfaction, 17 is patient focused, is readily calculated and understood, and is considered the “criterion standard” for the measurement of throughput and crowding.16 Length of stay was calculated separately for each emergency department and shift on each day as the mean length of stay from time of triage or registration, whichever came first, to final emergency department disposition (discharge or admission to an inpatient ward) of all patients who did not leave without being seen who presented at that department during that shift with a similar severity of illness based on triage score, including those admitted and waiting for beds in the hospital. Triage is done by nursing staff as soon as the patient arrives at the department to determine priority and degree of urgency. All Ontario emergency departments use a common, validated five level triage tool, the Canadian triage and acuity scale, with common training programmes for triage nurses.18 Triage scores were grouped as high acuity (1-3: resuscitation, emergent, and urgent) and low acuity (4-5: less urgent and non-urgent). If no similarly triaged patients presented during the shift, length of stay was calculated based on patients of the same category grouping (1-3 or 4-5) in the same department shift during the previous week (68 899 (0.47%) patients) or, in a few cases, in the same category grouping during the previous week 946 (0.01%) patients). Triage data were unavailable for a small proportion of patients (5030, 0.82%) who left without being seen (these patients leave the department before even being triaged). We excluded them from the main analyses. We chose mean length of stay over median as median length of stay tends to be less subject to variability, which we believe might be important in affecting shift level variations in processes of care. The mean is also a better measure to identify shifts with extremes in length of stay, such as when numerous admitted patients are kept, or boarded, in the emergency department, a factor known to be associated with crowding.19 Our measure of length of stay did not represent the length of stay for the individual patient but reflected the average waiting time for similar patients presenting during the same shift in which the patient presented.
Main outcome measures
We linked records of visits to emergency departments with outpatient and subsequent emergency department visits, hospital admissions, and death in the seven days after the index visit. We defined adverse events as death or a hospital admission within seven days after leaving the emergency department. This time frame was chosen to ensure that subsequent events were related to the index visit; others have observed that more than 80% of adverse events after discharge from an emergency department occur within the first 10 days.20
Patients’ records were linked with unique anonymised encrypted identifiers across multiple population based health administrative databases in Ontario containing information on all publicly insured, medically necessary hospital and physician services. Emergency department visits were identified through the national ambulatory care reporting system (NACRS); admission to hospital through the discharge abstract database; outpatient physician billings from the Ontario health insurance plan (OHIP); and deaths from a population based registry of all legal residents in Ontario. Neighbourhood income and community type were derived from Statistics Canada 2001 census estimates.
We used conditional logistic regression models, conditioning on emergency department as fixed effects, to analyse the effects of leaving without being seen at the patient level and length of stay by shift on adverse events. This allowed us to assess the effects of longer versus shorter waiting times within the same department, controlling for other factors.21 As these models cannot include factors at the level of the emergency department, we used unconditional logistic regression to test the association between annual rates of leaving without being seen, measured at the department level, and adverse events in patients; in these latter models, we used generalised estimating equation models to account for clustering of patients’ outcomes within emergency departments22 and controlled for other department characteristics (teaching status and annual patient volume). All models included length of stay and used the patient as the unit of analysis. This allowed us to control for individual risk factors and permitted inferences to outcomes for individual patients, even for exposures measured at the level of the emergency department. Models controlled for patients’ age group (0-1, 1-9, 10-17, 18-39, 40-65, 66-75, >75 years), sex, calendar month, weekend/holiday versus weekday, time of day (daytime (0801-1600), evening (1601-2400), night (2401-0800)), income fifth of the patients’ neighbourhood (including missing) and community type (rural, urban, missing), number of visits to an emergency department in the past year, and main complaint. The department level model for patients who left without being seen also controlled for mean time to physician for similarly triaged patient who did not leave without being seen during the same shift, measured as time from triage to being seen by a physician; we did not include this in the model testing leaving without being seen at the patient level as it is in the causal pathway of patients leaving without being seen. All analyses were stratified by Canadian triage and acuity scale category (1-3 (high acuity) and 4-5 (low acuity)). Statistical tests were computed at the P<0.05 and were two sided. We did not include the patient’s own length of stay as a separate variable in the models as we were investigating shift characteristics; however, it was included in the shift level calculation of length of stay in the emergency department. Missing data were handled as follows: missing fifth of neighbourhood income relates to the instability of the population in any given dissemination area (for instance, areas with high numbers of university students) and as such this category was kept separate and those patients were included. Similarly, missing data on community type reflect recent changes in neighbourhood boundaries that have not yet been mapped adequately by Statistics Canada and these patients were also analysed as a separate category. Missing mean time to physician was also retained as its own category. All analyses were performed with SAS for UNIX, version 9.2.1 (SAS Institute, Cary, NC). Conditional logistic models used the procedure PROC LOGISTIC with a STRATA statement and included fixed effects for each emergency department.
There were 21 925 275 visits during the study period, of which 1 487 094 were to emergency departments with low volume. Of the remaining 20 438 181 visits in 125 departments, 17 771 926 (87%) patients left without being seen or were seen and discharged. We excluded 3 220 373 visits within 30 days of a previous visit, leaving 14 551 553 visits, of which 617 011 were in patients who left without being seen and 13 934 542 were in patients who were seen and discharged.
The median (interquartile range) annual rate of those who left without being seen was 3.6% (2.5-4.8%). These patients were younger, had slightly lower acuity triage scores (table 1⇓), and attended more often during the evening shift and during shifts when mean length of stay and time to see a physician were longer (table 2⇓). They were more likely to live in urban areas (table 1⇓) and attend higher volume departments (table 2⇓).
Absolute rates of death and admission were much higher in high than in low acuity patients, and were lower in those who left without being seen than in those who were seen and discharged (table 3⇓). Rates of subsequent visits to emergency departments and physician offices or outpatient clinics were higher in patients who left without being seen. In all patients rates of death and admission to hospital increased with increasing length of stay in an emergency department (table 4⇓).
In multivariable analyses, the risk of death increased incrementally with each additional hour of mean waiting time per shift: the adjusted odds ratios (95% confidence interval) for death and admission were 1.79 (1.24 to 2.59) and 1.95 (1.79 to 2.13), respectively, among high acuity patients (fig 1⇓), and 1.71 (1.25 to 2.35) and 1.66 (1.56 to 1.76), respectively, among low acuity patients (fig 2⇓) for mean length of stay of ≥6 hours compared with <1 hour.
The adjusted risk of death was significantly lower for patients who left without being seen compared with those who were seen and discharged. The adjusted rate of hospital admission for patients who left without being seen was lower for low acuity patients (fig 2) but similar to those who were seen and discharged for high acuity patients (fig 1). The annual rate of leaving without being seen at the emergency department level was not associated with a clinically important risk of either death or admission (adjusted odds ratios 1.00 (0.97 to 1.02) for death and 0.98 (0.97 to 0.99) for admission in high acuity patients; 1.03 (0.99 to 1.06) for death and 0.99 (0.98 to1.00) for admission in low acuity patients).
Death was more likely in the three oldest age groups than in patients aged 18-39 for both high and low acuity patients. Admissions were higher in the three oldest age groups and in those aged under 1 year compared with those aged 18-39. Full model results can be found in the appendix on bmj.com.
Patients presenting to emergency departments during shifts with long mean waiting times might be at increased risk of death and admission in the subsequent seven days, regardless of acuity on presentation. As mean length of stay increases, so does the risk of subsequent adverse events. Although the absolute risk of death and admission for low acuity patients is smaller, the relative impact of increased length of stay seems to be slightly higher.
These results come from a large population based study that explored the safety implications of average waiting times in emergency departments among patients who go home after their visit. Such patients represented about 90% of the total in our study and comprise the majority in all emergency department systems in developed countries. We were able to show an increase in the risk of both mortality and admission to hospital associated with presenting during shifts with long mean lengths of stay. In contrast, patients who left without being seen were not at increased risk of adverse events compared with those who were seen and discharged, nor were patients who attend departments with high rates of leaving without being seen. Patients who left without being seen had slightly higher rates of follow-up with primary care physicians and return visits to the emergency department than those who were seen and discharged, which could have helped to mitigate adverse events. Some could have problems that resolve while they are waiting to be seen.
Comparison with other studies
Patients who leave without being seen have long been considered at risk of adverse events,23 24 25 but previous studies were small, single centre, or did not examine mortality.23 24 25 The absence of increased risk should be reassuring to emergency department clinicians and administrators, who have struggled with the uncertainty surrounding outcomes in such patients. Our results suggest that individual patients who leave without being seen are not at increased risk of adverse events, and that the emergency department level rate of leaving without being seen, taken alone, cannot be seen as a measure of patient safety.
On the other hand, our results suggest that presenting during shifts with a long mean length of stay could have serious safety implications for patients. We tested for associations between mean length of stay and adverse events both between and within institutions. Our findings build on previous research that has been largely restricted to patients admitted from the emergency department or with high acuity illnesses in tertiary care hospitals. An Australian study showed a 35% increase in 10 day mortality associated with higher emergency department occupancy among all emergency department patients in a single teaching hospital but only identified deaths occurring in the same hospital.26 Another study showed that mortality at two, seven, and 30 days was related to both emergency department and hospital crowding, but examined only patients admitted from the emergency department in three tertiary care hospitals.27 Studies of patients with acute myocardial infarction9 and pneumonia10 have found lower rates of evidence based time sensitive treatments associated with waiting times and crowding in emergency departments. These studies point to potential mechanisms of harm in specific high acuity illnesses, while other studies have shown delays in more general processes of care associated with emergency department crowding such as time to interpretation of diagnostic imaging.28
It is unlikely that delays in treatment alone account for adverse outcomes in the patients we studied, all of whom left the emergency department, most after being seen and discharged, implying that the intended care was completed. It is possible, however, that during shifts with long mean waiting times important processes of care are impeded and decision making altered.12 23 Examples might include reluctance to order time consuming tests or consultations and shortened observation periods (both of which could increase missed diagnoses), incomplete treatment, or inadequate planning and communication of care after discharge. The increased risk of adverse events for low acuity patients suggests that processes might be more likely to break down if patients are thought to be low risk. Yet patients initially thought to be low acuity can, with careful evaluation, be discovered to have serious illnesses and require hospital admission. For example, one large study found that about half of all patients with acute myocardial infarction were inappropriately triaged as low acuity when they arrived at the emergency department.29
Interventions to reduce waiting times and address crowding have been summarised elsewhere.30 These include improving access to inpatient beds, 5 use of physicians at triage,31 and dedicated fast track units for patients with minor complaints.32 More research should evaluate whether these efforts result in meaningful decreases in adverse events in all emergency department patients.
Our findings provide strong support for policies in Canada,6 Australia,7 and England5 targeting reductions in length of stay in emergency departments and call into question the recently announced plan to abandon English targets for lack of “clinical justification.”8 Our study suggests that there is clinical justification to reduce lengths of stay, although taken alone it cannot guide specific policy interventions or targets to achieve this goal. Policies to reduce length of stay are aligned with the Institute of Medicine’s patient safety initiative4 and could decrease important adverse events. The increasing risk to patients associated with increasing length of stay in an emergency department suggests that any reductions, regardless of magnitude, could benefit a wide array of patients. In our analysis, reducing mean length of stay by an average of one hour could have potentially decreased the number of deaths in our study in higher acuity patients (8625) by 558 (6.5%) and in lower acuity patients (2054) by 261 (12.7%). Further research needs to evaluate whether these benefits are in fact seen when waiting times are decreased.
Strengths and limitations of study
Strengths of our study include the large number of patients, inclusion of community and tertiary hospitals, linkage across healthcare sectors, and no loss to follow-up. Administrative health data are subject to some inaccuracies, but we limited our analyses to highly reliable data elements.33 Our findings might not be generalisable to other healthcare systems. The findings around patients who leave without being seen might not be the same in systems such as in the United States without universal access to other healthcare services, although we found that patients who left without being seen were only slightly more likely than those who were seen and discharged to seek office based outpatient care after an emergency department visit, suggesting that access to primary care for patients who leave without being seen in Ontario was not a major factor in limiting adverse events. A lower admission threshold in other systems might mean fewer high risk patients being discharged home and thus avoiding need for subsequent admission, although our results indicate the risk associated with long waiting times extends to low acuity patients as well, nearly all of whom are discharged home in all systems. Ontario’s rate of use of emergency departments is comparable with that in other jurisdictions,11 suggesting case mix is similar, although some other countries have reported higher rates of patients who leave without being seen.34 The organisation of care within emergency departments in Canada is similar to that in most developed countries, and crowding has become a common problem in many systems. Finally, as an observational study it is possible the association between waiting times and adverse events might have been subject to confounding. Although we have shown that the association exists within as well as between emergency departments, there could be local factors at play during shifts with long waiting times, such as differences in staffing, that might be unmeasured drivers of adverse events.
Conclusions and policy implications
Waiting times in emergency departments, as indicated by prolonged length of stay, are associated with important safety outcomes for patients, even for those who are well enough to leave the emergency department—the vast majority of patients. Contrary to popular belief, leaving without being seen was not associated with a higher risk of adverse events in the short term. Reducing adverse events attributable to long waiting times among patients who go home is probably best achieved by reducing the overall length of stay in emergency departments for all patients, rather than targeting patients who leave without being seen for review or follow-up. Further research on patient safety should evaluate whether quality improvement and performance measurement initiatives prioritising waiting times in emergency departments result in reductions of adverse events.
What is known on this topic
Long waiting times in the emergency department are a widespread problem in developed countries and known to be associated with delays in care and poor outcomes for sick patients who require time sensitive care or admission, or both
Most patients go home from the emergency department, usually after medical assessment, though up to 10% might leave without being seen by a physician
What this study adds
For every extra hour of mean length of stay in an emergency department, in similar patients at the time of presentation, there is an association with increased seven day mortality and admission to hospital in those who are discharged home or leave without being seen
Patients who leave without being seen do not have a higher adjusted risk of adverse outcomes compared with patients who are seen and discharged
Waiting times have important implications for patients who attend emergency departments, most of whom go home after their visit
Cite this as: BMJ 2011;342:d2983
We thank Kinwah Fung for help with the data analysis and Joel Ray for help in preparing the manuscript.
Contributors: AG was responsible for conception, design, analysis, funding, interpretation, drafting of article, and critical revision. MS and TS were involved in design, analysis, interpretation, and critical revision. MV was responsible for design, analysis, interpretation, drafting of article, and critical revision. AG is guarantor. All authors had full access to the data and take responsibility for its integrity and the accuracy of the analysis.
Funding: This study was supported by a grant from the Ontario Ministry of Health and Long Term Care and by the Institute for Clinical Evaluative Sciences, which is funded by an annual grant from the Ministry of Health and Long Term Care. The funding source had no involvement in the study design, analysis, interpretation or decision to submit this work.
Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: AG received financial support from the Ontario Ministry of Health and Long Term Care for the submitted work and she and MS receive salary support from the Canadian Institute for Health Research; the authors have no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: This study was approved by the research ethics board of Sunnybrook Health Sciences Centre.
Data sharing: The technical appendix, dataset creation plan/protocol and statistical code are available from the corresponding author at.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.