Objectives: As demand for emergency services outpaces available allocated resources, emergency department (ED) triage systems face increasing scrutiny. Longer waits for care make the use of reliable, valid triage systems imperative to patient safety. Little is known about the reliability and validity of triage systems in children. The purpose of this study was to evaluate the reliability and validity of the Emergency Severity Index version 3 (ESIv.3) triage algorithm in a pediatric population.
Methods: This two-phase investigation used both retrospective chart review and prospective, observational designs. Interrater reliability was evaluated using ED triage scenarios, a prospective cohort of ED patients presenting to triage, and retrospective triage assignments using the original triage note. ED triage nurses, nurse investigators, and physician investigators performed retrospective blinded triages using only the original triage note to assess reproducibility. In the second phase, validity was assessed using a retrospective analysis of observed resource use, ED length of stay, and hospitalization compared with resource utilization estimated at triage by the ESI.
Results: In the reliability phase, weighted kappa for ED nurse triage of standard scenarios ranged from 0.84 to 1.00, representing excellent agreement. Twenty ED pediatric patients were triaged simultaneously by an ED triage nurse and the nurse investigator. Weighted kappa was 0.82 (95% confidence interval = 0.66 to 0.98), also representing strong agreement between raters. When used for retrospective chart review, the weighted kappa statistics ranged from 0.42 to 0.84, representing poor to good agreement among the different categories of reviewers. During the validity phase, 510 patients were included in the final data analysis. Hospitalization, ED length of stay, and resource utilization were strongly associated with ESIv.3 category.
Conclusions: The ESI triage algorithm demonstrated reliability and validity between triage assignment and resource use in this group of ED pediatric patients.