Objectives: To 1) assess consistency in triage disposition among pediatric telephone triage nurses using computer-based algorithms and 2) determine agreement between nurse dispositions and protocol dispositions.
Design/methods: Fifteen nurses from the After Hours Telephone Care Program in Denver were randomly selected to receive mock calls from standardized patients. Each nurse received the same 15 scenarios. Reliability in triage disposition was assessed using the kappa statistic. Audiotapes of cases were reviewed if an urgent referral was warranted by the protocol but not given 20% or more of the time.
Results: Mean agreement among nurses for individual cases was 83% (range, 64%-100%). Overall interrater reliability among nurses for triage disposition was 0.46 (95% confidence interval, 0.43-0.49). Mean agreement between nurses' dispositions and protocol dispositions was 81% (range, 33%-100%). Audio review revealed no differences in length of call or information elicited between cases receiving urgent and nonurgent dispositions. Reasons for incorrect dispositions were 1) information necessary to make the disposition directed by the protocol was given and ignored and 2) nurses did not elicit the necessary information prescribed by the protocol.
Conclusions: Agreement regarding disposition decisions among call center nurses and between nurses and protocols was close to 80%. Disagreement with protocol dispositions occurred when nurses 1) did not follow protocols or 2) did not act on information provided by the parent. Our data suggest a need for additional attention to communication skills and to protocol adherence in training and ongoing quality improvement practices.