Reducing errors made by emergency physicians in interpreting radiographs: longitudinal study
BMJ 2000; 320 doi: https://doi.org/10.1136/bmj.320.7237.737 (Published 18 March 2000) Cite this as: BMJ 2000;320:737
Data supplement
Reliability modelling
Table 1 contains predictions of the potential adverse events (PAE) and the unmitigated potential adverse events (UPAE) that result from the rates of misinterpretation of the emergency physician and the radiologist, the effectiveness of the recall procedure, and the structure of the system in which these components interact. This appendix provides the basis for the numbers in the table and some suggestions on using reliability modelling to improve the safety of patients.
Method
The first figure (Figure A1) illustrates the chains of events that could occur in the system that was in place after the initial improvements but before the redesign. The diagram makes each chain of events leading to a PAE and UPAE visible by using a tree diagram and conditional probability. Each branch of the diagram contains three bits of information: an event, the frequency of the event given the preceding events, and the frequency of the chain of events including and preceding it. For example, one chain of events in the diagram is that the emergency physician reads the radiograph, a false negative error is made, and the radiologist identifies the error during the over read. Given that the emergency physician has read the radiograph and made a false negative interpretation, the frequency that the radiologist will identify it is 0.997. The frequency of this entire chain of events is 0.00718, which is (0.6)(0.012)(0.997).
Whenever a chain of events leads to a potential adverse event, the frequency of the chain is put in bold and marked with an asterisk. If the potential adverse event goes unidentified or unmitigated at a subsequent step another asterisk is added to the corresponding frequency. If the system mitigates the potential adverse event at some step in the chain, then the bold and asterisks are removed. When the chain of events reaches the point that the patient leaves the emergency department without the potential adverse event being mitigated, an exclamation point marks the frequency of the chain of events.
The frequency of potential adverse events and unmitigated potential adverse events can be computed by summing the frequencies of chain of events that led to them. From the diagram the frequency of potential adverse events is 0.0084. This corresponds to the rate of 8.4 potential adverse events per thousand cases in cell (d) of the table. The frequency of unmitigated adverse events is 0.00158 and corresponds to the rate in cell (e). The reader could reproduce the rates in cells (b) and (c) by substituting 0.03 for the frequency of a false negative interpretation by the emergency physician for 0.012 and recomputing the frequencies of these chains of events.
The second figure (Figure A2) contains the diagram of the chains of events for the system after redesign and produces frequencies of potential adverse events and unmitigated potential adverse events that correspond to the rates in cells (f) and (g).
Applications
In this case study we used the reliability modelling to predict retrospectively the effect of a reduced rate of error and a new structure of the system. The reliability modelling also can be used prospectively to predict the impact of potential changes to the system. Data on serious but rare adverse events provide slow feedback on the safety of a system. With some data on potential adverse events and some reasonable assumptions, modelling can help the designer to choose among alternative systems or courses of action. Modelling has the potential to predict which of the systems under consideration will be the safest. This seems preferable to evaluating data from injured patients.
Modelling can also guide decisions concerning where the next efforts at improvement should be focused. Comparing the recall rates in the last row of the table indicates that an effective recall process will have a substantial impact. Some data on the current level of effectiveness of the call back process would be a worthwhile.
Related articles
- This Week In The BMJ Published: 18 March 2000; BMJ 320 doi:10.1136/bmj.320.7237.0/b
- Practice Published: 03 April 2009; BMJ 338 doi:10.1136/bmj.b1046
See more
- An adolescent with disabling abdominal painBMJ December 07, 2016, 355 i6101; DOI: https://doi.org/10.1136/bmj.i6101
- Whole brain radiotherapy for brain metastasesBMJ December 05, 2016, 355 i6483; DOI: https://doi.org/10.1136/bmj.i6483
- Commitment to talking not just treatingBMJ December 05, 2016, 355 i6471; DOI: https://doi.org/10.1136/bmj.i6471
- Doctors need dragging out of their comfort zone to avert the antimicrobial crisis, says lordBMJ December 02, 2016, 355 i6517; DOI: https://doi.org/10.1136/bmj.i6517
- Association between screening and the thyroid cancer “epidemic” in South Korea: evidence from a nationwide studyBMJ November 30, 2016, 355 i5745; DOI: https://doi.org/10.1136/bmj.i5745
Cited by...
- Characteristics and determinants of high-risk unscheduled return visits to the emergency department
- Deep neural network improves fracture detection by clinicians
- Primary Care Physician Follow-up of Distal Radius Buckle Fractures
- Evaluation of clinically significant adverse events in patients discharged from a tertiary-care emergency department in Taiwan
- Radiological misinterpretations by emergency physicians in discharged minor trauma patients
- Cognitive interventions to reduce diagnostic error: a narrative review
- How reliable is your hospital? A qualitative framework for analysing reliability levels
- Practising safely in the foundation years
- An epistemology of patient safety research: a framework for study design and interpretation. Part 2. Study design
- Overestimation of clinical diagnostic performance caused by low necropsy rates
- Cranial computed tomography in trauma: the accuracy of interpretation by staff in the emergency department
- Relationship between tort claims and patient incident reports in the Veterans Health Administration
- Interpretation of trauma radiographs by radiographers and nurses in the UK: a comparative study
- Chest radiographs in the emergency department: is the radiologist really necessary?
- Improving care in accident and emergency departments
- Medical errors: a common problem
- Reducing error, improving safety
- Safe health care: are we up to it?