Multicentre trial to introduce the Ottawa ankle rules for use of radiography in acute ankle injuries
BMJ 1995; 311 doi: https://doi.org/10.1136/bmj.311.7005.594 (Published 02 September 1995) Cite this as: BMJ 1995;311:594- Ian Stiell, associate professora,
- George Wells, associate professora,
- Andreas Laupacis, associate professora,
- Robert Brison, associate professorb,
- Richard Verbeek, assistant professorc,
- Katherine Vandemheen, research coordinatora,
- C David Naylor, chief executive officerd
- aClinical Epidemiology Unit, Loeb Medical Research Institute, Ottawa, Ontario, Canada K1Y 4E9
- bDivision of Emergency Medicine, Queens University, Kingston, Ontario, Canada
- cDivision of Emergency Medicine, University of Toronto, Toronto, Ontario, Canada
- dfor the Multicentre Ankle Rule Study Group, Institute for Clinical Evaluative Sciences in Ontario, Toronto, Ontario, Canada
- Correspondence to: Dr Stiell.
- Accepted 26 June 1995
Abstract
Objective: To assess the feasibility and impact of introducing the Ottawa ankle rules to a large number of physicians in a wide variety of hospital and community settings over a prolonged period of time.
Design: Multicentre before and after controlled clinical trial.
Setting: Emergency departments of eight teaching and community hospitals in Canadian communities (population 10000 to 3000000).
Subjects: All 12777 adults (6288 control, 6489 intervention) seen with acute ankle injuries during two 12 month periods before and after the intervention.
Intervention: More than 200 physicians of varying experience were taught to order radiography according to the Ottawa ankle rules.
Main outcome measures: Referral for ankle and foot radiography.
Results: There were significant reductions in use of ankle radiography at all eight hospitals and within a priori subgroups: for all hospitals combined 82.8% control v 60.9% intervention (P<0.001); for community hospitals 86.7% v 61.7%; (P<0.001); for teaching hospitals 77.9% v 59.9%; (P<0.001); for emergency physicians 82.1% v 61.6%; (P<0.001); for family physicians 84.3% v 60.1%; (P<0.001); and for housestaff 82.3% v 60.1%; (P<0.001). Compared with patients without fracture who had radiography during the intervention period those who had no radiography spent less time in the emergency department (54.0 v 86.9 minutes; P<0.001) and had lower medical charges ($70.20 v $161.60; P<0.001). There was no difference in the rate of fractures diagnosed after discharge from the emergency department (0.5 v 0.4%).
Conclusions: Introduction of the Ottawa ankle rules proved to be feasible in a large variety of hospital and community settings. Use of the rules over a prolonged period of time by many physicians of varying experience led to a decrease in ankle radiography, waiting times, and costs without an increased rate of missed fractures. The multiphase methodological approach used to develop and implement these rules may be applied to other clinical problems.
Key messages
Key messages
The Ottawa ankle rules can be successfully applied by physicians of varying experience in many different settings to reduce incidence of ankle radiography
With proper application of the rules the risk of patient dissatisfaction or missed fractures is negligible
Widespread use of the Ottawa ankle rules would lead to large savings in health care costs
The multiphase methodological approach used to develop, validate, and implement these decision rules may be applied to other clinical problems
Introduction
Ankle and foot injuries are a common complaint among patients seen in emergency departments. Though only a few of these cases have suffered a fracture,1 2 3 4 5 6 nearly all typically undergo plain radiography of the ankle or foot, or both.7 8 9 10 11 12 To deal with this clinical problem decision rules for the use of radiography in acute ankle injuries have been recently developed13 14 and validated15 and have been found to be highly sensitive in identifying fractures. These Ottawa ankle rules are based on the assessment of ability to bear weight and areas of bone tenderness and allow physicians to determine quickly which patients are at negligible risk of fracture (figure).
We know of few clinical decision rules that have been studied to determine their impact on patient care in “usual clinical practice.”16 17 A recent study at a single hospital showed that implementation of the Ottawa ankle rules led to a significant reduction in the use of ankle radiography.18 This clinical trial was designed to assess whether the rules could be shown to reduce the use of radiography without affecting quality of care when used by many physicians of varying experience in a variety of different hospital settings.
Subjects and methods SUBJECTS
All adult patients with ankle injury seen in the emergency departments of the study hospitals during control (before) and intervention (after) periods of 12 months each were included in this controlled clinical trial. The eight hospitals were chosen because they were able to identify eligible cases retrospectively and because they represented various community population sizes (10000 to 3000000), hospital types (community, teaching), annual volumes of patients in emergency departments (20000 to 68000), and staffing patterns (emergency physician, family physician, house officers). Patients with acute ankle trauma from any mechanism of injury were eligible. “Ankle” was explicitly defined anatomically.15 19 The institutional research ethics committees approved the study.
INTERVENTION
We introduced the Ottawa ankle rules15 to the study hospitals before the intervention period by means of a single one hour lecture given by the principal investigator (IS) as well as handouts, pocket cards, and two posters mounted in each department. Physicians were asked to complete a data form and to distribute information sheets to patients. The decision to order radiography for individual patients was solely at the discretion of the treating physician.
OUTCOME MEASURES
For both study periods review of patients' eligibility was made from patients' records independently for each case by three members of the research team, and differences were resolved by consensus. Patients' records were identified (retrospectively for the control period and prospectively for the intervention period) in a consistent fashion by a combination of computerised retrieval from International Classification of Diseases (ninth revision) diagnostic codes and a search of daily census logs. The primary outcome measure, the proportion of patients referred for radiography of the ankle, was determined from radiology reports. Clinically important features were defined as fractures greater than three millimetres in breadth.
During the intervention period only, all patients without fracture were followed up by telephone call at 10 days if they had had neither ankle nor foot radiography or if they were seen during the first seven days of each month and did have radiography. Patients who had not improved according to explicit criteria regarding pain, ambulation, and ability to work were asked to return for assessment. Patients who had fractures diagnosed after discharge were questioned after six months about possible effects. For patients followed up by telephone calls the total charges for all emergency department and follow up physician visits and radiographic series were estimaed in 1993 US dollars.
STATISTICAL ANALYSIS
All patients who met the inclusion-exclusion criteria during the control and intervention periods were included in the analysis regardless of whether physicians completed a data collection form or were compliant with the decision rules. For each hospital separately the uncorrected χ2 analysis was used to test the primary hypothesis that there was no difference in the proportion of patients referred for a standard ankle radiographic series between the control and intervention study groups. We calculated 95% confidence intervals of the relative differences in referral rates for radiography between groups.20 Overall point and confidence interval estimates for the relative reductions were also derived. A similar analysis was used to test the secondary hypothesis that there was no difference in the referral for radiographic series of the foot.
We used χ2 analysis to compare the primary outcome between the control and intervention study periods within the a priori subgroups of hospital type (community, teaching) and physician type (emergency, family, housestaff). Comparisons of characteristics of patients and other outcomes were tested with χ2, Student's t test, or Mann-Whitney test as appropriate.
An absolute change of 10% for the referral rate for ankle radiography from the estimated baseline rate of 80% was considered to be clinically important. We assumed a statistical power of 80% and a two tailed 5% type I error so a sample size of 293 patients was estimated for each hospital during each study period.
Results
SUBJECTS
The 12777 eligible patients seen at the eight hospitals during the control (1 May 1991 to 30 April 1992) and intervention (1 January to 31 December 1993) periods were similar for all characteristics (table I).
REDUCTION IN RADIOGRAPHY
The overall proportion of patients referred for ankle radiography was 82.8% (5207/6288) during the control period and 60.9% (3955/6489) during the intervention period (P<0.001), with an observed relative reduction between periods of 26.4% (table II). The proportion referred for ankle radiography was significantly less at all eight hospitals for both hospital subgroups and for each physician subgroup.
Three hospitals had significant reductions and five had no significant change in referral for foot radiography. During the intervention period more patients did not have radiography (21.7% v 8.4%) and fewer patients had both ankle and foot series (9.2% v 21.8%). Furthermore, patients without fracture spent less time in the emergency department if they had no radiography compared with those who had radiography (54.0 (SD 42.0) v 86.9 (46.9) minutes; P<0.001).
COMPLIANCE BY PHYSICIANS
During the intervention period physicians completed the study data sheet for 77.1% (5003) of the 6489 patients in the study. The physicians accurately interpreted the rules (97.1% for ankle and 97.6% for foot) and satisfactorily complied with the rules (95.3% for ankle and 95.0% for foot). Radiography was performed but judged to have been unnecessary according to the rules in 4.9% of cases but in only 0.5% of cases because the patient insisted on radiography. Physicians indicated that they were uncomfortable with implementing the rules in 3.8% of cases.
Six (0.6%) of the 1090 important malleolar and midfoot fractures during the intervention period were diagnosed before discharge from the emergency department in cases in which the rules were interpreted to be negative (table III). Physicians had ordered radiography based on other clinical findings, primarily gross swelling.
FOLLOW UP OF SUBJECTS
Of 2171 patients in the radiography and no radiography groups, 93.6% (2032) were successfully reached by telephone, and there were no important differences in outcomes (table IV). The mean total charges for those who had no radiography in the emergency department were less than for those who had radiography ($70.20 (SD 52.1) v $161.60 (71.2); P<0.001).
Ten (0.5%) of the 2033 patients in the follow up group had a fracture diagnosed after discharge from the emergency department despite no repeat injury (table V). Three (0.4%) of 732 had undergone radiography in the emergency department. Seven (0.5%) of 1301 had received no radiography, but in only one of these had the rules been correctly applied. All 10 patients were contacted after six months and all had healed without delay or long term effects. No litigation was initiated. No data about missed fractures were available for the control period.
Discussion
We have shown that introduction of the Ottawa ankle rules led to significant reductions in the ordering of ankle radiographic series over a sustained period of time in a variety of community and hospital settings. These reductions were achieved by many physicians with differing experience in emergency medicine and who had not been involved in the development of the rules. Concomitant reductions in the use of the less common radiographic series of the foot were achieved in three hospitals. Physicians accurately interpreted the rules after briefteaching sessions and indicated low levels of discomfort with their application.21 Patients who did not undergo radiography were satisfied with their care and were no more likely to have a fracture missed in the emergency department than those who did undergo radiography.
The major benefits of introducing the Ottawa ankle rules are time savings for patients and cost savings for the health care system. Our data suggest the potential for large savings: the average medical charges for patients who had no radiography were estimated to be $90 less than for those patients without fracture who did undergo radiography. Ankle and foot radiographs are typical “little ticket” items,22 23 the many minor but high volume procedures which may collectively contribute as much to health care costs as “big ticket” items such as magnetic resonance imaging scans or coronary bypass surgery. In these eight hospitals alone more than 6000 patients with ankle injury were seen each year, so even a 25% reduction in radiography would translate into considerable savings.
This study represents the final phase of a multiphase project to develop and test the Ottawa ankle rules. To our knowledge very few other decision rules have been shown to alter clinical practice.17 The success of the rules may be attributed to the rigour with which they were derived and tested16 as well as to their clinical sensibility24--that is, their ease of use and their high sensitivity. The rules are easy to remember and present a simple “radiograph or no radiograph” decision which may be easier for busy clinicians to incorporate into their practice than a probability of fracture. Future studies should deal with the acceptability of the rules to physicians in other countries25 26 and methods of disseminating information about their use.27 28 29 30 31
We have developed rules with a sensitivity approaching 100%. Patients and physicians alike can be reassured that if the Ottawa ankle rules are properly applied and interpreted the chances of missing a clinically important fracture are remote. Fractures were diagnosed after discharge equally often in those who did and did not have radiography. In the former group, fractures were not diagnosed because either the physician misinterpreted the radiographs or the radiographs did not show a fracture on the first visit. In the group who did not have radiography, fractures were missed because the rules were not used or were misinterpreted and in one case because of gross swelling (table V). None of these patients suffered delayed healing or added morbidity.32 The most common errors in interpretation of the rules were disregarding the presence of medial malleolar tenderness or failing to palpate the entire distal 6 cm of the posterior edge of the fibula. The latter is especially important because some fractures exit posteriorly 5 or 6 cm proximal to the tip of the fibula.
The rules may be unreliable in cases when clinical assessment is difficult--for example, with altered mentation, intoxication, other painful injuries, diminished sensation in the lower extremities, or a language barrier. In this study physicians thought that the rules were unreliable in a few patients in whom gross swelling made palpation of the posterior edge of the malleolus impossible (table IV). Whether or not radiography is ordered patients must always be advised to seek follow up if their pain or ability to bear weight has not improved in five to seven days.
Our data indicate that patients with ankle injury are satisfied with care that does not include radiography. Integral to this satisfaction is adequate communication from the physician and use of printed instructions. If the physician has carefully applied the rules, documented the clinical findings, used good judgment, and provided advice on follow up the risk of patient dissatisfaction33 34 35 or litigation is extemely low even in the unlikely event of a missed fracture.
This study showed that the Ottawa ankle rules can be successfully applied by many different physicians in many settings to achieve considerable reductions in ankle radiography, waiting times, and health care costs without an increase in patient dissatisfaction or the rate of missed fractures. The multiphase methodological approach used to develop, validate, and implement these decision rules may be applied to other clinical problems.
We acknowledge the support and cooperation of the following investigators (MARS Study Group): Steven McMurray, Brockville General Hospital, Brockville; Alan Drummond, Peter Jechel, Great War Memorial Hospital, Perth; Terry O'Brien, Kingston General and Hotel Dieu Hospitals, Kingston; Edward Brown, Keith Greenway, Peel Memorial Hospital, Brampton; Wayne Helmer, Roger Hoag, Queensway-Carleton Hospital, Nepean; Raphael Shew, Smiths Falls Community Hospital, Smiths Falls; Andrew McDonald, Sunnybrook Health Science Center, Toronto, Ontario; Gary Greenberg, Ottawa, Ontario; Douglas McKnight, Vancouver, British Columbia.
Acknowledgments
We thank the many physicians at the participating hospitals for their cooperation in completing data forms and emergency department nursing and clerical staff for their help with the study.
Acknowledgments
We also thank the following MARS study staff members: Susan Delong, Sandra deMunnik, Mary McCarthy, and Helen Niezgoda (site research assistants); Fiona Campbell-Daigle, My-Linh Tran, and Elizabeth Yetisir (data management); Anne Gray and Jane Lockhart (administrative support).
Footnotes
-
Funding nstitute for Clinical Evaluative Sciences in Ontario, Toronto, Canada.
-
Conflict of interest he institute is cosponsored by the Ontario Ministry of Health and the Ontario Medical Association. IS and CDN are career scientists of the Ontario Ministry of Health. Endorsement by the supporting or sponsoring agencies is not implied.