Papers

Accuracy of Ottawa ankle rules to exclude fractures of the ankle and mid-foot: systematic review

Lucas M Bachmann, senior research fellow ^a, Esther Kolb, research fellow ^a, Michael T Koller, research fellow ^a, Johann Steurer, professor ^a, Gerben ter Riet, clinical epidemiologist ^b. 

^a Horten Centre, Zurich University, Postfach Nord, CH-8091 Zurich, Switzerland, ^b Academic Medical Center, Department of General Practice, Meibergdreef 15, 1105 AZ Amsterdam, Netherlands

Correspondence to: L M Bachmann lucas.bachmann{at}evimed.ch

	Abstract

Top Abstract Introduction Methods Results Discussion References

Objective: To summarise the evidence on accuracy of the Ottawa ankle rules, a decision aid for excluding fractures of the ankle and mid-foot.
Design: Systematic review.
Data sources: Electronic databases, reference lists of included studies, and experts.
Review methods: Data were extracted on the study population, the type of Ottawa ankle rules used, and methods. Sensitivities, but not specificities, were pooled using the bootstrap after inspection of the receiver operating characteristics plot. Negative likelihood ratios were pooled for several subgroups, correcting for four main methodological threats to validity.
Results: 32 studies met the inclusion criteria and 27 studies reporting on 15 581 patients were used for meta-analysis. The pooled negative likelihood ratios for the ankle and mid-foot were 0.08 (95% confidence interval 0.03 to 0.18) and 0.08 (0.03 to 0.20), respectively. The pooled negative likelihood ratio for both regions in children was 0.07 (0.03 to 0.18). Applying these ratios to a 15% prevalence of fracture gave a less than 1.4% probability of actual fracture in these subgroups.
Conclusion: Evidence supports the Ottawa ankle rules as an accurate instrument for excluding fractures of the ankle and mid-foot. The instrument has a sensitivity of almost 100% and a modest specificity, and its use should reduce the number of unnecessary radiographs by 30-40%.

What is already known on this topic Although most patients with ankle sprains who present to emergency departments undergo radiography, less than 15% have a fracture The Ottawa ankle rules is a clinical decision aid designed to avoid unnecessary radiography What this paper adds The Ottawa ankle rules is highly accurate at excluding ankle fractures after sprain injury

	Introduction

Top Abstract Introduction Methods Results Discussion References

The number of acute ankle sprains managed by lay people at sporting activities is unknown; however, general practitioners frequently encounter such injuries.¹ The management of ankle sprains is daily routine at emergency departments, and although most patients undergo radiography, fracture of the ankle or mid-foot occurs in less than 15%.^2-6 This small yield triggered the development of the Ottawa ankle rules in 1992.⁷ This instrument consists of a questionnaire for assessment of the ankle and foot.⁸ The ankle assessment covers the ability to walk four steps (immediately after the injury or at the emergency department) and notes localised tenderness of the posterior edge or tip of either malleolus (four spots). The mid-foot assessment covers the ability to walk and notes localised tenderness of the navicular or the base of the fifth metatarsal (fig 1). The instrument is designed to rule out fractures of the malleolus and the mid-foot. It has been validated and modified in several clinical settings.

When almost every patient entering the emergency department with an ankle sprain undergoes radiography, even modest values for specificity may imply large reductions in the number of radiographs needed. The instrument is therefore calibrated towards high sensitivity, at the expense of specificity to some extent. We conducted a systematic review on the accuracy of the Ottawa ankle rules.

	Methods

Top Abstract Introduction Methods Results Discussion References

We focused on studies in which the Ottawa ankle rules was used to diagnose fractures of the ankle or mid-foot. We electronically searched databases, checked the reference lists of included studies, and contacted experts and authors in the specialty (see appendix on bmj.com).

We searched Medline and Premedline (Ovid version; 1990 to present), Embase (Datastar version; 1990-2002), CINAHL (Winspirs version; 1990-2002), and the Cochrane Library (2002, issue 2). We explored the Science Citation Index database (Web of Science by Institute for Scientific Information), entering reference 7 of this paper. The search had no language restrictions.

We selected studies in a two stage process. Firstly, all abstracts or titles found by the electronic searches were independently scrutinised by JS and LMB. If a paper's eligibility was disputed, the paper was obtained and scrutinised. Next, we obtained copies of eligible papers. We used a checklist to assess that criteria for inclusion had been met. Minimal requirements for inclusion were assessment of the Ottawa ankle rules and the possibility of constructing at least a 2×2 table specifying the false positive rate and the false negative rate. Disagreements on eligibility of studies were resolved by consensus.

Methodological quality and statistical analysis
EK and LMB independently assessed the methods of data collection, patient selection, blinding and prevention of verification bias, and description of the instrument and reference standard.^9-14 Disagreements were resolved by consensus.

We calculated several pooled estimates of the negative likelihood ratio by successively increasing the number of methodological criteria required (table 1).

We calculated sensitivities, specificities, likelihood ratios, and their standard errors. Because the Ottawa ankle rules is calibrated towards high sensitivity, we were particularly interested in the pooled sensitivity (using the bootstrap) and in the pooled likelihood ratio of a negative result (using a random effects model)that is, how many times more likely it is to find a negative result among people with a fracture (1sensitivity) than among those without (specificity). To investigate sources of variation in the negative likelihood ratios, we looked at this variable in analyses stratified by variables related to clinical subgroups and study design. We calculated the Spearman rank correlation to assess variation in diagnostic threshold. We tested heterogeneity of sensitivities and specificities using ² tests, but the interpretation was hampered by small numbers of false negative results.¹⁵ After inspection of the receiver operating characteristics plot we decided to pool sensitivities, but not specificities (fig 2). We analysed the data with Stata 7.0. 

View larger version (56K): [in this window] [in a new window]   Fig 1.   Ottawa ankle rules

View larger version (34K): [in this window] [in a new window]   Fig 2.   Receiver operating characteristics plot of all included studies (39 2×2 tables)

	Results

Top Abstract Introduction Methods Results Discussion References

We identified 1085 studies from the electronic search, and we obtained full papers for 116. The reference lists of these studies revealed 15 additional articles. Overall, we analysed 32 studies meeting our inclusion criteria. ^{7 16-46} Contact with the first authors of these studies yielded no additional data.

Overall, 32 studies investigated the accuracy of the Ottawa ankle rules: 16 assessed the ankle, ^{7 16 18 26 28 30 31 33 34 37 39-43 46} 11 assessed the mid-foot, ^{7 16 18 28 30 33 40-43 46} and 10 investigated global accuracy, which included a combination of both assessments. ^{17 21-23 25 27 35 38 44 45}

The Ottawa ankle rules was developed to assist decision making in adults, but six studies reported on the accuracy of the instrument in children. ^{19 20 24 29 32 36} Several studies selectively included patients admitted to the hospital within 48 hours of a sprain instead of within one week. ^{21 24 31 36}

Pooled analyses
We excluded from the pooled estimates studies that collected data non-prospectively in addition to unknown blinding of the radiologist ^{17 37} and one abstract.⁴⁰ If studies compared the performance of different specialties using the rules, we analysed only the data on doctors' judgments. ^{31 35} We also excluded from the pooled analysis data on modifications of the rules. ^{27 28 41 44}

Overall, 27 studies were available for the pooled analysis: 12 on assessment of the ankle (13 2×2 tables), ^{7 16 18 26 30 31 33 34 39 42 43 46} eight on assessment of the mid-foot (nine 2×2 tables), ^{7 16 18 30 33 42 43 46} 10 on assessment of both the ankle and the mid-foot (10 2×2 tables), ^{21-23 25 27 32 38 44 45} and six on assessment of the ankle or mid-foot in children (seven 2×2 tables). ^{19 20 24 29 32 36}

Among these 27 studies describing 15 581 patients, 47 patients (0.3%) had a false negative result. Table 2 shows the studies' characteristics stratified by ankle, mid-foot, or combined assessment.

Sensitivity and specificity
Table 3 shows the pooled sensitivities and the distribution of specificities stratified by several characteristics. Sensitivities were consistently high but ranged from 99.6% (95% confidence interval 98.2% to 100.0%) in studies on application of the rules within 48 hours of injury to 96.4% (93.8% to 98.6%) in studies of combined assessment. The specificities ranged from 47.9% (interquartile range 42.3%-77.1%) in studies with a prevalence of fracture below the 25th centile of all studies to 26.3% (19.4%-34.3%) in studies of combined assessment.

Negative likelihood ratio
Table 4 shows pooled negative likelihood ratios for clinical subgroups and probabilities of fracture after a negative result, assuming a 15% prevalence of fracture. The post-test probability of fracture was lowest in those studies with prevalences below the 25th centile of all studies (0.7%, 0.35% to 1.90%) and highest in those studies with prevalences above the 75th centile of all studies (3.74%, 1.73% to 8.26%). As the pretest probability of fracture increases, the pooled negative likelihood ratio gets worse. In studies assessing the Ottawa ankle rules in children, the probability of fracture after a negative result was 1.22% (0.53% to 3.08%). A worse negative likelihood ratio was found in the studies that assessed both the ankle and the mid-foot.

Table 5 shows the likelihood ratios for three criteria believed to affect the accuracy of diagnosis. The features of ideal study design, such as consecutive entry and applying a radiography reference standard in all patients, were associated with slightly worse likelihood ratios.

Table 1 shows pooled negative likelihood ratios stratified for delay of patients being assessed (within or after 48 hours) and according to the quality items prospective data collection, enrolment of consecutive patients, blinding of assessor of radiographs, and definite diagnosis with radiography in all patients. Data on the use of the Ottawa ankle rules within 48 hours in adults are scarce. In children, the pooled negative likelihood ratio was 0.07, which seems low enough to be useful, although the evidence is sparse and the confidence interval correspondingly wide. The pooled likelihood ratios for assessment of the ankle and mid-foot are similar irrespective of methodological quality. Nevertheless, the estimates further towards the right side of the table are more likely to be valid.

	Discussion

Top Abstract Introduction Methods Results Discussion References

We summarised the accuracy of the Ottawa ankle rules for excluding fractures of the ankle and mid-foot in patients presenting to emergency departments with an acute ankle sprain. Less than 2% of patients in most subgroups who were negative for fracture according to the Ottawa ankle rules actually had a fracture.

As the Ottawa ankle rules is an instrument that is calibrated towards high sensitivity, we were particularly interested in the pooled sensitivity and the pooled likelihood ratio of a negative result. Specificity, however, is an indicator of the number of unnecessary radiographs that may be avoided with this decision rule. The variability in the specificities, which ranged from 10% to 79%, is surprising. ^{35 42}

We hypothesise that differences in clinical skills, interpretation of the test, and experience of staff performing the test influenced the accuracy of the Ottawa ankle rules. Only a few studies reported particulars of staff performing the test, stating, for instance, the number of years worked at a trauma emergency department. In addition, the expression of pain, which is crucial for the interpretation of the test, may have a cultural dimension. This could result in a higher false positive rate among patients with a relatively vivid expression of pain or a higher false negative rate among stoical individuals, unless the clinician shares the patient's cultural background. The subtlety of palpation technique might explain some of the large variation in false positive ratesthe percentages of patients who apparently indicated pain (or were unable to walk four steps) but had no fracture.

The Ottawa ankle rules was developed to avoid unnecessary radiography. The economic aspect of the test may be more complex. An obvious requirement of saving costs by means of the test is its application in clinical practice. A study on techniques for dissemination investigated the impact on requests for radiography of the ankle and foot in clinical practice after use of the instrument.⁴⁷ The study found that although clinicians widely recognised the test as a decision tool, its use and the change of clinical behaviour was limited. Clinicians aim to minimise the number of missed fractures and would therefore maximise sensitivity at all costs. Fear of a bad professional reputation or litigation might be an explanation. In contrast, a health insurer would be interested in the optimal balance between sensitivity and specificity of the instrument. Therefore, the practical question from the health authorities' point of view is, how should the instrument behave in order that clinicians will use it? Suppose, for example, that a sensitivity of 92% with a specificity of 85% maximised cost effectiveness. Suppose also that clinicians simply refuse to use the instrument at such a low sensitivity. In that case, it may be more useful to design the instrument such that, for example, 90% of clinicians will use it. To do this calculation would require knowing the distribution of the minimal sensitivities that the relevant clinicians are prepared to work with. Then the optimal cut-off point for sensitivity at which just enough clinicians would actually use it to make the test cost effective could be calculated.

Immediate access to radiography may further trigger requests for radiographs. So far the usefulness of the Ottawa ankle rules as a decision tool in primary care has not been assessed. Dissemination among general practitioners and people supervising sports activities may therefore be pertinent.

	Acknowledgments

We thank Pius Estermann (information specialist, University Hospital Zurich) for doing the literature searches and Afina Glas and Patrick Bossuyt (department of clinical epidemiology and biostatistics, University of Amsterdam) for commenting on an earlier draft.

Contributors: LMB and JS initiated the project. LMB, MTK, EK, and JS screened and extracted the data. LMB and MTK cross checked the extracted data. LMB and GtR analysed the data. All authors participated in discussing the results and in writing the paper. LMB will act as guarantor for the paper.

	Footnotes

Editorial by Heyworth

Funding: None.

Competing interests: None declared.

Examples of the search strategy and details of the included studies appear on bmj.com

	References

Top Abstract Introduction Methods Results Discussion References

1.	Smith GF, Madlon-Kay DJ, Hunt V. Clinical evaluation of ankle inversion injuries in family practice offices. J Fam Pract 1993; 37: 345-348[ISI][Medline].
2.	Brand DA, Frazier WH, Kohlhepp WC, Shea KM, Hoefer AM, Ecker MD, et al. A protocol for selecting patients with injured extremities who need x-rays. N Engl J Med 1982; 306: 333-339[Abstract].
3.	Brooks SC, Potter BT, Rainey JB. Inversion injuries of the ankle: clinical assessment and radiographic review. BMJ 1981; 282: 607-608[Medline].
4.	Dunlop MG, Beattie TF, White GK, Raab GM, Doull RI. Guidelines for selective radiological assessment of inversion ankle injuries. BMJ 1986; 293: 603-605[ISI][Medline].
5.	Lloyd S. Selective radiographic assessment of acute ankle injuries in the emergency department: barriers to implementation. CMAJ 1986; 135: 973-974[Medline].
6.	Sujitkumar P, Hadfield JM, Yates DW. Sprain or fracture? An analysis of 2000 ankle injuries. Arch Emerg Med 1986; 3: 101-106[ISI][Medline].
7.	Stiell IG, Greenberg GH, McKnight RD, Nair RC, McDowell I, Worthington JR. A study to develop clinical decision rules for the use of radiography in acute ankle injuries. Ann Emerg Med 1992; 21: 384-390[CrossRef][ISI][Medline].
8.	Ottawa ankle rules for ankle injury radiography www.ohri.ca/programs/clinical_epidemiology/OHDEC/ankle_rule/flash_ankle_rule.htm (accessed 20 Apr 2002).
9.	Cochrane Methods Group on systematic review of screening and diagnostic tests: recommended methods, last updated on 9 Feb, 1998. www.cochrane.org/cochrane/sadtdoc1.htm (accessed 22 Aug 2002).
10.	Irwig L, Tosteson AN, Gatsonis C, Lau J, Colditz G, Chalmers TC, et al. Guidelines for meta-analyses evaluating diagnostic tests. Ann Intern Med 1994; 120: 667-676[Abstract/Free Full Text].
11.	Khan KS, ter Riet G, Popay J, Nixon J, Kleijnen J. Study quality assessment (phase 5): conducting the review (stage II). In: CRD report No 4. Undertaking systematic reviews of research on effectiveness. , 2001:1-20. Last updated 2001 www.york.ac.uk/inst/crd/report4.htm (accessed 2 May 2002).
12.	Lijmer JG, Mol BW, Heisterkamp S, Bonsel GJ, Prins MH, van der Meulen JH, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA 1999; 282: 1061-1066[Abstract/Free Full Text].
13.	McAlister FA, Straus SE, Sackett DL. Why we need large, simple studies of the clinical examination: the problem and a proposed solution. CARE-COAD1 group. Clinical Assessment of the Reliability of the Examination-Chronic Obstructive Airways Disease Group. Lancet 1999; 354: 1721-1724[CrossRef][ISI][Medline].
14.	Reid MC, Lachs MS, Feinstein AR. Use of methodological standards in diagnostic test research. Getting better but still not good. JAMA 1995; 274: 645-651[Abstract].
15.	Deeks JJ. Systematic reviews of evaluation of diagnostic and screening tests. In: Egger M, Davey Smith G, Altman DG, eds. Systematic reviews in health care: meta-analysis in context. London: BMJ Publishing Group, 2001:248-282.
16.	Aginaga Jr B, Ventura HI, Tejera TE, Huarte SI, Cuende GA, Gomez GM, et al. [Validation of the Ottawa ankle rules for the efficient utilization of radiographies in acute lesions of the ankle]. [Spanish]. Atencion Primaria 1999; 24: 203-208.
17.	Allerston J, Justham D. Nurse practitioners and the Ottawa ankle rules: comparisons with medical staff in requesting X-rays for ankle injured patients. Accident Emerg Nurs 2000; 8(2): 110-115[CrossRef].
18.	Auleley GR, Kerboull L, Durieux P, Cosquer M, Courpied JP, Ravaud P. Validation of the Ottawa ankle rules in France: a study in the surgical emergency department of a teaching hospital. Ann Emerg Med 1998; 32: 14-18[CrossRef][ISI][Medline].
19.	Boutis K, Komar L, Jaramillo D, Babyn P, Alman B, Snyder B, et al. Sensitivity of a clinical examination to predict need for radiography in children with ankle injuries: a prospective study. Lancet 2001; 358: 2118-2121[CrossRef][ISI][Medline].
20.	Chande VT. Decision rules for roentgenography of children with acute ankle injuries. Arch Pediatr Adolesc Med 1995; 149: 255-258[Abstract].
21.	Chandra A, Schafmayer A. Clinical decision rules to rule out fractures after acute blunt ankle injuries. Prospective study in Germany for evaluation of the "Ottawa ankle rules." Unfallchirurg 2001; 104: 617-621[CrossRef][ISI][Medline].
22.	Garces P, Gurucharri S, Ibiricu C, Izuel M, Mozo J, Buil P, et al. [The Ottawa ankle guidelines: analysis of their validity as clinical decision guidelines in the indication of X-rays for ankle and/or middle-foot injuries]. [Spanish]. Atencion Primaria 2001; 28: 129-135.
23.	Glas AS, Pijnenburg BA, Lijmer JG, Bogaard K, de Roos M, Keeman JN, et al. Comparison of diagnostic decision rules and structured data collection in assessment of acute ankle injury. CMAJ 2002; 166: 727-733[Abstract/Free Full Text].
24.	Karpas A, Hennes H, Walsh-Kelly CM. Utilization of the Ottawa ankle rules by nurses in a pediatric emergency department. Acad Emerg Med 2002; 9(2): 130-133[CrossRef][ISI][Medline].
25.	Keogh SP, Shafi A, Wijetunge DB. Comparison of Ottawa ankle rules and current local guidelines for use of radiography in acute ankle injuries. J R Coll Surg Edinb 1998; 43: 341-343[ISI][Medline].
26.	Kerr L, Kelly AM, Grant J, Richards D, Odonovan P, Basire K, et al. Failed validation of a clinical decision rule for the use of radiography in acute ankle injury. NZ Med J 1994; 107: 294-295[ISI][Medline].
27.	Leddy JJ, Smolinski RJ, Lawrence J, Snyder JL, Priore RL. Prospective evaluation of the Ottawa ankle rules in a university sports medicine center. With a modification to increase specificity for identifying malleolar fractures. Am J Sports Med 1998; 26: 158-165[Abstract/Free Full Text].
28.	Leddy JJ, Kesari A, Smolinski RJ. Implementation of the Ottawa ankle rule in a university sports medicine center. Med Sci Sports Exerc 2002; 34: 57-62[CrossRef][ISI][Medline].
29.	Libetta C, Burke D, Brennan P, Yassa J. Validation of the Ottawa ankle rules in children. Emerg Med J 1999; 16: 342-344[Abstract].
30.	Lucchesi GM, Jackson RE, Peacock WF, Cerasani C, Swor RA. Sensitivity of the Ottawa rules. Ann Emerg Med 1995; 26: 1-5[CrossRef][ISI][Medline].
31.	Mann CJ, Grant I, Guly H, Hughes P. Use of the Ottawa ankle rules by nurse practitioners. Emerg Med J 1998; 15: 315-316[Abstract].
32.	McBride KL. Validation of the Ottawa ankle rules. Experience at a community hospital. Can Fam Physician 1997; 43: 459-465[ISI][Medline].
33.	Papacostas E, Malliaropoulos N, Papadopoulos A, Liouliakis C. Validation of Ottawa ankle rules protocol in Greek athletes: study in the emergency departments of a district general hospital and a sports injuries clinic. Br J Sports Med 2001; 35: 445-447[Abstract/Free Full Text].
34.	Perry S, Raby N, Grant PT. Prospective survey to verify the Ottawa ankle rules. Emerg Med J 1999; 16: 258-260[Abstract].
35.	Pigman EC, Klug RK, Sanford S, Jolly BT. Evaluation of the Ottawa clinical decision rules for the use of radiography in acute ankle and midfoot injuries in the emergency department: an independent site assessment. Ann Emerg Med 1994; 24: 41-45[ISI][Medline].
36.	Plint AC, Bulloch B, Osmond MH, Stiell I, Dunlap H, Reed M, et al. Validation of the Ottawa ankle rules in children with ankle injuries. Acad Emerg Med 1999; 6: 1005-1009[ISI][Medline].
37.	Rosin A, Sinopoli M. Impact of the Ottawa ankle rules in a US army troop medical clinic in South Korea. Milit Med 1999; 164: 793-794.
38.	Salt P, Clancy M. Implementation of the Ottawa ankle rules by nurses working in an accident and emergency department. Emerg Med J 1997; 14: 363-365[Abstract].
39.	Singh-Ranger G, Marathias A. Comparison of current local practice and the Ottawa ankle rules to determine the need for radiography in acute ankle injury. Accident Emerg Nurs 1999; 7: 201-206.
40.	Solomito AL, Singal BM, Radack M. Ankle radiography in the emergency department: a prospective validation of the Ottawa ankle rules [abstract.] Acad Emerg Med 1994; 193(1): A64.
41.	Springer BA, Arciero RA, Tenuta JJ, Taylor DC. A prospective study of modified Ottawa ankle rules in a military population. Am J Sports Med 2000; 28: 864-868[Abstract/Free Full Text].
42.	Stiell IG, Greenberg GH, McKnight RD, Nair RC, McDowell I, Reardon M, et al. Decision rules for the use of radiography in acute ankle injuries. Refinement and prospective validation. JAMA 1993; 269: 1127-1132[Abstract].
43.	Stiell IG, McKnight RD, Greenberg GH, McDowell I, Nair RC, Wells GA, et al. Implementation of the Ottawa ankle rules [see comments]. JAMA 1994; 271: 827-832[Abstract].
44.	Tay SY, Thoo FL, Sitoh YY, Seow E, Wong HP. The Ottawa ankle rules in Asia: validating a clinical decision rule for requesting X-rays in twisting ankle and foot injuries. J Emerg Med 1999; 17: 945-947[CrossRef][ISI][Medline].
45.	Verma S, Hamilton K, Hawkins HH, Kothari R, Singal B, Buncher R, et al. Clinical application of the Ottawa ankle rules for the use of radiography in acute ankle injuries: an independent site assessment. Am J Roentgenol 1997; 169: 825-827[Abstract/Free Full Text].
46.	Yuen MC, Sim SW, Lam HS, Tung WK. Validation of the Ottawa ankle rules in a Hong Kong ED. Am J Emerg Med 2001; 19: 429-432[CrossRef][ISI][Medline].
47.	Cameron C, Naylor CD. No impact from active dissemination of the Ottawa ankle rules: further evidence of the need for local implementation of practice guidelines. CMAJ 1999; 160: 1165-1168[Abstract].

(Accepted 2 December 2002)