Papers

Accuracy of Ottawa ankle rules to exclude fractures of the ankle and mid-foot: systematic review

Lucas M Bachmann, senior research fellow ^a, Esther Kolb, research fellow ^a, Michael T Koller, research fellow ^a, Johann Steurer, professor ^a, Gerben ter Riet, clinical epidemiologist ^b. 

^a Horten Centre, Zurich University, Postfach Nord, CH-8091 Zurich, Switzerland, ^b Academic Medical Center, Department of General Practice, Meibergdreef 15, 1105 AZ Amsterdam, Netherlands

Correspondence to: L M Bachmann lucas.bachmann{at}evimed.ch

	Abstract

Top Abstract Introduction Methods Results Discussion References

Objective: To summarise the evidence on accuracy of the Ottawa ankle rules, a decision aid for excluding fractures of the ankle and mid-foot.
Design: Systematic review.
Data sources: Electronic databases, reference lists of included studies, and experts.
Review methods: Data were extracted on the study population, the type of Ottawa ankle rules used, and methods. Sensitivities, but not specificities, were pooled using the bootstrap after inspection of the receiver operating characteristics plot. Negative likelihood ratios were pooled for several subgroups, correcting for four main methodological threats to validity.
Results: 32 studies met the inclusion criteria and 27 studies reporting on 15 581 patients were used for meta-analysis. The pooled negative likelihood ratios for the ankle and mid-foot were 0.08 (95% confidence interval 0.03 to 0.18) and 0.08 (0.03 to 0.20), respectively. The pooled negative likelihood ratio for both regions in children was 0.07 (0.03 to 0.18). Applying these ratios to a 15% prevalence of fracture gave a less than 1.4% probability of actual fracture in these subgroups.
Conclusion: Evidence supports the Ottawa ankle rules as an accurate instrument for excluding fractures of the ankle and mid-foot. The instrument has a sensitivity of almost 100% and a modest specificity, and its use should reduce the number of unnecessary radiographs by 30-40%.

What is already known on this topic Although most patients with ankle sprains who present to emergency departments undergo radiography, less than 15% have a fracture The Ottawa ankle rules is a clinical decision aid designed to avoid unnecessary radiography What this paper adds The Ottawa ankle rules is highly accurate at excluding ankle fractures after sprain injury

	Introduction

Top Abstract Introduction Methods Results Discussion References

The number of acute ankle sprains managed by lay people at sporting activities is unknown; however, general practitioners frequently encounter such injuries.¹ The management of ankle sprains is daily routine at emergency departments, and although most patients undergo radiography, fracture of the ankle or mid-foot occurs in less than 15%.^2-6 This small yield triggered the development of the Ottawa ankle rules in 1992.⁷ This instrument consists of a questionnaire for assessment of the ankle and foot.⁸ The ankle assessment covers the ability to walk four steps (immediately after the injury or at the emergency department) and notes localised tenderness of the posterior edge or tip of either malleolus (four spots). The mid-foot assessment covers the ability to walk and notes localised tenderness of the navicular or the base of the fifth metatarsal (fig 1). The instrument is designed to rule out fractures of the malleolus and the mid-foot. It has been validated and modified in several clinical settings. We conducted a systematic review on its accuracy.

	Methods

Top Abstract Introduction Methods Results Discussion References

We focused on studies in which the Ottawa ankle rules was used to diagnose fractures of the ankle or mid-foot. We electronically searched databases, checked the reference lists of included studies, and contacted experts and authors in the specialty (see appendix on bmj.com for examples of the search strategy).

We searched Medline and Premedline (1990 to present), Embase (1990-2002), CINAHL (1990-2002), and the Cochrane Library (2002, issue 2). We used the Science Citation Index database to identify studies citing reference 7 of this paper. The search had no language restrictions.

All abstracts or titles found by the electronic searches were independently scrutinised by JS and LMB. We then obtained copies of eligible papers. Minimal requirements for inclusion were assessment of the Ottawa ankle rules and the possibility of constructing at least a 2×2 table specifying the true positive rate and the true negative rate.

Methodological quality and statistical analysis
EK and LMB independently assessed the methods of data collection, patient selection, blinding and prevention of verification bias, and description of the instrument and reference standard.^9-14 Disagreements were resolved by consensus. We calculated several pooled estimates of the negative likelihood ratio by successively increasing the number of methodological criteria required (table 1).

We calculated sensitivities, specificities, likelihood ratios, and their standard errors. Because the Ottawa ankle rules is calibrated towards high sensitivity, we were particularly interested in the pooled sensitivity and in the pooled likelihood ratio of a negative resultthat is, how many times more likely it is to find a negative result among people with a fracture (1sensitivity) than among those without (specificity). After inspection of the receiver operating characteristics plot we decided to pool sensitivities, but not specificities, by using bootstrapping (fig 2).

View larger version (57K): [in this window] [in a new window]   Fig 1.   Ottawa ankle rules

	Results

Top Abstract Introduction Methods Results Discussion References

We identified 1085 studies. Thirty two studies investigated the accuracy of the Ottawa ankle rules: 16 assessed the ankle, 11 assessed the mid-foot, and 10 investigated global accuracy, which included a combination of both assessments (see bmj.com). The Ottawa ankle rules was developed to assist decision making in adults, but six reports reported on the accuracy of the instrument in children.

Pooled analyses
Overall, 27 studies were available for the pooled analysis: 12 on assessment of the ankle (13 2×2 tables), eight on assessment of the mid-foot (nine 2×2 tables), 10 on assessment of both the ankle and the mid-foot (10 2×2 tables), and six on assessment of the ankle or mid-foot in children (seven 2×2 tables).

Among these 27 studies describing 15 581 patients, 47 (0.3%) had a false negative result. The study characteristics stratified by ankle, mid-foot, or combined assessment are detailed on bmj.com.

View larger version (34K): [in this window] [in a new window]   Fig 2.   Receiver operating characteristics plot of all included studies (39 2×2 tables)

Sensitivity and specificity
Table 2 shows the pooled sensitivities and the distribution of specificities stratified by several characteristics. Sensitivities were consistently high but ranged from 99.6% in studies on application of the rules within 48 hours of injury to 96.4% in studies of combined assessment. The specificities ranged from 47.9% in studies with a prevalence of fracture below the 25th centile of all studies to 26.3% in studies of combined assessment.

Negative likelihood ratio
Table 3 shows pooled negative likelihood ratios for clinical subgroups and probabilities of fracture after a negative result, assuming a 15% prevalence of fracture. The post-test probability of fracture was lowest in those studies with prevalences below the 25th centile of all studies (0.7%, 0.35% to 1.90%) and highest in those studies with prevalences above the 75th centile of all studies (3.74%, 1.73% to 8.26%). As the pretest probability of fracture increases, the pooled negative likelihood ratio gets worse. In studies assessing the Ottawa ankle rules in children, the probability of fracture after a negative result was 1.22% (0.53% to 3.08%). A worse negative likelihood ratio was found in the studies that assessed both the ankle and the mid-foot. The features of ideal study design, such as consecutive entry and applying a radiography reference standard in all patients, were associated with slightly worse likelihood ratios. Meta-regression analyses did not show these differences to be significant.

	Discussion

Top Abstract Introduction Methods Results Discussion References

Less than 2% of patients in most subgroups who were negative for fracture of the ankle or mid-foot according to the Ottawa ankle rules actually had a fracture. Since the Ottawa ankle rules is an instrument that is calibrated towards high sensitivity, we were particularly interested in the pooled sensitivity and the pooled likelihood ratio of a negative result. Specificity, however, is an indicator of the number of unnecessary radiographs that may be avoided with this decision rule. The variability in the specificities, which ranged from 10% to 79%, is surprising. ^{15 16} The subtlety of palpation technique might explain some of the large variation in false positive ratesthe percentages of patients who apparently indicated pain (or were unable to walk four steps) but had no fracture.

The Ottawa ankle rules was developed to avoid unnecessary radiography. One study found that although clinicians widely recognised the test as a decision tool, its use and the change of clinical behaviour was limited.¹⁷ Clinicians aim to minimise the number of missed fractures and would therefore maximise sensitivity at all costs. Immediate access to radiography may further trigger requests for radiographs. So far the usefulness of the Ottawa ankle rules as a decision tool in the primary care setting has not been assessed. Dissemination among general practitioners and people supervising sport activities may therefore be pertinent.

	Acknowledgments

We thank Pius Estermann (information specialist, University Hospital Zurich) for doing the literature searches and Afina Glas and Patrick Bossuyt (department of clinical epidemiology and biostatistics, University of Amsterdam) for commenting on an earlier draft.

Contributors: See bmj.com

	Footnotes

Editorial by Heyworth

Funding: None.

Competing interests: None declared.

This is an abridged version; the full version is on bmj.com

Examples of the search strategy appear on bmj.com

	References

Top Abstract Introduction Methods Results Discussion References

1.	Smith GF, Madlon-Kay DJ, Hunt V. Clinical evaluation of ankle inversion injuries in family practice offices. J Fam Pract 1993; 37: 345-348[Web of Science][Medline].
2.	Brand DA, Frazier WH, Kohlhepp WC, Shea KM, Hoefer AM, Ecker MD, et al. A protocol for selecting patients with injured extremities who need x-rays. N Engl J Med 1982; 306: 333-339[Abstract].
3.	Brooks SC, Potter BT, Rainey JB. Inversion injuries of the ankle: clinical assessment and radiographic review. BMJ 1981; 282: 607-608[Abstract/Free Full Text].
4.	Dunlop MG, Beattie TF, White GK, Raab GM, Doull RI. Guidelines for selective radiological assessment of inversion ankle injuries. BMJ 1986; 293: 603-605[Abstract/Free Full Text].
5.	Lloyd S. Selective radiographic assessment of acute ankle injuries in the emergency department: barriers to implementation. CMAJ 1986; 135: 973-974[Medline].
6.	Sujitkumar P, Hadfield JM, Yates DW. Sprain or fracture? An analysis of 2000 ankle injuries. Arch Emerg Med 1986; 3: 101-106[Web of Science][Medline].
7.	Stiell IG, Greenberg GH, McKnight RD, Nair RC, McDowell I, Worthington JR. A study to develop clinical decision rules for the use of radiography in acute ankle injuries. Ann Emerg Med 1992; 21: 384-390[CrossRef][Web of Science][Medline].
8.	Ottawa ankle rules for ankle injury radiography www.ohri.ca/programs/clinical_epidemiology/OHDEC/ankle_rule/flash_ankle_rule.htm (accessed 20 Apr 2002).
9.	Cochrane Methods Group on systematic review of screening and diagnostic tests: recommended methods, last updated on 9 Feb, 1998. www.cochrane.org/cochrane/sadtdoc1.htm (accessed 22 Aug 2002).
10.	Irwig L, Tosteson AN, Gatsonis C, Lau J, Colditz G, Chalmers TC, et al. Guidelines for meta-analyses evaluating diagnostic tests. Ann Intern Med 1994; 120: 667-676[Abstract/Free Full Text].
11.	Khan KS, ter Riet G, Popay J, Nixon J, Kleijnen J. Study quality assessment (phase 5): conducting the review (stage II). In: CRD report No 4. Undertaking systematic reviews of research on effectiveness. , 2001:1-20. Last updated 2001 www.york.ac.uk/inst/crd/report4.htm (accessed 2 May 2002).
12.	Lijmer JG, Mol BW, Heisterkamp S, Bonsel GJ, Prins MH, van der Meulen JH, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA 1999; 282: 1061-1066[Abstract/Free Full Text].
13.	McAlister FA, Straus SE, Sackett DL. Why we need large, simple studies of the clinical examination: the problem and a proposed solution. CARE-COAD1 group. Clinical Assessment of the Reliability of the Examination-Chronic Obstructive Airways Disease Group. Lancet 1999; 354: 1721-1724[CrossRef][Web of Science][Medline].
14.	Reid MC, Lachs MS, Feinstein AR. Use of methodological standards in diagnostic test research. Getting better but still not good. JAMA 1995; 274: 645-651[Abstract/Free Full Text].
15.	Pigman EC, Klug RK, Sanford S, Jolly BT. Evaluation of the Ottawa clinical decision rules for the use of radiography in acute ankle and midfoot injuries in the emergency department: an independent site assessment. Ann Emerg Med 1994; 24: 41-45[Web of Science][Medline].
16.	Springer BA, Arciero RA, Tenuta JJ, Taylor DC. A prospective study of modified Ottawa ankle rules in a military population. Am J Sports Med 2000; 28: 864-868[Abstract/Free Full Text].
17.	Cameron C, Naylor CD. No impact from active dissemination of the Ottawa ankle rules: further evidence of the need for local implementation of practice guidelines. CMAJ 1999; 160: 1165-1168[Abstract].

(Accepted 2 December 2002)