Setting number of decimal places for reporting risk ratios: rule of four
BMJ 2015; 350 doi: https://doi.org/10.1136/bmj.h1845 (Published 27 April 2015) Cite this as: BMJ 2015;350:h1845
- T J Cole, professor of medical statistics
- tim.cole{at}ucl.ac.uk
- Accepted 10 February 2015
Summary points
Risk ratios are usually reported rounded to two decimal places; for a ratio of 1.00, this corresponds to a maximum absolute fractional rounding error of 0.5%
However, the error is larger for smaller ratios—for example, 5% for a ratio of 0.10—and small risk ratios need extra decimal places
A new evidence based reporting rule is proposed—the rule of four—which ensures that the rounding error is less than 1.3% for all ratios, whatever their value
The rule states: “Round the risk ratio to two significant digits if the leading non-zero digit is four or more, otherwise round to three;” it uses three decimal places for ratios in the range 0.040-0.399, two decimals for 0.40-3.99, one decimal for 4.0-39.9, and so on
An audit of 1250 risk ratios and confidence intervals from articles published in The BMJ in 2011-13 shows that just seven were reported to three decimal places, that 312 (25%) would be reported to other than two decimal places if the rule of four were used, and that it would reduce the mean fractional error from 0.8% to 0.5% (P<0.0001)
The rule of four is recommended as the better way to report risk ratios
Precision and rounding—decimal places and significant digits
Reporting of numerical data is an important element in medical research. Summary statistics are often reported to too many decimal places, leading to spurious precision and over-complicated presentation1; less often, too few decimal places are used, resulting in a lack of precision. Surprisingly, few guidelines on the subject exist. The SAMPL (Statistical Analyses and Methods in the Published Literature) guidelines on the EQUATOR Network repository (http://equator-network.org) say: “Report numbers . . . with an appropriate degree of precision. For ease of comprehension and simplicity, round to a reasonable extent.”2 One might call this the Goldilocks rounding principle—not too little, not too much. The EASE guidelines state that in tables “numbers should be given in (sic) 2-3 effective digits,”3 and APA Style requires consistently one or two decimal places.4
Precision is controlled by the degree of rounding used, expressed in terms of decimal places or significant figures. (Here the phrase significant digits is used in preference to significant figures or effective digits; note that it has nothing to do with statistical significance.) The number of decimal places is the number of digits that appear after the decimal point, whereas the number of significant digits is the total number of digits, ignoring the decimal point (and ignoring any leading zeros). Trailing zeros are assumed to be informative, not just placeholders, except in special cases (see “significant figures” in Wikipedia for a full explanation).
The number 1.2345 has four decimal places and five significant digits, whereas 123.45 has two decimal places and five significant digits. Shifting the decimal point affects the decimal places but not the significant digits, so significant digits are generally more flexible than decimal places.
Rounding inevitably introduces error; rounded to two decimal places 1.2345 becomes 1.23, so the rounding error is 0.0045. With two decimal places, the maximum possible rounding error is 0.005; this is so because, as 1.23 is rounded, the unrounded number must be closer to 1.23 than to 1.22 or 1.24—that is, lying between 1.225 and 1.235—giving an error of up to 0.005 on either side.
Reporting risk ratios
For risk ratios (that is, odds ratios, hazard ratios, relative risks, and rate ratios), which are the focus of this article, the question of precision and rounding is more complicated. The risk ratio scale is centred on unity, with ratios R and 1/R equal and opposite in effect size (for example, 0.1 and 10), yet in absolute terms the two are very different—0.1 is a hundredth of 10. So, in general, the effect of rounding error is greater the smaller the ratio. This tension is reflected in reporting guidelines for risk ratios: the Cochrane Style Guide says that “odds ratios, risk ratios, and standardized mean differences should be quoted to two decimal places;”5 Hopkins et al recommend “two meaningful digits when the value is 2.0 or more, otherwise two decimal places;”6 and Gerstman states that “odds ratios and relative risks should be reported to one decimal place.”7 The guidelines are clearly incompatible with each other.
The aim of this paper is to derive a new rule to improve the reporting of risk ratios. The paper compares current practice with two alternatives, introduces a new rule—the rule of four—and applies it to risk ratios from recent articles published in The BMJ.
Two decimal places rule
We start with the Cochrane Style Guide’s two decimal places rule,5 which given the Cochrane Collaboration’s global reach can be viewed as current practice. As already stated, the rounding error of 0.005 is larger in proportional terms when applied to smaller risk ratios. For a reported risk ratio of 1.00 the true value is in the range 1.00±0.005, corresponding to a maximum absolute fractional rounding error of 0.005/1.00×100 or 0.5%. From here on, this is called the fractional error. A ratio of 1.00 is at the centre of the risk ratio scale, so 0.5% is the “central” fractional error. However, the variation around this value is wide; for ratios of 0.10 and 10.00 the fractional errors are 5% and 0.05%, respectively; and for the extreme example of a risk ratio reported as 0.03 the fractional error is 17%.8
The top part of figure 1⇓ shows the relation between the fractional error and the risk ratio for different numbers of decimal places, in which the risk ratio scale is logged so that ratios R and 1/R are equidistant from 1. In general, the error rises as the ratio and the number of decimal places fall. The heavy line represents the two decimal places rule. The light horizontal and vertical lines indicate the central fractional error of 0.5% at ratio 1.00.
Fig 1 Percentage rounding error for three alternative rules linking decimal places (dp) to risk ratios, on basis of central error δ of 0.5%. Top: two decimal places; middle: three significant digits (3 dp for ratios 0.100-0.999, 2 dp for 1.00-9.99, and 1 dp for 10.0-99.9; bottom: rule of four (3 dp for 0.040-0.399, 2 dp for 0.40-3.99, and 1 dp for 4.0-39.9). Horizontal and vertical lines indicate central rounding error of 0.5% at ratio 1.00
Put simply, the two decimal places rule leads to small risk ratios having large fractional errors and excessive imprecision. So what to do about it? An obvious solution is to allow smaller ratios to have extra decimal places. However, the Goldilocks rounding principle also requires larger ratios to have correspondingly fewer decimals. The idea is that at certain values of the risk ratio—call them change points—the number of decimal places changes. But how many change points should there be, and what values should they take? The aim is to find a set of change points that control the mean fractional error, so that averaged across all possible risk ratios it equals the central fractional error of 0.5%.
Three significant digits rule
The simplest way to link the number of decimal places to the size of the ratio is to use a rule based on significant digits. Take the earlier example—the ratio 1.23 with two decimal places and three significant digits. Dividing and multiplying it by10 give 0.123 and 12.3, respectively, each still with three significant digits but with varying numbers of decimal places. So this corresponds to the rule “report to three significant digits,” which gives three decimal places for ratios in the range 0.100-0.999, two decimals for 1.00-9.99, and one decimal for 10.0-99.9. The same rule applies to more extreme ratios, such as 0.0123 or 123. The middle part of figure 1⇑ illustrates the rule, in which the fractional error rises as the risk ratio falls, but at each power of 10 (where the error peaks at 0.5%) the number of decimal places increases by one and the fractional error falls to 0.05%. So the change points are at the powers of 10 (0.1, 1, 10, 100, and so on), and the fractional error lies between 0.05% and 0.5%—more constant across the range of ratios than with the two decimal places rule (fig 1⇑, top). However, on average, the error is smaller than the 0.5% required, so that three significant digits are more precise than necessary.
Rule of four
The way to vary the mean fractional error is to multiply the change points in the middle part of figure 1⇑ by a suitable constant, and the bottom part of the figure illustrates this using the multiplier 0.4. The change points here are at 0.04, 0.4, 4, 40, and so on, and the curve is shifted left and up relative to the middle curve so that it straddles the 0.5% line. The mean error is 0.49%, very close to the required 0.5%, and the maximum error is 1.25%. The value 0.4 is very close to the optimal value of 0.39, which gives a mean error of exactly 0.5% (see appendix 1 and supplementary table). So this provides the basis for a simple rule with the required properties; its mean rounding error is close to 0.5%, and the maximum rounding error (which applies to ratios at the change points, such as 0.4 or 4) is 1.25% (supplementary table).
The rule can be stated as follows: “round the risk ratio to two significant digits if the leading non-zero digit is four or more, otherwise round to three.” A more formal definition is: “divide the risk ratio by four and round down to two significant digits, then report the ratio to that number of decimal places.” The rule of four uses three decimal places for ratios from 0.040 to 0.399, two decimals for 0.40-3.99, one decimal for 4.0-39.9, and so on. It thus matches the two decimal places rule for ratios in the range 0.40-3.99.
Risk ratios in The BMJ
To quantify current practice, I searched abstracts of papers published in The BMJ in 2011-13 and extracted all 1250 risk ratios and 95% confidence intervals (see appendix 2 for the search strategy). Figure 2⇓ plots each estimate and its confidence interval against the same estimate for the 1250 risk ratios, ranging in value from 0.03 (95% confidence interval 0.01 to 0.05)8 to 99 (37 to 266),9 with a median ratio of 1.14. The estimates (along the diagonal) are colour coded with the number of decimal places used and shifted vertically to separate them. Two decimal places were used most often (89%), one decimal place less so (10%), and three and no decimal places hardly at all (0.9%). The confidence intervals nearly always had the same number of decimal places as their estimate.
Fig 2 Risk ratio estimates and 95% confidence intervals published in The BMJ in 2011-13 (n=1250) plotted against the same estimates and colour coded by number of decimal places used. Inside central rectangle rule of four specifies two decimal places, whereas outside rectangle it specifies more or fewer than two
Figure 2⇑ also shows the region where the rule of four specifies two decimal places (0.40-3.99). Outside this region the rule would report ratios to more or fewer than two decimals, affecting 486 (13%) of the 3750 individual ratios (that is, estimates plus confidence intervals) and 312 (25%) of the 1250 confidence intervals that extended outside the region.
What would be the effect of the rule of four on mean precision? The fractional error of risk ratio R reported to dp decimal places is given by 102−dp/2R% (appendix 1). This allows the mean precision of the ratios as published and as based on the rule of four to be compared directly, by averaging across all the ratios and using the relevant decimal places in the formula. The mean error of the ratios as published was 0.76%, whereas reported to the rule of four it would be 0.49% (exactly matching the theoretical mean)—that is, one third less (P<0.0001, paired sign test). The rule of four materially increases precision.
The conclusions are clear: two decimal places are used too often and three decimal places too rarely (surprisingly, none of the small ratios was reported to three decimal places). Compared with the two decimal places rule, the rule of four would report a quarter of confidence intervals and an eighth of estimates differently. It is considerably more precise than current practice.
More or less precision
The rule as stated above uses two or three significant digits, but it applies equally on the basis of one or two, for when precision is less critical. For example, confidence intervals can often, especially if wide, be less precise than point estimates. The rule of four based on one/two significant digits: “round to one significant digit if the leading non-zero digit is four or more, otherwise round to two,” gives two decimal places for ratios from 0.04 to 0.39, one for 0.4-3.9, none for 4-39, and so on, with a mean rounding error of 5% and a maximum of 12.5% (10 times those for 0.4 in the supplementary table). Alternatively, when confidence intervals are very narrow, such as in logistic regression with continuous variables, three decimal places may be necessary.
Of the 1250 BMJ risk ratio estimates, 859 (69%) were statistically significant at 5%; that is, their 95% confidence intervals excluded 1.00. Reporting the confidence interval to one/two significant digits rather than two/three would render 15% of the 859 significant estimates insignificant (by rounding one or other limit to 1.00). Overall, however, for 90% of estimates significance would be unaffected by the reduction in precision. This confirms that in many cases confidence intervals can be reported to one/two significant digits.
The table⇓ shows a selection of risk ratios from figure 2⇑, reported both as published in The BMJ and rounded using the rule of four, with the estimates rounded to two/three significant digits and the confidence intervals to one/two. The two smallest ratios were very imprecise as published, and the table⇓ represents the missing digits for the rule of four with “x”. Note the trailing zeros in 0.20, 10, 70, and 270; in 0.20 and 10 they are significant digits, whereas in 70 and 270 they are placeholders.
Examples of risk ratios and 95% confidence intervals, as published in The BMJ and rounded to rule of four
Limitations and strengths
The rule of four has some limitations. One relates to risk ratios in tables where the number of decimal places will vary from ratio to ratio, and some people might view this as untidy. However, the “untidiness” can be turned to advantage. If the values are aligned by decimal point, the size of each ratio determines its position laterally, and this provides a simple visual cue to distinguish between ratios in the ranges 0.040-0.399, 0.40-3.99, 4.0-39.9, and so on.
Another limitation is that estimates may be reported to less precision than their lower confidence limit; for example, an estimate of 0.56 with lower limit 0.234 looks oddly over-precise. However, this can be avoided by reporting the confidence intervals to one/two significant digits rather than two/three, which simplifies presentation. In most cases it does not affect significance, and where it does the extra significant digit can be added.
The strengths of the rule of four compared with the two decimal places rule are that it avoids the risk of over-rounding small ratios, and overall it improves precision. It is also simple to implement. The audit shows that it is likely to affect reporting in around a quarter of cases.
Extending the SAMPL guidelines to include specific advice on reporting precision, including the rule of four as an option for reporting risk ratios, would be useful.2 The Cochrane Style Guide could similarly incorporate the rule of four in its guidance.5
The main message for authors is to think carefully about the appropriate number of decimal places to use, and not automatically to use two. That small ratios are so rarely reported to more than two decimal places is particularly surprising. The rule of four for risk ratios, together with a related set of reporting rules for other summary statistics,17 provides a simple and comprehensive framework to guide authors in the appropriate number of decimal places to use.
Conclusions
Small risk ratios are rarely reported to sufficient precision, and the rule of four provides a framework for rounding risk ratios appropriately. It rounds ratios to two or three significant digits and corresponds to two decimal places for ratios between 0.40 and 3.99. The rule of four improves precision by a third and is recommended as the better way to report risk ratios.
Notes
Cite this as: BMJ 2015;350:h1845
Footnotes
I am grateful to Doug Altman for originally raising the question of how best to report odds ratios and for his valuable comments on an earlier draft of the paper. I also thank my fellow statistics editors at The BMJ and statistical reviewers at the Archives of Disease in Childhood.
Funding: TJC is funded by Medical Research Council grant no MR/M012069/1.
Competing interests: I have read and understood the BMJ policy on declaration of interests and declare the following interests: none.
Provenance and peer review: Not commissioned; externally peer reviewed.
Disclaimer: The work was done in a private capacity, and the recommendations are not necessarily those of The BMJ.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See:http://creativecommons.org/licenses/by/4.0/