Letters

When can odds ratios mislead?

Odds ratios should be used only in case-control studies and logistic regression analyses

Avoidable systematic error in estimating treatment effects must not be tolerated

Authors' reply

Odds ratios should be used only in case-control studies and logistic regression analyses

EDITORExpressing the results of clinical trials and systematic reviews in terms of odds ratios can be more seriously misleading than Davies et al advise us.¹ They gave a correct analysis of situations in which odds ratios are used to describe increases in event rates, but their consideration of the more common situation, in which treatments reduce event rates, is short sighted. Here, effectiveness is more commonly expressed as the percentage relative risk reduction (100×(1relative risk)%) than the actual relative risk. The discrepancy between a relative risk reduction and the equivalent relative odds reduction (100×(1odds ratio)%) can be misleading. When event rates are high (commonly the case in trials and systematic reviews) the relative odds reduction can be many times larger than the equivalent relative risk reduction.

For example, Brent et al report results of a trial of a programme aimed at increasing the duration of breast feeding.² By three months 32/51(63%) women had stopped breast feeding in the intervention group, compared with 52/57(91%) in the control group. Whereas the relative risk reduction is 31% the relative odds reduction is 84%: nearly three times as large. The same problem can occur in systematic reviews: a summary of the results of seven trials of antimicrobial treatment on premature rupture of membranes showed a 49% relative odds reduction of delivery by seven days, whereas the relative risk reduction was only 19%.³

Although relative odds and relative risk reductions always go in the same direction, these discrepancies in magnitude are large enough to mislead. Good estimates of treatment effects are essential for clinicians to be able to balance the relative probabilities of the good and bad outcomes that could be caused by a treatment.

The only safe use of odds ratios is in case-control studies and logistic regression analyses, where they are the best estimates of relative risks that can be obtained. Theoretical mathematical arguments for using odds ratios in other circumstances have not been supported by empirical studies.

In clinical trials and systematic reviews of trials there is no reason for compromising interpretation by reporting results in terms of odds rather than risks. ^{4 5} Authors and journal editors should ensure that the results of trials and systematic reviews are reported as relative risks unless there is a convincing argument otherwise.

Jon Deeks, Medical statistician. 
Centre for Statistics in Medicine, Institute of Health Sciences, Oxford OX3 7LF

^a J.Deeks{at}icrf.icnet.uk

1. Davies HTO, Crombie IK, Tavakoli M. When can odds ratios mislead? BMJ 1998; 316: 989-991[Free Full Text]. (28 March.)
2. Brent NB, Redd B, Dworetz A, D'Amico F, Greenberg J. Breast-feeding in a low-income population. Arch Pediatr Adolesc Med 1995; 149: 788-803.
3. Mercer BM, Arheart KL. Antimicrobial therapy in expectant management of preterm premature rupture of membranes. Lancet 1995; 346: 1271-1279[Medline].
4. Sackett DL, Deeks JJ, Altman DG. Down with odds ratios! Evidence-Based Med 1996; 1: 164-166.
5. Sinclair JC, Bracken MB. Clinically useful measures of effect in binary analyses of randomized trials. J Clin Epidemiol 1994; 47: 881-890[Medline].

Avoidable systematic error in estimating treatment effects must not be tolerated

EDITORDavies et al conclude that "qualitative judgments based on interpreting odds ratios as though they were relative risks are unlikely to be seriously in error."¹ Statisticians may be satisfied with qualitative judgments, but doctors and patients must make quantitative judgments.

Relative risk and its complement, relative risk reduction, are widely used and well understood measures of treatment effect. Only case-control studies do not permit direct calculation of relative risk. Why then, when measures of treatment effect come from research that uses stronger designs, would clinicians accept odds ratios as being roughly equivalent to relative risks rather than demand to know the relative risk itself? If our goal is to provide as valid an estimate of a treatment effect as possible, why introduce any unnecessary systematic error?

Davies et al suggest that there is no important concern in interpreting an odds ratio of 0.66 (reduction in death after management in specialist stroke units) as if it were the relative risk (the true relative risk was 0.81 in their example). We disagree. How treatment effects are described influences doctors' perceptions of efficacy. ^{2 3} Moreover, the number needed to treat, a statistic widely used to express the clinical importance of treatment effects,⁴ is seriously underestimated (by 45%) when the odds ratio is interpreted as the relative risk (in their example, it would be calculated erroneously as 5.3 rather than the true 9.7).

Knowing the number of patients one needs to treat to prevent one patient having the adverse target event is particularly useful in deciding whether to treat. Clinicians will treat patients when the number needed to treat is lower than a threshold number at which benefits of treatment wholly offset adverse events attributable to it.⁵ Interpreting an odds ratio as if it were a relative risk introduces a systematic error in the estimation of the number needed to treat and hence in decisions on treatment: treatment will be recommended when it should not be.

The table shows the number needed to treat calculated erroneously from misinterpretation of the odds ratio as if it were the relative risk and correctly from the true relative risk. The calculations are done at high control event rates and over a range of odds ratios. When the control event rate is high, interpretation of the odds ratio as the relative risk results in a systematic and important underestimate of the number needed to treat.

When relative risk can be directly calculated, it should be. There is no reason to tolerate avoidable systematic error in estimating treatment effects.

Michael B Bracken, Professor. 
Yale University School of Medicine, Department of Epidemiology and Public Health, New Haven, CT, 06510, USA

John C Sinclair, Professor of paediatrics. 
McMaster University, Hamilton, Ontario, Canada L8N 3Z5

1. Davies HTO, Crombie IK, Tavakoli M. When can odds ratios mislead? BMJ 1998; 316: 989-991. (28 March.)
2. Forrow L, Taylor WC, Arnold RM. Absolutely relative: how research results are summarized can affect treatment decisions. Am J Med 1992; 92: 121-124[Medline].
3. Naylor CD, Chen E, Strauss B. Measured enthusiasm: does the method of reporting trial results alter perceptions of therapeutic effectiveness? Ann Intern Med 1992; 117: 916-921.
4. Sinclair JC, Bracken MB. Clinically useful measures of effect in binary analyses of randomized trials. J Clin Epidemiol 1994; 47: 881-889.
5. Guyatt GH, Sackett DL, Sinclair JC, Haywood R, Cook DJ, Cook RJ. Users' guides to the medical literature. IX. A method for grading health care recommendations. JAMA 1995; 274: 1800-1804[Medline].

Authors' reply

EDITORBoth letters make interesting points about odds ratios but do not actually uncover any shortcomings in our paper. We did not advocate the use of odds ratios. Instead our paper addressed the issue of how common events must be, and how big effect sizes must be, before the odds ratio becomes a misleading estimate of the relative risk. Our main aim was to put to rest the widespread misconception that the odds ratio is a good approximation to the relative risk only when rare events are being dealt with. Our conclusion was that "serious divergence between the odds ratio and the relative risk only occurs with large effects on groups at high initial risk."

In our paper we clarified this. So long as the event rate in both the intervention and the control groups is less than 30% and the effect size is no more than moderate (say, a halving or a doubling of risk) then interpreting an odds ratio as a relative risk will overestimate the size of the effect by less than one fifth. This is a far cry from the requirement that events be rare. The authors of the letters confirm that problems can arise with higher event ratesall their examples use unusually high rates of between 50% and 91%.

In the paper we were quite clear that we were concerned with broad qualitative judgments of treatment effects and not precise quantitative estimates of the size of any effect. Though it is true, as Bracken and Sinclair state, that "doctors and patients must make quantitative judgments" we should be wary of invoking too great a precision in making these judgments. Many factors may influence the observed effect size of a treatmentfor example, the nature of the group of patients studied, variations in the healthcare setting and concomitant care, and, of course, the play of chance.

On one thing we are in clear agreement: odds ratios can lead to confusion and alternative measures should be used when these are available. Authors reporting on prospective studies should be encouraged to report the actual relative risk or relative risk reduction. Better still, as Bracken and Sinclair point out, numbers needed to treat (which measure absolute benefit) are more useful when treatment decisions are made than either relative risks or odds ratios (which measure only relative benefit). Nevertheless, when odds ratios are encountered, guidance on their interpretation is of more use than outright rejection.

Huw Talfryn Oakley Davies, Lecturer in healthcare management. 
Manouche Tavakoli, Lecturer in health and industrial economics. 
Department of Management, University of St Andrews, St Andrews, Fife KY19 9AL

Iain Kinloch Crombie, Reader in epidemiology. 
Department of Epidemiology and Public Health, University of Dundee, Ninewells Hospital and Medical School, Dundee DD1 9SY