Education and debate

Expressing the magnitude of adverse effects in case-control studies: "the number of patients needed to be treated for one additional patient to be harmed"

Lise M Bjerre, doctoral candidate ^a,  Jacques LeLorier, chief ^b. 

^a Department of Epidemiology and Biostatistics, Faculty of Medicine, McGill University, Montreal, Canada H3A 1A2, ^b Pharmacoepidemiology and Pharmacoeconomy Research Unit, Centre de Recherche, Centre Hospitalier de l'Université de Montréal, Campus Hôtel-Dieu, Montreal, Canada H2W 1T8

Correspondence to: J LeLorier leloriej{at}ere.umontreal.ca

The need to express estimates of risk in an understandable manner is a challenge faced regularly by those who work with the results of epidemiological studies and try to convey their meaning to others. This is not an easy task, as is illustrated by the recent "pill scare" in the United Kingdom, in which there was much confusion over the clinical importance of the scientific information that was made public. Furthermore, practising clinicians also need a readily understandable tool for weighing the risks of various treatments. Ideally, this should be feasible without recourse to complicated statistical concepts. In this paper, we propose a simple and intuitively understandable method for expressing the results of case-control studies.

	Evidence from randomised controlled trials

Any intervention or exposure may have desirable and undesirable effects. Desirable effects are usually the intended effects of a treatment. These will often (at least for pharmacological interventions) have been established in randomised controlled trials before an agent is released onto the market and introduced into clinical practice.

In the context of randomised trials on the desirable effects of treatments, Sackett et al proposed a method for converting rate differences into a more intuitive quantity.^1-4 This quantity was named the number needed to treat (NNT=1/absolute risk reduction). It is the number of people who must be treated in order that one adverse event is prevented by the treatment at issue. The number needed to harm is an analogous concept proposed by Sackett and colleagues to express the probability of additional adverse events occurring in randomised controlled trials because of treatment.⁴ In both cases, the concept of rate difference is converted into a number of individuals, a more intuitively understandable quantity.

However, undesirable effects are often apparent only after an intervention has become part of clinical practice. It may therefore no longer be possible to study them in a randomised controlled trial, making it necessary to resort to the less ideal case-control study.

	Evidence from case-control studies

Case-control studies are more limited than randomised controlled trials with respect to the information they can yield because absolute estimates of risk for the exposure groups are not available from the study itself, unless the design is nested within a cohort or a primarily defined population. Consequently, estimates of risk are usually expressed in terms of the exposure odds ratio of cases compared with controls. Many doctors, however, are still unclear about the meaning of the odds ratio.

	NNTH derived from case-control studies

Inspired by the number needed to treat and number needed to harm concepts for randomised controlled trials, we developed a similar concept for case-control studies. We focused on the adverse effects of treatments, a topic of importance to practising clinicians. In this context, the main problem is that estimates of rate differences are usually not available.

Consistent with a modified nomenclature for the number needed to harm proposed recently by Altman,⁵ we put forward the concept of the "number of patients needed to be treated for one additional patient to be harmed" (NNTH). The NNTH is the number of people exposed to a given treatment such that on average and over a given follow up period one additional person experiences the adverse effect of interest because of the treatment. It expresses the additional absolute risk of an adverse effect conferred by a treatment and is therefore a useful and intuitively understandable decision making tool for practising clinicians.

where OR is the odds ratio provided by the case-control study and UER is the unexposed event rate (see appendix on BMJ website for the derivation). The behaviour of the NNTH is illustrated in table 1 for various arbitrary values of the odds ratio and unexposed event rate.

	Practical application of the NNTH

The application of the NNTH to real life examples is illustrated in table 2.^6-11 These examples were chosen because they cover a wide range of values for the odds ratio and the unexposed event rate. Obviously, the validity of the NNTH always depends on the odds ratio and unexposed event rate being unbiased estimators of their respective parameters.

These examples illustrate how the NNTH, by taking the unexposed event rate into account, can lead to an assessment of risk importance which differs from the assessment that could be concluded from the odds ratio alone. For example, the NNTH gives the clinician a better understanding of the risks of thromboembolism associated with third generation oral contraceptives compared with second generation products. Furthermore, the different NNTH values obtained from the different studies used in this example illustrate how a measure of absolute risk (the NNTH) is more informative than a measure of relative risk (the odds ratio) because, unlike the odds ratio, it takes into account the "background" risk of the outcome occurring in unexposed people. The NNTH also illustrates better the range of risk estimates obtained from different studies.

In the case of appetite suppressing drugs, an odds ratio of 23.1 for pulmonary hypertension does not by itself provide a clinician with the necessary information to weigh properly the risks and benefits. However, the NNTH, which in this case is 90 500 for one year, is a better decision making tool since it takes into account the rate of events in the unexposed population (unexposed event rate), which is very low in this example. Likewise, in the case of non-steroidal anti-inflammatory drugs and gastrointestinal bleeding and perforation, a NNTH of 650 is more understandable and informative than an odds ratio of 4.7.

	Advantages over the odds ratio

The NNTH is first and foremost a more intuitively understandable measure of risk than the odds ratio. In addition, it is a composite measure that takes into account not only the odds ratio but also the unexposed event rate. Consequently, it contains more information than the odds ratio alone and provides estimates of risk that better correspond with reality and with the intuitive assessment made by practising clinicians. For example, an odds ratio of 2 would weigh more heavily in the decision making process if it applied to an adverse event whose rate of occurrence in the unexposed population were 1 in 100 (NNTH=100) than if it were 1 in 100 000 (NNTH=100 000) (see table 1).

	Limitations of the NNTH

Obtaining the unexposed event rate
Unless the case-control study providing the odds ratio is nested within a cohort or a primarily defined population, it is necessary to go to other sources to estimate the unexposed event rate. These sources can be either the controls in randomised controlled trials or the unexposed subjects in cohort studies. However, it is not always possible to find a study that provides an appropriate estimate of the unexposed event rate.

Time dependency
In terms of duration of follow up for the occurrence of adverse events, the NNTH also calls for congruence between the studies from which the odds ratio and the unexposed event rate are derived. Thus, for the non-steroidal anti-inflammatory drugs example in table 2, the unexposed event rate, which is a monthly rate in the source article, had to be adjusted upward to fit with the five month follow up period for which the odds ratio was computed in the source case-control study. This extrapolation assumes that the rate of upper gastrointestinal bleeding is constant over time in unexposed people.

	Confidence intervals

The confidence interval is nearly always provided for the odds ratio. Thus, it might be tempting to proceed simply by applying the NNTH formula to the upper and lower bounds of the confidence interval of the odds ratio. This does not, however, take into account the uncertainty surrounding the unexposed event rate, which is also an estimate. Even if a confidence interval were available for the unexposed event rate, the problem of calculating variances and covariances for these estimators would remaina task complicated by the non-Gaussian distribution of the odds ratio.

An approximate confidence interval can be calculated for the NNTH by using the limits of the odds ratio's confidence interval (as described above) provided they are both greater than 1. If the lower limit of the confidence interval for the odds ratio is less than 1, then the NNTH formula used to calculate the upper limit of the approximate confidence interval for the NNTH becomes 1/(1-odds ratio)(unexplained event rate). If the lower limit of the confidence interval for the odds ratio is exactly 1, then the upper limit of the confidence interval for the NNTH is indeterminate.⁵ When interpreting approximate confidence intervals, bear in mind that the true confidence interval is inevitably larger than the one obtained by this method because of the additional uncertainty surrounding the unexposed event rate.

	Which risk is expressed?

The NNTH is an estimator of risk that focuses only on the additional risk conferred by a treatment; it does not express the total risk attributable to the combination of the background risk and the risk due to exposure. This should be taken into consideration when using the NNTH as an adjunct to clinical decision making

	Conclusion

In the context of case-control studies that focus on the adverse effects of treatments, we propose the NNTH as an aid to clinical decision making that is preferable to the odds ratio. As such, it is our hope that it may better convey the clinical importance of the results of epidemiological studies, thus helping to avoid future unnecessary confusion (and even panic) among clinicians, policy makers, and the public.

	Acknowledgments

We thank Professor Doug Altman (reviewer) and Drs Lucie Blais, Henri McQuay, Olli Miettinen, Andrew Moore, David Sackett, Sami Suissa, and the other reviewers for helpful comments on the manuscript, and Mrs Anita Massicotte for clerical assistance.

	Footnotes

Competing interests: None declared.

website extra: An appendix showing the derivation of the NNTH appears on the BMJ's website www.bmj.com

	References

1.	Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med 1998; 318: 1728-1733[Medline].
2.	Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology: a basic science for clinical medicine. 2nd ed. Toronto: Little, Brown, 1991.
3.	Sackett DL, Deeks JJ, Altman DG. Down with odds ratios! Evidence-Based Med 1996; 1: 164-166.
4.	Sackett DL, Richardson WS, Rosenberg W, Haynes RB. Evidence based medicine. London: Churchill Livingstone, 1997.
5.	Altman DG. Confidence intervals for the number needed to treat. BMJ 1998; 317: 1309-1312[Free Full Text].
6.	Spitzer WO, Lewis MA, Heinemann LAJ, Thorogood M, MacRae KD, for the Transnational Research Group on Oral Contraceptives and the Health of Young Women. Third generation oral contraceptives and risk of venous thromboembolic disorders: an international case-control study. BMJ 1996; 312: 83-88[Abstract/Free Full Text].
7.	Jick H, Jick SS, Gurewich V, Myers MW, Vasilakis C. Risk of idiopathic cardiovascular death and non-fatal venous thromboembolism in women using oral contraceptives with differing progestagen components. Lancet 1995; 346: 1589-1593[CrossRef][Medline].
8.	WHO Collaborative Study of Cardiovascular Disease and Steroid Hormone Contraception. Effect of different progestagens in low oestrogen oral contraceptives on venous thromboembolic disease. Lancet 1995; 346: 1582-1588[CrossRef][Medline].
9.	Abenhaim L, Moride Y, Brenot F, Rich S, Benichou J, Kurz X, et al. Appetite-suppressant drugs and the risk of primary pulmonary hypertension. N Engl J Med 1996; 335: 609-616[Abstract/Free Full Text].
10.	Garcia Rodriguez LA, Jick H. Risk of upper gastrointestinal bleeding and perforation associated with individual non-steroidal anti-inflammatory drugs. Lancet 1994; 343: 769-772[CrossRef][Medline].
11.	Carson JL, Strom BL, Soper KA, West SL, Morse ML. The association of nonsteroidal anti-inflammatory drugs with upper gastrointestinal tract bleeding. Arch Intern Med 1987; 147: 85-88[Abstract].

(Accepted 11 August 1999)