BMJ 1999;319:1492-1495 ( 4 December )

Education and debate

Calculating the number needed to treat for trials where the outcome is time to an event

Douglas G Altman, professor of statistics in medicine a,  Per Kragh Andersen, professor of biostatistics b. 

a Imperial Cancer Research Group Medical Statistics Group, Centre for Statistics in Medicine, Institute of Health Sciences, Headington, Oxford OX3 7LF, b Department of Biostatistics, University of Copenhagen, DK-2200 Copenhagen N, Denmark

Correspondence to: D G Altman d.altman{at}icrf.icnet.uk

The number of patients who need to be treated to prevent one additional event (number needed to treat; NNT) has become a widely used measure of treatment benefit derived from the results of randomised controlled trials with a binary outcome. 1 2 We show how to obtain a number needed to treat for studies where the primary outcome is the time to an event. We consider primarily the situation where there is no access to raw data, for example, when reviewing a published study, and also how to proceed when given the raw data.


Summary points

The number needed to treat is the number of patients who need to be treated to prevent one additional adverse outcome
This number (with confidence interval) is a clinically useful way to report the results of controlled trials
For any trial which has reported a binary outcome, the number needed to treat can be obtained as the reciprocal of the absolute difference in proportions of patients with the outcome of interest
In studies where the outcome of interest is the time to an event, calculations can be extended to show the number needed to treat at any time after the start of treatment



    Time to event data

As noted previously, for studies with binary outcome the number needed to treat will vary according to the length of follow up.3 For studies of survival this relation with time is more explicit. There is no single number needed to treat; rather it can be calculated at any time point after the start of treatment. Often there are one or two time points of particular clinical interest.

A time specific number needed to treat represents the number of patients who need to be given the treatment in question for one additional patient to survive to that time point---that is, to benefit from the treatment. To obtain an estimate of the number needed to treat together with a confidence interval, one of the following is needed: (a) an estimate of the survival probability in each group at one fixed time point, and either the number of patients "at risk" at that time---that is, not yet having experienced the event of interest---or the standard errors of the survival probabilities; or (b) the estimated hazard ratio and its standard error, and the estimated survival probability in the control group at a fixed time. Unfortunately, the reporting of results is often inadequate in studies of survival,4 and the required information is often not provided.

    Methods and examples

We will assume there are two treatment groups. The calculations relate to survival probabilities at a fixed time point after the start of the follow up period---that is, from the start of treatment. We consider three cases.

Only survival probabilities available
Suppose, firstly, that only a simple survival analysis has been performed, and that Kaplan-Meier survival curves have been generated. We denote the estimated survival probabilities in the active and control treatment groups at a chosen time point as Sa and Sc and will assume that the active drug is effective, so that Sa>Sc. The absolute risk reduction is estimated as Sa-Sc. If necessary, Sa and Sc can be estimated by careful measurement of a graph of the Kaplan-Meier survival curves. The number needed to treat is obtained simply as 1/(Sa-Sc), just as for trials with binary data.

The 95% confidence interval for the absolute risk reduction (ARR) is ARR±1.96 SE(ARR), where SE(ARR) is the standard error of the absolute risk reduction. If the limits of this confidence interval are Al and Au, then the 95% confidence interval for the number needed to treat is 1/Au to 1/Al.

When neither the standard error nor confidence interval for the absolute risk reduction is given, there are three options:

1. If confidence intervals for Sa and Sc are given, each standard error can be taken as one quarter of the width of the relevant confidence interval.

2. If the standard errors of Sa and Sc are given, SE(ARR) can be calculated as radical {[SE(Sa)]2 + [SE(Sc)]2}.

3. If standard errors or confidence intervals are not given, we need the numbers of patients still at risk (alive) at the time corresponding to the estimated probabilities, which we will call na and nc. These numbers are sometimes shown in the graph of survival; if not, they will have to be inferred. If there is little loss to follow up, the numbers at risk will be close to SaNa and ScNc, where Na and Nc are the numbers randomised to each group. Information about loss to follow up is, however, often missing.4 The standard error of the absolute risk reduction is radical [Sa2(1-Sa)/na + Sc2(1-Sc)nc], and a 95% confidence interval is obtained as above. If none of the preceding calculations is possible, then a confidence interval cannot be obtained for the number needed to treat.
 
(Credit: SUE SHARPLES)

Example
Overall, 279 patients with locally advanced rectal cancer were randomised to receive radiotherapy followed by surgery compared with surgery alone.5 The sample size calculation was on the basis of survival for 3 years. From figure 2 in the paper the three year survival rates were 62.2% and 46.8% for the two groups, with 59 and 43 patients still alive respectively. The above formula gives ARR=0.622-0.468=0.154, and SE(ARR)=radical [0.6222(1-0.622)/59+0.4682(1-0.468)/43]=0.072, giving a 95% confidence interval for the absolute risk reduction as 0.013 to 0.295. The number needed to treat at 3 years is thus 1/0.154=6.49 and its 95% confidence interval is 1/0.295 to 1/0.013, or 3.4 to 77.6. We thus estimate that giving patients radiotherapy before surgery would lead to one extra survivor at 3 years for every 6.5 patients treated. The confidence interval is very wide, however.

When the treatment effect is not statistically significant (P>0.05) the 95% confidence interval for the absolute risk reduction spans zero, and one limit of the confidence interval for the number needed to treat will be negative. In this case the inverse of the absolute risk reduction is often termed the number needed to harm (NNH).6 It is, however, more accurate to refer to the number needed to treat to benefit (NNTB) or to the number needed to treat to harm (NNTH).7 Difficulties in graphing the confidence interval are avoided by plotting the absolute risk reduction at suitable values and relabelling the axis,7 as illustrated below.

Survival probabilities and estimate of hazard ratio available
The hazard ratio is quite like a relative risk rather than an odds ratio,4 but it is not the same as a relative risk. Customary methods of analysis assume that this ratio is the same at all times after the start of treatment.

The log rank test provides the observed and expected numbers of events in each group. The hazard ratio is estimated as the ratio of the ratios of observed to expected numbers for the active and control groups. If the treatment is beneficial, the hazard ratio will be less than 1. Unfortunately, few authors provide the observed and expected numbers from this analysis.

The hazard ratio is more often available from a Cox regression, which is used in controlled trials to adjust the trial results for other prognostic variables. Here the regression coefficient for treatment (often denoted b or beta ) is the log hazard ratio. It follows that the hazard ratio is estimated as eb. Either the regression coefficient (b) or the hazard ratio (h=eb) may be quoted in a published paper.

If at some specified time, t, the survival probability in the control group is Sc(t) then the survival probability in the active group is [Sc(t)]h, where h is the hazard ratio comparing the treatment groups. The number needed to treat is estimated as: NTT=1/{[Sc(t)]h - Sc(t)} (equation 1)

where Sc(t) is obtained in one of the ways previously described. The number of patients at risk is not needed (the information is incorporated into the standard error of h). Note that h and the number needed to treat may depend on which other variables are included in the regression model and how they are coded, although in a randomised trial the differences should be small.

The 95% confidence interval for the number needed to treat is obtained from equation 1 by replacing h in turn by the two limits of the 95% confidence interval for h. If not given explicitly, the values can be obtained from the regression coefficient b (recall that h=eb) and its standard error as eb-1.96se(b) and eb+1.96se(b). The resulting confidence interval may be too narrow as it ignores the imprecision in the estimate of Sc(t). We return to this issue later. If we have results of a regression analysis but do not have any estimate of the control group survival probability Sc(t), we cannot estimate the number needed to treat.

Example
We use data from a randomised trial comparing intensive versus standard insulin treatment in patients with diabetes mellitus and acute myocardial infarction.8 From figure 1 in the paper, the control group mortality rates at 2 and 4 years were 0.33 and 0.49 respectively. The reported hazard ratio was h=0.72 with 95% confidence interval 0.55 to 0.92. The number needed to treat at 2 years is thus estimated as 1/(0.330.72-0.33)=8.32. The 95% confidence interval for the number needed to treat is obtained from equation 1 setting h to 0.55 and then 0.92, giving 4.7 to 32.7. 



View larger version (25K):
[in this window]
[in a new window]
  		Fig 1.   Kaplan-Meier plots of survival for 164 patients with non-small lung cancer treated with radiotherapy plus chemotherapy versus radiotherapy alone9

Raw data available
For researchers reporting the results of a trial, all the raw data will be available. Clearly it is possible to use any of the above methods to calculate a number needed to treat, either unadjusted or adjusted, as all of the statistics mentioned can be generated easily. We can also extend the method quite simply to generate a plot showing number needed to treat as a function of time rather than at a single time point.

Example
One hundred and seventy two patients with non-small cell lung cancer were randomised to receive either radiotherapy alone or in combination with chemotherapy.9 The raw data (with somewhat longer follow up) are given by Piantadosi.10 Figure 1 shows Kaplan- Meier curves of disease free survival for the two treatment groups, while the table shows the estimated number needed to treat, with 95% confidence intervals.


                              
View this table:
[in this window]
[in a new window]
 

Number needed to treat at various times after treatment for 164 patients with non-small cell lung cancer treated with radiotherapy plus chemotherapy versus radiotherapy alone9



View larger version (23K):
[in this window]
[in a new window]
  		Fig 2.   Number needed to treat to either benefit or harm with 95% confidence interval by time since treatment for 164 patients with non-small cell lung cancer treated with radiotherapy plus chemotherapy versus radiotherapy alone9 on basis of Cox regression model including only treatment

The table is based on simple comparison of the two treatment groups. Adjusted survival curves can be produced, often by Cox regression, to adjust a treatment comparison for various baseline variables. The number needed to treat can also be obtained from these adjusted analysis, again using equation 1. An example is shown in figure 2. If, as here, the treatment effect is statistically significant with P<0.05, the 95% confidence interval for the number needed to treat will exclude harmful effects at all times.

Even though the model assumes a constant hazard ratio (relative risk) for the comparison of two treatments, it is important to recognise that the number needed to treat will differ for subsets of patients with varying prognosis. It may be valuable to construct graphs like figure 2 for important subsets of patients, such as by stage or cell type in the example.

    Discussion

The need for absolute as well as relative measures of effect is increasingly recognised.2 The number needed to treat has recently become a quite popular way of reporting the results of clinical trials.1 The number needed to treat will usually tend to fall as the time from start of treatment increases. Sackett et al suggested a simple correction for length of follow up, in which the observed number needed to treat is multiplied by the ratio of the actual average duration of follow up to the duration of interest.3 This calculation assumes that the effect of treatment (relative risk reduction) is constant over time, and that events occur at a constant rate over time. Under these strong assumptions a number needed to treat of, say, 6 derived from a study in which patients were followed on average for 2 years would imply a number needed to treat of 3 if patients were followed for 4 years. Following this approach, Miller presented for several trials numbers needed to treat per year, calculated as the overall number needed to treat multiplied by the average length of follow up in years.11

When actual times to an event of interest are recorded, numbers needed to treat can be obtained as a function of follow up time. For many published papers it will be possible to use these methods to obtain numbers needed to treat, perhaps adjusted for other variables. This measure should be valuable for those reviewing papers for journals of secondary publication, with the number needed to treat calculated for one or two specific time points.

The confidence interval for the number needed to treat on the basis of the Cox model may be too narrow ("conservative") because the method ignores the uncertainty in the estimate of the survival probability. This deficiency applies equally to the confidence interval obtained for the number needed to treat derived from the log odds ratio estimated from a logistic regression model. There is no way around this problem when describing the number needed to treat from information given in a published paper. An unbiased confidence interval can be obtained from the raw data, but the method is rather complex and we have not presented it here.

The number needed to treat is valuable additional information that can be provided in reports of randomised trials where the outcome of interest was time to an event. We have shown how to calculate the number needed to treat for such studies in several ways. In general, it will better to make such calculations directly, rather than making the strong assumption that the risk reduction is constant over follow up time.

    Footnotes

   Funding: Activities of the Danish Epidemiology Service Centre are supported by a grant from the Danish National Research Foundation.

Competing interests: None declared.

    References

1. Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. BMJ 1995; 310: 452-454[Free Full Text].
2. Sackett DL, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine. How to practice and teach EBM. London: Churchill Livingstone, 1997:136-141, 168-70.
3. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology: a basic science for clinical medicine, 2nd ed. Boston: Little Brown, 1991:208.
4. Altman DG, De Stavola BL, Love SB, Stepniewska KA. Review of survival analyses published in cancer journals. Br J Cancer 1995; 72: 511-518[Medline].
5. Medical Research Council Rectal Cancer Working Party. Randomised trial of surgery alone versus radiotherapy followed by surgery for potentially operable locally advanced rectal cancer. Lancet 1996; 348: 1605-1610[Medline].
6. McQuay HJ, Moore RA. Using numerical results from systematic reviews in clinical practice. Ann Intern Med 1997; 126: 712-720[Abstract/Free Full Text].
7. Altman DG. Confidence intervals for the number needed to treat. BMJ 1998; 317: 1309-1312[Free Full Text].
8. Malmberg K, for the Diabetes Mellitus Insulin Glucose Infusion in Acute Myocardial Infarction (DIGAMI) Study Group. Prospective randomised study of intensive insulin treatment on long term survival after acute myocardial infarction in patients with diabetes mellitus. BMJ 1997; 314: 1512-1515[Abstract/Free Full Text].
9. Lad T, Rubinstein L, Sadeghi A. The benefit of adjuvant treatment for resected locally advanced non-small-cell lung cancer. J Clin Oncol 1988; 6: 9-17[Abstract].
10. Piantadosi S. Clinical trials. A methodologic approach. Chichester: John Wiley, 1997.
11. Miller DB. Secondary prevention for ischemic heart disease. Relative numbers needed to treat with different therapies. Arch Intern Med 1997; 157: 2045-2052[Abstract].

(Accepted 5 July 1999)

© BMJ 1999

This article has been cited by other articles:

  • Dodds, L., Fell, D. B., Joseph, K. S., Dewar, R., Scott, H., Platt, R., Aronson, K. J. (2008). Relationship of Time Since Childbirth and Other Pregnancy Factors to Premenopausal Breast Cancer Prognosis. Obstet Gynecol 111: 1167-1173 [Abstract] [Full text]  
  • Pisters, K. M.W., Evans, W. K., Azzoli, C. G., Kris, M. G., Smith, C. A., Desch, C. E., Somerfield, M. R., Brouwers, M. C., Darling, G., Ellis, P. M., Gaspar, L. E., Pass, H. I., Spigel, D. R., Strawn, J. R., Ung, Y. C., Shepherd, F. A. (2007). Cancer Care Ontario and American Society of Clinical Oncology Adjuvant Chemotherapy and Adjuvant Radiation Therapy for Stages I-IIIA Resectable Non Small-Cell Lung Cancer Guideline. JCO 25: 5506-5518 [Abstract] [Full text]  
  • Ravani, P., Parfrey, P. S., Dicks, E., Barrett, B. J. (2007). Clinical research of kidney diseases II: problems of study design. Nephrol Dial Transplant 22: 2785-2794 [Full text]  
  • Alho, O.-P., Koivunen, P., Penna, T., Teppo, H., Koskela, M., Luotonen, J. (2007). Tonsillectomy versus watchful waiting in recurrent streptococcal pharyngitis in adults: randomised controlled trial. BMJ 334: 939-939 [Abstract] [Full text]  
  • Schulz, H., Bohlius, J. F., Trelle, S., Skoetz, N., Reiser, M., Kober, T., Schwarzer, G., Herold, M., Dreyling, M., Hallek, M., Engert, A. (2007). Immunochemotherapy With Rituximab and Overall Survival in Patients With Indolent or Mantle Cell Lymphoma: A Systematic Review and Meta-analysis. JNCI J Natl Cancer Inst 99: 706-714 [Abstract] [Full text]  
  • Tleyjeh, I. M., Ghomrawi, H. M.K., Steckelberg, J. M., Hoskin, T. L., Mirzoyev, Z., Anavekar, N. S., Enders, F., Moustafa, S., Mookadam, F., Huskins, W. C., Wilson, W. R., Baddour, L. M. (2007). The Impact of Valve Surgery on 6-Month Mortality in Left-Sided Infective Endocarditis. Circulation 115: 1721-1728 [Abstract] [Full text]  
  • Montalescot, G., Antoniucci, D., Kastrati, A., Neumann, F. J., Borentain, M., Migliorini, A., Boutron, C., Collet, J.-P., Vicaut, E. (2007). Abciximab in primary coronary stenting of ST-elevation myocardial infarction: a European meta-analysis on individual patients' data with long-term follow-up. Eur Heart J 28: 443-449 [Abstract] [Full text]  
  • Kraemer, H. C. (2006). Correlation coefficients in medical research: from product moment correlation to the odds ratio. Stat Methods Med Res 15: 525-545 [Abstract]  
  • Kraemer, H. C., Mintz, J., Noda, A., Tinklenberg, J., Yesavage, J. A. (2006). Caution Regarding the Use of Pilot Studies to Guide Power Calculations for Study Proposals.. Arch Gen Psychiatry 63: 484-489 [Abstract] [Full text]  
  • Alho, O.-P., Teppo, H., Mantyselka, P., Kantola, S. (2006). Head and neck cancer in primary care: presenting symptoms and the effect of delayed diagnosis of cancer cases.. CMAJ 174: 779-784 [Abstract] [Full text]  
  • Boeckh, M., Kim, H. W., Flowers, M. E. D., Meyers, J. D., Bowden, R. A. (2006). Long-term acyclovir for prevention of varicella zoster virus disease after allogeneic hematopoietic cell transplantation--a randomized double-blind placebo-controlled study. Blood 107: 1800-1805 [Abstract] [Full text]  
  • Wellens, H. J., Pappone, C., Santinelli, V., Wellens, H. J., Pappone, C., Santinelli, V. (2005). When to Perform Catheter Ablation in Asymptomatic Patients With a Wolff-Parkinson-White Electrocardiogram. Circulation 112: 2201-2216 [Full text]  
  • Maier, P. C, Funk, J., Schwarzer, G., Antes, G., Falck-Ytter, Y. T (2005). Treatment of ocular hypertension and open angle glaucoma: meta-analysis of randomised controlled trials. BMJ 331: 134- [Abstract] [Full text]  
  • Bohlius, J., Langensiepen, S., Schwarzer, G., Seidenfeld, J., Piper, M., Bennett, C., Engert, A. (2005). Recombinant Human Erythropoietin and Overall Survival in Cancer Patients: Results of a Comprehensive Meta-analysis. JNCI J Natl Cancer Inst 97: 489-498 [Abstract] [Full text]  
  • de Lemos, M. L. (2005). NNT for studies with long-term follow-up. CMAJ 172: 613-613 [Full text]  
  • Barratt, A. L., Wyer, P. C., Guyatt, G., Simpson, J. M. (2005). NNT for studies with long-term follow-up. CMAJ 172: 613-615 [Full text]  
  • Neves-e-Castro, M. (2003). Menopause in crisis post-Women's Health Initiative? A view based on personal clinical experience. Hum Reprod 18: 2512-2518 [Abstract] [Full text]  
  • Muthu, V. (2003). The number needed to treat: problemsdescribing non-significant results. Evid. Based Ment. Health 6: 72-72 [Full text]  
  • Pitt, B., Remme, W., Zannad, F., Neaton, J., Martinez, F., Roniker, B., Bittman, R., Hurley, S., Kleiman, J., Gatlin, M., the Eplerenone Post-Acute Myocardial Infarction He, (2003). Eplerenone, a Selective Aldosterone Blocker, in Patients with Left Ventricular Dysfunction after Myocardial Infarction. NEJM 348: 1309-1321 [Abstract] [Full text]  
  • Wing, L. M.H., Reid, C. M., Ryan, P., Beilin, L. J., Brown, M. A., Jennings, G. L.R., Johnston, C. I., McNeil, J. J., Macdonald, G. J., Marley, J. E., Morgan, T. O., West, M. J., the Second Australian National Blood Pressure Stud, (2003). A Comparison of Outcomes with Angiotensin-Converting-Enzyme Inhibitors and Diuretics for Hypertension in the Elderly. NEJM 348: 583-592 [Abstract] [Full text]  
  • Annane, D., Sebille, V., Charpentier, C., Bollaert, P.-E., Francois, B., Korach, J.-M., Capellier, G., Cohen, Y., Azoulay, E., Troche, G., Chaumet-Riffaut, P., Bellissant, E. (2002). Effect of Treatment With Low Doses of Hydrocortisone and Fludrocortisone on Mortality in Patients With Septic Shock. JAMA 288: 862-871 [Abstract] [Full text]  
  • Attia, J, Page, J, Heller, R F, Dobson, A J (2002). Impact numbers in health policy decisions. J. Epidemiol. Community Health 56: 600-605 [Abstract] [Full text]  
  • Fouque, D., Wang, P., Laville, M., Boissel, J.-P. (2000). Low protein diets delay end-stage renal disease in non-diabetic adults with chronic renal failure. Nephrol Dial Transplant 15: 1986-1992 [Abstract] [Full text]  
Tools
Services
Online poll
Find out more

Rapid responses for this article

There are no rapid responses for this article.

Student BMJ

Risk of surgery for inflammatory bowel disease: record linkage studies

What can you learn from this BMJ paper? Read Leanne Tite's Paper+

www.student.bmj.com

Listen to the latest BMJ Interview