Education And Debate

Calculating the number needed to treat for trials where the outcome is time to an event

BMJ 1999; 319 doi: http://dx.doi.org/10.1136/bmj.319.7223.1492 (Published 04 December 1999) Cite this as: BMJ 1999;319:1492
  1. Douglas G Altman, professor of statistics in medicine (d.altman{at}icrf.icnet.uk)a,
  2. Per Kragh Andersen, professor of biostatisticsb
  1. a Imperial Cancer Research Group Medical Statistics Group, Centre for Statistics in Medicine, Institute of Health Sciences, Headington, Oxford OX3 7LF
  2. b Department of Biostatistics, University of Copenhagen, DK-2200 Copenhagen N, Denmark
  1. Correspondence to: Correspondence to: D G Altman
  • Accepted 5 July 1999

The number of patients who need to be treated to prevent one additional event (number needed to treat; NNT) has become a widely used measure of treatment benefit derived from the results of randomised controlled trials with a binary outcome 1 2 We show how to obtain a number needed to treat for studies where the primary outcome is the time to an event. We consider primarily the situation where there is no access to raw data, for example, when reviewing a published study, and also how to proceed when given the raw data.

Summary points

The number needed to treat is the number of patients who need to be treated to prevent one additional adverse outcome

This number (with confidence interval) is a clinically useful way to report the results of controlled trials

For any trial which has reported a binary outcome, the number needed to treat can be obtained as the reciprocal of the absolute difference in proportions of patients with the outcome of interest

In studies where the outcome of interest is the time to an event, calculations can be extended to show the number needed to treat at any time after the start of treatment

Time to event data

As noted previously, for studies with binary outcome the number needed to treat will vary according to the length of follow up.3 For studies of survival this relation with time is more explicit. There is no single number needed to treat; rather it can be calculated at any time point after the start of treatment. Often there are one or two time points of particular clinical interest.

A time specific number needed to treat represents the number of patients who need to be given the treatment in question for one additional patient to survive to that time point—that is, to benefit from the treatment. To obtain an estimate of the number needed to treat together with a confidence interval, one of the following is needed:(a) an estimate of the survival probability in each group at one fixed time point, and either the number of patients “at risk” at that time—that is, not yet having experienced the event of interest—or the standard errors of the survival probabilities; or (b) the estimated hazard ratio and its standard error, and the estimated survival probability in the control group at a fixed time Unfortunately, the reporting of results is often inadequate in studies of survival,4 and the required information is often not provided.

Methods and examples

We will assume there are two treatment groups. The calculations relate to survival probabilities at a fixed time point after the start of the follow up period—that is, from the start of treatment. We consider three cases

Only survival probabilities available

Suppose, firstly, that only a simple survival analysis has been performed, and that Kaplan-Meier survival curves have been generated. We denote the estimated survival probabilities in the active and control treatment groups at a chosen time point as Sa and Sc and will assume that the active drug is effective, so that Sa>Sc. The absolute risk reduction is estimated as Sa−Sc. If necessary, Sa and Sc can be estimated by careful measurement of a graph of the Kaplan-Meier survival curves The number needed to treat is obtained simply as 1/(Sa−Sc), just as for trials with binary data.

The 95% confidence interval for the absolute risk reduction (ARR) is ARR±1.96 SE(ARR), where SE(ARR) is the standard error of the absolute risk reduction. If the limits of this confidence interval are Al and Au, then the 95% confidence interval for the number needed to treat is 1/Au to 1/Al.

When neither the standard error nor confidence interval for the absolute risk reduction is given, there are three options:

  1. If confidence intervals for Sa and Sc are given, each standard error can be taken as one quarter of the width of the relevant confidence interval

  2. If the standard errors of Sa and Sc are given, SE(ARR) can be calculated as √{[SE(Sa)]2+[SE(Sc)]2}

  3. If standard errors or confidence intervals are not given, we need the numbers of patients still at risk (alive) at the time corresponding to the estimated probabilities, which we will call na and nc. These numbers are sometimes shown in the graph of survival; if not, they will have to be inferred. If there is little loss to follow up, the numbers at risk will be close to Sa Na and Sc Nc, where Na and Nc are the numbers randomised to each group. Information about loss to follow up is, however, often missing.4 The standard error of the absolute risk reduction is √[Sa2 (1−Sa)/na + Sc2 (1−Sc) nc], and a 95% confidence interval is obtained as above. If none of the preceding calculations is possible, then a confidence interval cannot be obtained for the number needed to treat.



Embedded Image

(Credit: SUE SHARPLES)

Example

Overall, 279 patients with locally advanced rectal cancer were randomised to receive radiotherapy followed by surgery compared with surgery alone.5 The sample size calculation was on the basis of survival for 3 years. From figure2 in the paper the three year survival rates were 62.2% and 46.8% for the two groups, with 59 and 43 patients still alive respectively. The above formula gives ARR=0.622−0.468=0.154, and SE(ARR)=√[0.6222 (1−0.622)/59+0.4682 (1−0.468)/43]=0.072, giving a 95% confidence interval for the absolute risk reduction as 0.013 to 0.295. The number needed to treat at 3 years is thus 1/0.154=6.49 and its 95% confidence interval is 1/0.295 to 1/0.013, or 3.4 to 77.6. We thus estimate that giving patients radiotherapy before surgery would lead to one extra survivor at 3 years for every 6.5 patients treated. The confidence interval is very wide, however.

When the treatment effect is not statistically significant (P>0.05) the 95% confidence interval for the absolute risk reduction spans zero, and one limit of the confidence interval for the number needed to treat will be negative. In this case the inverse of the absolute risk reduction is often termed the number needed to harm (NNH).6 It is, however, more accurate to refer to the number needed to treat to benefit (NNTB) or to the number needed to treat to harm (NNTH).7 Difficulties in graphing the confidence interval are avoided by plotting the absolute risk reduction at suitable values and relabelling the axis,7 as illustrated below.

Survival probabilities and estimate of hazard ratio available

The hazard ratio is quite like a relative risk rather than an odds ratio,4 but it is not the same as a relative risk Customary methods of analysis assume that this ratio is the same at all times after the start of treatment

The log rank test provides the observed and expected numbers of events in each group. The hazard ratio is estimated as the ratio of the ratios of observed to expected numbers for the active and control groups. If the treatment is beneficial, the hazard ratio will be less than 1. Unfortunately, few authors provide the observed and expected numbers from this analysis.

The hazard ratio is more often available from a Cox regression, which is used in controlled trials to adjust the trial results for other prognostic variables. Here the regression coefficient for treatment (often denoted b or β) is the log hazard ratio. It follows that the hazard ratio is estimated as eb. Either the regression coefficient (b) or the hazard ratio (h=eb) may be quoted in a published paper.

If at some specified time, t, the survival probability in the control group is Sc (t) then the survival probability in the active group is [Sc (t)]h, where h is the hazard ratio comparing the treatment groups. The number needed to treat is estimated as:NTT=1/{[Sc (t)]h - Sc (t)} (equation 1)

where Sc (t) is obtained in one of the ways previously described. The number of patients at risk is not needed (the information is incorporated into the standard error of h). Note that h and the number needed to treat may depend on which other variables are included in the regression model and how they are coded, although in a randomised trial the differences should be small.

The 95% confidence interval for the number needed to treat is obtained from equation 1 by replacing h in turn by the two limits of the 95% confidence interval for h. If not given explicitly, the values can be obtained from the regression coefficient b (recall that h=eb) and its standard error as eb-1.96se(b) and eb+1.96se(b). The resulting confidence interval may be too narrow as it ignores the imprecision in the estimate of Sc (t). We return to this issue later If we have results of a regression analysis but do not have any estimate of the control group survival probability Sc (t), we cannot estimate the number needed to treat.

Example

We use data from a randomised trial comparing intensive versus standard insulin treatment in patients with diabetes mellitus and acute myocardial infarction.8 From figure 1 in the paper, the control group mortality rates at 2 and 4 years were 0.33 and 0.49 respectively. The reported hazard ratio was h=0.72 with 95% confidence interval 0.55 to 0.92. The number needed to treat at 2 years is thus estimated as 1/(0.330.72-0.33)=8.32 The 95% confidence interval for the number needed to treat is obtained from equation 1 setting h to 0.55 and then 0.92, giving 4.7 to 32.7.

Fig 1.
Fig 1.

Kaplan-Meier plots of survival for 164 patients with non-small lung cancer treated with radiotherapy plus chemotherapy versus radiotherapy alone9

Raw data available

For researchers reporting the results of a trial, all the raw data will be available. Clearly it is possible to use any of the above methods to calculate a number needed to treat, either unadjusted or adjusted, as all of the statistics mentioned can be generated easily We can also extend the method quite simply to generate a plot showing number needed to treat as a function of time rather than at a single time point.

Example

One hundred and seventy two patients with non-small cell lung cancer were randomised to receive either radiotherapy alone or in combination with chemotherapy.1 The raw data (with somewhat longer follow up) are given by Piantadosi.10 Figure 1 shows Kaplan-Meier curves of disease free survival for the two treatment groups, while the table shows the estimated number needed to treat, with 95% confidence intervals.

Number needed to treat at various times after treatment for 164 patients with non-small cell lung cancer treated with radiotherapy plus chemotherapy versus radiotherapy alone9

View this table:
Fig 2.
Fig 2.

Number needed to treat to either benefit or harm with 95% confidence interval by time since treatment for 164 patients with non-small cell lung cancer treated with radiotherapy plus chemotherapy versus radiotherapy alone9 on basis of Cox regression model including only treatment

The table is based on simple comparison of the two treatment groups Adjusted survival curves can be produced, often by Cox regression, to adjust a treatment comparison for various baseline variables The number needed to treat can also be obtained from these adjusted analysis, again using equation 1. An example is shown in figure 2. If, as here, the treatment effect is statistically significant with P<0.05, the 95% confidence interval for the number needed to treat will exclude harmful effects at all times.

Even though the model assumes a constant hazard ratio (relative risk) for the comparison of two treatments, it is important to recognise that the number needed to treat will differ for subsets of patients with varying prognosis. It may be valuable to construct graphs like figure 2 for important subsets of patients, such as by stage or cell type in the example.

Discussion

The need for absolute as well as relative measures of effect is increasingly recognised.2 The number needed to treat has recently become a quite popular way of reporting the results of clinical trials.1 The number needed to treat will usually tend to fall as the time from start of treatment increases Sackett et al suggested a simple correction for length of follow up, in which the observed number needed to treat is multiplied by the ratio of the actual average duration of follow up to the duration of interest.3 This calculation assumes that the effect of treatment (relative risk reduction) is constant over time, and that events occur at a constant rate over time. Under these strong assumptions a number needed to treat of, say, 6 derived from a study in which patients were followed on average for 2 years would imply a number needed to treat of 3 if patients were followed for 4 years Following this approach, Miller presented for several trials numbers needed to treat per year, calculated as the overall number needed to treat multiplied by the average length of follow up in years.11

When actual times to an event of interest are recorded, numbers needed to treat can be obtained as a function of follow up time. For many published papers it will be possible to use these methods to obtain numbers needed to treat, perhaps adjusted for other variables. This measure should be valuable for those reviewing papers for journals of secondary publication, with the number needed to treat calculated for one or two specific time points.

The confidence interval for the number needed to treat on the basis of the Cox model may be too narrow (“conservative”) because the method ignores the uncertainty in the estimate of the survival probability This deficiency applies equally to the confidence interval obtained for the number needed to treat derived from the log odds ratio estimated from a logistic regression model. There is no way around this problem when describing the number needed to treat from information given in a published paper. An unbiased confidence interval can be obtained from the raw data, but the method is rather complex and we have not presented it here.

The number needed to treat is valuable additional information that can be provided in reports of randomised trials where the outcome of interest was time to an event. We have shown how to calculate the number needed to treat for such studies in several ways. In general, it will better to make such calculations directly, rather than making the strong assumption that the risk reduction is constant over follow up time.

Footnotes

  • Funding Activities of the Danish Epidemiology Service Centre are supported by a grant from the Danish National Research Foundation.

  • Competing interests None declared.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.