Education And Debate

The relation between treatment benefit and underlying risk in meta-analysis

BMJ 1996; 313 doi: (Published 21 September 1996) Cite this as: BMJ 1996;313:735
  1. Stephen J Sharp, research fellow in medical statisticsa,
  2. Simon G Thompson, professor of medical statistics and epidemiologyb,
  3. Douglas G Altman, headc
  1. a Medical Statistics Unit, London School of Hygiene and Tropical Medicine, London WC1E 7HT,
  2. b Department of Medical Statistics and Evaluation, Royal Postgraduate Medical School, London W12 0NN,
  3. c Imperial Cancer Research Fund Medical Statistics Group, Centre for Statistics in Medicine, Institute of Health Sciences, PO Box 777, Oxford OX3 7LF
  1. Correspondence to: Mr Sharp.
  • Accepted 21 June 1996

In meta-analyses of clinical trials comparing a treated group with a control group it has been common to ask whether the treatment benefit varies according to the underlying risk of the patients in the different trials, with the hope of defining which patients would benefit most and which least from medical interventions. The usual analysis used to investigate this issue, however, which uses the observed proportions of events in the control groups of the trials as a measure of the underlying risk, is flawed and produces seriously misleading results. This arises through a bias due to regression to the mean and will be particularly acute in meta-analyses which include some small trials or in which the variability in the true underlying risks across trials is small. Approaches which previously have been thought to be more appropriate are to substitute the average proportion of events in the control and treated groups as the measure of underlying risk or to plot the proportion of events in the treated group against that in the control group (L'Abbe plot). However, these are still subject to bias in most circumstances. Because of the potentially seriously flawed conclusions that can result from such analyses, they should be replaced either by statistically appropriate (but more complex) approaches or, preferably, by analyses which investigate the dependence of the treatment effect on measured baseline characteristics of the patients in each trial.

Where there are substantial clinical differences between the different trials of a meta-analysis and their patients, or substantial quantitative differences in the results from the different trials, a single overall summary estimate of treatment benefit has little practical applicability.1 An analysis which ignores this heterogeneity is clinically misleading and scientifically naive.2 Many authors have now emphasised the clinical and scientific importance of investigating potential sources of …

Sign in