Paper

Empirical assessment of effect of publication bias on meta-analyses

BMJ 2000; 320 doi: 10.1136/bmj.320.7249.1574 (Published 10 June 2000)
Cite this as: BMJ 2000;320:1574
  1. A J Sutton, Lecturer in medical statistics (ajs22{at}le.ac.uk)a,
  2. S J Duval, assistant professorb,
  3. R L Tweedie, headc,
  4. K R Abrams, senior lecturer in medical statisticsa,
  5. D R Jones, professor in medical statisticsa
  1. a Department of Epidemiology and Public Health, University of Leicester, Leicester LE1 6TP
  2. b Division of Epidemiology, School of Public Health, University of Minnesota, Minneapolis, USA
  3. c Department of Biostatistics, School of Public Health, University of Minnesota
  1. Correspondence to: A J Sutton
  • Accepted 2 March 2000

Abstract

Objective: To assess the effect of publication bias on the results and conclusions of systematic reviews and meta-analyses.

Design: Analysis of published meta-analyses by trim and fill method.

Studies: 48 reviews in Cochrane Database of Systematic Reviews that considered a binary endpoint and contained 10 or more individual studies.

Main outcome measures: Number of reviews with missing studies and effect on conclusions of meta-analyses.

Results: The trim and fill fixed effects analysis method estimated that 26 (54%) of reviews had missing studies and in 10 the number missing was significant. The corresponding figures with a random effects model were 23 (48%) and eight. In four cases, statistical inferences regarding the effect of the intervention were changed after the overall estimate for publication bias was adjusted for.

Conclusions: Publication or related biases were common within the sample of meta-analyses assessed. In most cases these biases did not affect the conclusions. Nevertheless, researchers should check routinely whether conclusions of systematic reviews are robust to possible non-random selection mechanisms.

Footnotes

  • Funding None declared.

  • Competing interests None.

  • Embedded Image Figures illustrating the method and funnel plots of all trials and a full table of results are available on the BMJ's website

Rapid responses

THIS WEEK'S POLL

Read related article

See previous polls

BMJ most popular