Education And Debate Systematic reviews in health care

Investigating and dealing with publication and other biases in meta-analysis

BMJ 2001; 323 doi: http://dx.doi.org/10.1136/bmj.323.7304.101 (Published 14 July 2001) Cite this as: BMJ 2001;323:101
  1. Jonathan A C Sterne (jonathan.sterne@bristol.ac.uk), senior lecturer in medical statistics,
  2. Matthias Egger, senior lecturer in epidemiology and public health medicine,
  3. George Davey Smith, professor of clinical epidemiology
  1. Medical Research Council Health Services Research Collaboration, Department of Social Medicine, University of Bristol, Bristol BS8 2PR
  1. Correspondence to: J A C Sterne

    This is the second in a series of four articles

    Studies that show a significant effect of treatment are more likely to be published, be published in English, be cited by other authors, and produce multiple publications than other studies.18 Such studies are therefore also more likely to be identified and included in systematic reviews, which may introduce bias.9 Low methodological quality of studies included in a systematic review is another important source of bias.10

    All these biases are more likely to affect small studies than large ones. The smaller a study the larger the treatment effect necessary for the results to be significant. The greater investment of time and money in larger studies means that they are more likely to be of high methodological quality and published even if their results are negative. Bias in a systematic review may therefore become evident through an association between the size of the treatment effect and study size—such associations may be examined both graphically and statistically.

    Summary points

    Asymmetrical funnel plots may indicate publication bias or be due to exaggeration of treatment effects in small studies of low quality

    Bias is not the only explanation for funnel plot asymmetry; funnel plots should be seen as a means of examining “small study effects” (the tendency for the smaller studies in a meta-analysis to show larger treatment effects) rather than a tool for diagnosing specific types of bias

    Statistical methods may be used to examine the evidence for bias and to examine the robustness of the conclusions of the meta-analysis in sensitivity analyses

    “Correction” of treatment effect estimates for bias should be avoided as such corrections may depend heavily on the assumptions made

    Multivariable models may be used, with caution, to examine the relative importance of different types of bias

    Graphical methods for detecting bias

    Funnel plots

    Funnel …

    Sign in

    Subscribe