Investigating and dealing with publication and other biases in meta-analysis
BMJ 2001; 323 doi: https://doi.org/10.1136/bmj.323.7304.101 (Published 14 July 2001) Cite this as: BMJ 2001;323:101
- Jonathan A C Sterne (jonathan.sterne@bristol.ac.uk), senior lecturer in medical statistics,
- Matthias Egger, senior lecturer in epidemiology and public health medicine,
- George Davey Smith, professor of clinical epidemiology
- Medical Research Council Health Services Research Collaboration, Department of Social Medicine, University of Bristol, Bristol BS8 2PR
- Correspondence to: J A C Sterne
This is the second in a series of four articles
Studies that show a significant effect of treatment are more likely to be published, be published in English, be cited by other authors, and produce multiple publications than other studies.1–8 Such studies are therefore also more likely to be identified and included in systematic reviews, which may introduce bias.9 Low methodological quality of studies included in a systematic review is another important source of bias.10
All these biases are more likely to affect small studies than large ones. The smaller a study the larger the treatment effect necessary for the results to be significant. The greater investment of time and money in larger studies means that they are more likely to be of high methodological quality and published even if their results are negative. Bias in a systematic review may therefore become evident through an association between the size of the treatment effect and study size—such associations may be examined both graphically and statistically.
Summary points
Asymmetrical funnel plots may indicate publication bias or be due to exaggeration of treatment effects in small studies of low quality
Bias is not the only explanation for funnel plot asymmetry; funnel plots should be seen as a means of examining “small study effects” (the tendency for the smaller studies in a meta-analysis to show larger treatment effects) rather than a tool for diagnosing specific types of bias
Statistical methods may be used to examine the evidence for bias and to examine the robustness of the conclusions of the meta-analysis in sensitivity analyses
“Correction” of treatment effect estimates for bias should be avoided as such corrections may depend heavily on the assumptions made
Multivariable models may be used, with caution, to examine the relative importance of different types of bias
Graphical methods for detecting bias
Funnel plots
Funnel …
Log in
Log in using your username and password
Log in through your institution
Subscribe from £184 *
Subscribe and get access to all BMJ articles, and much more.
* For online subscription
Access this article for 1 day for:
£50 / $60/ €56 (excludes VAT)
You can download a PDF version for your personal record.