Investigating and dealing with publication and other biases in meta-analysis

Search all BMJ education articles:

From

Limit by

Access to the full text of this article requires a subscription or payment. Please log in or subscribe below.

Jonathan A C Sterne (jonathan.sterne@bristol.ac.uk), senior lecturer in medical statistics,
Matthias Egger, senior lecturer in epidemiology and public health medicine,
George Davey Smith, professor of clinical epidemiology

Medical Research Council Health Services Research Collaboration, Department of Social Medicine, University of Bristol, Bristol BS8 2PR

Correspondence to: J A C Sterne

This is the second in a series of four articles

Studies that show a significant effect of treatment are more likely to be published, be published in English, be cited by other authors, and produce multiple publications than other studies.1^–8 Such studies are therefore also more likely to be identified and included in systematic reviews, which may introduce bias.9 Low methodological quality of studies included in a systematic review is another important source of bias.10

All these biases are more likely to affect small studies than large ones. The smaller a study the larger the treatment effect necessary for the results to be significant. The greater investment of time and money in larger studies means that they are more likely to be of high methodological quality and published even if their results are negative. Bias in a systematic review may therefore become evident through an association between the size of the treatment effect and study size—such associations may be examined both graphically and statistically.

Summary points

Asymmetrical funnel plots may indicate publication bias or be due to exaggeration of treatment effects in small studies of low quality

Bias is not the only explanation for funnel plot asymmetry; funnel plots should be seen as a means of examining “small study effects” (the tendency for the smaller studies in a meta-analysis to show larger treatment effects) rather than a tool for diagnosing specific types of bias

Statistical methods may be used to examine the evidence for bias and to examine the robustness of the conclusions of the meta-analysis in sensitivity analyses

“Correction” of treatment effect estimates for bias should be avoided as such corrections may depend heavily on the assumptions made

Multivariable models may be used, with caution, to examine the relative importance of different types of bias

Graphical methods for detecting bias

Funnel plots

Funnel …

Access to the full text of this article requires a subscription or payment

Article access

Article access for 1 day

Purchase this article for £20 $30 €32*

The PDF version can be downloaded as your personal record

* Prices do not include VAT

Education and debate

Systematic reviews in health care