Intended for healthcare professionals

  1. Research
  2. Recommendations for...
  3. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials
Research Methods & Reporting

Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials

BMJ 2011; 343 doi: https://doi.org/10.1136/bmj.d4002 (Published 22 July 2011) Cite this as: BMJ 2011;343:d4002
  1. Jonathan A C Sterne, professor1,
  2. Alex J Sutton, professor2,
  3. John P A Ioannidis, professor and director3,
  4. Norma Terrin, associate professor4,
  5. David R Jones, professor2,
  6. Joseph Lau, professor4,
  7. James Carpenter, reader5,
  8. Gerta Rücker, research assistant6,
  9. Roger M Harbord, research associate1,
  10. Christopher H Schmid, professor4,
  11. Jennifer Tetzlaff, research coordinator7,
  12. Jonathan J Deeks, professor8,
  13. Jaime Peters, research fellow9,
  14. Petra Macaskill, associate professor10,
  15. Guido Schwarzer, research assistant6,
  16. Sue Duval, assistant professor11,
  17. Douglas G Altman, professor12,
  18. David Moher, senior scientist7,
  19. Julian P T Higgins, senior statistician13
  1. 1School of Social and Community Medicine, University of Bristol, Bristol BS8 2PS, UK
  2. 2Department of Health Sciences, University of Leicester, Leicester, UK
  3. 3Stanford Prevention Research Center, Stanford University School of Medicine, Stanford, CA, USA
  4. 4Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, Boston, MA, USA
  5. 5Medical Statistics Unit, London School of Hygiene and Tropical Medicine, London, UK
  6. 6Institute of Medical Biometry and Medical Informatics, University Medical Center Freiburg, Germany
  7. 7Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
  8. 8School of Health and Population Sciences, University of Birmingham, Birmingham, UK
  9. 9Peninsula Medical School, University of Exeter, Exeter, UK
  10. 10School of Public Health, University of Sydney, NSW, Australia
  11. 11University of Minnesota School of Public Health, Minneapolis, MN, USA
  12. 12Centre for Statistics in Medicine, University of Oxford, Oxford, UK
  13. 13MRC Biostatistics Unit, Cambridge, UK
  1. Correspondence to: J A C Sterne jonathan.sterne{at}bristol.ac.uk
  • Accepted 21 February 2011

Funnel plots, and tests for funnel plot asymmetry, have been widely used to examine bias in the results of meta-analyses. Funnel plot asymmetry should not be equated with publication bias, because it has a number of other possible causes. This article describes how to interpret funnel plot asymmetry, recommends appropriate tests, and explains the implications for choice of meta-analysis model

The 1997 paper describing the test for funnel plot asymmetry proposed by Egger et al 1 is one of the most cited articles in the history of BMJ.1 Despite the recommendations contained in this and subsequent papers,2 3 funnel plot asymmetry is often, wrongly, equated with publication or other reporting biases. The use and appropriate interpretation of funnel plots and tests for funnel plot asymmetry have been controversial because of questions about statistical validity,4 disputes over appropriate interpretation,3 5 6 and low power of the tests.2

This article recommends how to examine and interpret funnel plot asymmetry (also known as small study effects2) in meta-analyses of randomised controlled trials. The recommendations are based on a detailed MEDLINE review of literature published up to 2007 and discussions among methodologists, who extended and adapted guidance previously summarised in the Cochrane Handbook for Systematic Reviews of Interventions.7

What is a funnel plot?

A funnel plot is a scatter plot of the effect estimates from individual studies against some measure of each study’s size or precision. The standard error of the effect estimate is often chosen as the measure of study size and plotted on the vertical axis8 with a reversed scale that places the larger, most powerful studies towards the top. The effect estimates from smaller studies should scatter more widely at the bottom, with the spread narrowing among larger studies.9 In the absence of bias and between …

View Full Text
Back to top