Uncertainty in heterogeneity estimates in meta-analyses
BMJ 2007; 335 doi: https://doi.org/10.1136/bmj.39343.408449.80 (Published 01 November 2007) Cite this as: BMJ 2007;335:914
- John P A Ioannidis, professor,
- Nikolaos A Patsopoulos, research associate,
- Evangelos Evangelou, research associate
- Clinical Trials and Evidence-Based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece
- Correspondence to: J P A Ioannidis jioannid{at}cc.uoi.gr
- Accepted 29 August 2007
Summary points
The extent of between study heterogeneity should be measured when interpreting results of meta-analyses
Meta-analyses rarely document uncertainty in estimates of heterogeneity
Our evaluation of a large number of meta-analyses shows a wide range of uncertainty about the extent of heterogeneity in most
Confidence intervals of I2 should be calculated and considered when interpreting meta-analyses
An important aim of systematic reviews and meta-analyses is to assess the extent to which different studies give similar or dissimilar results.1 Clinical, methodological, and biological heterogeneity are often topic specific, but statistical heterogeneity can be examined with the same methods in all meta-analyses. Therefore, the perception of statistical heterogeneity or homogeneity often influences meta-analysts and clinicians in important decisions. These decisions include whether the data are similar enough to combine different studies; whether a treatment is applicable to all or should be “individualised” because of variable benefits or harms in different types of patients; and whether a risk factor affects all people exposed or only select populations. How uncertain is the extent of statistical heterogeneity in meta-analyses? Moreover, is this uncertainty properly factored in when interpreting the results?
Evaluating heterogeneity between studies
Many statistical tests are available for evaluating heterogeneity between studies.2 3 Until recently, the most popular was Cochran's Q, a statistic based on the χ2 test.4 Cochran's Q usually has only low power to detect heterogeneity, however. It also depends on the number of studies and cannot be compared across different meta-analyses.2 3 Higgins and colleagues, in two highly cited papers,5 6 proposed the routine use of the I2 statistic. I2 is calculated as …
Log in
Log in using your username and password
Log in through your institution
Subscribe from £184 *
Subscribe and get access to all BMJ articles, and much more.
* For online subscription
Access this article for 1 day for:
£50 / $60/ €56 (excludes VAT)
You can download a PDF version for your personal record.