Intended for healthcare professionals

Research Methods & Reporting

A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis

BMJ 2014; 349 doi: (Published 24 September 2014) Cite this as: BMJ 2014;349:g5630

This article has a correction. Please see:

  1. Milo A Puhan1,
  2. Holger J Schünemann2,
  3. Mohammad Hassan Murad3,
  4. Tianjing Li4,
  5. Romina Brignardello-Petersen5,
  6. Jasvinder A Singh6,
  7. Alfons G Kessels7,
  8. Gordon H Guyatt2
  9. for the GRADE Working Group
  1. 1Epidemiology, Biostatistics and Prevention Institute–Epidemiology, Hirschengraben 84, Zurich 8001, Switzerland
  2. 2Department of Clinical Epidemiology and Biostatistics, McMaster University, Health Sciences Centre, Hamilton, Ontario L8N 3Z5, Canada
  3. 3Mayo Clinic–Preventive Medicine, Rochester MN, Minnesota 55905, USA
  4. 4Johns Hopkins Bloomberg School of Public Health–Epidemiology, Baltimore, Maryland, USA
  5. 5University of Toronto–Clinical Epidemiology and Health Care Research, Toronto, Ontario, Canada
  6. 6University of Alabama–Clinical Immunology and Rheumatology, Birmingham, Alabama, USA
  7. 7University of Maastricht, Maastricht, Netherlands
  1. Correspondence to: M A Puhan miloalan.puhan{at}
  • Accepted 22 August 2014

Network meta-analysis (NMA), combining direct and indirect comparisons, is increasingly being used to examine the comparative effectiveness of medical interventions. Minimal guidance exists on how to rate the quality of evidence supporting treatment effect estimates obtained from NMA. We present a four-step approach to rate the quality of evidence in each of the direct, indirect, and NMA estimates based on methods developed by the GRADE working group. Using an example of a published NMA, we show that the quality of evidence supporting NMA estimates varies from high to very low across comparisons, and that quality ratings given to a whole network are uninformative and likely to mislead.

Network meta-analysis (NMA) that simultaneously addresses the comparative effectiveness and/or safety of multiple interventions through combining direct and indirect estimates of effect is rapidly gaining popularity and influence.1 2 3 4 5 6 Although NMA approaches appear attractive,6 7 8 application of their results requires understanding the quality of the evidence. By quality of evidence, we mean the degree of confidence or certainty one can place in estimates of treatment effects.

NMA is sufficiently new that terminology differs between authors and continues to evolve. Box 1 presents a glossary of terms used in this article.

Box 1: Glossary of terms (in order they appear in the text)

  • Ranking—Ordering of treatments according to their relative effectiveness. The first ranked treatment is most likely to be the most effective treatment with respect to a particular outcome compared with the other treatments in the network

  • Direct estimates—Estimate of effect provided by a head-to-head comparison (such as trials of A versus B when A v B is the comparison of interest)

  • Indirect estimates—Estimate of effect provided by two or more head-to-head comparisons that share a common comparator (such as trials of A v C and trials of B v C when A v B …

View Full Text