GRADE approach to rate the certainty from a network meta-analysis: addressing incoherence

Romina Brignardello-Petersen; Reem A Mustafa; Reed A C Siemieniuk; M Hassan Murad; Thomas Agoritsas; Ariel Izcovich; Holger J Schünemann; Gordon H Guyatt; GRADE Working Group

doi:10.1016/j.jclinepi.2018.11.025

GRADE approach to rate the certainty from a network meta-analysis: addressing incoherence

J Clin Epidemiol. 2019 Apr:108:77-85. doi: 10.1016/j.jclinepi.2018.11.025. Epub 2018 Dec 5.

Authors

Romina Brignardello-Petersen¹, Reem A Mustafa², Reed A C Siemieniuk³, M Hassan Murad⁴, Thomas Agoritsas⁵, Ariel Izcovich⁶, Holger J Schünemann³, Gordon H Guyatt³; GRADE Working Group

Affiliations

¹ Department of Health Research Methods, Evidence and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4L8, Canada. Electronic address: brignarr@mcmaster.ca.
² Department of Health Research Methods, Evidence and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4L8, Canada; Division of Nephrology and Hypertension, Department of Medicine, University of Kansas Medical Center, 3901 Rainbow Blvd, MS3002, Kansas City, KS 66160, USA.
³ Department of Health Research Methods, Evidence and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4L8, Canada.
⁴ Mayo Clinic Evidence Based Practice Center, 200 1st Street SW, Rochester, MN 55905, USA.
⁵ Department of Health Research Methods, Evidence and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4L8, Canada; Division of General Internal Medicine and Division of Clinical Epidemiology, University Hospitals of Geneva, Switzerland, Rue Gabrielle-Perret-Gentil 4, 1211 Genève, Switzerland.
⁶ Internal Medicine Service, German Hospital, Pueyrredón 1640, Buenos Aires C1118AAT, Argentina.

PMID: 30529648
DOI: 10.1016/j.jclinepi.2018.11.025

Abstract

This article presents official guidance from the Grading of Recommendations Assessments, Development, and Evaluation (GRADE) working group on how to address incoherence when assessing the certainty in the evidence from network meta-analysis. Incoherence represents important differences between direct and indirect estimates that contribute to a network estimate. Bias due to limitations in study design or publication bias, indirectness, and intransitivity may be responsible for incoherence. Addressing incoherence requires a judgment regarding the importance of the impact on the network estimate. Reviewers need to be alert to the possibility of misguidedly arriving at excessively low ratings of certainty by rating down for both incoherence and other closely related GRADE domains. This article describes and illustrates each of these issues and provides explicit guidance on how to deal with them.

Keywords: Certainty in the evidence; GRADE; Incoherence; Inconsistency; Network meta-analysis; Quality of the evidence; Systematic reviews.

MeSH terms

Data Interpretation, Statistical
Evidence-Based Medicine / methods*
Evidence-Based Medicine / standards
GRADE Approach / methods*
Humans
Network Meta-Analysis*
Publication Bias
Systematic Reviews as Topic