The GRADE Working Group clarifies the construct of certainty of evidence

Monica Hultcrantz; David Rind; Elie A Akl; Shaun Treweek; Reem A Mustafa; Alfonso Iorio; Brian S Alper; Joerg J Meerpohl; M Hassan Murad; Mohammed T Ansari; Srinivasa Vittal Katikireddi; Pernilla Östlund; Sofia Tranæus; Robin Christensen; Gerald Gartlehner; Jan Brozek; Ariel Izcovich; Holger Schünemann; Gordon Guyatt

doi:10.1016/j.jclinepi.2017.05.006

The GRADE Working Group clarifies the construct of certainty of evidence

J Clin Epidemiol. 2017 Jul:87:4-13. doi: 10.1016/j.jclinepi.2017.05.006. Epub 2017 May 18.

Authors

Monica Hultcrantz¹, David Rind², Elie A Akl³, Shaun Treweek⁴, Reem A Mustafa⁵, Alfonso Iorio⁶, Brian S Alper⁷, Joerg J Meerpohl⁸, M Hassan Murad⁹, Mohammed T Ansari¹⁰, Srinivasa Vittal Katikireddi¹¹, Pernilla Östlund¹², Sofia Tranæus¹³, Robin Christensen¹⁴, Gerald Gartlehner¹⁵, Jan Brozek⁶, Ariel Izcovich¹⁶, Holger Schünemann⁶, Gordon Guyatt⁶

Affiliations

¹ Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU), S:t Eriksgatan 117, SE-102 33, Stockholm, Sweden; Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Tomtebodav. 18 A, SE-171 77, Stockholm, Sweden. Electronic address: monica.hultcrantz@sbu.se.
² Institute for Clinical and Economic Review, Two Liberty Square, 9(th) Floor, Boston, MA 02109, USA; Harvard Medical School Beth Israel Deaconess Medical Center, 330 Brookline Avenue Boston, MA 02215, USA.
³ Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada; Department of Internal Medicine, American University of Beirut Medical Center, P.O. Box 11-0236, Riad-El-Solh, Beirut 1107 2020, Lebanon.
⁴ Health Services Research Unit, University of Aberdeen, 3rd Floor, Health Sciences Building, Foresterhill, Aberdeen, AB25 2ZD, UK.
⁵ Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada; Department of Medicine, Division of Nephrology and Hypertension, University of Kansas Medical Center, 3901 Rainbow Blvd, MS3002, Kansas City, KS 66160, USA.
⁶ Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada; Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada.
⁷ EBSCO Health, EBSCO Publishing, 10 Estes Street, Ipswich, MA, 01938 USA; Department of Family and Community Medicine, School of Medicine, University of Missouri-Columbia, One Hospital Drive, Columbia, MO 65212, USA.
⁸ Cochrane Germany, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Breisacher Strasse 153,79110 Freiburg, Germany; Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité - U1153, Inserm / Université Paris Descartes; Cochrane France; Hôpital Hôtel-Dieu, 1 place du Parvis Notre Dame, 75181 Paris Cedex 04, France.
⁹ Mayo Clinic Evidence-based Practice Center, Mayo Clinic, 200 1st Street SW, Rochester, MN 55905, USA.
¹⁰ School of Epidemiology, Public Health and Preventive Medicine, Faculty of Medicine, University of Ottawa, 600 Peter Morand Crescent, Ottawa, Ontario K1G 5Z3, Canada.
¹¹ MRC/CSO Social & Public Health Sciences Unit, University of Glasgow, 200 Renfield Street, Glasgow, G2 3QB, UK.
¹² Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU), S:t Eriksgatan 117, SE-102 33, Stockholm, Sweden; Health Technology Assessment - Odontology (HTA-O), Faculty of Odontology, Malmö University, Carl Gustafs Väg 34, SE-214 21, Malmö, Sweden.
¹³ Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU), S:t Eriksgatan 117, SE-102 33, Stockholm, Sweden; Health Technology Assessment - Odontology (HTA-O), Faculty of Odontology, Malmö University, Carl Gustafs Väg 34, SE-214 21, Malmö, Sweden; Division of Cariology and Endodontology, Department of Dental Medicine, Karolinska institutet, Alfred Nobels Allé 8, SE-141 52, Huddinge, Sweden.
¹⁴ Musculoskeletal Statistics Unit, The Parker Institute, Bispebjerg and Frederiksberg Hospital, Nordre Fasanvej 57, DK-2000 Copenhagen F, Denmark.
¹⁵ Danube University, Department for Evidence-based Medicine and Clinical Epidemiology, Dr. Karl Dorrekstrasse 30, 3500 Krems, Austria; RTI International, Research Triangle Park, 3040 East Cornwallis Rd, Research Triangle Park 27709, NC, USA.
¹⁶ Internal Medicine Service, German Hospital, Pueyrredón 1640, Buenos Aires C1118AAT, Argentina.

Abstract

Objective: To clarify the grading of recommendations assessment, development and evaluation (GRADE) definition of certainty of evidence and suggest possible approaches to rating certainty of the evidence for systematic reviews, health technology assessments, and guidelines.

Study design and setting: This work was carried out by a project group within the GRADE Working Group, through brainstorming and iterative refinement of ideas, using input from workshops, presentations, and discussions at GRADE Working Group meetings to produce this document, which constitutes official GRADE guidance.

Results: Certainty of evidence is best considered as the certainty that a true effect lies on one side of a specified threshold or within a chosen range. We define possible approaches for choosing threshold or range. For guidelines, what we call a fully contextualized approach requires simultaneously considering all critical outcomes and their relative value. Less-contextualized approaches, more appropriate for systematic reviews and health technology assessments, include using specified ranges of magnitude of effect, for example, ranges of what we might consider no effect, trivial, small, moderate, or large effects.

Conclusion: It is desirable for systematic review authors, guideline panelists, and health technology assessors to specify the threshold or ranges they are using when rating the certainty in evidence.

Keywords: Certainty of evidence; GRADE; Guidelines; Health technology assessment; Systematic reviews; Thresholds.

The GRADE Working Group clarifies the construct of certainty of evidence

Authors

Affiliations

Abstract

Publication types

MeSH terms

Grants and funding