Intended for healthcare professionals

Research Methods & Reporting

Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients

BMJ 2015; 350 doi: https://doi.org/10.1136/bmj.h870 (Published 16 March 2015) Cite this as: BMJ 2015;350:h870
  1. Alfonso Iorio, associate professor of knowledge translation and medicine12,
  2. Frederick A Spencer, professor of medicine2,
  3. Maicon Falavigna, clinical epidemiologist3,
  4. Carolina Alba, clinical fellow4,
  5. Eddie Lang, academic and clinical department head5,
  6. Bernard Burnand, professor of clinical epidemiology and healthcare evaluation6,
  7. Tom McGinn, professor and chair of medicine7,
  8. Jill Hayden, associate professor8,
  9. Katrina Williams, professor of developmental medicine9,
  10. Beverly Shea, senior methodologist and clinical scientist1011,
  11. Robert Wolff, reviews manager12,
  12. Ton Kujpers, clinical epidemiologist13,
  13. Pablo Perel, clinical senior lecturer14,
  14. Per Olav Vandvik, associate professor15,
  15. Paul Glasziou, professor of evidence-based medicine16,
  16. Holger Schunemann, professor of clinical epidemiology and medicine12,
  17. Gordon Guyatt, professor of clinical epidemiology and biostatistics12
  1. 1Clinical Epidemiology and Biostatistics Department, McMaster University, Hamilton, ON, Canada
  2. 2Department of Medicine, McMaster University, Hamilton, ON, Canada
  3. 3National Institute of Health Technology Assessment, Federal University of Rio Grande do Sul Hospital Moinhos de Vento, Porto Alegre, Brazil
  4. 4Heart Failure and Transplant Program, Toronto General Hospital, University Health Network, Toronto, ON, Canada
  5. 5University of Calgary, Department of Emergency Medicine Alberta Health Services, Calgary Zone, AB, Canada
  6. 6Institute of Social and Preventive Medicine, Lausanne University Hospital, Lausanne, Switzerland
  7. 7North Shore-LIJ Health System, Hofstra North Shore-LIJ Medical School, Hempstead, NY, USA
  8. 8Department of Community Health and Epidemiology, Dalhousie University, Halifax, NS, Canada
  9. 9Department of Paediatrics, University of Melbourne; Developmental Medicine, Royal Children’s Hospital; Murdoch Childrens Research Institute, Melbourne, Australia
  10. 10Centre for Practice-Changing Research, Ottawa Hospital Research Institute, University of Ottawa, and Bruyère Research Institute, Ottawa, ON, Canada
  11. 11Institute for Clinical Evaluative Sciences, Toronto, ON, Canada
  12. 12Kleijnen Systematic Reviews Ltd, York, UK
  13. 13Department for Guideline Development, Dutch College of General Practitioners, Utrecht, Netherlands
  14. 14Department of Population Health, London School of Hygiene and Tropical Medicine, London, UK
  15. 15Department of Medicine, Innlandet Hospital Trust, Division Gjøvik, Norway
  16. 16Centre for Research in Evidence-Based Practice, Faculty of Health Sciences, Bond University, Gold Coast, Australia
  1. Correspondence to: A Iorio iorioa{at}mcmaster.ca
  • Accepted 12 December 2014

Summary points

Main concepts
  • The Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) approach defines quality of evidence as confidence in effect estimates; this conceptualization can readily be applied to bodies of evidence estimating the risk of future of events (that is, prognosis) in broadly defined populations

  • In the field of prognosis, a body of observational evidence (including single arms of randomized controlled trials) begins as high quality evidence

  • The five domains GRADE considers in rating down confidence in estimates of treatment effect—that is, risk of bias, imprecision, inconsistency, indirectness, and publication bias—as well as the GRADE criteria for rating up quality, also apply to estimates of the risk of future of events from a body of prognostic studies

  • Applying these concepts to systematic reviews of prognostic studies provides a useful approach to determine confidence in estimates of overall prognosis in broad populations

Lay summary
  • The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach to rating confidence in the results of research studies was initially developed for therapeutic questions

  • The GRADE approach considers study design (randomized trials versus non-randomized designs), risk of bias, inconsistency, imprecision, indirectness, and publication bias; size and trend in the effect are also considered

  • Observational studies looking at patients’ prognosis may provide robust estimates of the likelihood of undesirable or desirable outcomes in both treated and untreated patients

  • Patients will often find this information helpful in understanding the likely course of their disease, in planning their future, and in engaging in shared decision making with their healthcare providers

  • In a previous article, we examined factors that affect confidence in estimates of baseline risk (the risk of bad outcomes in untreated patients), providing examples of how this might influence the confidence in estimates of absolute treatment effect

  • This paper provides guidance for the use of the GRADE approach to determine confidence in …

View Full Text