GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: IntroductionBMJ 2016; 353 doi: https://doi.org/10.1136/bmj.i2016 (Published 28 June 2016) Cite this as: BMJ 2016;353:i2016
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Re: GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction
We thank Dr. Messori and colleagues for their interest in our recent publications about the Evidence to Decision (EtD) frameworks (1-3). As in their previous comment in PubMed Commons, to which we responded (http://www.ncbi.nlm.nih.gov/pubmed/27353417#comments), they question the real world application of these and other frameworks, as well as of GRADE methods in general.
As Dr. Messori and colleagues note, GRADE methods have been widely used not only in systematic reviews and clinical practice guidelines, but also in health technology assessments; specific frameworks have been developed also for coverage and health system decisions. Over 100 organisations use GRADE, including the Cochrane Collaboration, the World Health Organization and the National Institute for Health and Care Excellence. We believe this indicates that GRADE methods are well accepted and widely used.
Although the search strategy used by Dr. Messori and colleagues probably would benefit from a more exhaustive and structured development, we agree with Dr. Messori and colleagues that more publications about practical experience using GRADE, and particularly the GRADE EtD frameworks for coverage and health system and public health decisions, are needed. We encourage them and others to further evaluate GRADE and other methods for making judgements about the certainty of evidence and for going from evidence to decisions. The GRADE Working Group is continuing to develop these methods and welcomes empirical testing of them and suggestions for improvements.
1. Alonso-Coello P, Schunemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016.
2. Alonso-Coello P, Oxman AD, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ. 2016;353:i2089.
3. Schünemann HJ, Mustafa R, Brozek J, Santesso N, Alonso-Coello P, Guyatt G, et al; GRADE Working Group. GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health. J Clin Epidemiol. 2016 Feb 27. pii: S0895-4356(16)00136-0. doi: 10.1016/j.jclinepi.2016.01.032. [Epub ahead of print]
4. Messori A, Trippoli S, Marinai C. Structured methods of analysis and reporting: how many papers have reported the results of their application? BMJ 2016 (Rapid reply) http://www.bmj.com/content/353/bmj.i2016/rr
Competing interests: No competing interests
Structured methods of analysis and reporting: how many papers have reported the results of their application?
In the evaluation of therapeutic interventions, structured methods of analysis and reporting are increasingly being proposed, particularly in the field of oncology. In 2015, a task force of oncologists coordinated by the American Society of Clinical Oncology (ASCO) developed a complex consensus document (1,2) that examines - - through a fully structured approach - - the main determinants of effectiveness and of cost for current anti-cancer treatments. In the same year another project with similar characteristics was carried out by the European Society for Medical Oncology (ESMO) with a more specific focus on the magnitude of clinical benefits and no examination of costs (3). The GRADE method (4-6), firstly described in 2004, remains, however, the most widely recognised structured tool to devise therapeutic guidelines for any type of health-care intervention and to generate recommendations directed at clinicians or institutions. The recent article by Alonso-Coello et al (7) is of great interest because this paper tries to extend the application of the GRADE method and starts to cover also the field of decision-making.
In this framework, however, one reason of concern is suggested by an overview of the literature on this topic focused on a specific point of controversy (8). This overview showed that most applications of the GRADE method refer to studies conducted by academic researchers in a scientific context (typically, a systematic review or a meta-analysis undertaken for purely scientific purposes) whereas the use of this method is still limited to the level of decision-makers of health-care systems (8). As regards the experience conducted by the most authoritative institution that has adopted the GRADE method (i.e. WHO), a recent paper by Alexander and colleagues (9) has pointed out that many recommendations issued by WHO were not compliant with the criteria set by GRADE. One reason for this negative result might be that WHO panelists had a poor familiarity with GRADE (9). Even more interestingly, if one considers the ESMO tool or the ASCO method, virtually all of the published papers dealt with the description of the method, but no real-life applications of these tools have yet been reported in the literature (Messori and Trippoli, 2016, unpublished observations).
Alonso-Coello and colleagues (7) have pointed out that “more than 100 organisations globally, including the World Health Organization, the Cochrane Collaboration, and the National Institute for Health and Care Excellence (NICE) now use or have adopted the principles of the GRADE system”. However, despite the apparently large acceptance of this method by clinicians or institutions, the current literature only reports a limited number of experiences in which this method has been employed to make decisions in the context of a health-care system.
1. Schnipper LE, Davidson NE, Wollins DS, et al: American Society of Clinical Oncology statement: A conceptual framework to assess the value of cancer treatment options. J Clin Oncol 2015;33:2563-2577.
2. Schnipper LE, Davidson NE, Wollins DS, et al: Updating the ASCO value framework: Revisions and reflections in response to comments received. J Clin Oncol 2016, doi: 10.1200/JCO.2016.68.251
3. Cherny NI, Sullivan R, Dafni U, et al: A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: The European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann Oncol 2015;26:1547-1573.
4. Atkins D, Best D, Briss PA, et al. GRADE Working Group. Grading quality of evidence and strength of recommendations.BMJ 2004;328:1490.
5. Guyatt GH, Oxman AD, Vist GE, et al. GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ2008;336:924-
6. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol2011;64:383-94.
7. Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, Treweek S, Mustafa RA, Rada G, Rosenbaum S, Morelli A, Guyatt GH, Oxman AD; GRADE Working Group. GRADE Evidence to Decision (EtD) frameworks: asystematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016 Jun 28;353:i2016.
8. Trippoli S, Messori A. Structured methods of analysis and reporting: a quick literature analysis (Comment), PubMed Commons, published 30 June 2016. http://www.ncbi.nlm.nih.gov/pubmed/27353417#cm27353417_16441 Accessed 30 June 2016.
9. Alexander PE, Gionfriddo MR, Li SA, Bero L, Stoltzfus RJ, Neumann I, Brito JP,Djulbegovic B, Montori VM, Norris SL, Schünemann HJ, Thabane L, Guyatt GH. A number of factors explain why WHO guideline developers make strong recommendations inconsistent with GRADE guidance. J Clin Epidemiol. 2016 Feb;70:111-22.
Address correspondence to: Dr.Andrea Messori, HTA Unit, ESTAR, Regional Health Service, 50100 Firenze (Italy) Email: firstname.lastname@example.org
Competing interests: No competing interests