Going from evidence to recommendations
BMJ 2008; 336 doi: https://doi.org/10.1136/bmj.39493.646875.AE (Published 08 May 2008) Cite this as: BMJ 2008;336:1049
All rapid responses
The GRADE system (like other systems) for ranking the strength of a recommendation applies only to recommendations that are (i) simple (in the sense of not having multiple component subrecommendations), (ii) unambiguous, and (iii) directly reflect the balancing of pros and cons and uncertainties for a recommended action.
In my experience as a daily user of guidelines produced by many different organisations, many recommendations do not meet these constraints.
An example of a recommendation that cannot be GRADEd is:
=======================================================
If paracetamol or topical NSAIDs are insufficient at relieving pain, consider adding:
– opioid analgesics (consider the risks and benefits of prescribing opioids, particularly in elderly people)
– an oral NSAID/COX-2 inhibitor (see box 1) to the paracetamol.
--------------------------------------------------
Box 1 Treatment with oral NSAIDs/COX-2 inhibitors
– Offer a standard NSAID or a COX-2 inhibitor (but not etoricoxib 60 mg) as a first choice.
– Co-prescribe with a proton pump inhibitor (choose the agent with the lowest acquisition cost).
– Prescribe at the lowest effective dose for the shortest possible period of time.
– Owing to potential gastrointestinal, liver and cardio-renal toxicity:
o take into account individual patient risk factors, including age, when choosing the NSAID/COX-2 inhibitor and dose to be prescribed
o assess and/or monitor patient risk factors
o consider prescribing an alternative analgesic if the patient is already taking low-dose aspirin for another condition.
From the NICE guideline on Osteoarthritis [1]
=======================================================
This recommendation cannot be meaningfully GRADEd. For example, it contains:
(i) An incomplete list of options — additional options would be to add a topical NSAID to paracetamol, or paracetamol to a topical NSAID.
(ii) Several sub-recommendations at several levels.
(iii) Several hedges — “consider adding”; “consider the risks and benefits”; and in box 1: “take into account individual patient risk factors, including age”, “consider prescribing an alternative analgesic”. Hedging is a form of grading, and thus it does not make sense to grade it.
(iv) Unclear advice, e.g. “ Prescribe at the lowest effective dose for the shortest possible period of time”. But, no advice is given on how to find out what the “lowest effective dose” or “shortest possible time” is.
The GRADE system should, either include a caveat for guideline authors and users explaining which recommendations are not suitable for GRADEing. Or, preferably, the GRADE guidance should change the word “recommendation” to “conclusion”, and explain that the ranking of the conclusion refers to the weighing up of all the pros and cons (benefits, harms, costs, values, practicalities) and their uncertainties.
To ensure that the basis for guideline recommendations is transparent, guideline authors need a structured checklist analogous to the CONSORT [2] (and similar) checklists for research reports. This gap is not met by the AGREE guideline assessment instrument [3].
REFERENCES
[1] NICE guideline on Osteoarthritis: www.nice.org.uk/nicemedia/pdf/CG59NICEguideline.pdf
[2] CONSORT statement: www.consort-statement.org
[3] AGREE guideline assessment instrument: www.agreecollaboration.org
Competing interests: None declared
Competing interests: No competing interests
Is this 12 years old BMJ article outdated?
Dear Editor
Maybe this 12 years old BMJ article [1] is not outdated because it is still one of the first downloadable from the website (https://www.gradeworkinggroup.org/) of the inventors of GRADE (Grading of Recommendations Assessment, Development and Evaluation) who authored this BMJ article. It is also obvious that these definitions copied below from this article are quite clever: “GRADE classifies recommendations as strong or weak:
1.Strong recommendations mean that most informed patients would choose the recommended management and that clinicians can structure their interactions with patients accordingly,
2.Weak recommendations mean that patients’ choices will vary according to their values and preferences, and clinicians must ensure that patients’ care is in keeping with their values and preferences”.
In more recent articles, GRADE tended to replace the adjective “weak” by “conditional”, which is also clever in my view.
On the other hand, maybe this 12 years old BMJ article [1] is outdated because in GRADE leading members' recent real life, the words “most informed patients” may apparently not mean (as I have always believed they meant, because I tended to believe in the wisdom of the inventors of GRADE) “more than 95% of the informed patients at the very least”. For example, in quite recent guidelines endorsed by GRADE, it is stated: “for asymptomatic women aged 50 to 69 with an average risk of breast cancer, mammography screening is recommended in the context of an organised screening programme (strong recommendation, moderate certainty in the evidence)” [2, 3]. The authors of this recommendation also report “possibly important uncertainty about or variability in how much people value the main outcomes” [2, 3]. In patients' real life, this uncertainty or variability is shown by the fact that participation rates among women who are invited to take part in organized screening programs rarely (and barely) exceed 70%.
More recently leading members of GRADE insisted, again, that these recent breast-cancer guidelines strictly stick to the GRADE principles and methods [3]. Therefore maybe it is now time for the inventors of GRADE to update their 12 years old BMJ article, and clarify that “most informed patients” means in fact something like “more than 50% of the informed patients” [1]? (wich would be quite hazardous in my view)
References:
[1] Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, Schünemann HJ, GRADE working group. Going from evidence to recommendations. BMJ 2008;336:1049. https://www.bmj.com/content/336/7652/1049
[2] Schünemann HJ, Lerda D, Dimitrova N, et al; European Commission Initiative on Breast Cancer Contributor Group. Methods for development of the European Commission Initiative on Breast Cancer guidelines: recommendations in the era of guideline transparency. Ann Intern Med 2019;171:273-80. https://www.acpjournals.org/doi/10.7326/M18-3445
[3] Schünemann HJ, Alonso-Coello P, Gräwingholt A, Quinn C, Follmann M, Langendam M, Saz-Parkinson Z. Development of the European Commission Initiative on Breast Cancer Guidelines. Ann Intern Med 2020; 172: 72-73. https://www.acpjournals.org/doi/10.7326/L19-0621
Competing interests: No competing interests