Correcting a common misunderstanding regarding ESM0-MCBS grading
In their recent paper in the BMJ (1) Davis et al. have used the ESMO-MCBS to evaluate the level of clinical benefit of anticancer agents licensed between 2009-2013. While we laud them on this project and agree with the general conclusion that many treatments with very limited evidence for benefit are licensed, part of their analysis is based on a flawed understanding of the ESMO-MCBS scale.
The statement in the methods section: “Only scores of A or B (for treatments of curative intent), or 5 or 4 (for treatments used in the non-curative/palliative setting) are defined as clinically meaningful according to the ESMO framework.” is not an accurate representation. Indeed, it perpetuates a common misunderstanding of the ESMO-MCBS grading. While the ESM0-MCBS does distinguish high benefit from low benefit studies, it does not set a threshold for “clinical meaningfulness” (2-4).
While scores of A or B (for treatments of curative intent), or 5 or 4 (for treatments used in the non-curative/palliative setting) indicate “a high level of proven clinical benefit” or “substantial benefit”, this does not preclude that studies achieving slightly lower scores, such as a grade of 3 in the non-curative/palliative setting, may also provide clinically meaningful benefit. Indeed, a study correlating the decisions of the Israeli HTA body to ESMO-MCBS scoring found that in the non-curative setting, most medications with a score of >3 were approved for reimbursement whereas those with a score of <3 were very rarely approved (5). This experience would indicate that in a high income country, scores of >3 were usually judged by an experienced HTA body to be clinically meaningful, and worthy of reimbursement in a resource limited environment (5).
Accurate interpretation of the results generated by the ESMO-MCBS require methodological diligence in data acquisition, careful application of the tool, and a clear understanding of the meaning (and acknowledged limitations) of the grades generated (2-4).
1. Davis C, Naci H, Gurpinar E, Poplavska E, Pinto A, Aggarwal A. Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13. BMJ 2017;359:j4530 http://www.bmj.com/content/359/bmj.j.
2. Cherny N, Dafni U, Bogaerts J, Latino N, Piccart M, Pentheroudakis G, et al. ESMO-Magnitude of Clinical Benefit Scale Version 1.1. Ann Oncol 2017:mdx310, https://doi.org/10.1093/annonc/mdx310.
3. Cherny NI, Sullivan R, Dafni U, Kerst JM, Sobrero A, Zielinski C, et al. A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann Oncol 2015;26:1547-73.
4. Dafni U, Karlis D, Pedeli X, Bogaerts J, Pentheroudakis G, Tabernero J, et al. Detailed statistical assessment of the characteristics of the ESMO Magnitude of Clinical Benefit Scale (ESMO-MCBS) threshold rules. ESMO Open. 2017;2(4). e000216; DOI: 10.1136/esmoopen-2017-000216
5. Hammerman A, Greenberg-Dotan S, Feldhamer I, Birnbaum Y, Cherny NI. The ESMO-Magnitude of Clinical Benefit Scale for novel oncology drugs: correspondence with three years of reimbursement decisions in Israel. Exp Rev Pharmacoecon Outcomes Res 2017:1-4. doi: 10.1080/14737167.2017.1343146
Competing interests: No competing interests