We appreciate the paper by Barbui and Purgato reflecting on the processes for selection of antidepressant and antipsychotic medicines for the World Health Organization’s Model List of Essential Medicines List (WHO Model EML) and welcome their suggestions on how these decision-making processes could be improved.1 Some of the issues raised reflect internal discussions at WHO on how to strengthen the process.
As noted by Barbui and Purgato, the reforms implemented in 2001 substantially changed the way the Expert Committee reviews submissions.2 A requirement for systematic evidence synthesis and appraisal was introduced, along with the assessment of comparative cost and cost-effectiveness. In addition, the WHO made its processes for receiving and reviewing applications transparent, with all submitted data and reviews publicly available and opportunities for interested parties to comment.
It is unclear why Barbui and Purgato are concerned that “WHO does not report information on potential conflicts of interests of applicants”. The process of submitting an application is open to all parties, including medicine manufacturers, and the applicant must be disclosed. Thus, those reviewing the application are aware of the potential conflicts of interest. While the current procedures do not prohibit applications from applicants with conflicts of interests, there are rigorous procedures in place to assess and manage any potential conflicts of interests of Committee members.
Barbui and Purgato raise three major issues regarding the EML processes: the quality of the evidence in the submissions considered by the Committee; the transparency of the decision-making process and the criteria used in making the judgments; and the discord between Expert Committee decisions and the recommendations of individual WHO Departments.
The authors suggest that the use of GRADE is new to the EML process. One of the earliest applications using GRADE assessments of the quality of evidence was for the addition of lamotrigine, submitted to the Committee in 2009.3 It has been requested that “where appropriate [this] evidence of comparative effectiveness and safety should be presented in tabular form using Grading of Recommendations, Assessment, Development and Evaluation (GRADE) tables”.4 This has been done to a variable extent in the submissions received, also depending on the nature of the application. However, a GRADE approach is not always appropriate, for example for evaluating pharmacokinetic data, as was done when assessing the doses and dosage forms needed for the first-line treatment of tuberculosis in children; or there may be existing high quality systematic reviews that have not applied this methodology.
Although in this case Barbui and Purgato argue for strengthening evidentiary requirements, others have complained that the “lengthy, exhaustive process for applying for a listing can be discouraging [as] each component requires a separate detailed, complex application.”5 A balance must be reached between the perfect and the good.
To date, Expert Committee decision-making has been reported in the Technical Report Series (TRS), a formal requirement of WHO Expert Committees. WHO has been discussing ways of improving the quality of this reporting and the proposed approach of a more structured template has merit.
Consistency between EML decisions and WHO clinical guidelines issued by other WHO departments is highly desirable. As noted by the authors, after standardizing the application format and review process of the EML in 2001, formal guideline development processes were adopted by WHO in 2007.6 Where it exists, high quality evidence generated for guideline development has also been used for EML applications (see for example the applications for antiretrovirals).7
Barbui and Purgato have also suggested that WHO should specify comparators and outcomes of interest. However, this would be a major change of focus and more in line with regulatory activities than with selecting medicines for an essential medicines list.
The accompanying commentary by Millard et al. raises issues about the quality of available evidence and the influence of “vested interests” and “expert opinion” on the decisions taken by the Expert Committee.8 The authors argue that these components had more influence than available evidence on the decision to include misoprostol on the EML for prevention of postpartum haemorrhage (PPH) where oxytocin is not available or cannot be safely used. However, misoprostol was added for this purpose to the list in 2011 after the publication of an additional well conducted placebo-controlled trial in a community setting by Mobeen and colleagues, showing relative risks for prevention of moderate and severe PPH of 0.76 (95% CI 0.59–0.97) and 0.57 (95% CI 0.27–1.22), respectively.9 The Expert Committee (and the Cochrane Reviews that were included in the application) assessed the risks of bias of all available evidence and came to a different conclusion compared to the review referenced by Millard et al.10
The way forward
More interaction between the WHO Secretariat and applicants can ensure that the minimum standards for applications are met; this requires adequate resources for both WHO and applicants. Alignment between WHO guidelines and the EML has been the basic principle of the selection process, and WHO is working towards achieving better harmonisation of guideline development and consideration by the Expert Committee.
Assessing cost-effectiveness at a global level presents many challenges, however meaningful ways to include cost considerations in the selection processes for the Model List are needed. The methods of WHO CHOICE (CHOosing Interventions that are Cost-Effective) are being used to facilitate country-level cost-effectiveness assessments and are being applied to topics such as trastuzumab for breast cancer.11
WHO has a role in supporting countries in the evidence-based selection of medicines for national EMLs and in the implementation of these lists to guide medicines procurement and medicines use in practice. The Model EML is a key innovation and achievement in WHO history.12 In 2012, more than 120 countries reporting having national EMLs; others have adapted their EML or used similar evidence-based decision-making to choose medicines for medicines reimbursement lists. The importance of the Model EML was recognised in this year’s World Health Assembly resolution on access to medicines.13
The selection processes for EML medicines are not perfect. Yet the process in place is largely transparent and tries to strike a fair balance between methodological rigour, timeliness and other critical considerations including public health need and equity of access.
Nicola Magrini
Jane Robertson
Kees de Joncheere
Essential Medicines List Secretariat,
Essential Medicines and Health Products Department,
World Health Organization, Geneva
Lisa Bero
Chair of WHO Expert Committees on the Selection and Use of Essential Medicines in 2011 and 2013
Medicines Use and Health Outcomes,
Faculty of Pharmacy and Charles Perkins Centre,
The University of Sydney
1. Barbui C, Purgato M. Decisions on WHO's essential medicines need more scrutiny. BMJ 2014;349:g4798. http://www.bmj.com/content/bmj/349/bmj.g4798.
2. World Health Organization. WHO medicines strategy. Revised procedure for updating WHO’s Model List of Essential Drugs. http://www.who.int/selection_medicines/committees/subcommittee/2/eeb1098...
3. Review of the available evidence on lamotrigine for epilepsy for the WHO Model List of Essential medicines (2009). http://www.who.int/selection_medicines/committees/expert/17/application/...
4. Essential Medicines List (EML) 2015 – Commented application. Information to be included in an application for inclusion, change or deletion of a medicine in the WHO Model List of Essential Medicines. http://www.who.int/selection_medicines/committees/expert/20/Commented_EM...
5. Klein HG. Should blood be an essential medicine? N Engl J Med 2013;368:199-201.
6. Hill S, Pang T. Leading by example: a culture change at WHO. Lancet 2007;369:1842-4.
7. 12th Expert Committee on the Selection and Use of Essential Medicines, 15-19 April 2002. Guide to the submissions for antiretroviral drugs. http://archives.who.int/eml/expcom/expcom12/arvs_submissions.htm
8. Millard C, Brhlikova P, Pollock AM. Commentary: Evidence versus influence in the WHO procedure for approving essential medicines: misoprostol for maternal health. BMJ 2014;349:g4823. http://www.bmj.com/content/349/bmj.g4823
9. Mobeen N, Durocher J, Zuberi NF, Jahan N, Blum J, Wasim S, et al. Administration of misoprostol by trained traditional birth attendants to prevent postpartum haemorrhage in homebirths in Pakistan: a randomised placebo-controlled trial. BJOG 2011;118:353-61.
10. Chu CS, Brhlikova P, Pollock AM. Rethinking WHO guidance: review of evidence for misoprostol use in the prevention of postpartum haemorrhage. J R Soc Med 2012;105:336-47.
11. World Health Organization. CHOosing Interventions that are Cost-Effective (WHO-CHOICE). http://www.who.int/choice/cost-effectiveness/en/
12. World Health Organization. Continuity and Change — Implementing the third WHO Medicines Strategy 2008-2013. WHO Policy Perspectives on Medicines 2010. http://apps.who.int/medicinedocs/documents/s16880e/s16880e.pdf
13. World Health Assembly 2014. Resolution WHA67.22 - Access to Essential Medicines. http://apps.who.int/medicinedocs/documents/s21453en/s21453en.pdf
Competing interests:
No competing interests
16 August 2014
Nicola Magrini
Essential Medicines List Secretariat
Jane Robertson, Kees de Joncheere, Essential Medicine List Secretariat, Essential Medicines and Health Products Department, World Health Organization, Geneva, Lisa Bero, Chair of WHO Expert Committees on the Selection and Use of Essential Medicines in 2011 and 2013, Medicines Use and Health Outcomes, Faculty of Pharmacy and Charles Perkins Centre, The University of Sydney
Rapid Response:
We appreciate the paper by Barbui and Purgato reflecting on the processes for selection of antidepressant and antipsychotic medicines for the World Health Organization’s Model List of Essential Medicines List (WHO Model EML) and welcome their suggestions on how these decision-making processes could be improved.1 Some of the issues raised reflect internal discussions at WHO on how to strengthen the process.
As noted by Barbui and Purgato, the reforms implemented in 2001 substantially changed the way the Expert Committee reviews submissions.2 A requirement for systematic evidence synthesis and appraisal was introduced, along with the assessment of comparative cost and cost-effectiveness. In addition, the WHO made its processes for receiving and reviewing applications transparent, with all submitted data and reviews publicly available and opportunities for interested parties to comment.
It is unclear why Barbui and Purgato are concerned that “WHO does not report information on potential conflicts of interests of applicants”. The process of submitting an application is open to all parties, including medicine manufacturers, and the applicant must be disclosed. Thus, those reviewing the application are aware of the potential conflicts of interest. While the current procedures do not prohibit applications from applicants with conflicts of interests, there are rigorous procedures in place to assess and manage any potential conflicts of interests of Committee members.
Barbui and Purgato raise three major issues regarding the EML processes: the quality of the evidence in the submissions considered by the Committee; the transparency of the decision-making process and the criteria used in making the judgments; and the discord between Expert Committee decisions and the recommendations of individual WHO Departments.
The authors suggest that the use of GRADE is new to the EML process. One of the earliest applications using GRADE assessments of the quality of evidence was for the addition of lamotrigine, submitted to the Committee in 2009.3 It has been requested that “where appropriate [this] evidence of comparative effectiveness and safety should be presented in tabular form using Grading of Recommendations, Assessment, Development and Evaluation (GRADE) tables”.4 This has been done to a variable extent in the submissions received, also depending on the nature of the application. However, a GRADE approach is not always appropriate, for example for evaluating pharmacokinetic data, as was done when assessing the doses and dosage forms needed for the first-line treatment of tuberculosis in children; or there may be existing high quality systematic reviews that have not applied this methodology.
Although in this case Barbui and Purgato argue for strengthening evidentiary requirements, others have complained that the “lengthy, exhaustive process for applying for a listing can be discouraging [as] each component requires a separate detailed, complex application.”5 A balance must be reached between the perfect and the good.
To date, Expert Committee decision-making has been reported in the Technical Report Series (TRS), a formal requirement of WHO Expert Committees. WHO has been discussing ways of improving the quality of this reporting and the proposed approach of a more structured template has merit.
Consistency between EML decisions and WHO clinical guidelines issued by other WHO departments is highly desirable. As noted by the authors, after standardizing the application format and review process of the EML in 2001, formal guideline development processes were adopted by WHO in 2007.6 Where it exists, high quality evidence generated for guideline development has also been used for EML applications (see for example the applications for antiretrovirals).7
Barbui and Purgato have also suggested that WHO should specify comparators and outcomes of interest. However, this would be a major change of focus and more in line with regulatory activities than with selecting medicines for an essential medicines list.
The accompanying commentary by Millard et al. raises issues about the quality of available evidence and the influence of “vested interests” and “expert opinion” on the decisions taken by the Expert Committee.8 The authors argue that these components had more influence than available evidence on the decision to include misoprostol on the EML for prevention of postpartum haemorrhage (PPH) where oxytocin is not available or cannot be safely used. However, misoprostol was added for this purpose to the list in 2011 after the publication of an additional well conducted placebo-controlled trial in a community setting by Mobeen and colleagues, showing relative risks for prevention of moderate and severe PPH of 0.76 (95% CI 0.59–0.97) and 0.57 (95% CI 0.27–1.22), respectively.9 The Expert Committee (and the Cochrane Reviews that were included in the application) assessed the risks of bias of all available evidence and came to a different conclusion compared to the review referenced by Millard et al.10
The way forward
More interaction between the WHO Secretariat and applicants can ensure that the minimum standards for applications are met; this requires adequate resources for both WHO and applicants. Alignment between WHO guidelines and the EML has been the basic principle of the selection process, and WHO is working towards achieving better harmonisation of guideline development and consideration by the Expert Committee.
Assessing cost-effectiveness at a global level presents many challenges, however meaningful ways to include cost considerations in the selection processes for the Model List are needed. The methods of WHO CHOICE (CHOosing Interventions that are Cost-Effective) are being used to facilitate country-level cost-effectiveness assessments and are being applied to topics such as trastuzumab for breast cancer.11
WHO has a role in supporting countries in the evidence-based selection of medicines for national EMLs and in the implementation of these lists to guide medicines procurement and medicines use in practice. The Model EML is a key innovation and achievement in WHO history.12 In 2012, more than 120 countries reporting having national EMLs; others have adapted their EML or used similar evidence-based decision-making to choose medicines for medicines reimbursement lists. The importance of the Model EML was recognised in this year’s World Health Assembly resolution on access to medicines.13
The selection processes for EML medicines are not perfect. Yet the process in place is largely transparent and tries to strike a fair balance between methodological rigour, timeliness and other critical considerations including public health need and equity of access.
Nicola Magrini
Jane Robertson
Kees de Joncheere
Essential Medicines List Secretariat,
Essential Medicines and Health Products Department,
World Health Organization, Geneva
Lisa Bero
Chair of WHO Expert Committees on the Selection and Use of Essential Medicines in 2011 and 2013
Medicines Use and Health Outcomes,
Faculty of Pharmacy and Charles Perkins Centre,
The University of Sydney
1. Barbui C, Purgato M. Decisions on WHO's essential medicines need more scrutiny. BMJ 2014;349:g4798. http://www.bmj.com/content/bmj/349/bmj.g4798.
2. World Health Organization. WHO medicines strategy. Revised procedure for updating WHO’s Model List of Essential Drugs. http://www.who.int/selection_medicines/committees/subcommittee/2/eeb1098...
3. Review of the available evidence on lamotrigine for epilepsy for the WHO Model List of Essential medicines (2009). http://www.who.int/selection_medicines/committees/expert/17/application/...
4. Essential Medicines List (EML) 2015 – Commented application. Information to be included in an application for inclusion, change or deletion of a medicine in the WHO Model List of Essential Medicines. http://www.who.int/selection_medicines/committees/expert/20/Commented_EM...
5. Klein HG. Should blood be an essential medicine? N Engl J Med 2013;368:199-201.
6. Hill S, Pang T. Leading by example: a culture change at WHO. Lancet 2007;369:1842-4.
7. 12th Expert Committee on the Selection and Use of Essential Medicines, 15-19 April 2002. Guide to the submissions for antiretroviral drugs. http://archives.who.int/eml/expcom/expcom12/arvs_submissions.htm
8. Millard C, Brhlikova P, Pollock AM. Commentary: Evidence versus influence in the WHO procedure for approving essential medicines: misoprostol for maternal health. BMJ 2014;349:g4823. http://www.bmj.com/content/349/bmj.g4823
9. Mobeen N, Durocher J, Zuberi NF, Jahan N, Blum J, Wasim S, et al. Administration of misoprostol by trained traditional birth attendants to prevent postpartum haemorrhage in homebirths in Pakistan: a randomised placebo-controlled trial. BJOG 2011;118:353-61.
10. Chu CS, Brhlikova P, Pollock AM. Rethinking WHO guidance: review of evidence for misoprostol use in the prevention of postpartum haemorrhage. J R Soc Med 2012;105:336-47.
11. World Health Organization. CHOosing Interventions that are Cost-Effective (WHO-CHOICE). http://www.who.int/choice/cost-effectiveness/en/
12. World Health Organization. Continuity and Change — Implementing the third WHO Medicines Strategy 2008-2013. WHO Policy Perspectives on Medicines 2010. http://apps.who.int/medicinedocs/documents/s16880e/s16880e.pdf
13. World Health Assembly 2014. Resolution WHA67.22 - Access to Essential Medicines. http://apps.who.int/medicinedocs/documents/s21453en/s21453en.pdf
Competing interests: No competing interests