New drugs: where did we go wrong and what can we do better?
BMJ 2019; 366 doi: https://doi.org/10.1136/bmj.l4340 (Published 10 July 2019) Cite this as: BMJ 2019;366:l4340
All rapid responses
We thank Ilker Oztop for the interest in our publication.
All IQWiG assessment reports are publicly available on both IQWiG’s and the G-BA’s website. In addition to the full German-language report, IQWiG also provides English translations of the main parts of the assessment reports. These are available under the following link: https://www.iqwig.de/en/projects-results/projects.1057.html . A list of all assessments in the same format as in Table 1 is not available.
As described in Box 1 of the publication, the G-BA’s final decisions sometimes differ from IQWiG’s assessments due to further steps taken in the procedure after the publication of IQWiG’s assessment report (also described in Box 1). A comparison of the individual assessment outcomes between IQWiG and the G-BA is beyond the scope of the publication. All final decisions by the G-BA, including the G-BA’s rationale for the decision, are available on the G-BA’s website: https://www.g-ba.de/bewertungsverfahren/nutzenbewertung/ .
Competing interests: No competing interests
Thank you for this interesting article.
1) Is the full dataset of 216 IQWiG assessments available as supplementary documents - especially since the underlying data is already public? Would be great to be able to review the full curated set in the same format as in Table 1 where only select examples are listed
2) Box 1 lists aggregate G-BA assessments of the same 216 drugs IQWiG evaluated and in 72 instances (33% of all decisions) IQWiG ended up with different assessments:
Overall G-BA/IQWiG: 216/216
- No added benefit: 115/125
- Major benefit: 1/21 (after corrigendum)
- Considerable benefit: 55/33 (after corrigendum)
- Minor benefit: 33/19 (after corrigendum)
- Non-quantifiable: 12/16 (after corrigendum)
- Less benefit: 0/2
Would be great if the authors had highlighted and commented on these discordant decisions and provided the underlying data along with the IQWiG assessments.
Thank you again
Competing interests: No competing interests
Daniel Morales mentions potential advantages of having several new drugs of the same type. Unfortunately, the hope of lower prices due to competition is often not fulfilled [1]. Furthermore, even if a substantial impact on pricing existed, this still would not require the extensive parallel development of multiple similar drugs, as seen, for example, in immunotherapy [3]. We agree that different side effect profiles could represent advantages of similar drugs and this would be considered as an added benefit in our assessments. We do believe that new therapies such as orphan products or advanced therapy medicinal products (ATMPs) should in principle provide the same proof of benefit than more traditional treatments because patients using these products should also be able to receive sufficient information on their treatments. This is also specified in the orphan drug regulation. A review of ongoing studies in rare diseases confirmed that even in very small populations, controlled trials are possible [3].
François Houÿez and Virginie Hivert seem to conclude from our paper that we mainly ask for regulators to only approve drugs with a proven added benefit. This is not the case. At the same time, we do not think that the accelerated approval of drugs (with often high uncertainty concerning their benefit for patients) is per se an advantage, especially not in situations where other treatment options are available. There might be better ways to provide the new treatments required by patients. Our article aims to contribute to a discussion about the best approaches.
While we agree that not every drug can have an added benefit over available treatments, overall, we would like to see better drugs for patients. Furthermore, sufficient information should be available for patients to make their treatment decisions, for physicians to provide treatment recommendations, and for health care systems to ensure high quality and sustainable care. To this end, we compiled a number of suggestions and examples of successful contributions to the discussion. We feel that the examples provided by François Houÿez and Virginie Hivert underline rather than contradict a more proactive approach towards defining the needs of health care systems and actively supporting the required steps for drug development. Concerning drugs for rare diseases, our suggestion of full transparency for all study data would be particularly important.
We are not only addressing regulators, but also HTA agencies, health care decision makers, physicians, policy makers and society as a whole, including patients. We hope that an open discussion will improve treatment options for patients and health care in general. We thank the authors of the rapid responses for their contribution to this discussion.
Literature
[1] Gordon N, Stemmer SM, Greenberg D, Goldstein DA. Trajectories of Injectable Cancer Drug Costs After Launch in the United States. JCO 2018 36:4, 319-325
[2] Tang J, Shalabi, A, Hubbard-Lucey VM. Comprehensive analysis of the clinical immuno-oncology landscape. Ann Oncol. 2018 Jan 1;29(1):84-91
[3] Hee, S. W., et al. (2017). Does the low prevalence affect the sample size of interventional clinical trials of rare diseases? An analysis of data from the aggregate analysis of clinicaltrials.gov. Orphanet Journal of Rare Diseases 12(1): 44.
Competing interests: No competing interests
Wieseler and colleagues provide a critical opinion on the state of new drug development in Europe and the regulatory processes required when new medicines are approved, suggesting that flaws in regulatory requirements are a key reason for evidence gaps and that stricter regulatory measures prior to approval will be a solution for such problems.
Yet the traditional drug development process often takes between 10 and 15 years for a new medicine to reach the market. Under their suggestion, once approval is granted for a new type of medicine should industry then give up on similar products that have been in development for 10 or more years? There are potential advantages for having several new drugs. The first is competition, were multiple products may drive down cost, without which health systems could face a monopoly and potentially sustained high prices. The second is that a medicine which is similar does not mean it is the same. Side effect profiles may differ providing alternative treatment options for patients intolerant of individual products even if benefit remains the same.
Regulatory requirements for post-approval studies often form part of a medicines formal risk management plan and can vary with some studies mandatory and others not. Health Technology Assessment (HTA) organisations could themselves decide that reimbursement of medicines should not occur unless further evidence on the comparative benefit and safety is generated. Many newer medicines do not fit the traditional drug development system, such as orphan products or Advanced Therapy Medicinal Products. If large phase III clinical trials involving thousands of patients are always required for rare diseases before approval one may never obtain an answer or indeed access to the medicine. This is before we acknowledge that randomised clinical trials typically exclude large proportions of their target population who may substantially differ, meaning that further evidence will always be needed. Regulators and HTA need to be flexible to these modern challenges whilst still maintaining evidentiary standards, which is challenging.
Whilst it is acknowledged that this lack of evidence creates a significant problem for all involved, the solution probably lies with all stakeholders too. Initiatives such as the EMA-EUnetHTA that foster collaboration between the EMA and HTA in Europe are, in part, hoping to tackle this problem. Health care systems could also take a prime role in helping to determine the comparative benefit and safety of medicines through randomised policy decisions that would provide answers in a timely fashion. To use an analogy, whilst manufacturers provide detailed specifications about a cars performance in an idealised setting, it is often Top Gear that really tells us how they compare in challenging real world conditions.
Competing interests: No competing interests
Did we really go wrong?
Regulators assess the quality of medicines, and authorise them only when benefits seen so far outweigh the risks. Thereafter HTA experts determine where the medicines fits in the therapeutic arsenal and the patient population for whom the product can be reimbursed or covered. HTA constantly explained these are two different roles, and yet authors tend to conclude regulators are not considering the added therapeutic value of new products. Well, this is precisely the role of HTA, not of regulators (with the exception of the Significant Benefit as defined by the Regulation on Orphan Medicinal Products 141/2000 article 3.1 b).
While acknowledging that not all authorised medicines find their role in the armamentarium against diseases, EURORDIS disagrees with the authors’ opinion that regulators fail to focus on the needs of the patients when authorising medicines with minor benefit over existing options at the time of their authorisation.
In real life, treatment strategy trials, consensus conferences of experts, or real world evidence inform patients and doctors’ decisions on how to make the best use of authorised products, gaining experience and knowledge when more patients are treated after the authorisation. But for this to happen, the medicine has to be authorised in the first place, following the above mentioned regulatory assessment of quality, efficacy and safety.
A major risk all patients are facing is when regulators do not authorise a yet effective medicine. Beate Wieseler and colleagues tend to ignore that industry and regulators need to accelerate the development and the evaluation of medicines in the interest of patients. They evaluate if a medicine can work in the context of a clinical trial, i.e. if the benefits can outweigh the risks. If the case, then the product can be authorised and the medical field can learn how to best use it. It is not the scope of regulators to determine the full clinical added-value of medicines. A new medicine cannot be compared to all existing options in extenso otherwise their development would last for ever.
Here are some examples where patients and public health benefited from medicines that were authorised with apparently only minor benefit over existing treatments, if any:
• At the time they were placed on the market, none of the four anti-tuberculosis treatments used alone, i.e. rifampicin, isoniazid, pyrazinamide or ethambutol had a considerable or major added benefit compared to standard of care or compared to each other. Used one by one, they had no effect. Only when doctors started to use them in combination therapies (bi or triple therapy) and because they had been placed on the market, then patients could be cured and the bacteria eradicated from their body.
• Same for anti-HIV treatments, where none of them used individually had sustainable clinical benefit in terms of mortality or morbidity in the first period 1986-2000. Only years after their authorisation, HIV doctors started to combine them, using the latest information they had on HIV resistance profile to construct potent regimen. The second authorised reverse transcriptase inhibitor, didanosine, was more potent to increase the number of CD4 cells compared to zidovudine, but with a difference of only 11 cells per mm3 in favour of didanosine. Nothing clinically relevant, but thereafter all patients could start combining the two, and this had a clinical effect.
• In cancer, most of the progresses are step-by-step mortality gains of a few percent, where each new product can add something, even if minor, to available options. Mortality from all cancers in the EU declined annually by 1.5% in men since 1990 and by 0.8% in women. Post-authorisation clinical research using observational studies and treatment protocol comparisons helped oncologists to determine which different combination therapies were the most effective.
• In rare diseases, some products are authorised with a modest effect size or limited efficacy data, and high uncertainty, due to the difficulties to recruit adequate numbers of trial subjects. But, used in clinical practice thereafter, specialist doctors learn to distinguish which patients can benefit the most and post-authorisation studies can generate additional evidence.
• Many medicines find a new medical use years after having been placed on the market. Off-label use is s source of medical progress. Drugs can only be repositioned if they are authorised in the first place. A minor added benefit is not a reason to reject a new medicine from the market, as long as it outweighs the risks.
The European legislator is certainly concerned by patients who have no treatment option. Regulation 726/2004 article 14a.2 is a response to the situation of patients who have no satisfactory methods of diagnosis, prevention or treatment. For those patients, it is of utmost importance to make new products available. For other, research can continue to increase the treatment options. This is the sense of compassionate use schemes, conditional authorisation and /or adaptive authorisation. The role of HTA is then to determine which treatment can help the most, i.e. their relative efficacy and how much society is willing to invest. But a prerequisite for HTA to do their job is to have the medicines authorised for human use in the first place. Nowadays regulators and HTA experts propose joint scientific advice to increase chances that adequate data are submitted both to regulator and to HTA bodies at a later stage.
Competing interests: No competing interests
Re: New drugs: where did we go wrong and what can we do better?
New Evidence: Real-world data to better meet “the needs of patients”
In a recent analysis of newly approved medicinal products in Germany, Wieseler et al. of the Institute for Quality and Efficiency in Health Care (IQWiG), Germany’s health technology assessment agency, highlighted the underwhelming percentage of new drugs that demonstrated considerable or major added benefit as compared to the standard of care (54/216, 25%). Among the 216 drug approvals, there was no proof of added benefit for 125 of them, 106 of which were so deemed because the evidence package lacked the appropriate (or any) active comparator. In these value decisions, we feel there is a missed opportunity to consider real-world evidence (RWE) – a pragmatic pathway to understanding a drug’s performance in the relevant population.
The authors suggested that the frequent lack of demonstrated benefit is partially the result of regulatory requirements that allow placebo-controlled trials, and do not universally require active comparator evidence. While the mandate of regulatory agencies is focused on establishing efficacy (can it work in humans under best circumstances?) , HTAs, find information on incremental effectiveness (does it work?) against active comparators more informative for coverage decisions. This gap between regulatory efficacy evidence needs and the effectiveness evidence needs of HTAs — the so-called “efficacy-effectiveness” gap[1] — is dissatisfying for all parties particularly when it leads to delayed access of effective medications. RWE would offer the ability to bridge that gap to evaluate a therapy’s effectiveness in the German population, against comparators relevant to German clinical practice. Two relevant RWE strategies for bridging the gap are creating external controls and post-approval comparative effectiveness evidence to support continued reimbursement.
However, Germany rarely accepts RWE[2]. The Federal Joint Committee (G-BA) and IQWiG favor evidence from randomized controlled trials (i.e., efficacy data) with the German standard of care as the control. Non-interventional studies are only accepted under very limited circumstances, such as for orphan drugs. While we all love RCTs, this is a missed opportunity. As Pearl and Mackenize point out, we can use observational data to answer interventional questions, if we develop “a sufficiently strong and accurate causal model”[3]. Several decades of advancement of scientifically-rigorous methods, combined with increasing access to real-world data in countries worldwide, creates an opportunity to greatly expand the use of RWE for HTA decision-making[4].
To answer the causal questions most relevant to HTAs and the populations they serve, we can take the raw material of real-world data and craft from that real-world evidence. This process must respect several core tenets:
● RWE must be derived from fit-for purpose real-world data.
● RWE must be appropriate for decision-making (i.e., must be causal); to get to the “sufficiently strong and accurate model”, study designs should include a meaningful contrast, account for temporality, and include extensive bias reduction methods.
● RWE must be fully transparent to allow all stakeholders to understand and use the data[5,6].
Creating RWE is not without challenges. A newly approved drug may not have been on the market long enough to accumulate substantial history and would require analytical strategies like external control groups constructed with analytic methods like propensity score matching or weighting. For example, an external control group was used in the FDA accelerated approval of blinatumomab for the treatment of acute lymphoblastic leukemia, with standard of care as control derived from registry data[7].
HTAs often contemplate conditional reimbursement based on post-marketing study requirements which could be strengthened by RWE. The authors noted that six post-marketing RCTs were not completed by the time the re-assessments were due. Post-marketing RCTs necessitate substantial time and investment, which can be reduced with rapid-cycle RWE. In the examples where RWE was able to successfully predict the results of ongoing phase IV RCTs, the time to evidence was magnitudes shorter. [8,9] Furthermore, RWE allows for the flexibility to explore additional variations of the study implementation that is not possible in completed RCTs.
Several regulatory and HTA agencies are starting to explore expanding the use RWE for decision-making. America’s FDA[10] and Europe’s EMA[11] are establishing frameworks for their RWE programs, while on the payer side, HTA submissions are starting to incorporate RWE[12] as agencies such as the UK’s NICE[13] and Canada’s CADTH[14] explore ways to maximize RWE’s utility. Given this global movement in evidentiary standards, the time seems right for Germany and other countries to consider expanding the use of RWE for value decision-making.
References
[1] Nordon C, Karcher H, Groenwold RH, et al. The "Efficacy-Effectiveness Gap": Historical Background and Current Conceptualization. Value Health. 2016;19(1):75-81. doi: 10.1016/j.jval.2015.09.2938.
[2] Makady A, Ham RT, de Boer A, et al. Policies for Use of Real-World Data in Health Technology Assessment (HTA): A Comparative Study of Six HTA Agencies. Value Health. 2017;20(4):520-532. doi: 10.1016/j.jval.2016.12.003.
[3] Pearl J and Mackenzie D. Chapter 1: The Ladder of Causation. In: Book of Why: The New Science of Cause and Effect. New York: Basic Books 2018: 23-52.
[4] Hampson G, Towse A, Dreitlein WB, Henshall C, Pearson SD. Real-world evidence for coverage decisions: opportunities and challenges. J Comp Eff Res. 2018;7(12):1133-1143. doi: 10.2217/cer-2018-0066.
[5] Berger ML Sox H, Willke RJ, et al. Good Practices for Real‐World Data Studies of Treatment and/or Comparative Effectiveness: Recommendations from the Joint ISPOR‐ISPE Special Task Force on Real‐World Evidence in Health Care Decision Making. Value in Health 2017; 20: 1003-1008.
[6] Wang SV, Schneeweiss S, Berger ML, et al. on behalf of the joint ISPE-ISPOR Special Task Force on Real World Evidence in Health Care Decision Making. Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0. Pharmacoepidemiol Drug Saf. 2017 Sep;26(9):1018-1032. doi: 10.1002/pds.4295.
[7] BLINCYTO® (blinatumomab) [package insert]. Thousand Oaks, CA: Amgen; 2018. Accessed 15 August 2019. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/label/2018/125557s013lbl.pdf
[8] Patorno E, Schneeweiss S, Gopalakrishnan C, Martin D, Franklin JM. Using Real-World Data to Predict Findings of an Ongoing Phase IV Cardiovascular Outcome Trial - Cardiovascular Safety of Linagliptin vs. Glimepiride. Diabetes Care. 2019 Jun 25. pii: dc190069. doi: 10.2337/dc19-0069. [Epub ahead of print]
[9] Kim S, Solomon D, Rogers J, et al. Cardiovascular Safety of Tocilizumab Versus Tumor Necrosis Factor Inhibitors in Patients With Rheumatoid Arthritis: A Multi-Database Cohort Study. Arthritis Rheumatol. 2017 June;69(6):1154-1164.
[10] US Food & Drug Administration (FDA). Framework for FDA’s Real-world Evidence Program. Accessed 7 August 2019. Available from: https://www.fda.gov/science-research/science-and-research-special-topics....
[11] European Medicines Agency. EMA Regulatory Science to 2025. Accessed 15 August 2019. Available from: https://www.ema.europa.eu/en/about-us/how-we-work/regulatory-science-2025
[12] Makady A, van Veelen A, Jonsson P, et al. Using Real-world Data in Health Technology Assessment (HTA) Practice: A Comparative Study of Five HTA Agencies. Pharmacoeconomics. 2018; 36(3):359-368. doi: 10.1007/s40273-017-0596-z.
[13] National Institute for Health and Care Excellence (NICE). Consultation on the data and analytics statement of intent. Accessed 15 August 2019. Available from: https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/ni...
[14] Health Canada. Optimizing the Use of Real World Evidence to Inform Regulatory Decision-Making. Accessed 15 August 2019. Available from: https://www.canada.ca/en/health-canada/services/drugs-health-products/dr...
Competing interests: No competing interests