Re: New drugs: where did we go wrong and what can we do better?
New Evidence: Real-world data to better meet “the needs of patients”
In a recent analysis of newly approved medicinal products in Germany, Wieseler et al. of the Institute for Quality and Efficiency in Health Care (IQWiG), Germany’s health technology assessment agency, highlighted the underwhelming percentage of new drugs that demonstrated considerable or major added benefit as compared to the standard of care (54/216, 25%). Among the 216 drug approvals, there was no proof of added benefit for 125 of them, 106 of which were so deemed because the evidence package lacked the appropriate (or any) active comparator. In these value decisions, we feel there is a missed opportunity to consider real-world evidence (RWE) – a pragmatic pathway to understanding a drug’s performance in the relevant population.
The authors suggested that the frequent lack of demonstrated benefit is partially the result of regulatory requirements that allow placebo-controlled trials, and do not universally require active comparator evidence. While the mandate of regulatory agencies is focused on establishing efficacy (can it work in humans under best circumstances?) , HTAs, find information on incremental effectiveness (does it work?) against active comparators more informative for coverage decisions. This gap between regulatory efficacy evidence needs and the effectiveness evidence needs of HTAs — the so-called “efficacy-effectiveness” gap[1] — is dissatisfying for all parties particularly when it leads to delayed access of effective medications. RWE would offer the ability to bridge that gap to evaluate a therapy’s effectiveness in the German population, against comparators relevant to German clinical practice. Two relevant RWE strategies for bridging the gap are creating external controls and post-approval comparative effectiveness evidence to support continued reimbursement.
However, Germany rarely accepts RWE[2]. The Federal Joint Committee (G-BA) and IQWiG favor evidence from randomized controlled trials (i.e., efficacy data) with the German standard of care as the control. Non-interventional studies are only accepted under very limited circumstances, such as for orphan drugs. While we all love RCTs, this is a missed opportunity. As Pearl and Mackenize point out, we can use observational data to answer interventional questions, if we develop “a sufficiently strong and accurate causal model”[3]. Several decades of advancement of scientifically-rigorous methods, combined with increasing access to real-world data in countries worldwide, creates an opportunity to greatly expand the use of RWE for HTA decision-making[4].
To answer the causal questions most relevant to HTAs and the populations they serve, we can take the raw material of real-world data and craft from that real-world evidence. This process must respect several core tenets:
● RWE must be derived from fit-for purpose real-world data.
● RWE must be appropriate for decision-making (i.e., must be causal); to get to the “sufficiently strong and accurate model”, study designs should include a meaningful contrast, account for temporality, and include extensive bias reduction methods.
● RWE must be fully transparent to allow all stakeholders to understand and use the data[5,6].
Creating RWE is not without challenges. A newly approved drug may not have been on the market long enough to accumulate substantial history and would require analytical strategies like external control groups constructed with analytic methods like propensity score matching or weighting. For example, an external control group was used in the FDA accelerated approval of blinatumomab for the treatment of acute lymphoblastic leukemia, with standard of care as control derived from registry data[7].
HTAs often contemplate conditional reimbursement based on post-marketing study requirements which could be strengthened by RWE. The authors noted that six post-marketing RCTs were not completed by the time the re-assessments were due. Post-marketing RCTs necessitate substantial time and investment, which can be reduced with rapid-cycle RWE. In the examples where RWE was able to successfully predict the results of ongoing phase IV RCTs, the time to evidence was magnitudes shorter. [8,9] Furthermore, RWE allows for the flexibility to explore additional variations of the study implementation that is not possible in completed RCTs.
Several regulatory and HTA agencies are starting to explore expanding the use RWE for decision-making. America’s FDA[10] and Europe’s EMA[11] are establishing frameworks for their RWE programs, while on the payer side, HTA submissions are starting to incorporate RWE[12] as agencies such as the UK’s NICE[13] and Canada’s CADTH[14] explore ways to maximize RWE’s utility. Given this global movement in evidentiary standards, the time seems right for Germany and other countries to consider expanding the use of RWE for value decision-making.
References
[1] Nordon C, Karcher H, Groenwold RH, et al. The "Efficacy-Effectiveness Gap": Historical Background and Current Conceptualization. Value Health. 2016;19(1):75-81. doi: 10.1016/j.jval.2015.09.2938.
[2] Makady A, Ham RT, de Boer A, et al. Policies for Use of Real-World Data in Health Technology Assessment (HTA): A Comparative Study of Six HTA Agencies. Value Health. 2017;20(4):520-532. doi: 10.1016/j.jval.2016.12.003.
[3] Pearl J and Mackenzie D. Chapter 1: The Ladder of Causation. In: Book of Why: The New Science of Cause and Effect. New York: Basic Books 2018: 23-52.
[4] Hampson G, Towse A, Dreitlein WB, Henshall C, Pearson SD. Real-world evidence for coverage decisions: opportunities and challenges. J Comp Eff Res. 2018;7(12):1133-1143. doi: 10.2217/cer-2018-0066.
[5] Berger ML Sox H, Willke RJ, et al. Good Practices for Real‐World Data Studies of Treatment and/or Comparative Effectiveness: Recommendations from the Joint ISPOR‐ISPE Special Task Force on Real‐World Evidence in Health Care Decision Making. Value in Health 2017; 20: 1003-1008.
[6] Wang SV, Schneeweiss S, Berger ML, et al. on behalf of the joint ISPE-ISPOR Special Task Force on Real World Evidence in Health Care Decision Making. Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0. Pharmacoepidemiol Drug Saf. 2017 Sep;26(9):1018-1032. doi: 10.1002/pds.4295.
[7] BLINCYTO® (blinatumomab) [package insert]. Thousand Oaks, CA: Amgen; 2018. Accessed 15 August 2019. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/label/2018/125557s013lbl.pdf
[8] Patorno E, Schneeweiss S, Gopalakrishnan C, Martin D, Franklin JM. Using Real-World Data to Predict Findings of an Ongoing Phase IV Cardiovascular Outcome Trial - Cardiovascular Safety of Linagliptin vs. Glimepiride. Diabetes Care. 2019 Jun 25. pii: dc190069. doi: 10.2337/dc19-0069. [Epub ahead of print]
[9] Kim S, Solomon D, Rogers J, et al. Cardiovascular Safety of Tocilizumab Versus Tumor Necrosis Factor Inhibitors in Patients With Rheumatoid Arthritis: A Multi-Database Cohort Study. Arthritis Rheumatol. 2017 June;69(6):1154-1164.
[10] US Food & Drug Administration (FDA). Framework for FDA’s Real-world Evidence Program. Accessed 7 August 2019. Available from: https://www.fda.gov/science-research/science-and-research-special-topics....
[11] European Medicines Agency. EMA Regulatory Science to 2025. Accessed 15 August 2019. Available from: https://www.ema.europa.eu/en/about-us/how-we-work/regulatory-science-2025
[12] Makady A, van Veelen A, Jonsson P, et al. Using Real-world Data in Health Technology Assessment (HTA) Practice: A Comparative Study of Five HTA Agencies. Pharmacoeconomics. 2018; 36(3):359-368. doi: 10.1007/s40273-017-0596-z.
[13] National Institute for Health and Care Excellence (NICE). Consultation on the data and analytics statement of intent. Accessed 15 August 2019. Available from: https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/ni...
[14] Health Canada. Optimizing the Use of Real World Evidence to Inform Regulatory Decision-Making. Accessed 15 August 2019. Available from: https://www.canada.ca/en/health-canada/services/drugs-health-products/dr...
Competing interests:
No competing interests
16 August 2019
Ashley A Jaksa
VP Analytic Solutions
Jeremy A. Rassen, ScD, President and Chief Scientific Officer Aetion, Inc.
Rapid Response:
Re: New drugs: where did we go wrong and what can we do better?
New Evidence: Real-world data to better meet “the needs of patients”
In a recent analysis of newly approved medicinal products in Germany, Wieseler et al. of the Institute for Quality and Efficiency in Health Care (IQWiG), Germany’s health technology assessment agency, highlighted the underwhelming percentage of new drugs that demonstrated considerable or major added benefit as compared to the standard of care (54/216, 25%). Among the 216 drug approvals, there was no proof of added benefit for 125 of them, 106 of which were so deemed because the evidence package lacked the appropriate (or any) active comparator. In these value decisions, we feel there is a missed opportunity to consider real-world evidence (RWE) – a pragmatic pathway to understanding a drug’s performance in the relevant population.
The authors suggested that the frequent lack of demonstrated benefit is partially the result of regulatory requirements that allow placebo-controlled trials, and do not universally require active comparator evidence. While the mandate of regulatory agencies is focused on establishing efficacy (can it work in humans under best circumstances?) , HTAs, find information on incremental effectiveness (does it work?) against active comparators more informative for coverage decisions. This gap between regulatory efficacy evidence needs and the effectiveness evidence needs of HTAs — the so-called “efficacy-effectiveness” gap[1] — is dissatisfying for all parties particularly when it leads to delayed access of effective medications. RWE would offer the ability to bridge that gap to evaluate a therapy’s effectiveness in the German population, against comparators relevant to German clinical practice. Two relevant RWE strategies for bridging the gap are creating external controls and post-approval comparative effectiveness evidence to support continued reimbursement.
However, Germany rarely accepts RWE[2]. The Federal Joint Committee (G-BA) and IQWiG favor evidence from randomized controlled trials (i.e., efficacy data) with the German standard of care as the control. Non-interventional studies are only accepted under very limited circumstances, such as for orphan drugs. While we all love RCTs, this is a missed opportunity. As Pearl and Mackenize point out, we can use observational data to answer interventional questions, if we develop “a sufficiently strong and accurate causal model”[3]. Several decades of advancement of scientifically-rigorous methods, combined with increasing access to real-world data in countries worldwide, creates an opportunity to greatly expand the use of RWE for HTA decision-making[4].
To answer the causal questions most relevant to HTAs and the populations they serve, we can take the raw material of real-world data and craft from that real-world evidence. This process must respect several core tenets:
● RWE must be derived from fit-for purpose real-world data.
● RWE must be appropriate for decision-making (i.e., must be causal); to get to the “sufficiently strong and accurate model”, study designs should include a meaningful contrast, account for temporality, and include extensive bias reduction methods.
● RWE must be fully transparent to allow all stakeholders to understand and use the data[5,6].
Creating RWE is not without challenges. A newly approved drug may not have been on the market long enough to accumulate substantial history and would require analytical strategies like external control groups constructed with analytic methods like propensity score matching or weighting. For example, an external control group was used in the FDA accelerated approval of blinatumomab for the treatment of acute lymphoblastic leukemia, with standard of care as control derived from registry data[7].
HTAs often contemplate conditional reimbursement based on post-marketing study requirements which could be strengthened by RWE. The authors noted that six post-marketing RCTs were not completed by the time the re-assessments were due. Post-marketing RCTs necessitate substantial time and investment, which can be reduced with rapid-cycle RWE. In the examples where RWE was able to successfully predict the results of ongoing phase IV RCTs, the time to evidence was magnitudes shorter. [8,9] Furthermore, RWE allows for the flexibility to explore additional variations of the study implementation that is not possible in completed RCTs.
Several regulatory and HTA agencies are starting to explore expanding the use RWE for decision-making. America’s FDA[10] and Europe’s EMA[11] are establishing frameworks for their RWE programs, while on the payer side, HTA submissions are starting to incorporate RWE[12] as agencies such as the UK’s NICE[13] and Canada’s CADTH[14] explore ways to maximize RWE’s utility. Given this global movement in evidentiary standards, the time seems right for Germany and other countries to consider expanding the use of RWE for value decision-making.
References
[1] Nordon C, Karcher H, Groenwold RH, et al. The "Efficacy-Effectiveness Gap": Historical Background and Current Conceptualization. Value Health. 2016;19(1):75-81. doi: 10.1016/j.jval.2015.09.2938.
[2] Makady A, Ham RT, de Boer A, et al. Policies for Use of Real-World Data in Health Technology Assessment (HTA): A Comparative Study of Six HTA Agencies. Value Health. 2017;20(4):520-532. doi: 10.1016/j.jval.2016.12.003.
[3] Pearl J and Mackenzie D. Chapter 1: The Ladder of Causation. In: Book of Why: The New Science of Cause and Effect. New York: Basic Books 2018: 23-52.
[4] Hampson G, Towse A, Dreitlein WB, Henshall C, Pearson SD. Real-world evidence for coverage decisions: opportunities and challenges. J Comp Eff Res. 2018;7(12):1133-1143. doi: 10.2217/cer-2018-0066.
[5] Berger ML Sox H, Willke RJ, et al. Good Practices for Real‐World Data Studies of Treatment and/or Comparative Effectiveness: Recommendations from the Joint ISPOR‐ISPE Special Task Force on Real‐World Evidence in Health Care Decision Making. Value in Health 2017; 20: 1003-1008.
[6] Wang SV, Schneeweiss S, Berger ML, et al. on behalf of the joint ISPE-ISPOR Special Task Force on Real World Evidence in Health Care Decision Making. Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0. Pharmacoepidemiol Drug Saf. 2017 Sep;26(9):1018-1032. doi: 10.1002/pds.4295.
[7] BLINCYTO® (blinatumomab) [package insert]. Thousand Oaks, CA: Amgen; 2018. Accessed 15 August 2019. Available from: https://www.accessdata.fda.gov/drugsatfda_docs/label/2018/125557s013lbl.pdf
[8] Patorno E, Schneeweiss S, Gopalakrishnan C, Martin D, Franklin JM. Using Real-World Data to Predict Findings of an Ongoing Phase IV Cardiovascular Outcome Trial - Cardiovascular Safety of Linagliptin vs. Glimepiride. Diabetes Care. 2019 Jun 25. pii: dc190069. doi: 10.2337/dc19-0069. [Epub ahead of print]
[9] Kim S, Solomon D, Rogers J, et al. Cardiovascular Safety of Tocilizumab Versus Tumor Necrosis Factor Inhibitors in Patients With Rheumatoid Arthritis: A Multi-Database Cohort Study. Arthritis Rheumatol. 2017 June;69(6):1154-1164.
[10] US Food & Drug Administration (FDA). Framework for FDA’s Real-world Evidence Program. Accessed 7 August 2019. Available from: https://www.fda.gov/science-research/science-and-research-special-topics....
[11] European Medicines Agency. EMA Regulatory Science to 2025. Accessed 15 August 2019. Available from: https://www.ema.europa.eu/en/about-us/how-we-work/regulatory-science-2025
[12] Makady A, van Veelen A, Jonsson P, et al. Using Real-world Data in Health Technology Assessment (HTA) Practice: A Comparative Study of Five HTA Agencies. Pharmacoeconomics. 2018; 36(3):359-368. doi: 10.1007/s40273-017-0596-z.
[13] National Institute for Health and Care Excellence (NICE). Consultation on the data and analytics statement of intent. Accessed 15 August 2019. Available from: https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/ni...
[14] Health Canada. Optimizing the Use of Real World Evidence to Inform Regulatory Decision-Making. Accessed 15 August 2019. Available from: https://www.canada.ca/en/health-canada/services/drugs-health-products/dr...
Competing interests: No competing interests