Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13
Dr. Pignatti states it is “well known” that in many situations showing survival or quality of life benefit is not feasible. Unfortunately, the arguments he gives are not universally agreed upon, they are merely oft repeated.
Dr Pignatti states that survival benefits may be difficult to detect because subsequent lines of therapy “dilute” them. Step back and consider this argument. Even though median survival with metastatic cancer is short, and life limiting, and even though a new drug approved is a novel, costly therapy, that drug cannot improve survival because all of the prior drugs are good enough to result in the same survival. An analogy would be: you were running a marathon, and purchased a 100,000 pound energy drink and drank it at mile 2. Unfortunately, you complete the marathon in the exact same time you otherwise would. A critic points out that the drink had no value. It didn’t change the outcome. You then argue that is only because the effect of the energy drink was diluted over all the other miles. But that is precisely an argument why the energy drink was a waste. Its cost and side effects are not justified unless it improves an outcome that matters.
Dr. Pignatti also argues prolonging progression free survival (PFS) may delay the onset of symptoms. Here, he uses PFS as a surrogate for symptoms, and fails to provide data—in the form of surrogate validation studies—that proves PFS gains are accompanied by reduction in symptoms.
It is important to remember that PFS and response rate are typically based on the arbitrary change in tumor size on a CT scan. Either an increase of 20% (progression) or decrease of 30% (response). Almost no patient says “I feel bad” the moment their tumor grows from 19% to 21%. Nor do they sudden exclaim "I feel better" if the tumor is 31% rather than 29% smaller. Moreover, studies show variability in the measurement of tumors on scans by different readers [1]. For this reason, it is impossible to contend either of these metrics is something inherently valuable to patients. They are radiographic surrogates.
Dr. Pignatti argues that “dramatic activity” in the form of response rates may make further randomized trials non-feasible. This argument is unsound for a few reasons. First, the response rate in early clinical trials is almost always inflated over later trials [2]. The response rate tends to be smaller if you test the drug again. Second, there have been drugs with impressive response rates (>60%) that have failed in every single randomized trial they have latter been tested in, resulting in market withdrawal [3]. Most cancer drugs offer marginal benefits, and can and should be tested in randomized trials. In fact, gains in PFS are always measured in randomized trials, only response rate can be single armed.
Dr. Pignatti states quality of life is hard to measure. I agree, and support better and fairer ways to measure it. Moreover we have to acknowledge that good drugs improve both survival and quality of life, and this is the goal. Simply because quality of life is hard to measure however, does not justify allowing drugs to remain on the market for years with proof they only change imaging results.
Dr. Pignatti says that drugs that offer incremental benefits are still worthwhile. The major finding of the paper by Davis is that far too often we don’t know if drugs offer benefits at all. I personally support the use of provisional approval based on surrogates in certain situations, but that must come with a responsibility to show survival benefit post approval. You cannot have both lax approval and lax post-marketing commitments, which is what Davis and colleagues show.
Finally, Dr. Pignatti engages in speculation. The world is better off by having surrogate approvals. The truth is we do not know if allowing surrogate approvals benefits or harms patients because we do not know the counterfactual. While some may believe a world without surrogate approvals will result in delays for drug approval, the unanticipated consequence may be the industry will no longer pursue targets that merely result in changes to imaging scans, and instead focus on truly promising compounds. We may have fewer drugs, but the ones we have might be better. I do not know know this is true, and neither does anyone else. But just as we can speculate the world will be worse, we can also speculate it might be better. Both are just speculation.
The major point of Davis and colleagues is that while it may be ok to allow cancer drugs on the market based on a surrogate, it seems illogical to allow them to remain on the market 3, 5, or 7 years without proof of survival or quality of life benefit. At some point in the lifecycle of these drugs, this must be shown. The EMA has the obligation to make that happen. These comments raises concern that the EMA views their role differently.
Rapid Response:
Response to the comments from the EMA
I wish to respond to the person from the EMA.
Dr. Pignatti states it is “well known” that in many situations showing survival or quality of life benefit is not feasible. Unfortunately, the arguments he gives are not universally agreed upon, they are merely oft repeated.
Dr Pignatti states that survival benefits may be difficult to detect because subsequent lines of therapy “dilute” them. Step back and consider this argument. Even though median survival with metastatic cancer is short, and life limiting, and even though a new drug approved is a novel, costly therapy, that drug cannot improve survival because all of the prior drugs are good enough to result in the same survival. An analogy would be: you were running a marathon, and purchased a 100,000 pound energy drink and drank it at mile 2. Unfortunately, you complete the marathon in the exact same time you otherwise would. A critic points out that the drink had no value. It didn’t change the outcome. You then argue that is only because the effect of the energy drink was diluted over all the other miles. But that is precisely an argument why the energy drink was a waste. Its cost and side effects are not justified unless it improves an outcome that matters.
Dr. Pignatti also argues prolonging progression free survival (PFS) may delay the onset of symptoms. Here, he uses PFS as a surrogate for symptoms, and fails to provide data—in the form of surrogate validation studies—that proves PFS gains are accompanied by reduction in symptoms.
It is important to remember that PFS and response rate are typically based on the arbitrary change in tumor size on a CT scan. Either an increase of 20% (progression) or decrease of 30% (response). Almost no patient says “I feel bad” the moment their tumor grows from 19% to 21%. Nor do they sudden exclaim "I feel better" if the tumor is 31% rather than 29% smaller. Moreover, studies show variability in the measurement of tumors on scans by different readers [1]. For this reason, it is impossible to contend either of these metrics is something inherently valuable to patients. They are radiographic surrogates.
Dr. Pignatti argues that “dramatic activity” in the form of response rates may make further randomized trials non-feasible. This argument is unsound for a few reasons. First, the response rate in early clinical trials is almost always inflated over later trials [2]. The response rate tends to be smaller if you test the drug again. Second, there have been drugs with impressive response rates (>60%) that have failed in every single randomized trial they have latter been tested in, resulting in market withdrawal [3]. Most cancer drugs offer marginal benefits, and can and should be tested in randomized trials. In fact, gains in PFS are always measured in randomized trials, only response rate can be single armed.
Dr. Pignatti states quality of life is hard to measure. I agree, and support better and fairer ways to measure it. Moreover we have to acknowledge that good drugs improve both survival and quality of life, and this is the goal. Simply because quality of life is hard to measure however, does not justify allowing drugs to remain on the market for years with proof they only change imaging results.
Dr. Pignatti says that drugs that offer incremental benefits are still worthwhile. The major finding of the paper by Davis is that far too often we don’t know if drugs offer benefits at all. I personally support the use of provisional approval based on surrogates in certain situations, but that must come with a responsibility to show survival benefit post approval. You cannot have both lax approval and lax post-marketing commitments, which is what Davis and colleagues show.
Finally, Dr. Pignatti engages in speculation. The world is better off by having surrogate approvals. The truth is we do not know if allowing surrogate approvals benefits or harms patients because we do not know the counterfactual. While some may believe a world without surrogate approvals will result in delays for drug approval, the unanticipated consequence may be the industry will no longer pursue targets that merely result in changes to imaging scans, and instead focus on truly promising compounds. We may have fewer drugs, but the ones we have might be better. I do not know know this is true, and neither does anyone else. But just as we can speculate the world will be worse, we can also speculate it might be better. Both are just speculation.
The major point of Davis and colleagues is that while it may be ok to allow cancer drugs on the market based on a surrogate, it seems illogical to allow them to remain on the market 3, 5, or 7 years without proof of survival or quality of life benefit. At some point in the lifecycle of these drugs, this must be shown. The EMA has the obligation to make that happen. These comments raises concern that the EMA views their role differently.
[1] https://academic.oup.com/jnci/article/106/12/dju331/922126/Use-and-Misus...
[2] https://www.ncbi.nlm.nih.gov/pubmed/26259493
[3] https://jamanetwork.com/journals/jamainternalmedicine/article-abstract/1...
Competing interests: I am the editorialist