Rapid responses are electronic comments to the editor. They enable our users
to debate issues raised in articles published on bmj.com. A rapid response
is first posted online. If you need the URL (web address) of an individual
response, simply click on the response headline and copy the URL from the
browser window. A proportion of responses will, after editing, be published
online and in the print journal as letters, which are indexed in PubMed.
Rapid responses are not indexed in PubMed and they are not journal articles.
The BMJ reserves the right to remove responses which are being
wilfully misrepresented as published articles or when it is brought to our
attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not
including references and author details. We will no longer post responses
that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
A recent news item in the BMJ [1] reported that a trial of the anti-viral vaccines mAB114 and REGN-EB3 were ended early because they met early stopping criteria for effectiveness. This is indeed encouraging for patients suffering from Ebola, given its high mortality rate, and it is very understandable that health care workers would want to get these drugs to patients as rapidly as possible. However, stopping clinical trials prematurely based on interim analyses is fraught with difficulties. As pointed out in a forthcoming article, [2] trials that have been ended early tend to overestimate the effects of the intervention, as Pocock and Hughes [3] warned 30 years ago. This was reinforced by the systematic review by Montori et al. [4] In their review of 143 studies, they concluded that “Trials stopped early for benefit, particularly those with few events, often report treatment effects that are larger than typical of interventions that have been definitively studied” (p. 2209). That team also showed that the smaller the number of events, the larger the overestimates, and that even when there were 500 events, the overestimates were small but important. [5]
Complicating the picture even further, Ebola is not a single disease, but consists of five viruses, four of which are harmful to humans. [6] Decreasing the sample size by ending the study early may preclude sub-group analyses, such as determining if these treatments are effective for some of the viruses but not others, or if they are more effective for one sex rather than another.
Thus, if the past is any guide, we may expect that the magnitude of the effect of these drugs will decrease as they are administered to more patients. Needless to say, we hope that this does not occur, but we should not be overly surprised if it does.
References
1. Dyer O. Two Ebola treatments halve deaths in trial in DRC outbreak. BMJ 2019;366:l5140 doi: 10.1136/bmj.l5140.
2. Streiner DL. Should clinical trials be terminated early? Clinical Therapeutics in press.
3. Pocock SJ, Hughes MD. Practical problems in interim analyses, with particular regard to estimation. Control Clin Trials. 1989;10(suppl 4):209S-221S. doi.org/10.1016/0197-2456(89)90059-7
4. Montori VM, Devereaux PJ, Adhikari NKJ et al. Randomized trials stopped early for benefit: A systematic review. JAMA. 2005;294:2203-2209. doi:10.1001/jama.294.17.2203
5. Bassler D, Briel M, Montori VM, et al. Stopping randomized trials early for benefit and estimation of treatment effects: Systematic review and meta-regression analysis. JAMA. 2010;303:1180-1187. doi: 10.1001/jama.2010.310
6. Kuhn JH, Becker S, Ebihara H, Geisbert TW, Johnson KM, Kawaoka Y, et al. (December 2010). "Proposal for a revised taxonomy of the family Filoviridae: Classification, names of taxa and viruses, and virus abbreviations". Archives of Virology. 155 (12): 2083–103. doi: 10.1007/s00705-010-0814-x
Competing interests:
No competing interests
26 August 2019
David L. Streiner
Emeritus Professor
McMaster University
St. Joseph's Healthcare, Mountain Campus, Hamilton, Ontario, Canada L8N 3K7
Re: Two Ebola treatments halve deaths in trial in DRC outbreak
To the Editor:
A recent news item in the BMJ [1] reported that a trial of the anti-viral vaccines mAB114 and REGN-EB3 were ended early because they met early stopping criteria for effectiveness. This is indeed encouraging for patients suffering from Ebola, given its high mortality rate, and it is very understandable that health care workers would want to get these drugs to patients as rapidly as possible. However, stopping clinical trials prematurely based on interim analyses is fraught with difficulties. As pointed out in a forthcoming article, [2] trials that have been ended early tend to overestimate the effects of the intervention, as Pocock and Hughes [3] warned 30 years ago. This was reinforced by the systematic review by Montori et al. [4] In their review of 143 studies, they concluded that “Trials stopped early for benefit, particularly those with few events, often report treatment effects that are larger than typical of interventions that have been definitively studied” (p. 2209). That team also showed that the smaller the number of events, the larger the overestimates, and that even when there were 500 events, the overestimates were small but important. [5]
Complicating the picture even further, Ebola is not a single disease, but consists of five viruses, four of which are harmful to humans. [6] Decreasing the sample size by ending the study early may preclude sub-group analyses, such as determining if these treatments are effective for some of the viruses but not others, or if they are more effective for one sex rather than another.
Thus, if the past is any guide, we may expect that the magnitude of the effect of these drugs will decrease as they are administered to more patients. Needless to say, we hope that this does not occur, but we should not be overly surprised if it does.
References
1. Dyer O. Two Ebola treatments halve deaths in trial in DRC outbreak. BMJ 2019;366:l5140 doi: 10.1136/bmj.l5140.
2. Streiner DL. Should clinical trials be terminated early? Clinical Therapeutics in press.
3. Pocock SJ, Hughes MD. Practical problems in interim analyses, with particular regard to estimation. Control Clin Trials. 1989;10(suppl 4):209S-221S. doi.org/10.1016/0197-2456(89)90059-7
4. Montori VM, Devereaux PJ, Adhikari NKJ et al. Randomized trials stopped early for benefit: A systematic review. JAMA. 2005;294:2203-2209. doi:10.1001/jama.294.17.2203
5. Bassler D, Briel M, Montori VM, et al. Stopping randomized trials early for benefit and estimation of treatment effects: Systematic review and meta-regression analysis. JAMA. 2010;303:1180-1187. doi: 10.1001/jama.2010.310
6. Kuhn JH, Becker S, Ebihara H, Geisbert TW, Johnson KM, Kawaoka Y, et al. (December 2010). "Proposal for a revised taxonomy of the family Filoviridae: Classification, names of taxa and viruses, and virus abbreviations". Archives of Virology. 155 (12): 2083–103. doi: 10.1007/s00705-010-0814-x
Competing interests: No competing interests