Molnupiravir’s authorisation was prematureBMJ 2022; 376 doi: https://doi.org/10.1136/bmj.o443 (Published 03 March 2022) Cite this as: BMJ 2022;376:o443
- James M Brophy, professor of medicine and epidemiology
On 1 October 2021 Merck issued a press release1 reporting an interim analysis of Move-Out, a phase 3 randomised placebo controlled trial in unvaccinated adults with confirmed SARS-Co-V infection and mild-to-moderate symptoms outside hospital. The press release stated that molnupiravir, a nucleoside analogue that inhibits viral replication by mutagenesis, reduced risk of hospital admission or death by about 50% (P=0.0012) in the 29 days after infection.
The UK Medicines and Healthcare Products Regulatory Agency (MHRA) gave molnupiravir conditional marketing authorisation on 4 November 2021, based on the interim data underlying the press release.2 On 23 December, the US Food and Drug Administration (FDA) granted emergency use authorisation after seeing the trial’s full dataset.3 These authorisations, conducted behind closed doors, lack adequate scientific rigour, inhibit further necessary evaluations, and may ultimately lead to suboptimal resource allocation decisions.
Both emergency use and conditional marketing authorisations have a lower bar for efficacy than standard approvals. The language in the FDA’s press release makes this clear, stating: “The agency has determined that it is reasonable to believe that molnupiravir may be effective.”3 This raises questions about the strength and certainty of the evidence. The same lower standards underly the FDA’s controversial accelerated approvals of aducanumab for Alzheimer’s disease and eteplirsen for muscular dystrophy.45 The revolving door between regulatory employees and leading drug companies raises further concerns about conflicts of interest and scientific objectivity in regulatory decision making.6
Dangers of truncated trials
Move-Out was stopped prematurely for reasons that aren’t completely clear. When the trial was published, the authors reported a smaller absolute difference in primary outcome (−3%, 95% confidence interval −5.9 to −0.1, P=0.043)7 than quoted in Merck’s earlier press release (−6.8%, P=0.0012). As the upper limit of this 95% confidence interval approaches zero, these published results have borderline significance, indicating that even a small number of misclassified outcomes could overturn the significance of the findings.8 Robustness is further undermined by a modified intention-to-treat analysis that excluded 25 patients after randomisation. Whether a standard intention-to-treat analysis of all randomised patients would have produced significant results remains unclear.
The weakness of the evidence supporting molnupiravir has been partially obscured by the large relative benefit reported in Merck’s original press release. This created a cognitive bias by anchoring public and scientific opinion to a belief in substantial benefit. Press releases can also set unreasonable expectations among the public and policy makers, leading to demand for immediate access to poorly evaluated drugs.
Good evidence exists that prematurely terminated trials are more likely than non-truncated trials to overestimate effect sizes. In one meta-analysis, 91 truncated trials reported greater effect sizes than 424 matched non-truncated trials,9 independent of statistical stopping rules. Smaller truncated trials with less than 200 events (Move-Out had 112 events) reported a 63% greater reduction in relative risk than similar trials that were not truncated. Interestingly, Move-Out’s truncated relative risk reduction (0.5)1 is 66% higher than the non-truncated relative risk reduction (0.3),7 emphasising the dangers of making decisions based on a single prematurely terminated trial.
The real question for any trial is whether the findings are clinically as well as statistically significant. There are two main challenges in answering that question: firstly, determining how best to define clinical significance and, secondly, the fact that null hypothesis significance testing is inadequate for evaluating the clinical relevance of an effect. As Move-Out was powered to detect an absolute reduction of 6% in hospital admissions or death, we might reasonably accept this as the minimum clinically meaningful difference. The confidence interval around the published reduction in primary outcome (−3%, 95% CI −5.9 to −0.1)7 excludes 6%, leading to questions about the clinical importance of these findings.
Molnupiravir’s safety profile is also uncertain because Move-Out was underpowered, like most clinical trials, to detect clinically important side effects. Its mutagenic mode of action could in theory encourage emergence of additional SARS-CoV-2 variants. The generalisability of the results is also unclear as the trial population was unvaccinated and recruited before the most recent variants emerged, with 70% from low and middle income countries. It is not known whether reported benefits will be maintained across different populations with differing variants, ancillary treatments, and healthcare systems. Further uncertainty arises from two more recent studies of molnupiravir reporting no clinical benefit in either outpatients or inpatients with covid-19.1011
The importance of clinical benefit is highlighted by molnupiravir’s high costs. Five days’ treatment is expected to cost about $700 (£500; €600), and with the manufacturer estimating that demand will reach 10 million courses over the next year,1 global costs could reach several billion dollars annually. The opportunity costs associated with this agent, including the potential for more financial support for mass vaccinations in low and middle income countries, merit serious reflection.
Detailed cost effectiveness studies by independent health technology assessment groups must be done to avoid repeating past mistakes. Approval of oseltamivir (Tamiflu), an antiviral for early uncomplicated influenza, resulted in worldwide governmental spending exceeding $18bn (half on stockpiling) for a drug with no benefit other than reducing duration of symptoms by less than one day.12
The evidence for incorporating molnupiravir into routine practice is fragile. Premature regulatory authorisation and guideline recommendation13 on the basis of truncated and non-replicated trial findings and without full consideration of clinical, not statistical, significance and cost effectiveness falls far short of the wise stewardship of limited healthcare resources required during a global healthcare emergency. We deserve and should demand better.
Competing interests: The BMJ has judged that there are no disqualifying financial ties to commercial companies. The author declares no other interests. Further details of The BMJ policy on financial interests are here: https://www.bmj.com/sites/default/files/attachments/resources/2016/03/16-current-bmj-education-coi-form.pdf.
Provenance and peer review: Not commissioned; externally peer reviewed.
This article is made freely available for personal use in accordance with BMJ's website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.https://bmj.com/coronavirus/usage