Waste in covid-19 researchBMJ 2020; 369 doi: https://doi.org/10.1136/bmj.m1847 (Published 12 May 2020) Cite this as: BMJ 2020;369:m1847
- Paul P Glasziou, professor of evidence based medicine,
- Sharon Sanders, assistant professor,
- Tammy Hoffmann, professor of clinical epidemiology
- Correspondence to: P Glasziou
The medical research world is responding to the covid-19 pandemic at breathtaking speed. There has been a maelstrom of global research, with mixed consequences. Positives include the greater provision of open access to covid-19 studies, some increased collaboration, expedited governance and ethics approvals of new clinical studies, and wider use of preprints. But many problems have become evident. Before the pandemic, it was estimated that up to 85% of research was wasted because of poor questions, poor study design, inefficiency of regulation and conduct, and non or poor reporting of results.1 Many of these problems are amplified in covid-19 research, with time pressures and inadequate research infrastructure contributing.
An extraordinary number of covid-19 trials have been registered since the pandemic started. The National Library of Medicine registry ClinicalTrials.gov lists 1087 covid-19 studies, and though some will provide useful information, many are too small and poorly designed to be helpful, merely adding to the covid-19 noise. Of the 145 registered trials of hydroxychloroquine, for example, 32 have a planned sample size of ≤100, 10 have no control group, and 12 are comparative but non-randomised. Outcome measures vary widely, and only 50 seem to be multicentre. Strikingly, only one provides a protocol, and even limited registry details reveal unjustified outcome switching.2
The imbalance in trial topics is worrying, in particular the paucity of trials on non-drug interventions. Despite non-drug interventions being the mainstay of current mitigation,3 we could find just two trials of masks on ClinicalTrials.gov and none examining social distancing, quarantine effect or adherence, hand hygiene, or other non-drug interventions. Covid-19 research funding mirrors this woeful imbalance. A search of Covid-19 Research Project Tracker, a live database of funded covid-19 projects, found almost no primary research of the effects of non-drug interventions on transmissibility, compared with hundreds of drug intervention projects worth at least $74m (£60m; €67m).4
Preprints have provided valuable early access to study results. Postings in MedRxiv have increased over 400% (from 586 for the last 15 weeks of 2019 to 2572 for the first 15 weeks of 2020), while views and downloads have increased 100-fold.5 Many preprints are poorly reported, however. In systematically reviewing the proportion of asymptomatic covid-19 cases, we found the sample frame of most studies was unclear, missing cases were undocumented, and “asymptomatic” was undefined. We also identified disagreements between text and tables. Many such problems could be corrected before full publication (which doesn’t always follow), but poor reporting is complicating the research appraisal and synthesis already occurring.
Access to preprints has also led to irresponsible dissemination as flawed studies are picked up by the media. The preprint of the first reported study of hydroxychloroquine on 20 March 2020—a non-randomised study of 46 patients with inappropriate analyses6—has been cited 520 times, while a larger, randomised trial of hydroxychloroquine posted on MedRxiv on 14 April showing no benefits7 has received far less attention. The unbalanced media attention to the first study has triggered a wave of what is likely to be largely unnecessary or misdirected research: 135 hydroxychloroquine studies have been registered on ClinicalTrials.gov since 20 March.
Some replication of studies is important, but unnecessary duplication of studies is wasteful. The large number of registered trials evaluating hydroxychloroquine is one illustration, but waste is also occurring in other types of research. At least five systematic reviews of face masks for people in the community have occurred in parallel.89101112
Existing research infrastructure to enable collaboration and communication is extremely limited, with system cracks made more apparent by the pace and volume of covid-19 research. Registries do not exist for most study types. When there is a global rush to research a disease, a centralised, accessible portal (hosted by the World Health Organization for example) of all ongoing research and synthesis would be invaluable.
Several important research collaborations are engaged with covid-19 research. Perhaps most notably, the Coalition for Epidemic Preparedness Innovations (CEPI), which already had funding and coordinating mechanisms for vaccines, is developing and testing eight candidate vaccines in parallel. Similarly, the UK’s multicentre trials infrastructure has enabled the RECOVERY trial of four covid-19 treatments; it has recruited more than 9000 patients from 173 centres in less than two months.13 But there are few such examples, and coordination of many important areas of pandemic research has been lacking. Given the risk that a vaccine may be ineffective, partially effective, or delayed, there is an urgent need for a body similar to CEPI that could coordinate and support neglected research into non-drug interventions such as distancing, hand hygiene, masks, tracing, and environmental modifications, which have so far been the only effective means of control.
The massive waste in research that exists is not new but has been exacerbated by the pandemic inspired rush to research. While the poor quality of covid-19 research needs attention immediately, other problems must be addressed long term, and certainly before the next pandemic.
Competing interests: We have read and understood BMJ policy on declaration of interests and have no relevant interests to declare.
Provenance and peer review: Commissioned; not externally peer reviewed.
This article is made freely available for use in accordance with BMJ's website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.https://bmj.com/coronavirus/usage