Drug treatments for covid-19: living systematic review and network meta-analysisBMJ 2020; 370 doi: https://doi.org/10.1136/bmj.m2980 (Published 30 July 2020) Cite this as: BMJ 2020;370:m2980
- Reed AC Siemieniuk, methodologist, internist1 *,
- Jessica J Bartoszko, methodologist1 *,
- Long Ge, methodologist2 *,
- Dena Zeraatkar, methodologist1 *,
- Ariel Izcovich, methodologist, internist3,
- Elena Kum, methodologist1,
- Hector Pardo-Hernandez, methodologist45,
- Bram Rochwerg, methodologist, critical care physician16,
- Francois Lamontagne, methodologist, critical care physician7,
- Mi Ah Han, methodologist8,
- Qin Liu, professor910,
- Arnav Agarwal, methodologist, internist111,
- Thomas Agoritsas, methodologist, internist112,
- Derek K Chu, methodologist, immunologist16,
- Rachel Couban, librarian13,
- Andrea Darzi, methodologist1,
- Tahira Devji, methodologist1,
- Bo Fang, methodologist910,
- Carmen Fang, registered nurse14,
- Signe Agnes Flottorp, senior researcher1516,
- Farid Foroutan, methodologist117,
- Diane Heels-Ansdell, statistician1,
- Kimia Honarmand, methodologist, critical care physician18,
- Liangying Hou, medical doctor candidate2,
- Xiaorong Hou, librarian19,
- Quazi Ibrahim, statistician1,
- Mark Loeb, methodologist, infectious disease physician16,
- Maura Marcucci, methodologist, internist16,
- Shelley L McLeod, methodologist, assistant professor2021,
- Sharhzad Motaghi, methodologist1,
- Srinivas Murthy, clinical associate professor, pediatric critical care, infectious diseases physician22,
- Reem A Mustafa, associate professor, nephrologist123,
- John D Neary, methodologist, internist6,
- Anila Qasim, research associate1,
- Gabriel Rada, methodologist2425,
- Irbaz Bin Riaz, methodologist, internist26,
- Behnam Sadeghirad, assistant professor113,
- Nigar Sekercioglu, assistant professor1,
- Lulu Sheng, methodologist910,
- Ashwini Sreekanta, methodologist1,
- Charlotte Switzer, methodologist1,
- Britta Tendal, methodologist27,
- Lehana Thabane, professor1,
- George Tomlinson, senior biostatistician28,
- Tari Turner, senior research fellow27,
- Per O Vandvik, methodologist, internist14,
- Robin WM Vernooij, methodologist2930,
- Andrés Viteri-García, methodologist2431,
- Ying Wang, methodologist, pharmacist1,
- Liang Yao, methodologist1,
- Zhikang Ye, methodologist, pharmacist1,
- Gordon H Guyatt, methodologist, internist16,
- Romina Brignardello-Petersen, methodologist1
- 1Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main St W, Hamilton, ON L8S 4L8, Canada
- 2Evidence Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, Gansu, China
- 3Servicio de Clinica Médica del Hospital Alemán, Buenos Aires, Argentina
- 4Iberoamerican Cochrane Centre, Sant Pau Biomedical Research Institute (IIB Sant Pau), Barcelona, Spain
- 5CIBER de Epidemiología y Salud Pública (CIBERESP), Barcelona, Spain
- 6Department of Medicine, McMaster University, Hamilton, ON, Canada
- 7Department of Medicine and Centre de recherche du CHU de Sherbrooke, Sherbrooke, Quebec, Canada
- 8Department of Preventive Medicine, College of Medicine, Chosun University, Gwangju, Republic of Korea
- 9Cochrane China Network Affiliate, Chongqing Medical University, Chongqing, China
- 10School of Public Health and Management, Chongqing Medical University, Chongqing, China
- 11Department of Medicine, University of Toronto, Toronto, ON, Canada
- 12Division of General Internal Medicine & Division of Clinical Epidemiology, University Hospitals of Geneva, Geneva, Switzerland
- 13Department of Anesthesia, McMaster University, Hamilton, ON, Canada
- 14William Osler Health Network, Toronto, ON, Canada
- 15Norwegian Institute of Public Health, Oslo, Norway
- 16Institute of Health and Society, University of Oslo, Oslo, Norway
- 17Ted Rogers Center for Heart Research, Toronto General Hospital, ON, Canada
- 18Department of Medicine, Western University, London, ON, Canada
- 19College of Medical Informatics, Chongqing Medical University, Chongqing, China
- 20Schwartz/Reisman Emergency Medicine Institute, Sinai Health, Toronto, ON, Canada
- 21Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada
- 22Department of Pediatrics, Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada
- 23Department of Medicine, University of Kansas Medical Center, Kansas City, MO, USA
- 24Epistemonikos Foundation, Santiago, Chile
- 25UC Evidence Center, Cochrane Chile Associated Center, Pontificia Universidad Católica de Chile, Santiago, Chile
- 26Hematology and Oncology, Mayo Clinic Rochester, Rochester, MN, USA
- 27School of Public Health and Preventative Medicine, Monash University, Melbourne, Australia
- 28Department of Medicine, University Health Network, Toronto, ON, Canada
- 29Department of Nephrology and Hypertension, University Medical Center Utrecht, Utrecht, Netherlands
- 30Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- 31Centro de Investigación de Salud Pública y Epidemiología Clínica (CISPEC), Facultad de Ciencias de la Salud Eugenio Espejo, Universidad UTE, Quito, Ecuador
- *Joint first authors
- Correspondence to: R Siemieniuk
- Accepted 23 July 2020
- Final version accepted 4 September 2020
Objective To compare the effects of treatments for coronavirus disease 2019 (covid-19).
Design Living systematic review and network meta-analysis.
Data sources US Centers for Disease Control and Prevention COVID-19 Research Articles Downloadable Database, which includes 25 electronic databases and six additional Chinese databases to 10 August 2020.
Study selection Randomised clinical trials in which people with suspected, probable, or confirmed covid-19 were randomised to drug treatment or to standard care or placebo. Pairs of reviewers independently screened potentially eligible articles.
Methods After duplicate data abstraction, a Bayesian network meta-analysis was conducted. Risk of bias of the included studies was assessed using a modification of the Cochrane risk of bias 2.0 tool, and the certainty of the evidence using the grading of recommendations assessment, development and evaluation (GRADE) approach. For each outcome, interventions were classified in groups from the most to the least beneficial or harmful following GRADE guidance.
Results 35 trials with 16 588 patients met inclusion criteria; 12 (24.3%) trials and 6853 (41.3%) patients are new from the previous iteration. Twenty-seven randomised controlled trials were included in the analysis performed on 29 July 2020. Compared with standard care, glucocorticoids probably reduce death (risk difference 31 fewer per 1000 patients, 95% credible interval 55 fewer to 5 fewer, moderate certainty), mechanical ventilation (28 fewer per 1000 patients, 45 fewer to 9 fewer, moderate certainty), and duration of hospitalisation (mean difference −1.0 day, −1.4 to −0.6 days moderate certainty). The impact of remdesivir on mortality, mechanical ventilation, and length of hospital stay is uncertain, but it probably reduces duration of symptoms (−2.6 days −4.3 to −0.6 days, moderate certainty) and probably does not substantially increase adverse effects leading to drug discontinuation (3 more per 1000, 7 fewer to 43 more, moderate certainty). Hydroxychloroquine may not reduce risk of death (13 more per 1000, 15 fewer to 43 more, low certainty) or mechanical ventilation (19 more per 1000, 4 fewer to 45 more, moderate certainty). The certainty in effects for all other interventions was low or very low certainty.
Conclusion Glucocorticoids probably reduce mortality and mechanical ventilation in patients with covid-19 compared with standard care, whereas hydroxychloroquine may not reduce either. The effectiveness of most interventions is uncertain because most of the randomised controlled trials so far have been small and have important limitations.
Systematic review registration This review was not registered. The protocol is included as a supplement.
Readers’ note This article is a living systematic review that will be updated to reflect emerging evidence. Updates may occur for up to two years from the date of original publication. This version is update 1 of the original article published on 30 July 2020 (BMJ 2020;370:m2980), and previous versions can be found as data supplements. When citing this paper please consider adding the update number and date of access for clarity.
As of 19 August 2020, more than 22.1 million people have been infected with severe acute respiratory syndrome coronavirus virus 2 (SARS-CoV-2), the virus responsible for coronavirus disease 2019 (covid-19); of these, 781 000 have died.1 Despite global efforts to identify effective interventions for the prevention and treatment of covid-19, which have resulted in 2100 trials completed or underway,2 evidence for effective treatment remains limited.
Faced with the pressures of a global pandemic, healthcare workers around the world are prescribing drugs off-label for which there is only very low quality evidence. The result—and this certainly seems to be the case for the well publicised example of hydroxychloroquine—might be of no benefit but of appreciable harm. Timely evidence summaries and associated guidelines could ameliorate the problem.3 Clinicians, patients, guideline bodies, and government agencies are also facing the challenges of interpreting the results from trials that are being published at a rate never encountered previously. This environment makes it necessary to produce well developed summaries that distinguish more trustworthy evidence from less trustworthy evidence.
Living systematic reviews deal with the main limitation of traditional reviews—that of providing an overview of the relevant evidence only at a specific time.4 This is crucial in the context of covid-19, in which the best evidence is constantly changing. The ability of a living network meta-analysis to present a complete, broad, and updated view of the evidence makes it the best type of evidence synthesis to inform the development of practice recommendations. Network meta-analysis, rather than pairwise meta-analysis, provides useful information about the comparative effectiveness of treatments that have not been tested head to head. The lack of such direct comparisons is certain to limit inferences in the covid-19 setting. Moreover, the incorporation of indirect evidence can strengthen evidence in comparisons that were tested head to head.5
In this living systematic review and network meta-analysis we compare the effects of drug treatments for covid-19. This review is part of the BMJ Rapid Recommendations project, a collaborative effort from the MAGIC Evidence Ecosystem Foundation (www.magicproject.org) and The BMJ.6 This living systematic review and network meta-analysis will directly inform BMJ Rapid Recommendations6 on covid-19 treatments, initiated to provide trustworthy, actionable, and living guidance to clinicians and patients soon after new and potentially practice-changing evidence becomes available. The first covid-19 BMJ Rapid Recommendation considered the role of remdesivir7 (box 1). This living network meta-analysis is the second version. The first version is available in the supplementary material.
Linked resources in this BMJ Rapid Recommendations cluster
Rochwerg B, Agarwal A, Zeng L, et al. Remdesivir for severe covid-19: a clinical practice guideline. BMJ 2020;370:m2924, doi:10.1136/bmj.m2924
Rapid Recommendation on remdesivir for covid-19
Lamontagne F, Agoritsas T, Macdonald H, et al. A living WHO guideline on drugs for covid-19. BMJ 2020;370:m3379, doi:10.1136/bmj.m3379
Living WHO BMJ Rapid Recommendations guidance on drugs for covid-19
World Health Organization. Corticosteroids for COVID-19. Living guidance 2 September 2020. https://www.who.int/publications/i/item/WHO-2019-nCoV-Corticosteroids-2020.1
Siemieniuk RAC, Bartoszko JJ, Ge L, et al. Drug treatments for covid-19: living systematic review and network meta-analysis. BMJ 2020;370:m2980, doi:10.1136/bmj.m2980
Review and network meta-analysis of all available randomised trials that assessed drug treatments for covid-19
Expanded version of the methods, processes, and results with multilayered recommendations, evidence summaries, and decision aids for use on all devices
A protocol provides the detailed methods of this systematic review, including all updates (see supplementary file). We report this living systematic review following the guidelines of the preferred reporting items for systematic reviews and meta-analyses (PRISMA) checklist for network meta-analyses.8 A living systematic review is a cumulative synthesis that is updated regularly as new evidence becomes available.9 The linked BMJ Rapid Recommendations guideline panels approved all decisions relevant to data synthesis.
We included randomised clinical trials in people with suspected, probable, or confirmed covid-19 that compared drugs for treatment against one another or against no intervention, placebo, or standard care. We included trials regardless of publication status (peer reviewed, in press, or preprint) or language. No restrictions were applied based on severity of illness or setting and we included trials of Chinese medicines if the drug comprised one or more specific molecules with a defined molecular weight dosing.
We excluded randomised controlled trials evaluating vaccination, blood products, nutrition, traditional Chinese herbal medicines that include more than one molecule or a molecule without specific molecular weighted dosing, and non-drug supportive care interventions. Trials that evaluated these interventions were identified and categorised separately.
We perform daily searches from Monday to Friday in the US Centers for Disease Control and Prevention (CDC) COVID-19 Research Articles Downloadable Database for eligible studies—the most comprehensive database of covid-19 research articles.10 The database includes 25 bibliographic and grey literature sources: Medline (Ovid and PubMed), PubMed Central, Embase, CAB Abstracts, Global Health, PsycInfo, Cochrane Library, Scopus, Academic Search Complete, Africa Wide Information, CINAHL, ProQuest Central, SciFinder, the Virtual Health Library, LitCovid, WHO covid-19 website, CDC covid-19 website, Eurosurveillance, China CDC Weekly, Homeland Security Digital Library, ClinicalTrials.gov, bioRxiv (preprints), medRxiv (preprints), chemRxiv (preprints), and SSRN (preprints).
The daily searches are designed to match the update schedule of the database and to capture eligible studies the day of or the day after publication. To identify randomised controlled trials, we filtered the results from the CDC’s database through a validated and highly sensitive machine learning model.11 We tracked preprints of randomised controlled trials until publication and updated data to match that in the peer reviewed publication when discrepant and reconciled corrections and retractions existed.
In addition, we search six Chinese databases every two weeks: Wanfang, Chinese Biomedical Literature, China National Knowledge Infrastructure, VIP, Chinese Medical Journal Net (preprints), and ChinaXiv (preprints). We adapted the search terms for covid-19 developed by the CDC to the Chinese language. For the Chinese literature search, we also included search terms for randomised trials. The supplementary file includes the Chinese literature search strategy.
We monitor living evidence retrieval services on an ongoing basis. These included the Living Overview of the Evidence (L-OVE) COVID-19 Repository by the Epistemonikos Foundation12 and the Systematic and Living Map on COVID-19 Evidence by the Norwegian Institute of Public Health, in collaboration with the Cochrane Canada Centre at McMaster University.13
We searched all English information sources from 1 December 2019 to 10 August 2020, and the Chinese literature from conception of the databases to 10 August 2020.
Using a systematic review software, Covidence,14 pairs of reviewers, following training and calibration exercises, independently screened all titles and abstracts, followed by full texts of trials that were identified as potentially eligible. A third reviewer adjudicated conflicts.
For each eligible trial, pairs of reviewers, following training and calibration exercises, extracted data independently using a standardised, pilot tested data extraction form. Reviewers collected information on trial characteristics (trial registration, publication status, study status, design), patient characteristics (country, age, sex, smoking habits, comorbidities, setting and type of care, and severity of covid-19 symptoms for studies of treatment), and outcomes of interest (means or medians and measures of variability for continuous outcomes and the number of participants analysed and the number of participants who experienced an event for dichotomous outcomes). Reviewers resolved discrepancies by discussion and, when necessary, with adjudication by a third party. We updated the data collected from included preprints as soon as the peer review publication became available.
Outcomes of interest were selected based on importance to patients and were informed by clinical expertise in the systematic review team and in the linked guideline panel responsible for the BMJ Rapid Recommendations.7 The panel includes unconflicted clinical and methodology experts, recruited to ensure global representation, and patient-partners. All panel members rated outcomes from 1 to 9 based on importance to individual patients (9 being most important), and we included any outcome rated 7 or higher by any panel member. Selected outcomes included mortality (closest to 90 days), mechanical ventilation (total number of patients, over 90 days), adverse events leading to discontinuation (within 28 days), viral clearance (closest to 7 days, 3 days either way), admission to hospital, duration of hospital stay, intensive care unit (ICU) length of stay, duration of mechanical ventilation, time to symptom resolution or clinical improvement, and time to viral clearance. Viral clearance at seven days and time to viral clearance were included because both may be surrogates for transmissibility, although this is uncertain.15
Because of the inconsistent reporting observed across trials, we used a hierarchy for the outcome mechanical ventilation in which we considered the total number of patients who received ventilation over the study, if available, and the number of patients ventilated at the time point at which most of the patients were mechanically ventilated, if that is the only way in which this outcome was reported.
Risk of bias within individual studies
For each eligible trial, reviewers, following training and calibration exercises, used a revision of the Cochrane tool for assessing risk of bias in randomised trials (RoB 2.0)16 to rate trials as either at i) low risk of bias, ii) some concerns—probably low risk of bias, iii) some concerns—probably high risk of bias, or iv) high risk of bias, across the following domains: bias arising from the randomisation process; bias owing to departures from the intended intervention; bias from missing outcome data; bias in measurement of the outcome; bias in selection of the reported results, including deviations from the registered protocol; bias due to competing risks; and bias arising from early termination for benefit. We rated trials at high risk of bias overall if one or more domains were rated as some concerns—probably high risk of bias or as high risk of bias and as low risk of bias if all domains were rated as some concerns—probably low risk of bias or low risk of bias. Reviewers resolved discrepancies by discussion and, when not possible, with adjudication by a third party.
We conducted the network meta-analysis using a bayesian framework.17 In this report, we conducted a network meta-analysis of drug treatments for covid-19 that included all patients, regardless of severity of disease.
We summarised the effect of interventions on dichotomous outcomes using the odds ratio and corresponding 95% credible interval. For continuous outcomes, we used the mean difference and corresponding 95% credible interval in days for ICU length of stay, length of hospital stay, and duration of mechanical ventilation because we expected similar durations across randomised controlled trials. For time to symptom resolution and time to viral clearance, we first performed the analyses using the relative effect measure ratio of means and corresponding 95% credible interval before calculating the mean difference in days because we expected substantial variation between studies.18
Treatments were grouped into common nodes based on molecule and not on dose or duration. For intervention arms with more than one drug, we created a separate node and included drugs from the same class within the same node. Chloroquine and hydroxychloroquine were included in the same node for covid-19 specific effects and separated for disease independent adverse effects. We drew network plots using the networkplot command of Stata version 15.1 (StataCorp, College Station, TX), with thickness of lines between nodes and size of the nodes based on the inverse of the variance of the direct comparison.19
For most outcomes, we conducted network meta-analyses and pairwise meta-analyses using a bayesian framework with the same priors for the variance and effect parameters.17 We had initially planned to perform random effects network meta-analyses for all outcome; however, we decided to present fixed effects rather than random effects as the primary analytic method for several outcomes: mortality, mechanical ventilation, and time to symptom resolution. We conducted fixed effect network meta-analysis for these outcomes because i) for almost all comparisons, there were few RCTs and the heterogeneity parameter can be unstable in these circumstances and ii) comparisons including hydroxychloroquine and glucocorticoids were dominated by a single large trial (RECOVERY),2021 and iii) there were only two trials that examined remdesivir.2223 Random effects meta-analysis results are presented in full in the Supplementary material and highlighted in this document where they substantially differ from fixed effects. We used a plausible prior for variance parameter and a uniform prior for the effect parameter suggested in a previous study based on empirical data.24 For all analyses, we used three Markov chains with 100 000 iterations after an initial burn-in of 10 000 and a thinning of 10. We used node splitting models to assess local incoherence and to obtain indirect estimates.25 All network meta-analyses were performed using the gemtc package of R version 3.6.3 (RStudio, Boston, MA)26 and all pairwise meta-analyses using the bayesmeta package.17
In the first iteration of this living network meta-analysis, some treatment nodes with few total participants and few total events resulted in highly implausible and extremely imprecise effect estimates. We therefore decided to include only treatments that included at least 100 patients or had at least 20 events, based on our impression of the minimum number of patients/events to possibly provide meaningful results.
Certainty of the evidence
We assessed the certainty of evidence using the grading of recommendations assessment, development and evaluation (GRADE) approach for network meta-analysis.52728 Two people with experience in using GRADE rated each domain for each comparison separately and resolved discrepancies by consensus. We rated the certainty for each comparison and outcome as high, moderate, low, or very low, based on considerations of risk of bias, inconsistency, indirectness, publication bias, intransitivity, incoherence (difference between direct and indirect effects), and imprecision.28 Judgments of imprecision for this systematic review were made using a minimally contextualised approach, with a null effect as the threshold of importance.29 The minimally contextualised approach considers only whether credible intervals include the null effect and thus does not consider whether plausible effects, captured by credible intervals, include both important and trivial effects.29 To evaluate certainty of no benefit (or no effect), we used a 2% risk difference threshold of the 95% credible interval for mortality and mechanical ventilation. In other words, if the entire 95% credible interval was within 2% of the null effect, we would not rate down for imprecision. We decided on this preliminary threshold based on a survey of the authors. In future updates, it will be guided by a survey of patients and guideline panellists. We created GRADE evidence summaries (Summary of Findings tables) in the MAGIC Authoring and publication platform (www.magicapp.org) to provide user friendly formats for clinicians and patients and to allow re-use in the context of clinical practice guidelines for covid-19.
Interpretation of results
To facilitate interpretation of the results, we calculated absolute effects for outcomes in which the summary measure was an odds ratio or ratio of means. For the outcomes mortality and mechanical ventilation, we used baseline risks from the International Severe Acute Respiratory and Emerging Infection COVID-19 database.30 For all other outcomes, we used the median from all studies in which participants received standard of care to calculate the baseline risk for each outcome, with each study weighed equally. We calculated absolute effects using the transitive risks model31 using R2jags package in R.32
For each outcome, we classified treatments in groups from the most to the least effective using the minimally contextualised framework, which focuses on the treatment effect estimates and the certainty of the evidence.33
Subgroup and sensitivity analysis
We planned to perform subgroup analyses of preprints versus peer reviewed studies and high versus low risk of bias. In the future, we may perform additional subgroup analyses if directed by the linked independent BMJ Rapid Recommendation guideline panels; in this case there was no such direction. The RECOVERY trial published comparisons for glucocorticoids versus standard care and hydroxychloroquine versus standard care separately, with standard care groups that mostly overlapped.3435 For the primary analysis, we considered RECOVERY a three-arm trial because most of the patients randomised to the standard care arm were the same and the outcome event rates in the standard care arms were almost identical. In the analyses with RECOVERY as a single three-arm trial, we used the standard care group with more patients.21 We performed a sensitivity analysis that considered RECOVERY two independent two-arm trials.
Patient and public involvement
Patients were involved in the interpretation of results and the generation of parallel recommendations, as part of the BMJ Rapid Recommendations initiative.
After screening 8877 titles and abstracts and 154 full texts, 41 unique randomised controlled trials were identified that evaluated drug treatments as of 10 August 2020 (fig 1).22233637383940414243444546474849505152535455 Searches of living evidence retrieval services identified one additional eligible randomised controlled trial.56 Twenty-seven randomised controlled trials have been published in peer reviewed journals, and 14 only as preprints. Most of the trials were registered (37/41; 90%), published in English (37/41; 90%), and evaluated treatment in patients admitted to hospital with covid-19 (36/41; 88%). Just over one half of the trials were conducted in China (22/41; 54%). Of the 41 included drug trials, 10 evaluated treatment against active comparators, 24 evaluated treatment against standard care or placebo, and two evaluated different durations or doses of the same treatment. These analyses were performed on 29 July 2020 and include 27 randomised controlled trials.2223343940414243444647484950515253545758596061626364 Table 1 presents the characteristics of the included studies. Additional study characteristics, outcome data, and risk of bias assessments for each study are available in the supplementary file.
Several randomised controlled trials were not included in the analysis: two trials that evaluated different durations of the same drug, because both arms would have been classified within the same treatment node3745; one trial that compared lincomycin with azithromycin,67 because neither arm was connected to the network; two trials with insufficient data7077; and three trials that reported no outcomes of interest.687374 Table 2 describes the randomised controlled trials that were identified after the data analysis and that will be included in the next update.
Of the randomised controlled trials included in the analyses, three did not have publicly accessible protocols or registrations.677376 Of the trials with publicly accessible protocols or registrations, 22 reported results for one or more of our outcomes of interest that were not prespecified in protocols or registrations. No other discrepancies between the reporting of our outcomes of interest in trial reports and protocols or registrations were noted. One trial did not report outcomes in the groups as randomised; the authors shared outcome data with us in the groups as randomised.53
Eight studies were initially posted as preprints and subsequently published after peer review.353742444951525563646669717278 In one study, mortality was not reported in the preprint but was reported in the peer reviewed paper.4972 A trial that compared dexamethasone with standard care was published as a preprint52 and has since been published with additional events after peer review.35 Another trial that compared ribavirin, lopinavir-ritonavir, and the combination was included in our data analysis as a pre-print,42 but has since reported adverse events leading to discontinuation as an additional outcome in the peer reviewed publication.63 We will include this new outcome reported by the study in the next update. No substantive differences were found between the preprint and peer reviewed publications for the other five studies.
All analyses reached convergence based on trace plots and a Brooks-Gelman-Rubin statistic less than 1.05, except comparisons including favipiravir and umifenovir for mortality because no patients randomised to either of these drugs died.
Risk of bias in included studies
The supplementary material presents the assessment of risk of bias of the included studies for each outcome. Five studies were judged at low risk of bias in all domains.2223374360 All other studies had probably high or high risk of bias in the domains of randomisation or deviation from the intended interventions.
Effects of the interventions
The supplementary material presents the network plots depicting the interventions included in the network meta-analysis of each outcome. Figure 2 presents a summary of the effects of the interventions on the outcomes. The supplementary file also presents detailed relative and absolute effect estimates and certainty of the evidence for all comparisons and outcomes. We did not detect statistical incoherence in any of the network meta-analyses.
Twenty-three randomised controlled trials including 11 620 participants222334394041424446474849505253545557585960616263666972758485 reported mortality. The treatment nodes included in the network meta-analysis were favipiravir, glucocorticoids, hydroxychloroquine, hydroxychloroquine plus azithromycin, lopinavir-ritonavir, remdesivir, umifenovir, and standard care. Fixed effects network meta-analysis showed that fewer patients randomised to glucocorticoids (odds ratio 0.87, 95% credible interval 0.77 to 0.98; risk difference 31 fewer per 1000, 95% credible interval 55 fewer to 5 fewer; moderate certainty) and remdesivir (odds ratio 0.64, 0.43 to 0.94; risk difference 91 fewer per 1000, 154 fewer to 14 fewer; very low certainty) died than those randomised to standard care (fig 2). Patients randomised to hydroxychloroquine did not have a lower risk of death than those randomised to standard care (odds ratio 1.06, 0.93 to 1.21; risk difference 13 more per 1000, 16 fewer to 43 more; low certainty of no benefit). 95% credible intervals included both substantial benefit and harm for hydroxychloroquine plus azithromycin, and lopinavir-ritonavir (both very low certainty). Random effects network meta-analysis led to substantially wider credible intervals for all treatments; compared with standard care, glucocorticoids (odds ratio 0.89, 0.64 to 1.40), hydroxychloroquine (odds ratio 1.08, 0.77 to 1.60), and remdesivir (odds ratio 0.66, 0.41 to 1.09) (see supplementary material). The effect estimates were similar regardless of whether RECOVERY3435 was considered a single three-arm trial or two two-arm trials (see supplementary material).
Twelve randomised controlled trials including 9083 participants22233435394044464752535862668485 reported mechanical ventilation. The treatment nodes included in the network meta-analysis were glucocorticoids, hydroxychloroquine, hydroxychloroquine plus azithromycin, remdesivir, and standard care (fig 2). Compared with standard care, glucocorticoids probably reduce risk of mechanical ventilation (odds ratio 0.73, 0.58 to 0.92; risk difference 28 fewer per 1000, 45 fewer to 9 fewer; moderate certainty for risk of bias), while hydroxychloroquine probably does not reduce risk of mechanical ventilation (odds ratio 1.19, 0.96 to 1.47; risk difference 19 more per 1000, 4 fewer to 46 more; moderate certainty for risk of bias). Evidence for was less certain for remdesivir (odds ratio 0.78, 0.57 to 1.08; risk difference 23 fewer per 1000, 47 fewer to 8 more; low certainty) and hydroxychloroquine plus azithromycin (odds ratio 1.60, 0.86 to 2.93; risk difference 57 more per 1000, 15 fewer to 162 more; low certainty). Random effects network meta-analysis led to substantially wider credible intervals for all treatments; compared with standard care, glucocorticoids (odds ratio 0.78, 0.48 to 1.56), hydroxychloroquine (odds ratio 1.23, 0.76 to 2.18), hydroxychloroquine plus azithromycin (odds ratio 1.65, 0.72 to 3.88), and remdesivir (odds ratio 0.77, 0.43 to 1.36) (see supplementary material). The effects were similar regardless of whether RECOVERY was considered a single three-arm trial or two two-arm trials (see supplementary material).
Adverse events leading to discontinuation
Thirteen randomised controlled trials including 1938 participants222343444648495051545557586672757685 reported adverse effects leading to discontinuation of the study drug. The treatment nodes included in the network meta-analysis were hydroxychloroquine, remdesivir, and standard care. Moderate certainty evidence showed that remdesivir did not result in a substantial increase in adverse effects leading to drug discontinuation compared with standard care (odds ratio 1.27, 0.51 to 4.07; risk difference 4 more per 1000, 7 fewer to 43 more). Certainty in evidence for hydroxychloroquine was very low (fig 2).
Viral clearance at 7 days (3 days either way)
Eleven randomised controlled trials including 876 participants2339424749505154555657637285 measured viral clearance with polymerase chain reaction cut-off points. The treatment nodes included in the network meta-analysis were hydroxychloroquine, lopinavir-ritonavir, remdesivir, and standard care. We did not find any convincing evidence that any of the interventions increased the rate of viral clearance (fig 2). The certainty of the evidence was low for remdesivir compared with standard care, and very low for all other comparisons.
Admission to hospital
Two randomised controlled trials enrolling 551 participants5960 reported admission to hospital in patients who were outpatients at baseline. One study of hydroxychloroquine versus placebo was included.60 There were too few events to make any inferences with (odds ratio 0.52, 0.16 to 1.68; risk difference 19 fewer per 1000, 43 fewer to 26 more; low certainty) (fig 2).
Duration of hospital stay
Thirteen randomised controlled trials including 9631 participants2334353940424446525456575862636685 reported duration of hospital stay. The treatment nodes included in the network meta-analysis were glucocorticoids, hydroxychloroquine, hydroxychloroquine plus azithromycin, lopinavir-ritonavir, remdesivir, and standard care. Compared with standard care, duration of hospitalisation was shorter in patients who received glucocorticoids (mean difference −0.99 days, −1.36 to −0.64; moderate certainty) and lopinavir-ritonavir (mean difference −1.33 days, −2.38 to −0.29; low certainty). There was no evidence that hydroxychloroquine (very low certainty), hydroxychloroquine plus azithromycin (low certainty), or remdesivir (low certainty) decrease length of stay (fig 2).
ICU length of stay
Two randomised controlled trials enrolling 291 total participants reported length of ICU stay.3944 Neither study randomised at least 100 patients to receive the active drug, therefore no analyses were conducted for this outcome.
Duration of mechanical ventilation
Three randomised controlled trials enrolling 528 total participants233944 reported duration of mechanical ventilation. No active treatment node contained information on at least 100 patients, therefore no analyses were conducted for this outcome.
Time to symptom resolution
Fourteen randomised controlled trials including 2282 participants2223394041424344464749545663667285 reported time to symptom resolution. At least 100 patients received hydroxychloroquine, lopinavir-ritonavir, remdesivir, and standard care. Patients who received remdesivir (mean difference −2.62 days, 95% credible interval −4.30 to −0.56, moderate certainty), hydroxychloroquine (−4.68 days, −5.98 to −2.99, low certainty), and lopinavir-ritonavir (−1.12 days, −2.06 to −0.37, low certainty) had a shorter symptom duration than patients who received standard care.
This living systematic review and network meta-analysis provides a comprehensive overview of the evidence for drug treatments of covid-19 up to 29 July 2020 and a comprehensive list of drug trials to 10 August 2020. The certainty of the evidence for most of the comparisons was very low. Glucocorticoids probably reduce the risk of death and mechanical ventilation, and duration of hospitalisation, results driven almost entirely by the RECOVERY trial.52 Moderate certainty exists that remdesivir reduces both time to symptom resolution and duration of mechanical ventilation, but it remains uncertain whether remdesivir has any effect on mortality and other outcomes important to patients. Remdesivir was the only intervention where all the data came from randomised controlled trials sponsored by a pharmaceutical company. Direct evidence from randomised controlled trials in patients with covid-19 has so far provided little definitive evidence about adverse effects for most interventions.
Compared with the first iteration, there are several important updates (box 2). This update includes several more randomised trials comparing hydroxychloroquine with standard care/placebo. The evidence currently suggests that hydroxychloroquine may not reduce the risk of death, mechanical ventilation, or duration of hospitalisation. Patients who received hydroxychloroquine had a shorter time to symptom resolution than patients who received standard care, however this is very uncertain, and this outcome was not measured in the larger trials that did not show any benefit on related outcomes. Further, data from this review suggests that the degree to which hydroxychloroquine causes adverse effects is uncertain and it includes the possibility of substantial harm.
Summary of changes since last iteration
Twelve additional randomised trials (6853 participants)
Hydroxychloroquine with azithromycin and favipiravir are new interventions included in the analyses, but certainty is very low for the effects of these interventions
6460 participants were enrolled in nine additional randomised trials that included hydroxychloroquine
Increased confidence that hydroxychloroquine may be not reduce mortality (low certainty), mechanical ventilation (moderate certainty), or admission to hospital (low certainty)
New evidence that glucocorticoids probably reduce duration of hospital stay (moderate certainty)
Evidence for other interventions is similar to the previous version
Strengths and limitations of this review
Our search strategy and eligibility criteria were comprehensive, without restrictions on language of publication or publication status. To ensure expertise in all areas, our team is composed of clinical and methods experts who have undergone training and calibration exercises for all stages of the review process. To minimise problems with counterintuitive results, we anticipated challenges that arise in network meta-analysis when data are sparse.20 We assessed the certainty of the evidence using the GRADE approach and interpreted the results considering absolute effects. Many of the results for comparisons with sparse data were uninformative and were sometimes implausible. For that reason, we decided to report evidence on treatments for which at least 100 people were randomised or for which there were at least 20 events. In the future, when more data from more treatments are available, our classification of interventions from the most to the least effective will facilitate clear interpretation of results.
The main limitation of the systematic review is the very low quality of the evidence as a result of the sparse data currently available. As the many ongoing trials are completed, we anticipate that the effect estimates will become both plausible and informative as the quality of the evidence increases. Only five studies were judged to be at low risk of bias.2223374360 The most common limitation was lack of blinding, including in the largest trials.
Another limitation of this living systematic review and network meta-analysis is the limited quality of reporting. For some outcomes, the method in which the researchers measured and reported outcomes proved inconsistent across studies. This led the team to propose a hierarchy for the outcome mechanical ventilation, as described in the methods.
The living nature of our systematic review and network meta-analysis could conceivably (at least temporarily) amplify publication bias, because studies with promising results are more likely to be published and are published sooner than studies with negative results. The inclusion of preprints, many of which have negative results, might mediate this risk. Industry sponsored trials such as those for remdesivir and other patented drugs could be particularly at risk of publication bias, and positive results for these drugs might require more cautious interpretation than generic drugs tested in randomised controlled trials independent of industry influence. However, the inclusion of preprints in our network meta-analysis might introduce bias from simple errors and the reporting limitations of preprints. We include preprints because of the urgent need for information and because so many of the studies on covid-19 are published first as preprints.
For comparisons with sufficient data, the primary limitation of the evidence is lack of blinding, which might introduce bias through differences in co-interventions between randomisation groups. We chose to consider the treatment arms that did not receive an active experimental drug (ie, placebo or standard care) within the same node: it is possible that the unblinded standard care groups received systematically different co-interventions than groups randomised to receive a placebo. Direct comparisons in which the evidence is dominated by unblinded studies were rated down, consistent with GRADE, for risk of bias and that is reflected in the rating of the quality of evidence from the network estimate.86 It is also possible that study level meta-analysis might not detect important subgroup modification that would otherwise be detected within trial comparisons.87 For example, the RECOVERY trial suggested that patients with more severe disease might obtain a greater benefit from dexamethasone than patients with less severe disease.52
Our living systematic review and network meta-analysis will continue to inform the development of the BMJ Rapid Recommendations.67 An important difference in the methods for assessing the certainty of the evidence does, however, exist between the two. In this living systematic review and network meta-analysis, we use a minimally contextualised approach for rating the certainty of the evidence, whereas BMJ Rapid Recommendations uses a fully contextualised approach in which the thresholds of importance of magnitudes of effects depend on all other outcomes and factors involved in the decision.29 The contextualisation explains potential differences in the certainty of the evidence between the two. The limitations of potentially misleading results when the network is sparse, and the desirability of focusing on direct estimates from larger studies when this is the case, explain differences in the details of the estimates of effect in this network meta-analysis and in the associated guidelines for remdesivir.7
To date, we are aware of two other similar efforts to ours.8889 We decided to proceed independently to ensure that the results fully inform clinical decision making for the associated living guidance in BMJ Rapid Recommendations.6 We also include a more comprehensive search for the evidence and several differences in analytical methods, which we believe are best suited for this process. It is also important to evaluate the reproducibility and replicability of results from different scientific approaches.
We will periodically update this living systematic review and network meta-analysis. We expect data from several new large randomised trials that examined glucocorticoids, remdesivir, lopinavir and ritonavir, and hydroxychloroquine to be publicly available soon. The changes from each version will be highlighted for readers and the most updated version will be the one available in the publication platform. Previous versions will be archived in the supplementary material. This living systematic review and network meta-analysis will also be accompanied by an interactive infographic and a website for users to access the most updated results in a user-friendly format (magicapp.org).
Evidence from this living systematic review and network meta-analysis suggests that glucocorticoids probably reduce mortality and mechanical ventilation in patients with severe covid-19. Remdesivir probably reduces time to symptom resolution, but whether it has an impact on other patient-important outcomes such as mortality remains uncertain. Hydroxychloroquine may not reduce mortality or mechanical ventilation, and it seems unlikely to have any other benefits. The effects of most drug interventions are currently highly uncertain, and no definitive evidence exists that other interventions result in important benefits and harms for any outcomes.
What is already known on this topic
Despite huge efforts to identify effective drug interventions for coronavirus disease 2019 (covid-19), evidence for effective treatment remains limited
What this study adds
This living systematic review and network meta-analysis provides a comprehensive overview and assessment of the evidence published as of 29 July 2020 and will be updated periodically
The certainty of the evidence for most interventions tested thus far is low or very low
In patients with severe covid-19, glucocorticoids probably decrease mortality, mechanical ventilation, and duration of hospitalisation, while hydroxychloroquine may not reduce any of these
We thank Kevin Cheung and Paul Alexander (who was an author in the previous version of this review) for input and early contributions.
Contributors: RACS, JJB, LG, and DZ contributed equally to the systematic review and are joint first authors. RACS, JJB, DZ, LG, and RB-P were the core team leading the systematic review. JJB, RC, SAF, RWMV, SM, YW, ZY, IR, AD, TD, AI, AQ, CS, LY, FF, QL, XH, LS, BF, and AV-G identified and selected the studies. DZ, EK, NS, RWMV, AA, YW, KH, HP-H, MAH, CF, SLM, QL, AS, AQ, LY, and FF collected the data. LG, BS, LH, QI, DH-A, GHG, GT, and LT analysed the data. RB-P, HPH, AI, RAM, TD, NS, and DC assessed the certainty of the evidence. SLM, FL, BR, TA, POV, GHG, MM, JDN, ML, TT, BT, FF, and GR provided advice at different stages. RACS, RB-P, and GHG drafted the manuscript. All authors approved the final version of the manuscript. RACS is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funder: This study was supported by the Canadian Institutes of Health Research (grant CIHR-IRSC:0579001321).
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Canadian Institutes of Health Research; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not applicable. All the work was developed using published data.
Data sharing: No additional data available.
RACS affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
Dissemination to participants and related patient and public communities: The infographic and MAGICapp decision aids (available at www.magicapp.org/) were created to facilitate conversations between healthcare providers and patients or their surrogates. The MAGICapp decision aids were co-created with people who have lived experience of covid-19.
Provenance and peer review: Not commissioned; externally peer reviewed.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.