Intended for healthcare professionals

CCBYNC Open access
Research

Drug treatments for covid-19: living systematic review and network meta-analysis

BMJ 2020; 370 doi: https://doi.org/10.1136/bmj.m2980 (Published 30 July 2020) Cite this as: BMJ 2020;370:m2980
  1. Reed AC Siemieniuk, methodologist, internist1 *,
  2. Jessica J Bartoszko, methodologist1 *,
  3. Long Ge, methodologist2 *,
  4. Dena Zeraatkar, methodologist1 *,
  5. Ariel Izcovich, methodologist, internist3,
  6. Hector Pardo-Hernandez, methodologist45,
  7. Bram Rochwerg, methodologist, critical care physician16,
  8. Francois Lamontagne, methodologist, critical care physician7,
  9. Mi Ah Han, methodologist8,
  10. Elena Kum, methodologist1,
  11. Qin Liu, professor910,
  12. Arnav Agarwal, methodologist, internist111,
  13. Thomas Agoritsas, methodologist, internist112,
  14. Paul Alexander, methodologist, assistant professor1,
  15. Derek K Chu, methodologist, immunologist16,
  16. Rachel Couban, librarian13,
  17. Andrea Darzi, methodologist1,
  18. Tahira Devji, methodologist1,
  19. Bo Fang, methodologist910,
  20. Carmen Fang, registered nurse14,
  21. Signe Agnes Flottorp, senior researcher1516,
  22. Farid Foroutan, methodologist117,
  23. Diane Heels-Ansdell, statistician1,
  24. Kimia Honarmand, methodologist, critical care physician3,
  25. Liangying Hou, medical doctor candidate2,
  26. Xiaorong Hou, librarian18,
  27. Quazi Ibrahim, statistician1,
  28. Mark Loeb, methodologist, infectious disease physician16,
  29. Maura Marcucci, methodologist, internist16,
  30. Shelley L McLeod, methodologist, assistant professor1920,
  31. Sharhzad Motaghi, methodologist1,
  32. Srinivas Murthy, clinical associate professor, pediatric critical care, infectious diseases physician21,
  33. Reem A Mustafa, associate professor, nephrologist122,
  34. John D Neary, methodologist, internist3,
  35. Anila Qasim, research associate1,
  36. Gabriel Rada, methodologist2324,
  37. Irbaz Bin Riaz, methodologist, internist25,
  38. Behnam Sadeghirad, assistant professor113,
  39. Nigar Sekercioglu, assistant professor1,
  40. Lulu Sheng, methodologist910,
  41. Charlotte Switzer, methodologist1,
  42. Britta Tendal, methodologist26,
  43. Lehana Thabane, professor1,
  44. George Tomlinson, senior biostatistician27,
  45. Tari Turner, senior research fellow26,
  46. Per O Vandvik, methodologist, internist14,
  47. Robin WM Vernooij, methodologist2829,
  48. Andrés Viteri-García, methodologist2330,
  49. Ying Wang, methodologist, pharmacist1,
  50. Liang Yao, methodologist1,
  51. Zhikang Ye, methodologist, pharmacist1,
  52. Gordon H Guyatt, methodologist, internist16,
  53. Romina Brignardello-Petersen, methodologist1
  1. 1Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main St W, Hamilton, ON L8S 4L8, Canada
  2. 2Evidence Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, Gansu, China
  3. 3Servicio de Clinica Médica del Hospital Alemán, Buenos Aires, Argentina
  4. 4Iberoamerican Cochrane Centre, Sant Pau Biomedical Research Institute (IIB Sant Pau), Barcelona, Spain
  5. 5CIBER de Epidemiología y Salud Pública (CIBERESP), Barcelona, Spain
  6. 6Department of Medicine, McMaster University, Hamilton, ON, Canada
  7. 7Department of Medicine and Centre de recherche du CHU de Sherbrooke, Sherbrooke, Quebec, Canada
  8. 8Department of Preventive Medicine, College of Medicine, Chosun University, Gwangju, Republic of Korea
  9. 9Cochrane China Network Affiliate, Chongqing Medical University, Chongqing, China
  10. 10School of Public Health and Management, Chongqing Medical University, Chongqing, China
  11. 11Department of Medicine, University of Toronto, Toronto, ON, Canada
  12. 12Division of General Internal Medicine & Division of Clinical Epidemiology, University Hospitals of Geneva, Geneva, Switzerland
  13. 13Department of Anesthesia, McMaster University, Hamilton, ON, Canada
  14. 14William Osler Health Network, Toronto, ON, Canada
  15. 15Norwegian Institute of Public Health, Oslo, Norway
  16. 16Institute of Health and Society, University of Oslo, Oslo, Norway
  17. 17Ted Rogers Center for Heart Research, Toronto General Hospital, ON, Canada
  18. 18College of Medical Informatics, Chongqing Medical University, Chongqing, China
  19. 19Schwartz/Reisman Emergency Medicine Institute, Sinai Health, Toronto, ON, Canada
  20. 20Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada
  21. 21Department of Pediatrics, Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada
  22. 22Department of Medicine, University of Kansas Medical Center, Kansas City, MO, USA
  23. 23Epistemonikos Foundation, Santiago, Chile
  24. 24UC Evidence Center, Cochrane Chile Associated Center, Pontificia Universidad Católica de Chile, Santiago, Chile
  25. 25Hematology and Oncology, Mayo Clinic Rochester, Rochester, MN, USA
  26. 26School of Public Health and Preventative Medicine, Monash University, Melbourne, Australia
  27. 27Department of Medicine, University Health Network, Toronto, ON, Canada
  28. 28Department of Nephrology and Hypertension, University Medical Center Utrecht, Utrecht, Netherlands
  29. 29Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
  30. 30Centro de Investigación de Salud Pública y Epidemiología Clínica (CISPEC), Facultad de Ciencias de la Salud Eugenio Espejo, Universidad UTE, Quito, Ecuador
  31. *Joint first authors
  1. Correspondence to: R Siemieniuk reed.siemieniuk{at}medportal.ca
  • Accepted 23 July 2020

Abstract

Objective To compare the effects of treatments for coronavirus disease 2019 (covid-19).

Design Living systematic review and network meta-analysis.

Data sources US Centers for Disease Control and Prevention COVID-19 Research Articles Downloadable Database, which includes 25 electronic databases and six additional Chinese databases to 20 July 2020.

Study selection Randomised clinical trials in which people with suspected, probable, or confirmed covid-19 were randomised to drug treatment or to standard care or placebo. Pairs of reviewers independently screened potentially eligible articles.

Methods After duplicate data abstraction, a bayesian random effects network meta-analysis was conducted. Risk of bias of the included studies was assessed using a modification of the Cochrane risk of bias 2.0 tool, and the certainty of the evidence using the grading of recommendations assessment, development and evaluation (GRADE) approach. For each outcome, interventions were classified in groups from the most to the least beneficial or harmful following GRADE guidance.

Results 23 randomised controlled trials were included in the analysis performed on 26 June 2020. The certainty of the evidence for most comparisons was very low because of risk of bias (lack of blinding) and serious imprecision. Glucocorticoids were the only intervention with evidence for a reduction in death compared with standard care (risk difference 37 fewer per 1000 patients, 95% credible interval 63 fewer to 11 fewer, moderate certainty) and mechanical ventilation (31 fewer per 1000 patients, 47 fewer to 9 fewer, moderate certainty). These estimates are based on direct evidence; network estimates for glucocorticoids compared with standard care were less precise because of network heterogeneity. Three drugs might reduce symptom duration compared with standard care: hydroxychloroquine (mean difference −4.5 days, low certainty), remdesivir (−2.6 days, moderate certainty), and lopinavir-ritonavir (−1.2 days, low certainty). Hydroxychloroquine might increase the risk of adverse events compared with the other interventions, and remdesivir probably does not substantially increase the risk of adverse effects leading to drug discontinuation. No other interventions included enough patients to meaningfully interpret adverse effects leading to drug discontinuation.

Conclusion Glucocorticoids probably reduce mortality and mechanical ventilation in patients with covid-19 compared with standard care. The effectiveness of most interventions is uncertain because most of the randomised controlled trials so far have been small and have important study limitations.

Systematic review registration This review was not registered. The protocol is included as a supplement.

Readers’ note This article is a living systematic review that will be updated to reflect emerging evidence. Updates may occur for up to two years from the date of original publication.

Introduction

As of 24 July 2020, more than 15.6 million people have been infected with severe acute respiratory syndrome coronavirus virus 2 (SARS-CoV-2), the virus responsible for coronavirus disease 2019 (covid-19); of these, 636 000 have died.1 Despite huge efforts to identify effective interventions for the prevention and treatment of covid-19, which have resulted in almost 1800 trials completed or underway,2 evidence for effective treatment remains limited.

Faced with the pressures of a global pandemic, healthcare workers around the world are prescribing drugs off-label for which there is only very low quality evidence. The result—and this certainly seems to be the case for the well publicised example of hydroxychloroquine—might be of no benefit but of appreciable harm. Timely evidence summaries and associated guidelines could ameliorate the problem.3 Clinicians, patients, guideline bodies, and government agencies are also facing the challenges of interpreting the results from trials that are being published at a rate never encountered previously. This environment makes it necessary to produce well developed summaries that distinguish more trustworthy evidence from less trustworthy evidence.

Living systematic reviews and network meta-analyses deal with the main limitation of traditional reviews—that of providing an overview of the relevant evidence only at a specific time.4 This is crucial in the context of covid-19, in which the best evidence is constantly changing. The ability of a living network meta-analysis to present a complete, broad, and updated view of the evidence makes it ideal to inform the development of practice recommendations. Network meta-analysis, rather than pairwise meta-analysis, provides useful information about the comparative effectiveness of treatments that have not been tested head to head. The lack of such direct comparisons is certain to limit inferences in the covid-19 setting. Moreover, the incorporation of indirect evidence can strengthen evidence in comparisons that were tested head to head.5

In this living systematic review and network meta-analysis we compare the effects of drug treatments for covid-19. This review is part of the BMJ Rapid Recommendations project, a collaborative effort from the MAGIC Evidence Ecosystem Foundation (www.magicproject.org) and The BMJ. Our living systematic review and network meta-analysis will directly inform BMJ Rapid Recommendations6 on covid-19 treatments, initiated to provide trustworthy, actionable, and living guidance to clinicians and patients soon after new and potentially practice-changing evidence becomes available. This systematic review informs a BMJ Rapid Recommendation (box 1).

Box 1

Linked resources in this BMJ Rapid Recommendations cluster

  • Rochwerg B, Agarwal A, Zeng L, et al. Remdesivir for severe covid-19: a clinical practice guideline. BMJ 2020;370:m2924

  • Summary of the results from the Rapid Recommendation process

  • Siemieniuk RAC, Bartoszko JJ, Ge L, et al. Drug treatments for covid-19: living systematic review and network meta-analysis. BMJ 2020;370:m2980, doi:10.1136/bmj.m2980

  • Review and network meta-analysis of all available randomised trials that assessed drug treatments for covid-19

  • Expanded version of the methods, processes, and results with multilayered recommendations, evidence summaries, and decision aids for use on all devices

RETURN TO TEXT

Methods

A protocol provides the detailed methods of this systematic review, including all updates (see supplementary file). We report this living systematic review following the guidelines of the preferred reporting items for systematic reviews and meta-Analyses (PRISMA) checklist for network meta-analyses.7 A living systematic review is a cumulative synthesis that is updated regularly as new evidence becomes available.8 The linked BMJ Rapid Recommendations guideline panels approved all decisions relevant to data synthesis.

Eligibility criteria

We included randomised clinical trials in people with suspected, probable, or confirmed covid-19 that compared drugs for treatment against one another or against no intervention, placebo, or standard care. We included trials regardless of publication status (peer reviewed, in press, or preprint) or language. No restrictions were applied based on severity of illness or setting and we included trials of Chinese medicines if the drug comprised one or more specific molecules with a defined molecular weight dosing.

We excluded randomised controlled trials evaluating vaccination, blood products, nutrition, traditional Chinese herbal medicines that include more than one molecule or a molecule without specific molecular weighted dosing, and non-drug supportive care interventions. Trials including patients with covid-19 that evaluated these interventions were identified and categorised separately.

Information sources

We perform daily searches from Monday to Friday in the US Centers for Disease Control and Prevention (CDC) COVID-19 Research Articles Downloadable Database for eligible studies—the most comprehensive database of covid-19 research articles.9 The database includes 25 bibliographic and grey literature sources: Medline (Ovid and PubMed), PubMed Central, Embase, CAB Abstracts, Global Health, PsycInfo, Cochrane Library, Scopus, Academic Search Complete, Africa Wide Information, CINAHL, ProQuest Central, SciFinder, the Virtual Health Library, LitCovid, WHO covid-19 website, CDC covid-19 website, Eurosurveillance, China CDC Weekly, Homeland Security Digital Library, ClinicalTrials.gov, bioRxiv (preprints), medRxiv (preprints), chemRxiv (preprints), and SSRN (preprints).

The daily searches are designed to match the update schedule of the database and to capture eligible studies the day of or the day after publication. To identify randomised controlled trials, we filtered the results from the CDC’s database through a validated and highly sensitive machine learning model.10 We tracked preprints of randomised controlled trials until publication and updated data to match that in the peer reviewed publication when discrepant and reconciled corrections and retractions existed.

In addition, we search six Chinese databases every two weeks basis: Wanfang, Chinese Biomedical Literature, China National Knowledge Infrastructure, VIP, Chinese Medical Journal Net (preprints), and ChinaXiv (preprints). We adapted the search terms for covid-19 developed by the CDC to the Chinese language. For the Chinese literature search, we also included search terms for randomised trials. The supplementary file includes the Chinese literature search strategy.

We monitor living evidence retrieval services on an ongoing basis. These included the Living Overview of the Evidence (L-OVE) COVID-19 Repository by the Epistemonikos Foundation and the Systematic and Living Map on COVID-19 Evidence by the Norwegian Institute of Public Health, in collaboration with the Cochrane Canada Centre at McMaster University.11

We searched all English information sources from 1 December 2019 to 20 July 2020, and the Chinese literature from conception of the databases to 20 July 2020.

Study selection

Using a systematic review software, Covidence,12 pairs of reviewers, following training and calibration exercises, independently screened all titles and abstracts, followed by full texts of trials that were identified as potentially eligible. A third reviewer adjudicated conflicts.

Data collection

For each eligible trial, pairs of reviewers, following training and calibration exercises, extracted data independently using a standardised, pilot tested data extraction form. Reviewers collected information on trial characteristics (trial registration, publication status, study status, design), patient characteristics (country, age, sex, smoking habits, comorbidities, setting and type of care, and severity of covid-19 symptoms for studies of treatment), and outcomes of interest (means or medians and measures of variability for continuous outcomes and the number of participants analysed and the number of participants who experienced an event for dichotomous outcomes). Reviewers resolved discrepancies by discussion and, when necessary, with adjudication by a third party. We updated the data collected from included studies when they were published as a preprint and as soon as the peer review publication became available in studies initially included as preprints.

Outcomes of interest were selected based on importance to patients and were informed by clinical expertise in the systematic review team and in the linked guideline panel responsible for the BMJ Rapid Recommendations.13 The panel includes unconflicted clinical experts, recruited to ensure global representation, and patient-partners. Outcomes were rated from 1 to 9 based on importance to individual patients (9 being most important), and we included any outcome rated 7 or higher by any panel member. Selected outcomes included mortality (closest to 90 days), mechanical ventilation (total number of patients, over 90 days), adverse events leading to discontinuation (within 28 days), viral clearance (closest to 7 days, 3 days either way), duration of hospital stay, intensive care unit (ICU) length of stay, time to symptom resolution or clinical improvement, and time to viral clearance. Viral clearance at seven days and time to viral clearance were included because both may be surrogates for transmissibility, although this is uncertain.14

Because of the inconsistent reporting observed across trials, in the updates we will use a hierarchy for the outcome mechanical ventilation in which we will include information from the total number of patients who received ventilation over a period if available (as done for this analysis), but we will also include the number at the time when most of the patients were mechanically ventilated if that is the only way in which this outcome is reported.

Risk of bias within individual studies

For each eligible trial, reviewers, following training and calibration exercises, used a revision of the Cochrane tool for assessing risk of bias in randomised trials (RoB 2.0)15 to rate trials as either at i) low risk of bias, ii) some concerns—probably low risk of bias, iii) some concerns—probably high risk of bias, or iv) high risk of bias, across the following domains: bias arising from the randomisation process; bias owing to departures from the intended intervention; bias from missing outcome data; bias in measurement of the outcome; bias in selection of the reported results, including deviations from the registered protocol; and bias arising from early termination for benefit. We rated trials at high risk of bias overall if one or more domains were rated as some concerns—probably high risk of bias or as high risk of bias and as low risk of bias if all domains were rated as some concerns—probably low risk of bias or low risk of bias. Reviewers resolved discrepancies by discussion and, when not possible, with adjudication by a third party.

Data synthesis

We conducted the network meta-analysis using a bayesian framework.16 In this report, we conducted a network meta-analysis of drug treatments for covid-19 that included all patients, regardless of severity of disease.

Summary measures

We summarised the effect of interventions on dichotomous outcomes using the odds ratio and corresponding 95% credible interval. For continuous outcomes, we used the mean difference and corresponding 95% credible interval in days for ICU length of stay and duration of mechanical ventilation because we expected similar durations across randomised controlled trials. For time to symptom resolution and length of hospital stay, we first performed the analyses using the relative effect measure ratio of means and corresponding 95% credible interval before calculating the mean difference in days because we expected substantial variation between studies.17

Treatment nodes

Treatments were grouped into common nodes based on molecule and not on dose or duration. For intervention arms with more than one drug, we created a separate node and included drugs from the same class within the same node. Chloroquine and hydroxychloroquine were included in the same node for covid-19 specific effects and separated for disease independent adverse effects. We drew network plots using the networkplot command of Stata version 15.1 (StataCorp, College Station, TX), with thickness of lines between nodes and size of the nodes based on the number of studies.18

Statistical analysis

For most outcomes, we conducted random effects network meta-analyses using a bayesian framework with the same priors for the variance and effect parameters.16 For networks with particularly sparse outcomes, we conducted fixed effect network meta-analysis.19 We used a plausible prior for variance parameter and a uniform prior for the effect parameter suggested in a previous study based on empirical data.20 For all analyses, we used three Markov chains with 100 000 iterations after an initial burn-in of 10 000 and a thinning of 10. We used node splitting models to assess local incoherence and to obtain indirect estimates.21 All network meta-analyses were performed using the gemtc package of R version 4.0.0 (RStudio, Boston, MA).22

Some treatment nodes with few total participants and few total events resulted in highly implausible and extremely imprecise effect estimates. We therefore decided to include only treatments that enrolled at least 100 patients or had at least 20 events. For this iteration, the analyses included treatment nodes with fewer than 100 patients and 20 events, but the results are not reported.

Certainty of the evidence

We assessed the certainty of evidence using the grading of recommendations assessment, development and evaluation (GRADE) approach for network meta-analysis.52324 Two people with experience in using GRADE rated each domain for each comparison separately and resolved discrepancies by consensus. We rated the certainty for each comparison and outcome as high, moderate, low, or very low, based on considerations of risk of bias, inconsistency, indirectness, publication bias, intransitivity, incoherence (difference between direct and indirect effects), and imprecision.24 Judgments of imprecision for this systematic review were made using a minimally contextualised approach, with a null effect as the threshold of importance.25 The minimally contextualised approach considers only whether credible intervals include the null effect and thus does not consider whether plausible effects, captured by credible intervals, include both important and trivial effects.25 We created GRADE evidence summaries (Summary of Findings tables) in the MAGIC Authoring and publication platform (www.magicapp.org) to provide user friendly formats for clinicians and patients and to allow re-use in the context of clinical practice guidelines for covid-19.

Interpretation of results

To facilitate interpretation of the results, we calculated absolute effects for outcomes in which the summary measure was an odds ratio or ratio of means. For the outcomes mortality and mechanical ventilation, we used baseline risks from the International Severe Acute Respiratory and Emerging Infection COVID-19 database.26 For all other outcomes, we used the median from all studies in which participants received standard of care to calculate the baseline risk for each outcome, with each study weighed equally. We calculated absolute effects using the transitive risks model27 using R2jags package in R.28

For each outcome, we classified treatments in groups from the most to the least effective using the minimally contextualised framework, which focuses on the treatment effect estimates and the certainty of the evidence.29

Subgroup and sensitivity analysis

When a comparison was dominated by a single study (defined as >90% contribution in fixed effects), we conducted our primary analysis with a fixed effects model for that comparison.19 We planned to perform subgroup analyses of preprints versus peer reviewed studies and high versus low risk of bias. We will perform additional subgroup analyses in the future if directed by the linked independent Rapid Recommendation guideline panels; in this case there was no such direction.

Patient and public involvement

Patients were involved in the interpretation of results and the generation of parallel recommendations, as part of the BMJ Rapid Recommendations initiative.

Results

After screening 7285 titles and abstracts and 122 full texts, 32 unique randomised controlled trials were identified that evaluated drug treatments as of 20 July 2020 (fig 1).30313233343536373839404142434445464748495051 Searches of living evidence retrieval services identified one additional eligible randomised controlled trial.52 Eighteen randomised controlled trials have been published in peer reviewed journals, and 14 only as preprints. Most of the trials were registered (30/32; 94%), published in English (30/32; 94%), and evaluated treatment in patients admitted to hospital with covid-19 (28/32; 88%). Just over one half of the trials were conducted in China (18/32; 56%). Of the 32 included drug trials, six evaluated treatment against active comparators, 18 evaluated treatment against standard care or placebo, and two evaluated different durations or doses of the same treatment. Our analyses were performed on 26 June 2020 and include 20 randomised controlled trials.3134353637383941424344454647484950Table 1 presents the characteristics of the included studies. Additional study characteristics, outcome data, and risk of bias assessments for each study are available in the supplementary file.

Table 1

Study characteristics

View this table:

Several randomised controlled trials were not included in the analysis: two trials that evaluated different durations of the same drug, because both arms would have been classified within the same treatment node3240; one trial that compared lincomycin with azithromycin,55 because neither arm was connected to the network; 10 trials that compared technetium 99m-methyl diphosphonate (99mTC-MDP),62 azvudine,63 colchicine,64 febuxostat,65 hydroxychloroquine,6667686970 and hydroxychloroquine with darunavir-cobicistat6869 because they were identified, or the data were available, after the analysis was completed. Table 2 describes the randomised controlled trials that were identified after the data analysis and that will be included in the next update.

Table 2

Randomised trials identified after data analysis, which will be included in the next update

View this table:

Of the randomised controlled trials included in the analyses, two did not have publicly accessible protocols or registrations.5561 Of the trials with publicly accessible protocols or registrations, 16 reported results for one or more of our outcomes of interest that were not prespecified in protocols or registrations. No other discrepancies between the reporting of our outcomes of interest in trial reports and protocols or registrations were noted. One trial did not report outcomes in the groups as randomised; the authors shared outcome data with us in the groups as randomised.49

Five studies were initially posted as preprints and subsequently published after peer review.32394448515456575859 In one study, mortality was not reported in the preprint but was reported in the peer reviewed paper.4459 A trial that compared dexamethasone with standard care was published as a preprint before our data analysis48 and has since been published with additional events after peer review.56 No substantive differences were found between the preprint and peer reviewed publications for the other three studies.

All analyses reached convergence based on trace plots and a Brooks-Gelman-Rubin statistic less than 1.05. Two randomised controlled trials that studied glucocorticoids differed substantially in size (the Randomised Evaluation of COVID-19 therapy (RECOVERY) trial enrolled 6425 patients48 and GLUCOCOVID 6349), thus we performed a fixed effects analysis for the direct pairwise analysis for the outcomes that were reported in both of the trials (mortality and mechanical ventilation). This analysis was separate from the network meta-analyses, which was conducted with random effects. Owing to insufficient data, we did not conduct any of the subgroup or sensitivity analyses specified in the protocol (see supplementary file). For comparisons between treatments with at least 100 patients or 20 events, the effects were similar whether or not we included treatments with fewer patients and events in the network meta-analyses (see supplementary file).

Risk of bias in included studies

The supplementary material presents the assessment of risk of bias of the included studies for each outcome. Two studies were judged at low risk of bias in all domains.3145 All other studies had probably high or high risk of bias in the domains of randomisation or deviation from the intended interventions.

Effects of the interventions

The supplementary material presents the network plots depicting the interventions included in the network meta-analysis of each outcome. Figure 2 presents a summary of the effects of the interventions on the outcomes. The supplementary file also presents detailed relative and absolute effect estimates and certainty of the evidence for all comparisons and outcomes. No statistical incoherence was detected in any of the network meta-analyses.

Fig 2
Fig 2

Summary of effects of interventions on outcomes. Numbers are absolute risk differences (95% credible intervals) per 1000 patients or mean differences (95% credible intervals) in days compared with standard care (SC). Empty cells represent no evidence for the specific intervention. Grey cells represent fewer than 100 patients randomised to the intervention for the outcome. ICU=intensive care unit

Mortality

Fifteen randomised controlled trials including 8654 participants3134353637394142434445464849507172 reported mortality. The treatment nodes included in the network meta-analysis were glucocorticoids, hydroxychloroquine, lopinavir-ritonavir, remdesivir, umifenovir, and standard care. The network estimates did not reveal a convincing reduction for any of these interventions compared with standard care. The certainty of the evidence was low for the comparison between remdesivir and standard of care, and very low for all other comparisons (fig 2). For glucocorticoids, the direct estimate was more credible than the network estimate (moderate certainty versus very low certainty) because the direct estimate was more precise. The network estimate (relative risk), which considers heterogeneity of the entire network, was 0.84 (95% credible interval 0.52 to 1.36). The direct pairwise meta-analysis of two trials of glucocorticoids versus standard care4849 suggested a probable reduction in mortality with glucocorticoids (relative risk 0.88 (95% credible interval 0.80 to 0.97), risk difference 37 fewer per 1000 patients (95% credible interval 63 fewer to 11 fewer), moderate certainty for risk of bias).

Mechanical ventilation

Eight randomised controlled trials that enrolled 6953 participants3134353941424548497172 reported mechanical ventilation in patients who were not receiving mechanical ventilation at baseline. The treatment nodes included in the network meta-analysis were glucocorticoids, remdesivir, and standard care (fig 2). The network estimate for glucocorticoids was very low certainty because of serious imprecision (relative risk 0.71, 95% credible interval 0.29 to 1.73). The direct pairwise meta-analysis for glucocorticoids versus standard care4849 resulted in higher certainty and suggested a probable reduction with glucocorticoids versus standard care (relative risk 0.74 (95% credible interval 0.59 to 0.93), risk difference 30 fewer per 1000 patients (95% credible interval 48 fewer to 8 fewer), moderate certainty for risk of bias).

Adverse events leading to discontinuation

Eleven randomised controlled trials including 1875 participants3138394143444546475072 reported adverse effects leading to discontinuation of the study drug. The treatment nodes included in the network meta-analysis were hydroxychloroquine, remdesivir, and standard care. Moderate certainty evidence showed that remdesivir did not result in any additional harm beyond standard care and low certainty evidence showed that hydroxychloroquine increased the risk of adverse events compared with standard care (fig 2).

Viral clearance at 7 days (3 days either way)

All 10 randomised controlled trials that cumulatively enrolled 856 participants34374244454647505272 measured viral clearance with polymerase chain reaction cut-off points. The treatment nodes included in the network meta-analysis were hydroxychloroquine, lopinavir-ritonavir, remdesivir, and standard care. We did not find any convincing evidence that any of the interventions increased the rate of viral clearance (fig 2). The certainty of the evidence was low for remdesivir compared with standard care, and very low for all other comparisons.

Duration of hospital stay

Eight randomised controlled trials including 855 participants34353739414548505272 reported duration of hospital stay. The treatment nodes included in the network meta-analysis were lopinavir-ritonavir, remdesivir, and standard care. Patients who received lopinavir-ritonavir had fewer days of hospital stay than patients who received standard care, but the effect estimate included no difference (risk difference −1.42 days, 95% credible interval −3.03 to 0.02, low certainty; fig 2). Remdesivir did not seem to reduce the duration of hospital stay (low certainty).

ICU length of stay

Two randomised controlled trials including 280 participants reported length of ICU stay.3439 The treatments evaluated were lopinavir-ritonavir and interferon beta-1 versus standard care. Standard care was the only treatment node with at least 100 patients and therefore no analyses were performed for this outcome.

Duration of mechanical ventilation

Three randomised controlled trials including 557 participants343945 reported duration of mechanical ventilation. The treatment nodes included in the meta-analysis were remdesivir and standard care. Moderate certainty evidence showed that remdesivir reduces the duration of mechanical ventilation compared with standard care (mean difference −5.15 days, 95% credible interval −8.28 to −2.02; fig 2).

Time to symptom resolution

Thirteen randomised controlled trials including 2282 participants3134353637383941424445505272 reported time to symptom resolution. At least 100 patients received hydroxychloroquine, lopinavir-ritonavir, remdesivir, and standard care. Patients who received remdesivir (mean difference −2.58 days, 95% credible interval −4.32 to −0.54, moderate certainty), hydroxychloroquine (−4.53 days, −5.98 to −2.99, low certainty), and lopinavir-ritonavir (−1.22 days, −2.00 to −0.37, low certainty) had a shorter symptom duration than patients who received standard care.

Time to viral clearance

Ten randomised controlled trials including 684 participants35374142444647505272 found no convincing evidence that any of the interventions reduced the time to viral clearance. At least 100 patients received hydroxychloroquine, lopinavir-ritonavir, remdesivir, and standard care. The certainty of the evidence was very low for all comparisons (fig 2).

Discussion

This living systematic review and network meta-analysis provides a comprehensive overview of the evidence for drug treatments of covid-19 up to 20 July 2020. The certainty of the evidence for most of the comparisons was very low. The only intervention that probably reduces mortality and mechanical ventilation is glucocorticoids, a result driven entirely by the RECOVERY trial.48 Remdesivir is the only intervention in which moderate certainty exists supporting benefits for both time to symptom resolution and duration of mechanical ventilation, but it remains uncertain whether remdesivir has any effect on mortality and other outcomes important to patients. Remdesivir was the only intervention where all the data came from randomised controlled trials sponsored by a pharmaceutical company. Direct evidence from randomised controlled trials in patients with covid-19 has so far provided little definitive evidence about adverse effects for most interventions.

Hydroxychloroquine might increase the risk of adverse events leading to drug discontinuation compared with the other interventions. Notably, this iteration of the living network meta-analysis did not include four recently published randomised controlled trials on hydroxychloroquine compared with standard care.66676873 RECOVERY, the largest randomised controlled trial on hydroxychloroquine, suggests that hydroxychloroquine might not reduce mortality and might increase length of hospital stay.66 These data will be included in the next update. No convincing evidence was found that the other interventions resulted in benefits or harms compared with standard of care.

Strengths and limitations of this review

Our search strategy and eligibility criteria were comprehensive, without restrictions on language of publication, and provide an overview of the current evidence. To ensure expertise in all areas, our team is composed of clinical and methods experts who have undergone training and calibration exercises for all stages of the review process. To minimise problems with counterintuitive results, in our data analysis plan we anticipated challenges that arise in network meta-analysis when data are sparse.19 We assessed the certainty of the evidence using the GRADE approach and interpreted the results considering absolute effects. Many of the results for comparisons with sparse data were uninformative and were sometimes implausible. For that reason, we decided to report evidence on treatments for which at least 100 people were randomised. In the future, when more data from more treatments are available, our classification of interventions from the most to the least effective will facilitate clear interpretation of results.

The main limitation of the systematic review is the very low quality of the evidence as a result of the sparse data currently available. As the many ongoing trials are completed, we anticipate that the effect estimates will quickly become both plausible and informative as the quality of the evidence increases. Only two studies were judged to be at low risk of bias.313258 The most common limitation was lack of blinding, including in the largest trials.

Another limitation of this living systematic review and network meta-analysis is the limited quality of reporting. For some outcomes, the method in which the researchers measured and reported outcomes proved inconsistent across studies, and thus such studies could not be included in the network meta-analyses. This led the team to propose a hierarchy for the outcome mechanical ventilation, as described in the methods. Our decision to exclude treatment nodes with fewer than 100 patients or 20 events was made retrospectively because including some treatment options with small numbers of patients or events led to implausible results. We will continue to use this approach prospectively in updates of this living systematic review and network meta-analysis.

The living nature of our systematic review and network meta-analysis could conceivably (at least temporarily) amplify publication bias, because studies with promising results are more likely to be published and are published sooner than studies with negative results. The inclusion of preprints, many of which have negative results, might mediate this risk. Industry sponsored trials such as those for remdesivir and other patented drugs could be particularly at risk of publication bias, and positive results for these drugs might require more cautious interpretation than generic drugs tested in randomised controlled trials independent of industry influence. However, the inclusion of preprints in our network meta-analysis might introduce bias from simple errors and the reporting limitations of preprints. We include preprints because of the urgent need for information and because so many of the studies on covid-19 are published first as preprints.

For comparisons with sufficient data, the primary limitation of the evidence is lack of blinding, which might introduce bias through differences in co-interventions between randomisation groups. We chose to consider the treatment arms that did not receive an active experimental drug (ie, placebo or standard care) within the same node: it is possible that the unblinded standard care groups received systematically different co-interventions than groups randomised to receive a placebo. Direct comparisons in which the evidence is dominated by unblinded studies were rated down, consistent with GRADE, for risk of bias and that is reflected in the rating of the quality of evidence from the network estimate.74 It is also possible that study level meta-analysis might not detect important subgroup modification that would otherwise be detected within trial comparisons.75 For example, the RECOVERY trial suggested that patients with more severe disease might obtain a greater benefit from dexamethasone than patients with less severe disease.48

Our living systematic review and network meta-analysis is informing the development of the BMJ Rapid Recommendations.6 An important difference in the methods for assessing the certainty of the evidence does, however, exist between the two. In this living systematic review and network meta-analysis, we use a minimally contextualised approach for rating the certainty of the evidence, whereas BMJ Rapid Recommendations uses a fully contextualised approach in which the thresholds of importance of magnitudes of effects depend on all other outcomes and factors involved in the decision.25 The contextualisation explains potential differences in the certainty of the evidence between the two. The limitations of potentially misleading results when the network is sparse, and the desirability of focusing on direct estimates from larger studies when this is the case, explain differences in the details of the estimates of effect in this network meta-analysis and in the associated guidelines for remdesivir.13

To date, we are aware of two other similar efforts to ours.7677 We decided to proceed independently to ensure that the results fully inform clinical decision making for the associated living guidance in BMJ Rapid Recommendations.6 We also include a more comprehensive search for the evidence and several differences in analytical methods, which we believe are best suited for this process. It is also important to evaluate the reproducibility and replicability of results from different scientific approaches.

We will periodically update this living systematic review and network meta-analysis. The changes from each version will be highlighted for readers and the most updated version will be the one available in the publication platform. Previous versions will be archived in the supplementary material. This living systematic review and network meta-analysis will also be accompanied by an interactive infographic and a website for users to access the most updated results in a user friendly format (magicapp.org).

Conclusions

Evidence from this living systematic review and network meta-analysis suggests that glucocorticoids probably reduce mortality and mechanical ventilation in patients with severe covid-19. Remdesivir probably reduces length of hospital stay. The effects of most drug interventions are currently highly uncertain, and no definitive evidence exists that other interventions result in important benefits and harms for any outcomes.

What is already known on this topic

  • Despite huge efforts to identify effective drug interventions for coronavirus disease 2019 (covid-19), evidence for effective treatment remains limited

What this study adds

  • This living systematic review and network meta-analysis provides a comprehensive overview and assessment of the evidence published as of 20 July 2020 and will be updated periodically

  • The certainty of the evidence for most interventions tested thus far is low or very low

  • In patients with severe covid-19, glucocorticoids probably decrease mortality and mechanical ventilation

  • Hydroxychloroquine, lopinavir-ritonavir, and remdesivir might reduce the time to symptom resolution

Acknowledgments

We thank Kevin Cheung for input and early contributions.

Footnotes

  • Contributors: RACS, JJB, LG, and DZ contributed equally to the systematic review and are joint first authors. RACS, JJB, DZ, LG, and RB-P were the core team leading the systematic review. JJB, RC, SAF, RWMV, PA, SM, YW, ZY, IR, AD, TD, AI, AQ, CS, LY, FF, QL, XH, LS, BF, and AV-G identified and selected the studies. DZ, EK, NS, RWMV, AA, YW, KH, HP-H, MAH, CF, SLM, QL, AQ, LY, and FF collected the data. LG, BS, LH, QI, DH-A, GHG, GT, and LT analysed the data. RB-P, HPH, AI, RAM, TD, NS, and DC assessed the certainty of the evidence. SLM, FL, BR, TA, POV, GHG, MM, JDN, ML, TT, BT, FF, and GR provided advice at different stages. RACS, RB-P, and GHG drafted the manuscript. All authors approved the final version of the manuscript. RACS is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funder: This study was supported by the Canadian Institutes of Health Research (grant CIHR-IRSC:0579001321).

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Canadian Institutes of Health Research; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

  • Ethical approval: Not applicable. All the work was developed using published data.

  • Data sharing: No additional data available.

  • RACS affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

  • Dissemination to participants and related patient and public communities: The infographic and MAGICapp decision aids (available at www.magicapp.org/) were created to facilitate conversations between healthcare providers and patients or their surrogates. The MAGICapp decision aids were co-created with people who have lived experience of covid-19.

  • RACS affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

http://creativecommons.org/licenses/by-nc/4.0/

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

References

View Abstract