Prophylaxis against covid-19: living systematic review and network meta-analysisBMJ 2021; 373 doi: https://doi.org/10.1136/bmj.n949 (Published 26 April 2021) Cite this as: BMJ 2021;373:n949
- Jessica J Bartoszko, methodologist1 *,
- Reed A C Siemieniuk, methodologist, internist1 *,
- Elena Kum, methodologist1 *,
- Anila Qasim, methodologist1 *,
- Dena Zeraatkar, methodologist1 *,
- Long Ge, methodologist2 *,
- Mi Ah Han, methodologist3,
- Behnam Sadeghirad, assistant professor14,
- Arnav Agarwal, methodologist, internist15,
- Thomas Agoritsas, methodologist, internist16,
- Derek K Chu, methodologist, immunologist17,
- Rachel Couban, librarian4,
- Andrea J Darzi, methodologist1,
- Tahira Devji, methodologist1,
- Maryam Ghadimi, methodologist1,
- Kimia Honarmand, methodologist, critical care physician8,
- Ariel Izcovich, methodologist, internist9,
- Assem Khamis, data analyst10,
- Francois Lamontagne, methodologist, critical care physician11,
- Mark Loeb, methodologist, infectious disease physician17,
- Maura Marcucci, methodologist, internist17,
- Shelley L McLeod, methodologist, assistant professor1213,
- Sharhzad Motaghi, methodologist1,
- Srinivas Murthy, clinical associate professor, paediatric critical care, infectious diseases physician14,
- Reem A Mustafa, methodologist, nephrologist15,
- John D Neary, methodologist, internist7,
- Hector Pardo-Hernandez, methodologist1617,
- Gabriel Rada, methodologist1819,
- Bram Rochwerg, methodologist, critical care physician17,
- Charlotte Switzer, methodologist1,
- Britta Tendal, methodologist20,
- Lehana Thabane, professor1,
- Per O Vandvik, methodologist, internist21,
- Robin W M Vernooij, methodologist2223,
- Andrés Viteri-García, methodologist1824,
- Ying Wang, methodologist, pharmacist1,
- Liang Yao, methodologist1,
- Zhikang Ye, methodologist, pharmacist1,
- Gordon H Guyatt, methodologist, internist17,
- Romina Brignardello-Petersen, methodologist1
- 1Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- 2Evidence Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, Gansu, China
- 3Department of Preventive Medicine, College of Medicine, Chosun University, Gwangju, Republic of Korea
- 4Department of Anesthesia, McMaster University, Hamilton, ON, Canada
- 5Department of Medicine, University of Toronto, Toronto, ON, Canada
- 6Division of General Internal Medicine & Division of Clinical Epidemiology, University Hospitals of Geneva, Geneva, Switzerland
- 7Department of Medicine, McMaster University, Hamilton, ON, Canada
- 8Department of Medicine, Western University, London, ON, Canada
- 9Servicio de Clinica Médica del Hospital Alemán, Buenos Aires, Argentina
- 10Wolfson Palliative Care Research Centre, Hull York Medical School, York, UK
- 11Department of Medicine and Centre de recherche du CHU de Sherbrooke, Sherbrooke, QC, Canada
- 12Schwartz/Reisman Emergency Medicine Institute, Sinai Health, Toronto, ON, Canada
- 13Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada
- 14Department of Pediatrics, Faculty of Medicine, University of British Columbia, Vancouver, BC, Canada
- 15Department of Medicine, University of Kansas Medical Center, Kansas City, MO, USA
- 16Iberoamerican Cochrane Centre, Sant Pau Biomedical Research Institute (IIB Sant Pau), Barcelona, Spain
- 17CIBER de Epidemiología y Salud Pública (CIBERESP), Barcelona, Spain
- 18Epistemonikos Foundation, Santiago, Chile
- 19C Evidence Center, Cochrane Chile Associated Center, Pontificia Universidad Católica de Chile, Santiago, Chile
- 20School of Public Health and Preventative Medicine, Monash University, Melbourne, Australia
- 21Institute of Health and Society, University of Oslo, Oslo, Norway
- 22Department of Nephrology and Hypertension, University Medical Center Utrecht, Utrecht, Netherlands
- 23Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- 24Centro de Investigación de Salud Pública y Epidemiología Clínica (CISPEC), Facultad de Ciencias de la Salud Eugenio Espejo, Universidad UTE, Quito, Ecuador
- *Joint first authors
- Correspondence to: R A C Siemieniuk McMaster University Medical Centre 1280 Main Street West 2C Area Hamilton, ON, L8S 4K1, Canada @rsiemieniuk on Twitter) (or
- Accepted 7 April 2021
Objective To determine and compare the effects of drug prophylaxis on SARS-CoV-2 infection and covid-19.
Design Living systematic review and network meta-analysis.
Data sources World Health Organization covid-19 database, a comprehensive multilingual source of global covid-19 literature to 25 March 2021, and six additional Chinese databases to 20 February 2021.
Study selection Randomised trials of people at risk of covid-19 who were assigned to receive prophylaxis or no prophylaxis (standard care or placebo). Pairs of reviewers independently screened potentially eligible articles.
Methods Random effects bayesian network meta-analysis was performed after duplicate data abstraction. Included studies were assessed for risk of bias using a modification of the Cochrane risk of bias 2.0 tool, and certainty of evidence was assessed using the grading of recommendations assessment, development, and evaluation (GRADE) approach.
Results The first iteration of this living network meta-analysis includes nine randomised trials—six of hydroxychloroquine (n=6059 participants), one of ivermectin combined with iota-carrageenan (n=234), and two of ivermectin alone (n=540), all compared with standard care or placebo. Two trials (one of ramipril and one of bromhexine hydrochloride) did not meet the sample size requirements for network meta-analysis. Hydroxychloroquine has trivial to no effect on admission to hospital (risk difference 1 fewer per 1000 participants, 95% credible interval 3 fewer to 4 more; high certainty evidence) or mortality (1 fewer per 1000, 2 fewer to 3 more; high certainty). Hydroxychloroquine probably does not reduce the risk of laboratory confirmed SARS-CoV-2 infection (2 more per 1000, 18 fewer to 28 more; moderate certainty), probably increases adverse effects leading to drug discontinuation (19 more per 1000, 1 fewer to 70 more; moderate certainty), and may have trivial to no effect on suspected, probable, or laboratory confirmed SARS-CoV-2 infection (15 fewer per 1000, 64 fewer to 41 more; low certainty). Owing to serious risk of bias and very serious imprecision, and thus very low certainty of evidence, the effects of ivermectin combined with iota-carrageenan on laboratory confirmed covid-19 (52 fewer per 1000, 58 fewer to 37 fewer), ivermectin alone on laboratory confirmed infection (50 fewer per 1000, 59 fewer to 16 fewer) and suspected, probable, or laboratory confirmed infection (159 fewer per 1000, 165 fewer to 144 fewer) remain very uncertain.
Conclusions Hydroxychloroquine prophylaxis has trivial to no effect on hospital admission and mortality, probably increases adverse effects, and probably does not reduce the risk of SARS-CoV-2 infection. Because of serious risk of bias and very serious imprecision, it is highly uncertain whether ivermectin combined with iota-carrageenan and ivermectin alone reduce the risk of SARS-CoV-2 infection.
Systematic review registration This review was not registered. The protocol established a priori is included as a supplement.
Readers’ note This article is a living systematic review that will be updated to reflect emerging evidence. Updates may occur for up to two years from the date of original publication.
As of 30 March 2021, more than 127 million people have been infected with SARS-CoV-2, the virus responsible for covid-19; of these, more than 2.7 million have died.1 Cases and deaths continue to rise as SARS-CoV-2 variants of concern become increasingly widespread. Because of vaccine hesitancy, contraindications to receiving the vaccine, and potential reduced vaccine effectiveness against these variants of concern, drug and antibody prophylaxis, if effective, will be an important intervention against covid-19.2 Drugs used as pre-exposure prophylaxis in a high risk population and post-exposure prophylaxis are effective and commonly used for other viruses, including HIV and hepatitis B.3 If effective against covid-19, these drugs could also have a monumental impact worldwide to prevent SARS-CoV-2 infection and attenuate disease, especially in those at high risk of death. Researchers around the world are therefore enrolling participants in randomised trials of drugs and antiviral antibodies for prophylaxis against covid-19.
Clinicians, patients, guideline bodies, and government agencies face challenges in interpreting the results from trials that are being published at a rate never encountered previously. This environment necessitates well developed summaries that can distinguish between trustworthy and untrustworthy evidence.
Living systematic reviews and network meta-analyses resolve an important limitation of traditional systematic reviews, which provide an overview of the relevant evidence only at a specific time.4 The ability of a living network meta-analysis to present a complete, broad, and up-to-date view of the evidence makes it ideal to inform the development of practice recommendations, ideally in the form of living clinical practice guidelines.456 Network meta-analysis, rather than pairwise meta-analysis, provides useful information about the comparative effectiveness of interventions that have not been tested head to head. The lack of such direct comparisons is certain to limit inferences in the covid-19 setting; therefore, network meta-analysis is critical to inform the selection of the best drug among all alternative options. Moreover, the incorporation of indirect evidence can strengthen evidence from comparisons that were tested head to head.7
In this living systematic review and network meta-analysis, we compare the effects of drug prophylaxis for covid-19. This living network meta-analysis will, similar to our established living network meta-analysis on covid-19 treatment,8 directly inform living World Health Organization guidelines on drugs for covid-19,56 a collaborative effort between WHO and the MAGIC Evidence Ecosystem Foundation (www.magicproject.org), inspired by BMJ Rapid Recommendations.9 This review will inform trustworthy, actionable, and living guidance to clinicians caring for patients with covid-19 (also see box 1).
Linked resources in this BMJ Rapid Recommendations cluster
Lamontagne F, Agoritsas T, Siemieniuk R, et al. A living WHO guideline on drugs to prevent covid-19. BMJ 2021;372:n526, doi:10.1136/bmj.n526
World Health Organization. Drugs to prevent COVID-19: A WHO living guideline. 2021. https://apps.who.int/iris/bitstream/handle/10665/339877/WHO-2019-nCoV-prophylaxes-2021.1-eng.pdf
Summary of the results from the Rapid Recommendation process
Bartoszko JJ, Siemieniuk RAC, Kum E, et al. Prophylaxis against covid-19: living systematic review and network meta-analysis. BMJ 2021;373:n949, doi:10.1136/bmj.n949
Review and network meta-analysis of all available randomised trials that assessed drug prophylaxis for covid-19
Expanded version of the methods, processes, and results with multilayered recommendations, evidence summaries, and decision aids for use on all devices
The protocol provides detailed methods of this systematic review (see supplementary file). We report this living systematic review following the guidelines of the preferred reporting items for systematic reviews and network meta-analyses (PRISMA) checklist.10 A living systematic review is a cumulative synthesis that is updated regularly as new evidence becomes available.411 The linked BMJ Rapid Recommendations methods team approved all decisions relevant to data synthesis.
We included randomised trials in participants at risk of covid-19 and that compared different drugs or therapeutic doses of vitamins or antibodies for prophylaxis against each other or against no prophylaxis (placebo or standard care, as defined by the researchers), with no restriction on language of publication. We included studies of people pre-exposure or post-exposure to covid-19 and at low risk for covid-19 (eg, non-exposed community member) or high risk (eg, member of a household where one or more people tested positive for SARS-CoV-2, contact of index case, healthcare worker, or long term care resident).
Randomised trials were excluded if they were published only as press releases or evaluated vaccination, nutrition, or non-drug supportive care interventions. We synthesise randomised trials that evaluated drug, antibody, and cellular treatments in people with covid-19 in separate living network meta-analyses.8
From Monday to Friday we performed daily searches for eligible studies in the WHO covid-19 database—a comprehensive multilingual source of global literature on covid-19.12 We also performed daily searches (Monday to Friday) in the US Centers for Disease Control and Prevention (CDC) COVID-19 Research Articles Downloadable Database before its merger with the WHO covid-19 database on 9 October 2020.13 The WHO database includes, but is not limited to, 25 bibliographic and grey literature sources: Medline (Ovid and PubMed), PubMed Central, Embase, CAB Abstracts, Global Health, PsycInfo, Cochrane Library, Scopus, Academic Search Complete, Africa Wide Information, CINAHL, ProQuest Central, SciFinder, the Virtual Health Library, LitCovid, WHO covid-19 website, CDC covid-19 website, Eurosurveillance, China CDC Weekly, Homeland Security Digital Library, ClinicalTrials.gov, bioRxiv (preprints), medRxiv (preprints), chemRxiv (preprints), and SSRN (preprints). The supplementary file includes the WHO literature search strategy, which is maintained and tested on an ongoing basis by two WHO research librarians and one US CDC research librarian.
We designed the daily searches to match the update schedule of the database and to capture eligible studies the day of or the day after publication. To identify randomised trials, we filtered the results through RobotSearch, a validated and highly sensitive machine learning model.14 We tracked preprints of randomised trials until publication and, when discrepant, updated data to match those in the peer reviewed publication. When needed, we reconciled multiple versions of preprints, post hoc analyses, corrections, and retractions.
In addition, monthly searches, utilising search terms for covid-19 developed by the CDC and adapted for the Chinese language by native speakers, were performed in six Chinese databases: Wanfang Data, Chinese Biomedical Literature, China National Knowledge Infrastructure, VIP Chinese Science Journals Database, Chinese Medical Journal Net (preprints), and ChinaXiv (preprints). The Chinese literature search also included search terms for randomised trials. The supplementary file includes the Chinese literature search strategy.
Our search strategy also included monitoring, on an ongoing basis, living evidence retrieval services, including the Living Overview of the Evidence (L-OVE) covid-19 Repository by the Epistemonikos Foundation and in collaboration with the Cochrane Canada Centre at McMaster University, the Systematic and Living Map on covid-19 Evidence by the Norwegian Institute of Public Health.1516
The search included WHO information sources from 1 December 2019 to 25 March 2021, and the Chinese literature from conception of the databases to 20 February 2021.
Study selection and data collection
Using a systematic review software, Covidence,17 after training and calibration exercises, pairs of reviewers independently screened all titles and abstracts, followed by full texts of trials that were identified as potentially eligible. A third reviewer adjudicated conflicts.
For each eligible trial, after training and calibration exercises, pairs of reviewers extracted data independently using a standardised, pilot tested data extraction form. Reviewers collected information on trial characteristics (trial registration, publication status, study status, design), participant characteristics (country, age, sex, comorbidities), exposure characteristics (exposure status, exposure duration, high risk group), and outcomes of interest (means or medians and measures of variability for continuous outcomes, and number of participants analysed and number of participants who experienced an event for dichotomous outcomes). Reviewers resolved discrepancies by discussion and, when necessary, with adjudication by a third party.
The review team selected outcomes of interest based on importance to patients, and these were informed by clinical expertise in the systematic review team and the linked guideline panel responsible for the WHO living guideline on drugs to prevent covid-19.6 The panel, detailed in the guideline, includes unconflicted clinical experts and patient partners who were recruited to ensure global representation. We rated outcomes from 1 to 9 based on importance to individual patients (9 being most important), and we included any outcome rated 7 or higher by any panel member. This process resulted in choice of several outcomes: laboratory confirmed SARS-CoV-2 infection; a composite of suspected, probable, or laboratory confirmed SARS-CoV-2 infection; admission to hospital (within 28 days of randomisation); mortality (closest to 90 days); adverse effects leading to drug discontinuation (within 28 days of randomisation); and time to symptom resolution or clinical improvement in the subset of participants who became infected with SARS-CoV-2. In anticipation of hydroxychloroquine prophylaxis being included in the linked WHO recommendation, we analysed drug specific adverse effects from trials reporting on hydroxychloroquine compared with standard care or placebo. The supplementary file includes the results of pairwise meta-analyses and related subgroup analyses for cardiac toxicity and non-serious gastrointestinal adverse effects.
Because of inconsistent reporting across trials, when possible we preferentially extracted participant characteristics and outcome data for participants with a negative polymerase chain reaction (PCR) test result for SARS-CoV-2 infection at baseline. If authors did not report data separately for those who were PCR negative for SARS-CoV-2 at baseline, we extracted data from all participants, regardless of their PCR status at baseline.
Risk of bias within individual studies
For each eligible trial, after training and calibration exercises, reviewers used a revised version of the Cochrane tool for assessing risk of bias in randomised trials (RoB 2.0)18 to rate trials at the outcome level as either: low risk of bias; some concerns—probably low risk of bias; some concerns—probably high risk of bias; or high risk of bias, across the following domains: bias arising from the randomisation process; bias due to departures from the intended intervention; bias due to missing outcome data; bias in measurement of the outcome; bias in selection of the reported results, including deviations from the registered protocol; bias arising from early termination owing to benefit; and bias arising from competing risks. We rated trials at high risk of bias overall if one or more domains were rated as some concerns—probably high risk of bias, or as high risk of bias and as low risk of bias overall if all domains were rated as some concerns—probably low risk of bias or low risk of bias. Reviewers resolved discrepancies by discussion and, when not possible, with adjudication by a third party.
The effect of interventions on dichotomous outcomes were summarised using odds ratios and corresponding 95% credible intervals. To mitigate results with highly implausible and extremely imprecise effect estimates, the analyses included only prophylactic drugs associated with at least 100 participants or 20 events, regardless of the number of studies in which the drug was assessed or the number of participants who received the drug in each study.8 The analysis plan included, data permitting, adjustment for cluster randomisation.
We created nodes for each prophylactic drug (or combination of drugs), independent of dose or duration. Standard care and placebo arms across included trials were combined into a single node for analyses. The networkplot command of Stata version 15.1 (StataCorp, College Station, TX) provided software for network plots in which the inverse variance of the direct comparison determined the thickness of lines between nodes and the size of nodes.19
For outcomes with sufficient data (ie, laboratory confirmed SARS-CoV-2 infection, and suspected, probable, or laboratory confirmed SARS-CoV-2 infection), we performed random effects network-meta analysis using the R package gemtc20 and used three Markov chains with 100 000 iterations after an initial burn-in of 10 000 and a thinning of 10. Node splitting models provided methods to obtain indirect estimates and to assess local heterogeneity.21 For all other outcomes, we performed random effects bayesian meta-analysis using bayesmeta package in RStudio version 3.5.3 (R Studio, Boston, MA).22 An empirical study provided the basis for choosing a plausible prior for the variance parameter and a uniform prior for the effect parameter.23 Outcomes were analysed regardless of whether they were prespecified in a protocol or trial registration. The foundational R codes are publicly available at https://github.com/covid19lnma/prophylaxis_NMA.
Certainty of the evidence
The grading of recommendations assessment, development, and evaluation (GRADE) approach for network meta-analysis provided the framework for assessing the certainty of evidence.724 Two methodologists with experience in using GRADE rated each domain for each comparison and outcome separately and resolved discrepancies by discussion. Criteria for rating the certainty for each comparison and outcome as high, moderate, low, or very low, included considerations of risk of bias, inconsistency, indirectness, publication bias, intransitivity, incoherence (difference between direct and indirect effects), and imprecision (eg, from lack of power as a result of very low event rates). When the credible interval suggested an effect, we used optimal information size to inform the GRADE rating for imprecision.24 Judgments of imprecision for this systematic review were made using a minimally contextualised approach.25 This approach considers whether credible intervals include the null effect, or, when the point estimate is close to the null effect, whether the credible interval lies within the boundaries of small but important benefit and harm.
We rated the certainty of no important effect for the outcomes laboratory confirmed infection; suspected, probable, or laboratory confirmed infection; admission to hospital; and mortality. Pending data from quantitative studies of patient values, we chose thresholds of small but important effects of 0.5% for mortality, 3% for infection (whether laboratory confirmed or not), and 1% for admission to hospital. We rated the certainty of an increase or decrease in adverse effects leading to discontinuation of a drug using the null effect as a threshold. GRADE evidence summaries (Summary of Findings tables) in the MAGIC Authoring and Publication Platform (www.magicapp.org) provided user friendly formats for clinicians and patients and allowed reuse in the context of clinical practice guidelines for covid-19, such as the WHO living guidelines.56 Interim updates and additional study data will appear on our website (www.covid19lnma.com).
Interpretation of results
To facilitate interpretation of the results, we calculated absolute effects for outcomes in which the summary measure was an odds ratio. For mortality, we used the event rate among all participants randomised to standard care or placebo to calculate the baseline risk. For all other outcomes, we used the median event rate in the standard care or placebo arms to calculate the baseline risk. We categorised the interventions from the most to the least effective using the minimally contextualised GRADE guidance to drawing conclusions from network meta-analyses, which considers the effect estimates and the certainty of the evidence.25
Subgroup and sensitivity analysis
The analysis plan included performing subgroup analyses of pre-exposure versus post-exposure studies, preprints versus peer reviewed studies, and high versus low risk of bias studies when at least two studies were present for each subgroup. We plan on performing network meta-regression to explore if duration of prophylactic drug use might modify the relative effect of the drug on adverse effects leading to discontinuation, hypothesising that if the drug is active at the time of exposure to covid-19, it will have a greater relative effect. The linked independent WHO guideline panels might in the future direct additional subgroup analyses; in this first report, the panel provided direction to perform subgroup analyses according to duration and dose of drug prophylaxis. The Instrument to assess the Credibility of Effect Modification ANalyses in randomised controlled trials and meta-analyses (ICEMAN) provides the methodology for, whenever statistical evidence of a subgroup effect exists, assessing the credibility of the subgroup hypothesis.26
Patient and public involvement
As part of the WHO living guidelines and BMJ Rapid Recommendations initiative, patients participated in defining clinical questions and rating the importance of outcomes for this systematic review and were also involved in the interpretation of results and the generation of parallel recommendations.
As of 25 March 2021, after screening 35 106 titles and abstracts and 671 full texts, 11 unique randomised trials reporting on prophylactic drugs were eligible—six evaluated hydroxychloroquine, two evaluated ivermectin, and one each evaluated ivermectin combined with iota-carrageenan, ramipril, and bromhexine hydrochloride (fig 1).2728293031323334353637 Searches of living evidence retrieval services identified two of these eligible randomised trials.3133 No randomised trials that reported on vitamins or antibodies as prophylaxis in people at risk of covid-19 were identified. The supplementary file includes a table of excluded full texts.
Of the 11 eligible randomised trials, seven were published in peer reviewed journals27282930323435 and three only as preprints,313337 and one was a clinical trial registry that reported results.36 All trials were registered, published in English, and compared prophylactic drugs with standard care or placebo.2728293031323334353637 Six evaluated prophylactic drugs in participants without documented exposure to covid-19 (pre-exposure)273031323637 and four evaluated prophylactic drugs in participants with documented exposure to covid-19 (post-exposure).28293435 One trial evaluated prophylactic ivermectin in participants both pre-exposure and post-exposure to covid-19.33 Five trials—four evaluating hydroxychloroquine28293234 and one evaluating bromhexine hydrochloride37—reported on adherence to drug prophylaxis. Full adherence to drug prophylaxis ranged from 75%28 to 95%.29Table 1 presents the characteristics of the 11 included studies, including prophylactic drug dose and duration of use and extent and setting of participant exposure to covid-19.
Seven trials reported results for one or more outcomes of interest that were not prespecified in protocols or registrations.27303133343537 The protocols and trial reports included no other discrepancies in reporting of the outcomes of interest. Two trials were initially posted as preprints and subsequently published in full after peer review.2932 One trial published multiple iterations of the related preprint.33 Another trial published results on a trial registry and subsequently published as a peer reviewed publication.35 Of the trials with multiple published versions, three showed discrepancies in patient baseline characteristics.293233 Two trials showed discrepancies in outcome reporting.2935 Four trials had discrepancies that affected risk of bias ratings.29323335 The supplementary file presents additional study characteristics, outcome data, and reporting differences between versions of study preprints, peer reviewed publications, or both.
Analyses were performed on 18 January 2021 and included nine trials that evaluated hydroxychloroquine and ivermectin (with and without iota-carrageenan) prophylaxis against no prophylaxis (standard care or placebo).282930313233343536 Two trials—one of ramipril27 and one of bromhexine hydrochloride37—were not included in the network meta-analysis because they each enrolled fewer than 100 participants and observed fewer than 20 events in the prophylaxis arm. Since molecule, not drug dose or duration of use, dictates choice of nodes, the two active arms were combined with one included three arm trial (hydroxychloroquine once weekly or twice weekly).32 In this report, because the authors did not report the intracluster correlation coefficient, cluster randomisation could not be adjusted for in one analysed trial that evaluated hydroxychloroquine.29 A post hoc analysis with one of the analysed trials that evaluated hydroxychloroquine did not include information beyond what was already reported in the original peer reviewed publication of the trial.2838
Risk of bias in included studies
The supplementary material presents the assessment of risk of bias of the 11 included studies for each outcome. Five studies evaluating hydroxychloroquine showed low risk of bias across all outcomes.2830313234 Six studies showed high risk of bias overall owing to lack of blinding.272933353637 Of these six studies, five2733353637 had insufficient detail on allocation concealment and one terminated early owing to benefit.35
Effects of the interventions
The supplementary material presents the network and forest plots depicting the interventions included in the network meta-analysis of each outcome. The supplementary file also presents detailed relative and absolute effect estimates and certainty of the evidence for all comparisons and outcomes. Statistical incoherence was not detected in any of the comparisons or outcomes. Five trials compared hydroxychloroquine with placebo2830313234 and four trials—one of hydroxychloroquine,29 one of ivermectin combined with iota-carrageenan,36 and two of ivermectin alone3335—compared drug prophylaxis with standard care, defined as no specific treatment,2935 standard biosecurity care,36 and personal protective measures.33Figure 2 presents a summary of the effects of hydroxychloroquine, ivermectin combined with iota-carrageenan, and ivermectin alone on the outcomes.
Laboratory confirmed SARS-CoV-2 infection
Eight trials that reported on laboratory confirmed SARS-CoV-2 infection in 5728 participants with a negative PCR test result at baseline were included in random effects bayesian network meta-analysis.2829303132333436 The network nodes comprised hydroxychloroquine, ivermectin combined with iota-carrageenan, ivermectin alone, and standard care or placebo. Hydroxychloroquine, compared with standard care or placebo, probably does not reduce the risk of laboratory confirmed infection (odds ratio 1.03 (95% credible interval 0.71 to 1.47); risk difference 2 more per 1000 participants (95% credible interval 18 fewer to 28 more); moderate certainty of evidence because of serious risk of bias). It was very uncertain whether ivermectin combined with iota-carrageenan, when compared with standard care, reduces the risk of laboratory confirmed infection (0.12 (0.03 to 0.38); 52 fewer per 1000 participants (58 fewer to 37 fewer); very low certainty of evidence because of serious risk of bias and very serious imprecision; fig 2). It was also very uncertain whether ivermectin alone, when compared with standard care, reduces the risk of laboratory confirmed infection (0.16 (0.02 to 0.73); 50 fewer per 1000 participants (59 fewer to 16 fewer); very low certainty of evidence because of serious risk of bias and very serious imprecision).
Suspected, probable, or laboratory confirmed SARS-CoV-2 infection
Four trials reported on the composite of suspected, probable, or laboratory confirmed SARS-CoV-2 infection in 4531 participants and were included in random effects bayesian network meta-analysis consisting of nodes for hydroxychloroquine, ivermectin, and standard care or placebo.28293235 For the outcome suspected, probable, or laboratory confirmed infection, low certainty evidence suggests that hydroxychloroquine, compared with standard care or placebo, may have trivial to no effect (0.90 (0.58 to 1.31); 15 fewer per 1000 participants (64 fewer to 41 more); the certainty of evidence was rated down because of serious risk of bias and imprecision. Owing to very low certainty of evidence, the effect of ivermectin, compared with standard care, in reducing the risk of suspected, probable, or laboratory confirmed infection remains very uncertain (0.06 (0.02 to 0.13); 159 fewer per 1000 participants (165 fewer to 144 fewer); the certainty of evidence was rated down because of serious risk of bias and very serious imprecision.
Five trials reported hospital admission in 5659 participants randomised to hydroxychloroquine or standard care or placebo.2829303234 Trials on other prophylactic drugs eligible for analysis did not report on the outcome hospital admission, precluding network meta-analysis. Hydroxychloroquine has trivial to no effect on hospital admission when compared with standard care or placebo (0.87 (0.42 to 1.77); 1 fewer per 1000 participants (3 fewer to 4 more); high certainty of evidence; fig 2).
Five trials reported mortality in 5153 participants randomised to hydroxychloroquine, ivermectin, standard care, or placebo.2829303235 Although data were sufficient to perform network meta-analysis, the network did not converge; therefore, results are presented from pairwise meta-analyses. Hydroxychloroquine has trivial to no effect on mortality compared with standard care or placebo (0.70 (0.24 to 1.99); 1 fewer per 1000 participants (2 fewer to 3 more); high certainty of evidence; fig 2). Because no deaths occurred in the one ivermectin trial reporting mortality, its effect on this patient important outcome is very uncertain.
Adverse effects leading to drug discontinuation
Four trials reported adverse effects leading to drug discontinuation in 3616 participants randomised to hydroxychloroquine, standard care, or placebo.28293031 Trials on other prophylactic drugs eligible for analysis did not report any adverse effects leading to drug discontinuation, precluding network meta-analysis. Hydroxychloroquine probably increases adverse effects compared with standard care or placebo (2.34 (0.93 to 6.08); 19 more per 1000 participants (1 fewer to 70 more); moderate certainty of evidence because of imprecision; fig 2).
Time to symptom resolution or clinical improvement
No randomised trials reported on time to symptom resolution or clinical improvement in the subset of participants who developed SARS-CoV-2 infection.
Insufficient data precluded subgroup analysis for trials randomising participants to ivermectin alone or to ivermectin combined with iota-carrageenan versus standard care. Thus, subgroup analysis in the current study was limited to hydroxychloroquine trials only. No statistical evidence was found of differences in laboratory confirmed SARS-CoV-2 infection; suspected, probable, or laboratory confirmed SARS-CoV-2 infection, hospital admission, or adverse effects leading to discontinuation between pre-exposure303132 and post-exposure282934 studies or based on hydroxychloroquine dosing regimens (see supplementary file). Extremely low event rates precluded investigation of subgroup effects for mortality.
This living systematic review and network meta-analysis provides a comprehensive overview of the evidence for prophylaxis against covid-19 up to 25 March 2021 and directly informs WHO living guidelines on prophylaxis.6 The comparisons for which evidence was informative were hydroxychloroquine versus no prophylaxis, ivermectin combined with iota-carrageenan versus no prophylaxis, and ivermectin versus no prophylaxis. Hydroxychloroquine probably increases adverse effects leading to drug discontinuation (moderate certainty evidence). For other outcomes (laboratory confirmed SARS-CoV-2 infection, hospital admission, and mortality), the certainty of evidence for study results was moderate and high—none of which support any benefit from hydroxychloroquine. Owing to serious risk of bias and very serious imprecision, it is highly uncertain whether ivermectin combined with iota-carrageenan and ivermectin alone reduces the risk of patient important outcomes.
These findings are consistent with those reported in a meta-analysis of hydroxychloroquine prophylaxis compared with no prophylaxis, which did not find any statistical evidence of a benefit with hydroxychloroquine prophylaxis for the patient important outcomes of SARS-CoV-2 infection, hospital admission, and mortality.39 The study concluded that hydroxychloroquine is likely to increase the risk of adverse effects.39 Furthermore, the data are consistent with three meta-analyses404142 and one network meta-analysis8 evaluating ivermectin as treatment for covid-19. In contrast with other meta-analyses, we rated the certainty as very low because of serious risk of bias and very serious imprecision.8404142
The risk of death is much lower in people at risk of covid-19 than in those with a diagnosis of covid-19.8 Similarly, the risk of SARS-CoV-2 infection varies depending on pre-exposure or post-exposure status and setting. Research on prophylaxis therefore necessitates large trials or a focus on the highest risk populations to detect a possible benefit on outcomes of most importance to patients. Also, rare but important harms might not be detected by randomised trials unless extremely large numbers of participants are enrolled, they are diligently followed-up to ascertain these outcomes, or patients at greater risk are included. Guideline panels, which independently rate the certainty of the evidence, therefore must consider the trade-offs between concluding probably no benefit and meaningful adverse effects and waiting for more precise data. For example, the WHO living guidelines issued a strong recommendation against hydroxychloroquine for prophylaxis in covid-19.6
Strengths and limitations of this review
This network meta-analysis on prophylaxis for covid-19 incorporates the most up-to-date evidence on hydroxychloroquine, ivermectin combined with iota-carrageenan, and ivermectin alone. It adds to our living systematic review on drugs for covid-19 and directly informs the WHO living guidelines, together constituting major innovations in the evidence ecosystem.6
The search strategy was comprehensive, with explicit eligibility criteria and no restrictions on language of publication. To ensure expertise in all areas, our team includes clinical and methods experts who have undergone training and calibration exercises for all stages of the review process. To avoid spurious findings, we prespecified that we would only analyse interventions to which at least 100 people had been randomised or 20 events had been observed. Two trials—one of ramipril27 and one of bromhexine hydrochloride37—were therefore omitted from the network meta-analysis, necessitated by the priority to avoid problems that arise from network meta-analysis of sparse data (uninformative and implausible results).8
The GRADE approach provided the structure for rating certainty of evidence and interpreting the results considering absolute effects. To rate the GRADE domain of imprecision, we prespecified thresholds of effect that most would consider small but important. In the absence of empirical data, these thresholds represent our collective experience but are, to a large extent, arbitrary. People placing a larger or smaller value on certain outcomes might reasonably make different inferences about the certainty of evidence for no important effect. For example, people who consider that the smallest important effect in mortality is increasing or reducing 2 per 1000 or more deaths, would rate down the certainty of the evidence due to imprecision and conclude that hydroxychloroquine probably does not have an effect on this outcome (ie, moderate certainty evidence).
This review has some limitations. Conclusions for ivermectin are based on very low certainty evidence and we therefore anticipate that future studies evaluating ivermectin for prophylaxis might substantially change the results, particularly for outcomes of infection and mortality.43 One cluster randomised trial did not report the design effect or the intracluster correlation coefficient itself necessary to calculate the design effect, precluding adjustment in analyses, and therefore potentially leading to falsely narrow credible intervals.29 Cluster sizes were, however, small, making substantial bias unlikely.
The living nature of our systematic review and network meta-analysis could amplify publication bias because studies with promising results are more likely to be published and are published sooner than studies with negative results. Given the failure of hydroxychloroquine trials to show benefit, this is not a concern for hydroxychloroquine. This is, however, a concern for the evidence to date on ivermectin, for which most of the data has not been peer reviewed. With the inclusion of this data in network meta-analysis from one preprint,33 and one clinical trial registry reporting results,36 we found evidence of large positive effects; however, bias from simple errors and reporting limitations might have been introduced. We include these data, regardless of publication status and risk of bias, because of the urgent need for information and because so many of the studies on covid-19 are published first as preprints.
Another limitation of the evidence to date is lack of blinding, which might introduce bias through differences in co-interventions between randomisation groups, especially when the outcomes clinically suspected and probable infection are measured, and adverse effects lead to discontinuation of the drug. We chose to consider the treatment arms that did not receive an active experimental drug (ie, placebo or standard care) within the same node: it is possible that unblinded standard care groups2933353637 might have received systematically different co-interventions or changed their personal protective behaviours compared with groups randomised to receive a placebo.2830313234 Laboratory confirmation mitigates risk of bias from lack of blinding in outcome measurement; however, the availability of diagnostic testing differs across health systems, warranting the additional use of a symptomatic case definition for infection. This was the case for most participants, including healthcare workers, enrolled in one study in the United States, which risked overestimating the incidence of infection in its use of a symptomatic definition for infection.2844
We will periodically update this living systematic review and network meta-analysis. The changes from each version will be highlighted for readers, and the most updated version will be the one available in the publication platform. Previous versions will be archived in the supplementary material. This living systematic review and network meta-analysis will also be accompanied by an interactive infographic and a website for users to access the most updated results in a user friendly format (https://app.magicapp.org/public/guideline/L6RxYL, www.covid19lnma.com).
This living systematic review and network meta-analysis on prophylactic drugs for covid-19 provides evidence that hydroxychloroquine has trivial to no effect on mortality and hospital admission, probably increases the risk of adverse effects, and probably does not reduce the risk of laboratory confirmed SARS-CoV-2 infection. We are very uncertain if ivermectin with or without iota-carrageenan reduces the risk of SARS-CoV-2 infection and mortality owing to serious risk of bias and very serious imprecision, and the effect estimates are likely to change substantially with additional evidence from ongoing trials. No other drug has been studied in large enough trials to make any inferences about the effects of prophylaxis on covid-19.
What is already known on this topic
Prophylaxis is effective and commonly used for viruses, including HIV and hepatitis B
Several interventions are being assessed as potential prophylaxis agents for covid-19, including repurposed drugs and antiviral based antibodies
If effective against covid-19, prophylaxis could also have a major impact worldwide to prevent infection and attenuate disease, especially in those at high risk of death
What this study adds
Hydroxychloroquine did not reduce the rate of SARS-CoV-2 infection, admission to hospital, or mortality
More people discontinued hydroxychloroquine because of adverse events
As studies on ivermectin so far have been small it remains very uncertain whether ivermectin reduces SARS-CoV-2 infection
DKC is an emerging clinician-scientist research fellow, supported by the Canadian Allergy, Asthma and Immunology Foundation, the Canadian Society of Allergy and Clinical Immunology, and AllerGen NCE (the Allergy, Genes and Environment Network, funded through the Canadian Institutes of Health Research).
Contributors: RACS, JJB, LG, AQ, EK, and DZ contributed equally to the systematic review and are joint first authors. RACS, JJB, DZ, LG, EK, AQ, and RB-P were the core team leading the systematic review. JJB, RC, RWMV, SM, YW, ZY, CS, LY, MG, and AV-G identified and selected the studies. DZ, EK, RWMV, AA, YW, KH, HP-H, MAH, SLM, AQ, and LY collected the data. AQ, LG, AK, BS, GHG, and LT analysed the data. RB-P, HPH, AI, RAM, TD, and DC assessed the certainty of the evidence. SLM, FL, BR, TA, POV, GHG, MM, JDN, ML, BT, and GR provided advice at different stages. JJB, RACS, RB-P, and GHG drafted the manuscript. All authors approved the final version of the manuscript. RACS is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding: This study was supported by the Canadian Institutes of Health Research (grant CIHR-IRSC:0579001321; grant MM1-174897).
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: BS reports receiving funding from PIPRA AG (www.pipra.ch) to conduct a systematic review and individual patient data meta-analysis on predictors of postoperative delirium in elderly people in 2020-21; funding from Mitacs Canada, Accelerate internship in partnership with Nestlé Canada to support his graduate student stipend from 2016 to 2018. Mitacs is a national, not-for-profit organisation that has designed and delivered research and training programmes in Canada, working with universities, companies, and both federal and provincial governments; and funding from the International Life Sciences Institute (ILSI)-North America to support his graduate work for his 2015 academic year. In 2016-17, BS worked part-time for the Cornerstone Research Group (CRG), a contract research organisation. The ILSI funding and being employed by CRG are outside the required three year period requested on the ICJME form. ML reports personal fees and non-financial support from Sanofi, grants and personal fees from Seqirus, and personal fees from Pfizer and Medicago outside the submitted work; and is co-investigator on an Anti-Coronavirus Therapies (ACT) randomised trial of covid-19 treatment. LG reports grants from the Ministry of Science and Technology of China, outside the submitted work. All other authors report no financial relationships with any organisations that might have an interest in the submitted work in the previous three years and no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not required.
Data sharing: No additional data available.
The manuscript’s guarantor (RACS) affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
Dissemination to participants and related patient and public communities: The infographic and MAGICapp decision aids (available at www.magicapp.org/) were created to facilitate conversations between healthcare providers and patients or their surrogates. The MAGICapp decision aids were co-created with people who have lived experience of covid-19.
Provenance and peer review: Not commissioned; externally peer reviewed.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.