The effects of excluding treatments from network meta-analyses: survey
BMJ 2013; 347 doi: https://doi.org/10.1136/bmj.f5195 (Published 05 September 2013) Cite this as: BMJ 2013;347:f5195
- Edward J Mills, associate professor12,
- Steve Kanters, biostatistician1,
- Kristian Thorlund, associate professor23,
- Anna Chaimani, PhD student4,
- Areti-Angeliki Veroniki, PhD student4,
- John P A Ioannidis, professor and director2
- 1Faculty of Health Sciences, University of Ottawa, Ottawa, Canada
- 2Stanford Prevention Research Center, Stanford University, Stanford, USA
- 3Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada
- 4Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece
- Correspondence to: E J Mills Faculty of Health Sciences, University of Ottawa, 43 Templeton Street, Ottawa, Canada K1N6X1 edward.mills{at}uottawa.ca
- Accepted 4 August 2013
Abstract
Objective To examine whether the exclusion of individual treatment comparators, including placebo/no treatment, affects the results of network meta-analysis.
Design Survey of networks with individual trial data.
Data sources PubMed and communication with authors of network meta-analyses.
Study selection and methods We included networks that had five or more treatments, contained at least two closed loops, had at least twice as many studies as treatments, and had trial level data available. Investigators abstracted information about study design, participants, outcomes, network geometry, and the exclusion of eligible treatments.
Results Among 18 eligible networks involving 757 randomised controlled trials with 750 possible treatment comparisons, 11 had upfront decided not to consider all treatment comparators and only 10 included placebo/no treatment nodes. In 7/18 networks, there was at least one node whose removal caused a more than 1.10-fold average relative change in the estimated treatments effects, and switches in the top three treatments were observed in 9/18 networks. Removal of placebo/no treatment caused large relative changes of the treatment effects (average change 1.16-3.10-fold) for four of the 10 networks that had originally included placebo/no treatment nodes. Exclusion of current uncommonly used drugs resulted in substantial changes of the treatment effects (average 1.21-fold) in one of three networks on systemic treatments for advanced malignancies.
Conclusion Excluding treatments in network meta-analyses sometimes can have important effects on their results and can diminish the usefulness of the research to clinicians if important comparisons are missing.
Footnotes
The authors appreciate the contribution of Georgia Salanti for providing data and comments on protocol and analysis.
Contributors: EJM, SK, KT, and JPAI conceived the study design. AC, A-AV, and SK acquired the data. SK, KT, and EJM conducted the analyses. EJM, SK, KT, and JPAI wrote the manuscript and all contributed to the writing. All authors approved the final manuscript. EJM is guarantor for the study.
Funding: This study was supported by the Drug Safety and Effectiveness Network (DSEN) of the Canadian Institutes of Health Research. No funding agency has seen this study.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not required.
Data sharing: The technical appendix and statistical code are available from the corresponding author at jioannid@stanford.edu.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.