Impact of searching clinical trial registries in systematic reviews of pharmaceutical treatments: methodological systematic review and reanalysis of meta-analysesBMJ 2017; 356 doi: https://doi.org/10.1136/bmj.j448 (Published 17 February 2017) Cite this as: BMJ 2017;356:j448
- Marie Baudard, PhD student1 2,
- Amélie Yavchitz, physician and researcher1 2 3,
- Philippe Ravaud, professor and director1 2 3 4 5,
- Elodie Perrodeau, statistician1 2 3 4,
- Isabelle Boutron, professor1 2 3 4
- 1Université Paris Descartes, Sorbonne Paris Cité, Paris, France
- 2Centre de Recherche Epidémiologie et Statistique, INSERM U1153, Paris, France
- 3Cochrane France, Paris, France
- 4Centre d’Épidémiologie Clinique, Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, 75004 Paris, France
- 5Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, NY, USA
- Correspondence to: A Yavchitz
- Accepted 18 January 2017
Objective To evaluate the impact of searching clinical trial registries in systematic reviews.
Design Methodological systematic review and reanalyses of meta-analyses.
Data sources Medline was searched to identify systematic reviews of randomised controlled trials (RCTs) assessing pharmaceutical treatments published between June 2014 and January 2015. For all systematic reviews that did not report a trial registry search but reported the information to perform it, the World Health Organization International Trials Registry Platform (WHO ICTRP search portal) was searched for completed or terminated RCTs not originally included in the systematic review.
Data extraction For each systematic review, two researchers independently extracted the outcomes analysed, the number of patients included, and the treatment effect estimated. For each RCT identified, two researchers independently determined whether the results were available (ie, posted, published, or available on the sponsor website) and extracted the data. When additional data were retrieved, we reanalysed meta-analyses and calculated the weight of the additional RCTs and the change in summary statistics by comparison with the original meta-analysis.
Results Among 223 selected systematic reviews, 116 (52%) did not report a search of trial registries; 21 of these did not report the information to perform the search (key words, search date). A search was performed for 95 systematic reviews; for 54 (57%), no additional RCTs were found and for 41 (43%) 122 additional RCTs were identified. The search allowed for increasing the number of patients by more than 10% in 19 systematic reviews, 20% in 10, 30% in seven, and 50% in four. Moreover, 63 RCTs had results available; the results for 45 could be included in a meta-analysis. 14 systematic reviews including 45 RCTs were reanalysed. The weight of the additional RCTs in the recalculated meta-analyses ranged from 0% to 58% and was greater than 10% in five of 14 systematic reviews, 20% in three, and 50% in one. The change in summary statistics ranged from 0% to 29% and was greater than 10% for five of 14 systematic reviews and greater than 20% for two. However, none of the changes to summary effect estimates led to a qualitative change in the interpretation of the results once the new trials were added.
Conclusions Trial registries are an important source for identifying additional RCTs. The additional number of RCTs and patients included if a search were performed varied across systematic reviews.
We thank Laura Smales (BioMedEditing, Toronto, Canada) for English language proofreading and Carolina Riveros for data extraction.
Contributors: MB, AY, IB, and PR conceived, designed the study, and wrote the protocol MB, IB, and AY wrote the first draft. AY, IB, EP, and PR contributed to the writing of the manuscript. MB, AY, EP, and IB analysed the data. AY and IB had access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. MB and AY contributed equally to the manuscript. AY is the guarantor.
Funding: This study received no funding.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not required.
Data sharing: All data from this study—including literature searches, additional explanatory material, and data extraction forms—are available on request.
Transparency: The lead author (AY) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.