Outcome reporting bias in clinical trials: why monitoring mattersBMJ 2017; 356 doi: https://doi.org/10.1136/bmj.j408 (Published 14 February 2017) Cite this as: BMJ 2017;356:j408
- John PA Ioannidis, professor1,
- Arthur L Caplan, professor and director2,
- Rafael Dal-Ré, director3
- 1Departments of Medicine, Health Research and Policy, and Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Palo Alto, CA 94305, USA
- 2Division of Medical Ethics, School of Medicine, NYU Langone Medical Center, New York, NY 10016, USA
- 3Clinical Research, BUC (Biosciences UAM+CSIC) Program, International Campus of Excellence, Universidad Autónoma de Madrid, Ciudad Universitaria de Cantoblanco, Madrid 28049, Spain
- Correspondence to: J P A Ioannidis
Selective reporting of outcomes can bias clinical trials.1 This happens when investigators publish some prespecified outcomes but not others, depending on the nature and direction of the results. They may also add outcomes that were not prespecified or modify some of the prespecified outcomes in subtle or major ways. These selective reporting practices tend to make trial results more attractive but also spurious.
The proposed solution has been trial registration, including the explicit listing of prespecified outcomes before launch, and the transparent description of all changes that occur afterwards. The BMJ asks the guarantor of research to sign a declaration of transparency at submission, stating that the manuscript is an “honest, accurate, and transparent account” of the methods and results, including explanations of “any discrepancies from the study as planned” (http://www.bmj.com/about-bmj/resources-authors/). We discuss efforts to identify discrepancies between trial protocols and published papers and outline possible quality control measures for the future.
From October 2015 to January 2016 the Centre for Evidence Based Medicine Outcome Monitoring Project (COMPare) team checked articles published in the top five general medicine journals based on impact factor—New England Journal of Medicine, Lancet, JAMA, The BMJ, and Annals of Internal Medicine—against protocols and registries pre-dating trial launch.2 Only nine of the 67 clinical trials published (13%) were “perfectly reported,” that is all primary and secondary outcomes were the same in the protocols or registries and the articles. COMPare found that investigators didn’t report the results of 354 outcomes that were prespecified in protocols or registries and that they added—without flagging it to the editor and reviewers—357 outcomes that were not prespecified. COMPare …