Effect of revealing authors’ conflicts of interests in peer review: randomized controlled trial
BMJ 2019; 367 doi: https://doi.org/10.1136/bmj.l5896 (Published 06 November 2019) Cite this as: BMJ 2019;367:l5896Commercial influence in health: from transparency to independence
Click here to read the complete collection
Linked Editorial
Declaring interests and restoring trust in medicine
- Leslie K John, Marvin Bower associate professor of business administration1,
- George Loewenstein, Herbert Simon university professor of economics and psychology2,
- Andrew Marder, independent consultant1,
- Michael L Callaham, professor emeritus of emergency medicine3
- 1Harvard Business School, Harvard University, Boston, MA, USA
- 2Department of Social and Decision Sciences, Carnegie Mellon University, Pittsburgh, PA, USA]
- 3Emergency Medicine, University of California at San Francisco, San Francisco, CA, USA
- Correspondence to: L K John ljohn{at}hbs.edu (or @lesliekjohn on Twitter)
- Accepted 26 September 2019
Abstract
Objective To assess the effect of disclosing authors’ conflict of interest declarations to peer reviewers at a medical journal.
Design Randomized controlled trial.
Setting Manuscript review process at the Annals of Emergency Medicine.
Participants Reviewers (n=838) who reviewed manuscripts submitted between 2 June 2014 and 23 January 2018 inclusive (n=1480 manuscripts).
Intervention Reviewers were randomized to either receive (treatment) or not receive (control) authors’ full International Committee of Medical Journal Editors format conflict of interest disclosures before reviewing manuscripts. Reviewers rated the manuscripts as usual on eight quality ratings and were then surveyed to obtain “counterfactual scores”—that is, the scores they believed they would have given had they been assigned to the opposite arm—as well as attitudes toward conflicts of interest.
Main outcome measure Overall quality score that reviewers assigned to the manuscript on submitting their review (1 to 5 scale). Secondary outcomes were scores the reviewers submitted for the seven more specific quality ratings and counterfactual scores elicited in the follow-up survey.
Results Providing authors’ conflict of interest disclosures did not affect reviewers’ mean ratings of manuscript quality (Mcontrol=2.70 (SD 1.11) out of 5; Mtreatment=2.74 (1.13) out of 5; mean difference 0.04, 95% confidence interval –0.05 to 0.14), even for manuscripts with disclosed conflicts (Mcontrol= 2.85 (1.12) out of 5; Mtreatment=2.96 (1.16) out of 5; mean difference 0.11, –0.05 to 0.26). Similarly, no effect of the treatment was seen on any of the other seven quality ratings that the reviewers assigned. Reviewers acknowledged conflicts of interest as an important matter and believed that they could correct for them when they were disclosed. However, their counterfactual scores did not differ from actual scores (Mactual=2.69; Mcounterfactual=2.67; difference in means 0.02, 0.01 to 0.02). When conflicts were reported, a comparison of different source types (for example, government, for-profit corporation) found no difference in effect.
Conclusions Current ethical standards require disclosure of conflicts of interest for all scientific reports. As currently implemented, this practice had no effect on any quality ratings of real manuscripts being evaluated for publication by real peer reviewers.
Footnotes
Contributors: LKJ contributed to the design of the study, supervised data collection, interpreted the results, and wrote the manuscript. GL contributed to the design of the study, interpretation of the results, and revision of the manuscript. AM did the data analysis and contributed to the interpretation of the results and revision of the manuscript. MLC conceived the study and contributed to its design, implemented and supervised data collection, interpreted the results, and revised the manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. LKJ and MLC are the guarantors.
Funding: This research was funded using LKJ’s annual research budget allocation from Harvard Business School.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: The study was approved by the Institutional Review Board of Harvard University (IRB: 24032). All participants gave informed consent before taking part in the research.
Transparency statement: The lead author (the manuscript’s guarantor) affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained..
Data sharing: No additional data available.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.