Prepublication histories and open peer review at The BMJ

Over the past 15 years peer reviewers for The BMJ have shown, by signing their reviews and declaring to authors and editors any relevant competing interests, that they are unafraid of transparent scientific discourse. Now we are opening up our process to make our reviewers’ role as authors’ critical friends visible to all.

From this autumn on thebmj.com all research articles, and certain scholarly articles in The BMJ’s Analysis section, will have an article tab marked “Reviews.” Clicking on this will open the article’s prepublication history, comprising all signed reviews (including those by statisticians and patient peer reviewers), previous versions of the article, the study protocol for any clinical trial, the report from The BMJ’s manuscript committee meeting, and the authors’ responses to the editors’ and reviewers’ comments. As now, reviewers will not be able to make private comments to editors, except in the rare case when a reviewer wants to express concerns about the scientific integrity of the work (www.bmj.com/about-bmj/resources-reviewers/guidance-peer-reviewers).

Such open peer review should increase the accountability of reviewers and editors, at least to some extent. Importantly, it will also give due credit and prominence to the vital work of peer reviewers. At present, peer review activities are under-recognised in the academic community. We hope that reviewers will find this increased visibility helpful when demonstrating the extent and impact of their academic work and that they and others will cite and share their reviews as a learning resource.

Greater accountability and transparency are clearly laudable goals.1 2 3 But is there a downside to open peer review? Does it, for instance, provide “more scope for power relationships to favour ‘the great and the good,’” as Karim Khan, editor of the British Journal of Sports Medicine, feared?4 Or might it produce a flurry of spurious criticism motivated by commercial interest, academic jealousy, or pettiness? Such problems may occur, but we think the good consequences of more open editorial and peer review practices will outweigh any harms. One beneficial result may be that access to prepublication histories will encourage readers and other interested parties to participate in the self correction processes that are vital to the credibility of medical research.

Randomised controlled trials conducted at The BMJ since the turn of the millennium found that removing anonymity improved the tone and constructiveness of reviews without detriment to scientific and editorial value. One of the trials also found that telling reviewers that prepublication histories might be posted online did not affect the quality of peer review.5 6 These positive outcomes may reflect The BMJ’s position as a general medical journal that is relatively free from academic turf wars and the fact that editors, not reviewers, decide whether to accept or reject submissions. However, in the trial, reviewers who were asked to look at a paper that was randomised to online prepublication history were moderately more likely to decline the assignment or not to reply.6 We will keep a close eye out for such a trend and will act quickly to solicit further reviews when needed.

A recent study investigated the potential of open peer review to improve the reporting of randomised trials.7 The authors looked at changes in reporting of items on the CONSORT (Consolidated Standards of Reporting Trials) checklist between the original and final versions of 93 randomised controlled trials in BioMed Central’s series of open access journals in medicine (www.biomedcentral.com/authors/bmcseries), which have been posting prepublication histories since 2000. They also looked at changes requested by peer reviewers and at authors’ subsequent responses. Most changes had a positive effect on reporting of the trials’ methods and results and on toning down of conclusions. Reviewers missed some important deficiencies in the articles, however, and in 15 of the 93 articles their advice actively worsened the reporting.

This study’s authors did not know the extent to which these findings might be generalisable to other journals with different editorial and peer review processes. We hope that the open review processes at The BMJ and BMJ Open (which has successfully implemented prepublication histories of well over 2500 articles so far (http://bmjopen.bmj.com)) will provide equally fertile ground for study.

This action follows other important steps that The BMJ has taken to ensure the integrity of the research we publish. These include prospective registration of clinical trials, the routine use of research reporting guidelines and checklists, open (that is, signed) peer review, campaigning for data sharing and linking to datasets in the Dryad digital repository, and endorsement of the RIAT (Restoring Invisible and Abandoned Trials) initiative to publish or correct abandoned or misreported clinical trials.8 Posting prepublication histories is another part—but not the conclusion—of a process aimed at restoring trust in the clinical research enterprise. We will closely monitor the effects of our new, more open review and editorial processes and refine or modify them as needed. We invite comments and suggestions from our readers.