Intended for healthcare professionals

Head To Head

Is open peer review the fairest system? Yes

BMJ 2010; 341 doi: https://doi.org/10.1136/bmj.c6424 (Published 16 November 2010) Cite this as: BMJ 2010;341:c6424
  1. Trish Groves, deputy editor
  1. 1BMJ, London WC1H 9JR, UK
  1. tgroves{at}bmj.com

Trish Groves argues that telling authors who has reviewed their paper has helped to make the process fairer, but Karim Khan (doi:10.1136/bmj.c6425) is concerned that it stops reviewers from being completely frank

When I arrived at the BMJ in 1989 I often had to cut (with scissors) and paste (with glue) the highlights of a reviewer’s report on to another sheet of paper and photocopy it. Only then could I enclose the review with my letter to an author, as the original was too rude or destructive. At the same time, numerous studies of peer review were finding that it was riddled with systematic biases— against authors, their institutions, their nationality, and their sex. Such abuses of anonymity were so common that Drummond Rennie, then deputy editor of JAMA, called for reviews to be signed, so that journals would link “privilege and duty, by reminding the reviewer that with power comes responsibility: that the scientist invested with the mantle of the judge cannot be arbitrary in his or her judgment and must be a constructive critic.”1 Although JAMA did not adopt open (signed) review, the BMJ did: in 1999.

Why didn’t we adopt double blind review, withholding reviewers’ names and also removing authors’ identity from reviewed articles? Firstly, because evidence, including that from randomised controlled trials, had found no benefit of such blinded review.1 More importantly, the BMJ’s editors felt strongly that withholding reviewers’ names was an unfair and kafkaesque system that seriously limited transparency. We hoped that unmasked reviewers would rise to the challenge and act as critical friends, unafraid of face to face (virtual) scientific discourse. Dream on, said the critics: if you ask reviewers to sign their reports they will perch on the fence and blandly say nothing much. They were wrong, as a BMJ randomised controlled trial2 and now more than a decade’s experience have shown.

Open debate

Perhaps open peer review has succeeded at the BMJ because we make it clear that editors, not reviewers, decide whether to accept or reject submissions. Or has it worked because the BMJ is a general journal that doesn’t serve just one specialty or have to rely on one group’s goodwill or patronage? The same applies to the Journal of the Royal Society of Medicine, which also uses signed open review. PLoS Medicine has taken a slightly different path. In the first two years editors expected all reviewers to provide either a signed review or a reason for not signing, but in 2007 the policy changed to one of encouraging open review while honouring any requests for anonymity. None of the other big general medical journals has even tried open review.

The argument that open review won’t work within the closer circle of a specialist journal—particularly one owned by a professional body or society—was refuted by a randomised controlled trial conducted at the British Journal of Psychiatry.3 And the specialist journals in the BMC series (set up by Fiona Godlee, now editor of the BMJ, who was BioMed Central’s first editorial director for medicine) have opened the peer review process wider still. They post alongside each accepted article its prepublication history, comprising the original submitted version plus the signed reviewers’ reports.4

Across academia there are many other models of journal peer review. Varying definitions of “open” review include asking a wider community to come to the journal’s website and rate articles.5 Open review at the BMJ currently means that all reviewers sign their reports, declare their competing interests, and desist from making additional covert comments to the editors.6 Most BMJ authors and reviewers seem happy with this approach, and some actively applaud it. Reviewers generally make detailed and constructive comments; only a handful of reviewers each year cite open review as their reason for declining BMJ assignments, and we have had no reports of serious adverse events arising from open review. Furthermore, a randomised controlled trial has shown that it is feasible to publish reviewers’ comments without detriment to reviews’ quality.7 The BMJ will start to post prepublication histories from mid-2011,8 and our new sister journal BMJ Open is using the same approach from launch (http://blogs.bmj.com/bmjopen/).

Giving credit

Open peer review has another advantage that might persuade more specialist, small, or local journals to take the plunge: it gives reviewers credit that goes well beyond seeing their names on a thank you list in the last journal of the year. Signed reviews are seen by authors and other reviewers who are often important peers, senior figures, or rising stars. A well researched and constructively worded review can be a great help, even if the journal then rejects the article. Once authors have recovered from rejection they can use that criticism to submit a revised article more successfully to the next journal, or to appeal. Many authors will then appreciate and remember the careful reviewer whose contribution made a difference. In addition, reviewers’ reports can be excellent learning tools, both for their substantive content and for their style: they can show how to conduct peer review well. Peer review is a linchpin of science, yet it is too often a thankless task reliant on underappreciated skills learnt without teaching materials. For all of these reasons, putting a name to the anonymous face is surely a good thing.

Notes

Cite this as: BMJ 2010;341:c6424

Footnotes

  • Research, doi:10.1136/bmj.c5729
  • Competing interests: The author has completed the unified competing interest form at www.icmje.org/coi_disclosure.pdf (available on request from the author) and declares no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

  • Provenance and peer review: Commissioned; not externally peer reviewed.

References

View Abstract