Susan van Rooyen, Fiona Godlee, Stephen Evans, Nick Black, Richard Smith
van Rooyen S, Godlee F, Evans S, Black N, Smith R.
Effect of open peer review on quality of reviews and on reviewers'recommendations: a randomised trial
BMJ 1999; 318 :23
doi:10.1136/bmj.318.7175.23
Open peer review: no need for change
Sir
I read with interest the paper by van Rooyen et al (1) regarding the effect of open peer review on
quality of reviews submitted to the BMJ. I referee for about ten journals
(though not the BMJ) and I see it as my responsibility to alert editors
(whose sole decision it is to accept/reject papers) to poor quality
papers. Papers that are poorly written, poorly thought out, poor in terms
of analysis and those papers which are plain fraudulent (although I have
come across only one case of fraud in ten years or so of refereeing).
My concern with open peer review is that reviewers may be more
inclined to be less critical of manuscripts and therefore point the editor
to 'acceptance'. This is indeed what the authors found in their study.
Given that we are under pressure to publish I would advise a note of
caution against journal editors bringing in open peer review willy nilly.
Occasionally, disputes arise
between editors, reviewers and authors (most usually after a paper is
rejected). I can envisage that overcoming such disputes may be much more
of a problem if the reviewer is known to the author. A 'rejected' author
may be in a stronger position to challenge a referee is s/he is known to
them. What might happen if the same referee should reject the same author
for a different study (in the same or indeed different journal)? Disputes
could mount, become near on impossible to solve with the editor more
likely to side with the referee.
Some journals ask authors to suggest possible referees. Although
editors are under no obligation to use them, would this practice continue
under open peer review? Could an author suggest whom not to referee?
There is an issue of response time. Although the BMJ has a fast turn
around of papers this is not necessarily the case for the more specialist
journals. The authors found that 'Reviewers randomised to be asked to be
identified were 12% (95% confidence interval 0.2% to 24%) more likely to
decline to review than reviewers randomised to remain anonymous (35% v
23%)'. Such a refusal 'rate' could lead to a much lengthier review
process than already occurs. I'm sure most
of us have had cause to complain about lengthy delays by reviewers at some
stage or other. The authors rightly point
out that the more specialist journals have less of a referee
pool to call upon anyway.
If a reviewer declines to review, would the editor be obliged to tell
the author whom refused, and why?
Finally, in case the authors are wondering, I prefer to remain
anonymous when reviewing papers while at the same time preferring to know
the authors' names. Indeed, until '...an instrument capable of measuring
the most interesting and least accessible outcome of all - manuscript
quality' (2) come into being this is
how I would prefer it to remain.
Yours faithfully
Alan S Rigby,
Senior Lecturer in Statistics & Epidemiology,
Chartered Statistician,
Division of Child Health,
Sheffield Children's Hospital,
University of Sheffield,
Sheffield S10 2TH
References
1. van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer
review on quality of reviews and on reviewers' recommendations: a
randomised trial. Br Med J 1999:18: 23-27.
2. Goldbeck-Wood S. Evidence on peer review - scientific quality
control or smokescreen? Br Med J 1999: 318:44-45.
Competing interests: No competing interests