Double blind peer reviews are fairer and more objective, say academics
BMJ 2008; 336 doi: https://doi.org/10.1136/bmj.39476.357280.DB (Published 31 January 2008) Cite this as: BMJ 2008;336:241
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
The survey that you report cannot be relied upon. Although 3040
responses were available for analysis, these represented only 7.7% of the
authors and editors to whom questionnaires were sent. It is very
surprising that so many academics and editors declined to reply, and one
wonders if the questionnaire was overlong or in some way was thought to be
either boring or offensive. We cannot claim to know the preferences of the
majority of authors or editors.
For serious scientific journals, peer review is essential before a
paper can be published. Post-publication review is valuable but is limited
in scope, because only a fraction of readers are qualified to review a
complex article on an abstruse subject, and only a limited number of post-
publication reviews can be accepted for the Correspondence columns of
journals. Pre-publication review acts as a filter provided by experts so
that high standards of scientific reporting can be maintained.
Double blind peer review is the best available method. The reviewers
need to be anonymous so that they can express honest opinions,
particularly when recommending rejection of a paper, without fear of
repercussions. The authors - and the source - of a paper should be
anonymous, because reviewers can be influenced by the prestige (or
otherwise) of an author or of an institution. Personal acrimony and
academic jealousy are regrettable but very real and anonymity helps to
exclude them.
It is true that peer review cannot altogether exclude plagiarism,
fraud or misconduct. Editors, knowing the names of individuals and
institutions, are somewhat better placed to detect such aberrations.
Security might be improved by ruling that papers should be certified by
the dean or director of an institution when they are submitted. Only the
most cynical would think it likely that many institutions might have
crooked researchers and a corrupt dean or director!
Competing interests:
None declared
Competing interests: No competing interests
Irrespective of statistical surveys double-blind peer-reviewed
clinical trials can only be as good as the prevailing level of knowledge.
They can never be absolute and hence are only as good as is reasonably
practicable. Knowledge can be used positively or negatively depending upon
the perceived state of knowledge of a subject. It must therefore be
recognised that the current system of clinical trials is open to bias and
abuse. That is not to say that the system should be scrapped in favour of
a less onerous system as proposed in e.g. complementary health research,
but that that there should be greater emphasis upon improving their
validity i.e. understanding why it is necessary to recruit as little as 1-
2% of clinical trial volunteers for their inclusion in a clinical trial.
Such statistics reflects very badly upon the theoretical and statistical
basis for such studies bearing in mind that the drug is subsequently
licensed for use in the wider population.
That there are significant theoretical limitations is recognised by
organisations across the EU including here in the UK where the NIHR and
MHRA are seeking new and better ways to evaluate the performance and
safety of drugs.
To emphasise the situation take some statistics available from
reputable researchers published in peer-reviewed journals which indicate
that medical diagnosis is circa 50% accurate (ref 1) and that drugs have
circa 50% efficacy (ref 2). This theoretical and technological deficit can
only be addressed by looking more widely than solely in the area of
biomedical research i.e. to investigate the neuro-regulatory role of the
brain and its ability to regulate the physiological systems - which
regulate organ, cell and molecular biochemistry.
Goverment is influenced by academia and by industry however a great
deal of academic research is funded, directly or indirectly, by industry.
This creates an 'industry-roadblock' which influences or limits
technological progress.
Graham Ewing
1. British Medical Journal 18th March, 2000
2. Spear, B., Heath-Chiozzi, M., Huff, J. (2001) Clinical
Applications of Pharmacogenetics. Trends in Molecular Medicine, 7:5:201-
204.
Competing interests:
co-author of the book 'Virtual Scanning - a new generation of healthcare - beyond biomedicine'
Competing interests: No competing interests
I am very surprised at both the headline and the way in which this
survey was reported. The survey had a response rate of only 7.7%. So on
what basis have you justified a headline of 'double blind peer reviews are
fairer and more objective, say academics'? In fact, 71% of the 7.7% who
responded said that, which by a very rough calculation amounts to fewer
than 6% of the people contacted. The survey is so unrepresentative as to
be of no interest. Indeed, by reporting it in this way, you have made it
misleading. This was worthy of a tabloid newspaper.
Competing interests:
None declared
Competing interests: No competing interests
A fistful of dollars: the key to better reviews is reward.
To enhance peer review under any model whether single, double, or
open, we must introduce direct rewards for referees.
By far, the most compelling aspects of the PRC survey related to the
effectiveness of peer review and whether referees should be paid. Most
respondents were satisfied with peer review, none supported the radical
proposition that peer review was unnecessary, yet there was only limited
support for more direct rewarding of those that do the service of review
(i.e. Table 14 only 35% supported payment). As editors however, we have
experienced increasingly difficulty in securing even 2 reviews for a
manuscript. Similar to the Wild West, enforcement of ideals within the
peer-review system in science is variable by sub-discipline and often with
limited reward and substantial risk. Journal editors, or the sheriffs in
each town, are often unpaid and commit an inordinate amount of time to
review, process comments, and improve manuscripts. These individuals pay
a cost in terms of available time for active research and may be subject
to negative treatment by rejected authors when they in turn submit.
Referees, or the deputies, perform an equally valid service by ensuring
that the quality of the research is appropriate for publication, correct,
and a significant contribution to the literature. These individuals also
pay a cost in terms of time and may or may not be protected depending on
the masking system adopted by the journal. Furthermore, both sets of
individuals and the peer-review system not only provide checks on quality
or social control but act as a form of constructive dialogue in science
prior to dissemination to a wider audience. We propose a paradigm shift.
Reviewing is a discourse, enhances science, and in and of itself should be
viewed as a scientific activity.
Peer review is not a ‘problem’ that requires punishments to the tardy
or those whom decline reviews but rather a valuable mechanism the
scientific community uses to assign merit and improve papers.
Furthermore, even with penalties imposed, good referees would not be
rewarded. Positive incentives or compensation in some form is another
clear set of solutions that may address concerns associated with the
review process such as securing an appropriate number of referees,
protracted lag times, accountability, or quality. Incentives could
include the opportunity for referees to publish comments alongside the
publication (thereby recognizing their contribution and promoting
scientific discourse), prominent publication by journals of review service
by individuals, compensation in the form of awards for top referees that
do excellent service (either in terms of numbers or quality of the
gunslinger), or similar to airline reward programs, journals could offer
frequent, reliable referees priority service (i.e. accelerated review
times) when they next submit a paper. We expect that the latter may
already be in place; however, journal editors should consider making
rewards a transparent component of the system. In light of the increasing
pressure to use metrics to comparatively rank scientists, we suggest that
referees report referee service, view other reviews of a given manuscript
to calibrate their comments and further effective discourse in science,
and develop a portfolio of reviews which would be useful for merit and
tenure exercises. Regardless of the incentives that could be added to the
process, recognition of the men/women ‘with no names’ needs to be
acknowledged more prominently. Hopefully, most of us are one and the same
simply switching hats between roles of hero and villain.
Competing interests:
None declared
Competing interests: No competing interests