Editor's Choice | This Week in BMJ | Press releases
BMJ No 7122 Volume 315 Education and debate Saturday 13 December 1997
Your letter failed to win a place...Eyal ShaharLetters that comment on published work are treated differently from the original article itself. They are rarely subject to peer review, and scientific explanations are not usually given when they are rejected. Professor Shahar argues that this is unjustified and counterproductive to scientific inquiry, and that criticism of published work should be subject to peer review. The quality of published scientific work is evaluated at
least twice - by a handful of reviewers and editors d The voice of the reader is heard through letters written by the
relatively few who formalise their critique in writing. Letters are
sometimes better thought out than the original article. They may
identify inaccuracies that were missed by formal peer review or uncover
flaws in design, analysis, or interpretation. Peer review does not
preclude error.(1) Although letter columns are considered
important by editorial boards,(3) the fate of correspondence
on published work is rarely determined by peer review. The editor(s)
usually make the decision whether to publish, and rejection notes to
authors are often standardised and contain little, if any, scientific
explanation for the decision. Vague statements such as "in the face
of fierce competition, your letter failed to win a place" or "many
worthwhile contributions must be declined simply for lack of space"
are typical.
The example I provide below illustrates the shortcomings of
editorial practices. It is a "letter to the editor" (coauthored by
Paul G McGovern) that was rejected with no specific explanation. Since
the letter challenges unequivocally the main conclusion of an article,
either the challenge or the original conclusion must have been faulty.
The letter can, therefore, only be rejected on the assumption that its
contents are faulty. I am asking the reader to judge this assertion
(ignoring the question of whether the letter's content is true or
false). I also provide the letter as a test case of my proposal for
peer review of correspondence, inviting concrete evaluation of its
content that will either support or oppose the editorial verdict in
this case.
Sir The study by Andreotti et al was an observational cohort study of 23
patients who had suffered a myocardial infarction. Of these patients,
fourteen had experienced preinfarction angina and 9 had not
("exposed" and "unexposed" groups, respectively, in
epidemiological terminology.) The only acceptable inference from this
small study is that exposure to unstable angina before a myocardial
infarction was associated with more rapid reperfusion and smaller
infarcts than were observed in the absence of prodromal unstable
angina. That the two groups of patients happened to be treated with
tissue plasminogen activator (and by other means) further defines the
cohort characteristics, but is irrelevant to the question of whether
thrombolytic therapy is more effective in the presence of preinfarction
angina than in its absence. As the authors acknowledge, reperfusion
occurs spontaneously during the course of myocardial infarction and therefore its rate is far from being
entirely determined by treatment with tissue plasminogen activator. For
example, Andreotti et al would have observed exactly the same results
if treatment with tissue plasminogen activator had identical effects in
both groups of patients yet infarctions that follow unstable angina are
associated with faster rates of spontaneous reperfusion (and tend to be
smaller) than infarctions without prodromal unstable angina.
To show a differential effect of treatment with tissue plasminogen
activator in the presence (versus absence) of preinfarction angina, one
should demonstrate a statistically significant interaction between two
effects: the effect of treatment with tissue plasminogen activator in
myocardial infarction patients who had preinfarction angina and the
effect of treatment with tissue plasminogen activator in those who did
not. Each of these effects can only be estimated by comparing the
reperfusion rate and infarction size in patients treated with tissue
plasminogen activator to these measures in patients who were not,
preferably by a randomised design. Unfortunately, such a design is no
longer feasible since it is ethically unacceptable to withhold
thrombolytic therapy from patients who should receive it.
Only two legitimate reasons exist for the journal to reject this
letter - most of its content was judged faulty or the editor(s)
preferred to publish another letter with a similar message. Lack of
space should not justify rejection because space should be made
available for corrections, even at the expense of delaying the
publication of new original articles. I saw no letter with a similar
message in follow up correspondence.(5,6)
When a manuscript is rejected by a journal, the authors may get it
published elsewhere. When a letter criticising a published article is
rejected no such remedy is usually available. In this sense, an
erroneous editorial decision to reject a letter may be more damaging to
scientific progress than an erroneous decision to reject a
manuscript.
The differential treatment of scientific correspondence and manuscripts
is not unique to scientific journals. Scientists rarely cite criticism
of original research,(2) and academic institutions give
little or no credit for published letters.
The most truthful message in any particular case cannot be deduced from
some general rule of importance, even if there were an empirical way of
substantiating such a rule. What is important is not the origin of the
message (for example, authors of a manuscript or authors of a letter)
but the message itself. Is it scientific or perhaps pseudoscientific?
Does it survive logical criticism or not? Differential treatment of
scientific communications introduces a potential prejudice into the
search for objective knowledge.
Peer review of manuscripts is based on criteria such as clarity,
validity, originality, and relevance. Peer review of correspondence
could follow the same path. Letters to the editor (including the reply
of the authors of the original article, which usually escapes
rejection) should be evaluated for their scientific merit, and their
fate should be determined on specific grounds. A letter may be
rejected, for example, because its argument is judged to be rhetorical,
its content faulty, or the thoughts of the author poorly articulated.
Unexplained decisions leave too much room for speculation and,
sometimes, suspicion.
Opponents of my suggestion for peer review of correspondence may argue
that the process is lengthy and that it is essential to publish follow
up correspondence quickly, while the original article is still fresh in
the reader's memory. Scientific progress, however, is not a race
against an arbitrary deadline.
It might be argued that formulating specific criteria for evaluating
letters might be difficult. But a scientific communication - that is,
one that claims to advance objective knowledge - should lend itself to
critical appraisal, above and beyond just "feeling" for its merit.
Some might claim that peer review for letters could lead to an
infinite, regressive process of publishing letters that comment on
letters, and that such a process would have to be stopped arbitrarily
anyway. Most debate, however, fades naturally away after one or two
rounds, and if it does not, peer review should be responsible for
identifying reiterative stages of a correspondence and for stopping it.
Remember too that arbitrary termination also happens with peer review
of manuscripts since the reviewers' critiques are not s In a recent article, Bhopal and Tonks asked, "If published critical
comment is considered integral to research should it not be peer
reviewed?"(2) My answer is "Yes, it should." Editors
would do justice to science if they solicited peer review of
correspondence, including peer review of the reply from the authors of
the original article. Editors who object should provide authors of
rejected letters to the editor with their own scientific review to
support their decision.
Neil McIntyre, Karl Popper
I thank Lori Vitelli, Maureen Smith, Jacqueline Dekker, the
anonymous reviewer, and the editorial board of the BMJ for helpful
comments on earlier versions of the article.
(Accepted 4 March 1997) Division of Epidemiology,
email: shahar@epivax.epi.umn.edu
References
1 Spodick D H. The peer review system and the editor's
correspondence. Arch Intern Med 1981;141:1121.
2 Bhopal R S, Tonks A. The role of letters in reviewing research:
always look for letters that follow original papers. BMJ
1994;308:1582-3.
3 International Committee of Medical Journal Editors. Statements
from the Vancouver Group. BMJ 1989;299:1394-5.
4 Andreotti F, Pasceri V, Hackett D R, Davies G J, Haider A W,
Maseri A. P
5 Migrino R Q, Moliterno D J, Topol E J. Preinfarction angina.
N Engl J Med 1996;335:59.
6 Kloner R A, Gibson M, Cannon C, Braunwald E. Preinfarction
angina. N Engl J Med 1996;335:59-60.
|