Intended for healthcare professionals

CCBYNC Open access

Adequacy of authors’ replies to criticism raised in electronic letters to the editor: cohort study

BMJ 2010; 341 doi: (Published 10 August 2010) Cite this as: BMJ 2010;341:c3926
  1. Peter C Gøtzsche, director1,
  2. Tony Delamothe, editor2,
  3. Fiona Godlee, editor2,
  4. Andreas Lundh, PhD student1
  1. 1Nordic Cochrane Centre, Rigshospitalet and University of Copenhagen, Dept 3343, Blegdamsvej 9, DK-2100 Copenhagen, Denmark
  2. 2BMJ, BMA House, Tavistock Square, London
  1. Correspondence to: P C Gøtzsche pcg{at}
  • Accepted 1 June 2010


Objective To investigate whether substantive criticism in electronic letters to the editor, defined as a problem that could invalidate the research or reduce its reliability, is adequately addressed by the authors.

Design Cohort study.

Setting BMJ between October 2005 and September 2007.

Inclusion criteria Research papers generating substantive criticism in the rapid responses section on

Main outcome measures Severity of criticism (minor, moderate, or major) as judged by two editors and extent to which the criticism was addressed by authors (fully, partly, or not) as judged by two editors and the critics.

Results A substantive criticism was raised against 105 of 350 (30%, 95% confidence interval 25% to 35%) included research papers, and of these the authors had responded to 47 (45%, 35% to 54%). The severity of the criticism was the same in those papers as in the 58 without author replies (mean score 2.2 in both groups, P=0.72). For the 47 criticisms with replies, there was no relation between the severity of the criticism and the adequacy of the reply, neither as judged by the editors (P=0.88 and P=0.95, respectively) nor by the critics (P=0.83; response rate 85%). However, the critics were much more critical of the replies than the editors (average score 2.3 v 1.4, P<0.001).

Conclusions Authors are reluctant to respond to criticisms of their work, although they are not less likely to respond when criticisms are severe. Editors should ensure that authors take relevant criticism seriously and respond adequately to it.


  • We thank the rapid respondents for their judgments and comments and Juliet Walker (BMJ), for administrative help.

  • Contributors: PCG conceived the idea, wrote the draft protocol and manuscript, and did the data analyses. He is guarantor. TD and FG commented on the protocol and evaluated the selected responses and replies. AL collected and screened the research papers and did a supplementary analysis of letters in the print issues. All authors contributed to the manuscript.

  • Funding: This study received no funding. The Nordic Cochrane Centre and BMJ provided in-house resources.

  • Competing interests: All authors have completed the unified competing interest form at (available on request from the corresponding author) and declare that all authors had: (1) no financial support for the submitted work from anyone other than their employer; (2) no financial relationships with commercial entities that might have an interest in the submitted work; (3) no spouses, partners, or children with relationships with commercial entities that might have an interest in the submitted work; (4) TD and FG are editors of the BMJ.

  • Ethical approval: Not required.

  • Data sharing: Data sharing: A full dataset is available at These data may only be used for replication of the analyses published in this paper or for private study. Express written permission must be sought from the authors for any other data use. Consent was not obtained from study participants, and we therefore needed to delete spontaneous comments and also the judgements made by the rapid responders. Researchers are welcome to contact us if they wish to know which specific criticisms we selected for the individual papers.

  • Accepted 1 June 2010

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: and

View Full Text