Intended for healthcare professionals

CCBYNC Open access
Research

Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial

BMJ 2011; 343 doi: https://doi.org/10.1136/bmj.d6783 (Published 22 November 2011) Cite this as: BMJ 2011;343:d6783
  1. E Cobo, senior statistics editor and senior statistical lecturer12,
  2. J Cortés, statistical researcher2,
  3. J M Ribera, general secretary and chief of clinical haematology department1345,
  4. F Cardellach, general secretary and professor of internal medicine16,
  5. A Selva-O’Callaghan, editorial committee member and senior lecturer in internal medicine137,
  6. B Kostov, statistical researcher 8,
  7. L García, statistical researcher2,
  8. L Cirugeda, statistical researcher9,
  9. D G Altman, professor of statistics in medicine10,
  10. J A González, senior statistical lecturer2,
  11. J A Sànchez, senior statistical lecturer2,
  12. F Miras, statistical researcher2,
  13. A Urrutia, editorial committee member and senior lecturer in internal medicine134,
  14. V Fonollosa, editorial committee member and professor of internal medicine137,
  15. C Rey-Joly, current editor and professor of internal medicine134,
  16. M Vilardell, editor in chief and professor of internal medicine137
  1. 1Medicina Clínica, Elsevier-Barcelona, Barcelona 08021, Spain
  2. 2Universitat Politècnica Catalunya, Barcelona
  3. 3Universitat Autònoma de Barcelona, Barcelona
  4. 4Hospital Germans Trias I Pujol, Badalona, Spain
  5. 5José Carreras Leukaemia Research Institute, Catalan Institute of Oncology, ICO, Badalona
  6. 6Universitat de Barcelona and Hospital Clínic, Barcelona
  7. 7Vall D’Hebron Hospital, Barcelona
  8. 8Primary Health Care Center Les Corts, GESCLINIC, Barcelona
  9. 9Centre for Research in Environmental Epidemiology, Barcelona
  10. 10Centre for Statistics in Medicine, University of Oxford, Oxford, UK
  1. Correspondence to: E Cobo erik.cobo{at}upc.edu
  • Accepted 23 September 2011

Abstract

Objective To investigate the effect of an additional review based on reporting guidelines such as STROBE and CONSORT on quality of manuscripts.

Design Masked randomised trial.

Population Original research manuscripts submitted to the Medicina Clínica journal from May 2008 to April 2009 and considered suitable for publication.

Intervention Control group: conventional peer reviews alone. Intervention group: conventional review plus an additional review looking for missing items from reporting guidelines.

Outcomes Manuscript quality, assessed with a 5 point Likert scale (primary: overall quality; secondary: average quality of specific items in paper). Main analysis compared groups as allocated, after adjustment for baseline factors (analysis of covariance); sensitivity analysis compared groups as reviewed. Adherence to reviewer suggestions assessed with Likert scale.

Results Of 126 consecutive papers receiving conventional review, 34 were not suitable for publication. The remaining 92 papers were allocated to receive conventional reviews alone (n=41) or additional reviews (n=51). Four papers assigned to the conventional review group deviated from protocol; they received an additional review based on reporting guidelines. We saw an improvement in manuscript quality in favour of the additional review group (comparison as allocated, 0.25, 95% confidence interval –0.05 to 0.54; as reviewed, 0.33, 0.03 to 0.63). More papers with additional reviews than with conventional reviews alone improved from baseline (22 (43%) v eight (20%), difference 23.6% (3.2% to 44.0%), number needed to treat 4.2 (from 2.3 to 31.2), relative risk 2.21 (1.10 to 4.44)). Authors in the additional review group adhered more to suggestions from conventional reviews than to those from additional reviews (average increase 0.43 Likert points (0.19 to 0.67)).

Conclusions Additional reviews based on reporting guidelines improve manuscript quality, although the observed effect was smaller than hypothesised and not definitively demonstrated. Authors adhere more to suggestions from conventional reviews than to those from additional reviews, showing difficulties in adhering to high methodological standards at the latest research phases. To boost paper quality and impact, authors should be aware of future requirements of reporting guidelines at the very beginning of their study.

Trial registration and protocol Although registries do not include trials of peer review, the protocol design was submitted to sponsored research projects (Instituto de Salud Carlos III, PI081903).

Footnotes

  • We thank David Moher, Umair Mallick, and Julie Morris for their helpful suggestions to the manuscript; Matthew Elmore for English editing; and Mercedes Belmonte for managing the manuscript follow-up.

  • Contributors: EC (guarantor) and JMR had the original idea, and with FC, AS, DGA, AU, VF, CRJ, and MV, designed the study. JMR, FC, and AS managed and monitor the manuscript flow. EC did the additional review based on reporting guidelines. BK, LG, and JC rated the manuscript quality. BK and LG assessed the suggestion relevance and adherence. LC managed the database, classified, merged, and masked the reviewer suggestions. JC and JAS programmed and did the analysis. FM and JAG designed, programmed, and maintained the allocation algorithm. EC, JAG, and JC designed the figures. JAG and JC audited the database, statistical programming, and editing process. All authors contributed to and approved the final manuscript.

  • All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf and declare that: JC, LG, LC, BK, and FM are supported by the “Bioestadística para no estadísticos” learning programme; they have no relationships with companies that might have an interest in the submitted work in the previous 3 years; their spouses, partners, or children have no financial relationships that may be relevant to the submitted work; and they have no non-financial interests that may be relevant to the submitted work.

  • Ethics approval: both authors and referees were informed that their material might be used to assess the quality of the editorial process.

  • Data sharing: Technical appendix, statistical code, and dataset available from the corresponding author at erik.cobo{at}upc.edu.

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

View Full Text