BMJ  2004;328:673 (20 March), doi:10.1136/bmj.38023.700775.AE (published 2 March 2004)

Paper

Effects of training on quality of peer review: randomised controlled trial

¹ BMJ Editorial Office, BMA House, Tavistock Square, London WC1H 9JR, ² London School of Hygiene and Tropical Medicine, London WC1E 7HT

Objective To determine the effects of training on the quality of peer review.

Design Single blind randomised controlled trial with two intervention groups receiving different types of training plus a control group.

Setting and participants Reviewers at a general medical journal.

Interventions Attendance at a training workshop or reception of a self taught training package focusing on what editors want from reviewers and how to critically appraise randomised controlled trials.

Main outcome measures Quality of reviews of three manuscripts sent to reviewers at four to six monthly intervals, evaluated using the validated review quality instrument; number of deliberate major errors identified; time taken to review the manuscripts; proportion recommending rejection of the manuscripts.

Results Reviewers in the self taught group scored higher in review quality after training than did the control group (score 2.85 v 2.56; difference 0.29, 95% confidence interval 0.14 to 0.44; P = 0.001), but the difference was not of editorial significance and was not maintained in the long term. Both intervention groups identified significantly more major errors after training than did the control group (3.14 and 2.96 v 2.13; P < 0.001), and this remained significant after the reviewers' performance at baseline assessment was taken into account. The evidence for benefit of training was no longer apparent on further testing six months after the interventions. Training had no impact on the time taken to review the papers but was associated with an increased likelihood of recommending rejection (92% and 84% v 76%; P = 0.002).

Conclusions Short training packages have only a slight impact on the quality of peer review. The value of longer interventions needs to be assessed.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

This article has been cited by other articles:

• Vetter, N. (2007). Editing the JPH. J Public Health (Oxf) 29: 215-217 [Full text]  
• Sheiman, R. G. (2007). The RSNA Reviewer Mentorship Program. Radiology 244: 631-632 [Full text]  
• Carpenter, J. R., Kenward, M. G., White, I. R. (2007). Sensitivity analysis after multiple imputation under missing at random: a weighting approach. Stat Methods Med Res 16: 259-275 [Abstract]  
• White, I. R, Carpenter, J., Evans, S., Schroter, S. (2007). Eliciting and using expert opinions about dropout bias in randomized controlled trials. Clin Trials 4: 125-139 [Abstract]  
• Tite, L., Schroter, S. (2007). Why do peer reviewers decline to review? A survey. J. Epidemiol. Community Health 61: 9-12 [Abstract] [Full text]  
• Tite, L., Schroter, S. (2006). Evidence based publishing. BMJ 333: 366-366 [Full text]  
• Smith, R. (2006). Peer review: a flawed process at the heart of science and journals.. JRSM 99: 178-182 [Full text]  
• Schroter, S., Tite, L., Hutchings, A., Black, N. (2006). Differences in Review Quality and Recommendations for Publication Between Peer Reviewers Suggested by Authors or by Editors. JAMA 295: 314-317 [Abstract] [Full text]  
• Savulescu, J, Viens, A M (2005). What makes the best medical ethics journal? A North American perspective. J. Med. Ethics 31: 591-597 [Abstract] [Full text]  
• Davidoff, F. (2004). Improving peer review: who's responsible?. BMJ 328: 657-658 [Full text]  
• Schroter, S., Groves, T. (2004). BMJ training for peer reviewers. BMJ 328: 658-658 [Full text]  

Rapid Responses:

Read all Rapid Responses

Effect of training on quality of peer review

Robert West
bmj.com, 26 Mar 2004 [Full text]

Data-analysis errors

Eric N Grosch
bmj.com, 12 Apr 2004 [Full text]

Were there fatal order effects in this trial?

Trisha Greenhalgh
bmj.com, 19 Apr 2004 [Full text]