What do we know about peer review?

In 2011 the House of Commons Science and Technology Committee held an inquiry into peer review and sought written and oral evidence from a wide range of experts and organisations. The BMJ Group's written evidence reviewed the latest research evidence on the strengths and weaknesses of peer review as a quality control mechanism for scientists, publishers, and the public including:

• measures to strengthen peer review
• the processes by which reviewers with the requisite skills and knowledge are identified, in particular as the volume of multi-disciplinary research increases
• the impact of IT and greater use of online resources on the peer review process
• possible alternatives to peer review.

Training package for BMJ peer reviewers

Objectives

Do you review work for the BMJ, or are you thinking about becoming a BMJ reviewer? If so, we hope you will use this training pack. It will help you to learn more about peer review, and to understand what makes a review really useful to editors and authors.

The pack includes PowerPoint presentations and written exercises. Much of the material here relates to the general art of peer review, but we have also included specific guidance on what the BMJ needs from you.

We developed this pack for use in a randomised controlled trial of peer reviewer training. Now you can use it as you wish; for your own learning or to teach others. There are four objectives:

• To inform participants on the state of peer review research
• To make clear what constitutes a good review
• To help participants understand what matters to editors about reviews
• To give participants help in producing a good review

Objective One: To inform participants on the state of peer review research

Download What we know about peer review (Microsoft PowerPoint presentation, 261 KB)

Further reading:

• Rennie R. Editorial peer review:its development and rationale (PDF - 71 KB). In Godlee F, Jefferson T, editors. Peer review in health sciences. Second edition. London: BMJ Books, 2003:1-13.
• Overbeke J, Wager E. The state of evidence: what we know and what we don't know about journal peer review (PDF - 89KB). In Godlee F, Jefferson T, editors. Peer review in health sciences. Second edition. London: BMJ Books, 2003:45-61.
• Fletcher RH, Fletcher SW. The effectiveness of editorial peer review (PDF - 82 KB). In Godlee F, Jefferson T, editors. Peer review in health sciences. Second edition. London: BMJ Books, 2003:62-75.
• Martyn C. Peer review: some questions from Socrates (PDF - 46 KB). In Godlee F, Jefferson T, editors. Peer review in health sciences. Second edition. London: BMJ Books, 2003:322-8.
• Smith R. The future of peer review (PDF - 82 KB). In Godlee F, Jefferson T, editors. Peer review in health sciences. Second edition. London: BMJ Books, 2003:329-46.
• WAME website has a good bibliography, including original research on peer review
• JAMA website for abstracts and articles from the Second, Third, and Fourth Peer Review Congresses.

Objective Two: To make clear what constitutes a good review

What editors want (Microsoft PowerPoint presentation - 244 KB)

Further reading:

• Moher D, Jadad AR. How to peer review a manuscript (PDF - 51 KB). In Godlee F, Jefferson T, editors. Peer review in health sciences. Second edition. London: BMJ Books, 2003:183-90.
• Wager E, Godlee F, Jefferson T. How to survive peer review (PDF - 261 KB). London: BMJ Books, 2002.

Objective Three: To help participants understand what matters to editors about reviews

Below are three reviews of manuscripts recently published in the BMJ. Having read the presentation on what editors want from reviewers, we would like you to read these three reviews and note their strengths and weaknesses. This exercise should take approximately 15 minutes. Having noted the strengths and weaknesses of each review, read our critique of each review from the editorial perspective.

• Referee’s report for Paper A
• Referee’s report for Paper B
• Referee’s report for Paper C

Objective Four: To give participants help in producing a good review

We would like you to do a practice rapid review of the paper titled: Magnetic bracelets for relieving pain in lower-limb osteoarthritis: a randomised controlled trial. To help guide your review, we have also provided a question sheet, and links to our standard Guidance for Reviewers and the CONSORT statement. You should spend approximately 30 minutes on this exercise. It is not a "formal" review and we do not want you to send us your review.

• Manuscript to review (PDF - 114 KB)
• Question sheet to help guide your review (Microsoft Word - 22KB)
• General guidance for BMJ peer reviewers
• CONSORT statement

This paper was published in the BMJ in 2004. Below are links to the reviews of the submitted version (and an extra review comissioned for this training package) and also the published version of this paper. Please do not read these reviews until after you have completed the exercise above.

• Reviews of magnetic bracelets paper (Microsoft Word - 52KB)
• Published version of magnetic bracelets paper (PDF - 145KB)