Intended for healthcare professionals

Guidance for BMJ Patient and Public Reviewers

Patient and public review at The BMJ

If you're a patient living with disease or have experienced a significant illness or medical condition, a carer of a patient, a patient advocate acting on behalf of a patient group, or you play a leading part in advocating for patient participation and partnership in healthcare we'd like to invite you to take part in a unique initiative. The BMJ has committed to improving the relevance and patient centredness of its research, education, analysis, and editorial articles by asking patients to comment on them. We need your help to make these changes.

If you already review for The BMJ as a researcher or clinician, but you are also interested in reviewing as a patient, carer, or patient advocate, you can do this too.

Patient and Public review is a new initiative for The BMJ. We are taking the lead here, and we hope other publishers will follow. We are working hard to make our processes as straightforward and smooth as possible for patient and public reviewers and welcome suggestions for how we could do this better.

The peer review process

If the journal's editors think that a paper submitted to the research section of the journal might be suitable for publication they send the paper out to specialist experts who research, practise, and publish in the same area, asking them to comment on whether the research is done well and if it provides an important contribution to scientific knowledge. For more information about what we ask reviewers of research papers to do, see our guidance for traditional peer reviewers. The scientists and healthcare professionals who assess the papers are called reviewers, and the whole process is called peer review. The aim of peer review is to identify and reject poor quality articles and unoriginal studies, and pursue and promote good ones. We also send other types of invited and submitted articles including analysis, education and editorial articles out for peer review Through providing feedback and constructive criticism to authors, reviewers help improve the quality, clarity and impact of papers.

Review of research papers

The BMJ now invites patients, carers and members of the public to review all clinical research articles alongside peer reviewers. We receive around 3500 research papers a year, but about 80% of these are rejected by editors and not sent for external peer review (see Figure). For example, they might be deemed too specialist for a general medical journal or lack originality. About 20% are sent for external peer review (usually to at least two reviewers) and are thus eligible for patient and public review. We eventually accept less than 5% of all our research submissions. This means that many patient and public reviewers find that they are asked to comment on papers that end up being rejected by The BMJ. Some of these will have positive reviews but decisions on publication are based on many factors and rest with the editors not the reviewers. Decisions are always based on more than one reviewers’ report and specialist statistical review as required.



What to include in your review of research papers?

Patient and public reviewers do not need medical or scientific training. What we seek and value from you is your perspective on a slightly different set of questions than those posed to traditional peer reviewers. They include the following:
• Are the study's aims and the issue and questions that the paper addresses relevant and important to patients and carers?
• Are there any areas that you find relevant as a patient or carer that are missing or need to come across more clearly?
• From your perspective as a patient or carer, would the treatment, intervention studied, or guidance given be feasible and work in practice? What challenges might it present that should be considered?
• Are the outcomes measured in the study or described in the paper outcomes that are important to patients and carers? Are there others that should have been considered?
• Do you have any suggestions that might help the author(s) strengthen their paper to make it more patient centred, and make it more useful for doctors and other healthcare professionals to share and discuss with patients?
• Can you comment on the level of patient involvement in the research described, and if and how it could have been improved? The BMJ requires authors to state if and how they involved patients and the public in setting the research question(s), the design, implementation and dissemination of the study and authors are asked to include this information in the methods section of the manuscript. If there was no patient and public involvement we would welcome your ideas on how this could have been done. We hope this will help authors and readers think of the best ways to include patients and the public in their future research and advance patient and public involvement in the research enterprise.

Reviewers of all kinds sometimes worry whether they have something useful to contribute to an academic article. The following are quotes from a few of our patient and public reviewers:

"I recently reviewed a paper for The BMJ and as a non-academic I was terrified of saying what I actually thought of it – I agonised over the words.... I wanted to be constructive, challenging, and polite, but the bottom line was that I felt that the authors of the paper were in an academic bubble and very divorced from what I experience, read, and talk about in real life. A huge relief then to see the other reviewer felt the same way! It was a steep learning curve and a big leap to have faith in my own views and not be afraid to share these with the authors and The BMJ's editorial team."

“Initially I was worried that my lack of in-depth knowledge of statistical methods would make my comments seem petty or too simplistic. Yet when I read expert reviewers comments I saw that they picked up on some of my points which gave me the confidence to stick to my guns and write anything I felt was important to say.”

“It takes time in my estimation to get into a place where as a patient I am confident reviewing the work of academic scholars. At first I took the stance of "who am I to challenge them," but recognized after the first review I am the patient with the experience and perspective that can help to inform the practice of research.”

Why review for us?

By reviewing you will be giving the editors a valued "real life" perspective on the patient focused aspects of selected manuscripts, drawing on your own experience of a particular topic, condition, or intervention. It's an opportunity to have a voice in shaping the way researchers design and report research and clinicians practise medicine. We see it as an important way to further health professionals’ understanding on what is most important, and of benefit, to patients. Here’s what some of our patient and public reviewers think about taking part:

“I have already suggested to a number of other advocates they should apply to become a reviewer. I find the process very inclusive and it is one of the few places where I feel we are invited for the right reasons. We aren't tokens in the BMJ process, we are equal and valued voices.”

“I appreciate the opportunity....... I felt my opinion really counted and made a difference (which hasn't been the case in some of the other 'patient involvement' projects I have been involved in).”

We thank all our reviewers by giving them a year's free online subscription to The BMJ. We also name and thank them on our website every year. Reviewers have told us that they value both access to the journal and a formal acknowledgement of their input and support.

How to register as a patient and public reviewer for The BMJ

All our peer reviewing is conducted using an online database called ScholarOne Manuscripts™. This database was designed before we started patient and public review and is rather inflexible. Because some patient and public reviewers reported that setting up an account was confusing, we register an account on your behalf. To do this we need you to complete a short registration form accessed through this link. The form asks you to provide information about your experiences as a patient, carer or patient advocate so we can match appropriate papers to your areas of interest.

As a general medical journal we receive many papers about general medical issues and also acute conditions. These broad based papers might be about patient and carers’ experience of accessing and using services, their involvement in decisions about how services are designed and delivered, priority setting in health care, ethical debates or other areas. If you are willing to review papers on general subjects, such as these, as well as those directly related to the conditions you have experience of, please indicate this in the relevant question in the patient and public reviewer registration form. It is likely that we would then be able to call on your help a little more often than we currently do or are likely to do. That said, if the requests do not fit areas you feel comfortable reviewing the invitation to review can always be turned down. It is helpful if you can do this as early as possible so editors can find other reviewers. As a volunteer you have no obligation to review anything you are uncomfortable with or are too busy to take on.

Once the above form has been submitted, we will upload the key information you provide onto our online database (ScholarOne Manuscripts™) and set up your account. You will receive an automated email to confirm it’s been set up and notifying you of your user ID. To complete the account registration you will need to click on the link in the email to log you into our database and set your own password.

What happens if you are selected to do a patient review?

If we select you to review, we will send you an invitation by email. You will see the title of the paper, the names of the authors, and for research papers a short summary (abstract) of its contents. You will be asked to accept or decline the invitation to review either by email or by clicking a link in the email. If you are unable to review please let us know, and if you are able to suggest an alternative patient and public reviewer that would be most helpful.

All unpublished manuscripts are confidential documents. If we invite you to review an article, please do not discuss it with anyone. All reviewers are asked to provide their opinion within two weeks of being sent a paper. Most traditional peer reviewers provide opinions of 500-1000 words, but it is quality, not quantity, that counts, and we are seeking an informed independent viewpoint presented in a clear and constructive way.

The BMJ operates an open peer review system and asks all reviewers to sign their reports so that authors know who has reviewed their work. This includes patient and public reviewers. In addition it is now an established policy to publish all reviewers' signed reviews on our website (bmj.com) alongside the published articles, so that readers can see them. We appreciate that there may be circumstances when that will prove difficult for some patient and public reviewers and we are willing to discuss this with individual reviewers (please contact Amy on aprice@bmj.com).

We also ask our reviewers to declare all competing interests. A competing interest exists when judgment concerning a primary interest (such as whether a paper is accepted) might be influenced by a secondary interest (for example, the researcher is your doctor, or you've been paid to be a patient advocate by the sponsor of the study). Read more on competing interests here.

If you would like to try to complete the review but feel you need more guidance please email Amy Price, The BMJ Patient Editor | Research and Evaluation, (aprice@bmj.com) and Amy will find someone to work with you on the review.

What happens to your review?

We do not specifically ask reviewers to comment on the suitability of the paper for publication. Comments, from both patient and public reviewers and traditional peer reviewers, are used to guide editors' views about the paper under consideration. But the decision to accept or reject a paper is made by the editors. Reviewers may well disagree on the importance and relevance of the paper, and it is the editors' job to weigh up the various comments, and together with colleagues reach a decision. It is therefore important that all reviewers provide reasons for their views. Once a final decision has been made (and this may take several months), all reviewers are copied into the detailed decision letter sent to the authors, so they can see the editors’ views and the other reviewers' comments. When papers are published in The BMJ, the reviewers’ comments are made available for all readers online at bmj.com alongside the paper. There is a variable time lag between the decision to accept a paper and its subsequent publication. Regrettably we don’t have an automated way of alerting you to when the paper is published.

Editors have limited ability to give personal feedback to reviewers because of the volume of papers they have to handle under tight time constraints. A lack of such feedback is not related to the quality of your review. All reviewers’ opinions are valued by us and can potentially improve the author’s paper.

Sample reviews of research papers to help you get started

Here are some links to patient and public reviews of published research papers. We have selected reviews on different topics with different styles. You should bear in mind that these are examples, not set guides on style and format. More examples of reviews can now be found on bmj.com alongside the published research articles (click on the “Peer Review” tab to see all reviews of the article).

Research paper: Patient and public review 1
Research paper: Patient and public review 2
Research paper: Patient and public review 3
Research paper: Patient and public review 4
Research paper: Patient and public review 5
Research paper: Patient and public review 6
Research paper: Patient and public review 7
Research paper: Patient and public review 8
Research paper: Patient and public review 9
Research paper: Patient and public review 10

Are there any reasons why I should decline to review this article?

The BMJ takes competing interests seriously to maintain the independence of the review process. We therefore suggest that you decline to review the paper if you:

(a) are or have been a patient of one of the authors
(b) are working with the authors on another project or have worked closely with them in the past
(c) have a relationship with the author(s) such that you do not feel at liberty to express your true impressions of their work
(d) are too unwell or life circumstances would make it difficult for you to complete the review in time.

If you remain uncertain about whether you should review a paper that has been sent to you please email the editor who sent you the invitation and explain your concerns.

Whom should I contact if I need help?

The email inviting you to review contains a link that will take you to ScholarOne Manuscripts™ where you read the paper and submit your review.

Please note that the box for your review on ScholarOne Manuscripts™ appears very small on the screen so you will find it easiest if you write and save your review in word processing software, such as Microsoft Word, and then copy and paste it into the relevant box or upload it as an attachment. We would also advise you to use a pc, laptop or tablet and not a mobile phone to view this software.

With whom will you share my data?

After you have registered as a reviewer, we will set up an account for you on our manuscript tracking system (ScholarOne Manuscripts ™) and will enter keywords to try to reflect the information you provided us with on registration. This database is accessed by The BMJ's editors and editorial staff and very occasionally by external researchers working with us on specific projects under contract. Your data will be held on our database and accessed by BMJ Publishing Group’s editors and staff and very occasionally by researchers working on specific research projects under contract. We will not share your data with other parties, not specified above without your permission. You have the right to ask for your data to be updated or removed at any time.

BMJ is committed to protecting and safeguarding your personal data. Our recently updated Privacy Notice explains your rights under the new law. We’ve also updated our cookie policy

Whom should I contact if I need help?

If you run into any problems or have any questions that haven't been answered here, please email the editor who sent you the paper or our patient editor (Amy Price at aprice@bmj.com). If you have any problems logging on to our online database and submitting your review please email Sue Minns at papersadmin@bmj.com.

Frequently asked questions

Where can I find out more information about peer review in general?
Some additional resources:
Wikipedia article on peer review
Reviewer guidelines—Elsevier
"How to review a paper"—Benos et al 2003
Top 10 tips for peer reviewers—Guardian
A graduate student guide to peer review—American Psychological Association
How to be a peer reviewer—Political Science
Peer review – tips for junior reviewers - BMC
How to peer review - Springer Nature

How many reviews will I be expected to review?

This depends on your area of expertise and the content of the papers we receive. If we don't receive any papers that match your experience then we will not contact you. We do keep track of reviewer activity and will try to send you only a few reviews a year. It’s possible that over the course of a year you will not receive any papers for we cannot predict what will get submitted to us.

Who will see my review?

As explained above, The BMJ has an open review policy so you should be aware that if the article you review is accepted for publication that it will be publicly available on our website (bmj.com) for all to see and will include your name. If the paper is not accepted for publication only the authors of the paper and the editor will see your signed comments.

What happens if I am unable or don’t want to review an article?

If we send you an invitation to review a paper, but you are unable to review it for any reason, you can simply decline the invitation by clicking on the “decline” link in the email we send you. To avoid delays for authors, it is helpful if you decline an invitation to review quickly. If you don't decline or accept the invitation to review, you may receive an automated reminder. If you decline an invitation to review, your account will still be active on our database, and we will still invite you to review in future when we receive a suitable paper.

If you accept an invitation to review but on reading the full manuscript feel that you don’t have the expertise to review it, please let the editor who invited you know as soon as possible so they can invite another reviewer in your place.

It can be difficult for us to identify appropriate reviewers for some topics. If you have registered to review papers on general medical issues, you may receive regular invitations to review papers that you feel are inappropriate or beyond your area of expertise. You shouldn’t feel obliged to try to review these papers if you don’t feel comfortable in doing so.

Why was I sent an email saying my review was no longer needed?

We are committed to making fast decisions on articles submitted to us so that authors are not delayed. As such, there might be an occasion where you are invited to do a review, but we then withdraw the request so that the paper can be pushed on to the next stage. This is not an indication that we do not value your opinion, merely a reflection of our commitment to expedite the peer review process. To ensure we get enough reviewers for each paper we sometimes need to send out multiple invitations. Reviewers that are quick to accept the invitation will be sent the paper for review and the remaining invitations will be withdrawn.

Why do I receive invitations to review papers on topics outside of my areas of experience?

As a general medical journal we receive lots of papers on general topics of medicine eg auto immune disease, prevention medicine, diet, genetics, sexual health and maternity, to name a few. However, many patient and public reviewers have chronic diseases and have indicated that they want to review papers about these diseases/conditions. To try to meet this gap and ensure that we have a patient and public reviewer for as many relevant papers as possible, we ask our patient and public reviewers when they register with us to indicate if they are willing to also review more general topics. If you have indicted this, you may receive a number of emails to review papers that feel outside of your own experience. If you feel the papers are too far removed from your own experience or you don’t want to do the reviews, please simply decline the invitations in the emails received. Editors will recognise that the paper may be outside of your experience and that you may not feel confident reviewing it and don’t mind you declining the invitation. If you feel the paper would be challenging but you would like to try to review it, you can email Amy Price on aprice@bmj.com for guidance.

Why is the box where I write my review on the database so small?

This box expands as you write your review, so although it may appear small it should be sufficient. You might find it easier to write your review in MS Word and copy and paste it into this box.

Should I offer an opinion as to whether I think the paper should be published or not?

We do not ask any of our reviewers to comment on the paper's suitability for publication as this is an editorial decision.

Can I discuss the contents of a paper with other people?

No, papers are submitted to us in confidence, and we need you to respect this until the paper is published.

What happens if reviewers do not agree with each other?

It's not unusual for reviewers to offer conflicting advice. We will consider the opinions in all the reviews we receive and weigh them up alongside the views of other editors.

Do I need to comment on spelling mistakes and grammatical errors?

We do not need you to point out spelling mistakes or grammatical errors as we have a team of editors who correct this when they are preparing the final manuscript for publication.

Do I need to comment on the statistics?

We do not expect you to comment on the statistical approaches reported in the studies, unless you happen to have this expertise. We have a team of statistical reviewers who do this for all papers we are considering accepting, as many academic reviewers lack expertise in this area too.

Can I ask for an extension on the time to review?

If you find you need a little longer to review a paper than the standard two weeks you can email the editor who invited you to review and request an extension. We try to avoid this as we try to act quickly in the interests of our authors, but appreciate that sometimes requests come at busy times. It is also fine to just decline an invitation to review if it is not a convenient time for you. Declining to review won’t influence the chances of you being invited to review other papers in the future.

Other frequently asked questions

How do I access my bmj.com annual subscription?

As a thank you for completing a review, all reviewers receive an annual online subscription to bmj.com (the definitive version of the journal). You will receive a link to activate your 12 month online subscription to bmj.com once the final decision has been made on the manuscript you reviewed. Look out for an email with "BMJ Reviewer Thank You" in the email subject line. There will be a delay between you submitting your review and the final decision on the paper as it may go through several more rounds of scrutiny and the thank you email is only activated once the final decision is made on the database.

Can I be a patient and public reviewer if I already review for The BMJ as a researcher or clinician?

If you are already a peer reviewer for The BMJ, but would like to also register as a patient and public reviewer, we will need you to have two accounts on our database. This is because the database is restricted in the number of keywords we can use to match your expertise and patient experience to relevant papers. Having two accounts will avoid us confusing your role as researcher/clinician with your role as a patient and public reviewer. We suggest that you use a personal email address for your patient and public reviewer account and your work/professional email address for your academic peer reviewing. If you have any questions then please email aprice@bmj.com.

How many patient and public reviewers do you have and how often do they review?

We currently have around 800 registered patient and public reviewers. Over 85% of the research papers sent out for review between January-March 2018 included an invitation to a PPI reviewer (up from 68% in 2017) and figures from 2017 showed that PPI reviews were obtained in around a quarter of papers. We view this as a great achievement. We have found that PPI reviewers take a similar amount of time to respond to invitations and to complete their reviews as peer reviewers. In fact, they are often faster! Our hardest challenge is matching PPI reviewers’ experience with the wide range of topics we are sent and commission papers on.

What value does patient and public review add at The BMJ?

Patient and public reviews help editors decide whether to publish papers and add to the advice we give authors on how to revise and improve their papers. Feedback from authors suggests that they too value patient and public reviewers’ comments even when their papers are not accepted for publication by The BMJ, for they help refine their papers for publication in other journals.

We are often asked what value patient and public review adds at The BMJ. Our response is firstly that we seek to partner with patients and the public because we see it as an ethical imperative essential to improving the quality, safety, value, and sustainability of health systems. How patient and public review adds value is an important question, but not an easy one to answer. It’s worth noting that there is little evidence to support the effectiveness of academic peer review - despite its widely accepted role in the scientific process. We plan to do some research on the changes made to manuscripts following patient and public review and will share examples with you when we have done this.

Are there other opportunities for patients and the public to contribute to The BMJ?

Yes, there are several ways that you can do this including writing for the following two patient and public generated series.

BMJ Opinion

BMJ Opinion is a digital platform for comment and opinion written by The BMJ's international community of readers, authors, and editors and we welcome submissions from patients and carers. Most of these are published in our Patient Perspectives series (https://blogs.bmj.com/bmj/category/patient-perspectives). Guidance on writing these articles is here.

What Your Patient is Thinking series (WYPITs)

The WYPITs series is led, edited, and written by patients and their carers with the guidance of a BMJ editor. They contain messages that are thought provoking, and of important educational value for healthcare professionals. Examples of published WYPITs are here https://www.bmj.com/specialties/what-your-patient-thinking and guidance here.

Partnership in Practice

We also welcome articles co written with healthcare professionals in a new series on our digital platform which seeks to showcase good examples where productive partnerships have been forged in clinical practice, policy, education and research. [Link to rationale and guidance]