Intended for healthcare professionals

Guidance for BMJ Patient and Public Reviewers

Patient and public review at The BMJ

If you're a patient or carer of a patient living with a long term medical condition, or have experienced a significant illness or are a patient advocate acting on behalf of a patient group, or advocating for patient participation and partnership in healthcare we'd like to invite you to support the BMJ's commitment to embed patient and public review of papers alongside peer review. The BMJ has instigated patient and public review as part of its drive to co produce its content with patients and the public and strong commitment to increasing the relevance and patient centredness of its research, education, and scholarly comment articles. We need your help to do this.

If you already review for The BMJ as a researcher or clinician, but you are also interested in reviewing as a patient, carer, or patient advocate, you can do this too.

The BMJ started Patient and Public review of papers in 2014 and while we have taken the lead it is encouraging to note that other journals are also starting to do this. We seek to make our processes as straightforward as possible - suggestions on how to improve them are always welcome.

The peer review process

If the journal's editors think that a paper submitted to the research section of the journal might be suitable for publication they send the paper out to specialist experts who research, practise, and publish in the same area, asking them to comment on whether the research is done well and if it provides an important contribution to scientific knowledge. For more information about what we ask reviewers of research papers to do, see our guidance for traditional peer reviewers. The scientists and healthcare professionals who assess the papers are called reviewers, and the whole process is called peer review. The aim of peer review is to identify and reject poor quality articles and unoriginal studies, and pursue and promote good ones. We also send other types of invited and submitted articles including analysis, education and editorial articles out for peer review Through providing feedback and constructive criticism to authors, reviewers help improve the quality, clarity and impact of papers.

Review of research papers

The BMJ now invites patients, carers and members of the public to review all clinical research articles alongside peer reviewers. We receive around 3500 research papers a year, but about 80% of these are rejected by editors and not sent for external peer review (see Figure). For example, they might be deemed too specialist for a general medical journal or lack originality. About 20% are sent for external peer review (usually to at least two reviewers) and are thus eligible for patient and public review. We eventually accept less than 5% of all our research submissions. This means that many patient and public reviewers find that they are asked to comment on papers that end up being rejected by The BMJ. Some of these will have positive reviews but decisions on publication are based on many factors and rest with the editors not the reviewers. Decisions are always based on more than one reviewers’ report and specialist statistical review as required.



What to include in your review of research papers?

Patient and public reviewers do not need medical or scientific training. What we seek and value from you is your perspective on a slightly different set of questions than those posed to traditional peer reviewers. They include the following:

• Are the questions the paper addresses relevant and important to patients and/or carers?
• Are there topics or issues that are missing, or need to be highlighted more?
• Is the treatment or intervention suggested or guidance given something which patients/carers can readily take up? or does it present challenges?
• Are the outcomes described/measured in the study important to patients/carers? Are there others that should have been considered?
• Do you have any suggestions that might help the author(s) strengthen their paper and make it more useful for doctors to share and discuss with patients/ carers?
• Do you think the level of patient/carer involvement in the study could have been improved? If there was none do you have ideas on how they might have done so?

Reviewers of all kinds sometimes worry whether they have something useful to contribute to an academic article. The following are quotes from a few of our patient and public reviewers:

"I recently reviewed a paper for The BMJ and as a non-academic I was terrified of saying what I actually thought of it – I agonised over the words.... I wanted to be constructive, challenging, and polite, but the bottom line was that I felt that the authors of the paper were in an academic bubble and very divorced from what I experience, read, and talk about in real life. A huge relief then to see the other reviewer felt the same way! It was a steep learning curve and a big leap to have faith in my own views and not be afraid to share these with the authors and The BMJ's editorial team."

“Initially I was worried that my lack of in-depth knowledge of statistical methods would make my comments seem petty or too simplistic. Yet when I read expert reviewers comments I saw that they picked up on some of my points which gave me the confidence to stick to my guns and write anything I felt was important to say.”

“It takes time in my estimation to get into a place where as a patient I am confident reviewing the work of academic scholars. At first I took the stance of "who am I to challenge them," but recognized after the first review I am the patient with the experience and perspective that can help to inform the practice of research.”

Why review for us?

By reviewing you will be giving the editors a valued "real life" perspective on the patient focused aspects of selected manuscripts, drawing on your own experience of a particular topic, condition, or intervention. It's an opportunity to have a voice in shaping the way researchers design and report research and clinicians practise medicine. We see it as an important way to further health professionals’ understanding on what is most important, and of benefit, to patients. Here’s what some of our patient and public reviewers think about taking part:

“I have already suggested to a number of other advocates they should apply to become a reviewer. I find the process very inclusive and it is one of the few places where I feel we are invited for the right reasons. We aren't tokens in the BMJ process, we are equal and valued voices.”

“I appreciate the opportunity....... I felt my opinion really counted and made a difference (which hasn't been the case in some of the other 'patient involvement' projects I have been involved in).”

We thank all our reviewers by giving them a year's free online subscription to The BMJ. We also name and thank them on our website every year. Reviewers have told us that they value both access to the journal and a formal acknowledgement of their input and support.

How to register as a patient and public reviewer for The BMJ

All our peer reviewing is conducted using an online database called ScholarOne Manuscripts™. This database was designed before we started patient and public review and is rather inflexible. Because some patient and public reviewers reported that setting up an account was confusing, we register an account on your behalf. To do this we need you to complete a short registration form accessed through this link. The form asks you to provide information about your experiences as a patient, carer or patient advocate so we can match appropriate papers to your areas of interest.

As a general medical journal we receive many papers about general medical issues and also acute conditions. These broad based papers might be about patient and carers’ experience of accessing and using services, their involvement in decisions about how services are designed and delivered, priority setting in health care, ethical debates or other areas. If you are willing to review papers on general subjects, such as these, as well as those directly related to the conditions you have experience of, please indicate this in the relevant question in the patient and public reviewer registration form. It is likely that we would then be able to call on your help a little more often than we currently do or are likely to do. That said, if the requests do not fit areas you feel comfortable reviewing the invitation to review can always be turned down. It is helpful if you can do this as early as possible so editors can find other reviewers. As a volunteer you have no obligation to review anything you are uncomfortable with or are too busy to take on.

Once the above form has been submitted, we will upload the key information you provide onto our online database (ScholarOne Manuscripts™) and set up your account. You will receive an automated email with this subject heading “BMJ - Account Modified in ScholarOne Manuscripts” to confirm it’s been set up and notifying you of your user ID. To complete the account registration you will need to click on the link in the email to log you into our database and set your own password. There may be a delay between the submission of your registration form and receiving the email about your account.

What happens if you are selected to do a patient review?

If we select you to review, we will send you an invitation by email. You will see the title of the paper, the names of the authors, and for research papers a short summary (abstract) of its contents. You will be asked to accept or decline the invitation to review either by email or by clicking a link in the email. If you are unable to review please let us know, and if you are able to suggest an alternative patient and public reviewer that would be most helpful.

All unpublished manuscripts are confidential documents. If we invite you to review an article, please do not discuss it with anyone. All reviewers are asked to provide their opinion within two weeks of being sent a paper. Most traditional peer reviewers provide opinions of 500-1000 words, but it is quality, not quantity, that counts, and we are seeking an informed independent viewpoint presented in a clear and constructive way.

The BMJ operates an open peer review system and asks all reviewers to sign their reports so that authors know who has reviewed their work. This includes patient and public reviewers. In addition it is now an established policy to publish all reviewers' signed reviews on our website (bmj.com) alongside the published articles, so that readers can see them. We appreciate that there may be circumstances when that will prove difficult for some patient and public reviewers and we are willing to discuss this with individual reviewers (please contact Amy on aprice@bmj.com).

We also ask our reviewers to declare all competing interests. A competing interest exists when judgment concerning a primary interest (such as whether a paper is accepted) might be influenced by a secondary interest (for example, the researcher is your doctor, or you've been paid to be a patient advocate by the sponsor of the study). Read more on competing interests here.

If you would like to try to complete the review but feel you need more guidance please email Amy Price, The BMJ Patient Editor | Research and Evaluation, (aprice@bmj.com) and Amy will find someone to work with you on the review.

What happens to your review?

We do not specifically ask reviewers to comment on the suitability of the paper for publication. Comments, from both patient and public reviewers and traditional peer reviewers, are used to guide editors' views about the paper under consideration. But the decision to accept or reject a paper is made by the editors. Reviewers may well disagree on the importance and relevance of the paper, and it is the editors' job to weigh up the various comments, and together with colleagues reach a decision. It is therefore important that all reviewers provide reasons for their views. Once a final decision has been made (and this may take several months), all reviewers are copied into the detailed decision letter sent to the authors, so they can see the editors’ views and the other reviewers' comments. When papers are published in The BMJ, the reviewers’ comments are made available for all readers online at bmj.com alongside the paper. There is a variable time lag between the decision to accept a paper and its subsequent publication. Regrettably we don’t have an automated way of alerting you to when the paper is published.

Editors have limited ability to give personal feedback to reviewers because of the volume of papers they have to handle under tight time constraints. A lack of such feedback is not related to the quality of your review. All reviewers’ opinions are valued by us and can potentially improve the author’s paper.

Sample reviews of research papers to help you get started

Here are some links to patient and public reviews of published research papers. We have selected reviews on different topics with different styles. You should bear in mind that these are examples, not set guides on style and format. More examples of reviews can now be found on bmj.com alongside the published research articles (click on the “Peer Review” tab to see all reviews of the article).

Research paper: Patient and public review 1
Research paper: Patient and public review 2
Research paper: Patient and public review 3
Research paper: Patient and public review 4
Research paper: Patient and public review 5
Research paper: Patient and public review 6
Research paper: Patient and public review 7
Research paper: Patient and public review 8
Research paper: Patient and public review 9
Research paper: Patient and public review 10

Are there any reasons why I should decline to review this article?

The BMJ takes competing interests seriously to maintain the independence of the review process. We therefore suggest that you decline to review the paper if you:

(a) are or have been a patient of one of the authors
(b) are working with the authors on another project or have worked closely with them in the past
(c) have a relationship with the author(s) such that you do not feel at liberty to express your true impressions of their work
(d) are too unwell or life circumstances would make it difficult for you to complete the review in time.

If you remain uncertain about whether you should review a paper that has been sent to you please email the editor who sent you the invitation and explain your concerns.

How do I submit my review?

The email inviting you to review contains a link that will take you to ScholarOne Manuscripts™ where you read the paper and submit your review.

Please note that the box for your review on ScholarOne Manuscripts™ appears very small on the screen so you will find it easiest if you write and save your review in word processing software, such as Microsoft Word, and then copy and paste it into the relevant box or upload it as an attachment. We would also advise you to use a pc, laptop or tablet and not a mobile phone to view this software.

Who should I contact if I need help with my review or registration?

If you run into any problems or have any questions that haven't been answered here, please email the editor who sent you the paper or our patient editor for research (Amy Price at aprice@bmj.com). If you have any problems logging on to our online database and submitting your review please email Sue Minns at papersadmin@bmj.com.

Data protection

BMJ is committed to protecting and safeguarding your personal data. Our recently updated Privacy Notice (https://www.bmj.com/company/your-privacy) explains your legal rights. You can also look at our Cookie Policy to find out what cookies we use and why (https://www.bmj.com/company/legal-information/bmj-cookie-policy).

For the purposes of any applicable data protection laws including the EU General Data Protection Regulation 2016/679 (GDPR), BMJ is the data controller of your personal data. We are registered with the ICO on the Data Protection Public Register under registration number Z7607533.

If you have any questions about these rights or how your personal data is used by us, you should contact the Data Protection Lead using the details below:
• Post: Data Protection Lead, BMJ, BMA House, Tavistock Square, London, WC1H 9JR
• Email: dataprotectionlead@bmj.com

How will my personal data be collected?

BMJ collects your personal data when you register with us and when you submit a review.

What data will you collect from me?

We will collect and store the following categories of personal data:
• biographical information such as your name, title;
• your contact details including address, email address and phone number;
• your opinions, assessments and experience provided in your review;
• where appropriate (as part of your review if you choose to disclose it), your age and gender;
• any competing interests that may relate to an article you have submitted or reviewed; and
• online data when you visit our websites or use our apps including your Internet Protocol (IP) address, location, browser type and version, operating system; and details about how you use our website and apps (for detailed information on the cookies we use and the purposes for which we use them please see our Cookie Policy).

We also collect and store the following special categories of personal data (which includes health information and other sensitive personal data that you may choose to share with us, either in your registration form or in your reviews of manuscripts):
• health information;
• racial and ethnic origin;
• genetic data;
• sexual orientation and sexual and reproductive health practices;
• religious and philosophical beliefs.

How will my data be used?

• Our Databases: After you have registered as a reviewer, we will set up an account for you on our manuscript tracking system (ScholarOne Manuscripts™) and your information will be stored. Your reviewer profile will be linked to specific keywords to enable your profile to be searched more effectively by users of our databases (data profiling). The keywords linked to your profile will be chosen to try to reflect the information you provide us with on registration and may include the names of any health conditions you have disclosed. Our databases containing your profile is accessed by editors and editorial staff in connection with BMJ’s products and services (which includes The BMJ as well as other publications and platforms provided by BMJ) and the third parties mentioned below so that you may be invited to review an article.

• Review: If your review is accepted and the manuscript you have reviewed is accepted for publication, your review (including any personal data and special category personal data you disclose in your review) will be published on our website (bmj.com) alongside the published article.

How long will you store my data?

• Personal data supplied during the registration process: Your registration personal data will be stored by us indefinitely. However, you have the right to ask for your data to be removed at any time and we will email you annually to remind you about your right to unsubscribe.
• Additional personal data supplied in connection with your review: If your review is published, any personal data you disclose in that review will be available on our website indefinitely (unless you ask for your review to be removed at any time). However, once your review has been in the public domain its content may no longer be within our control and consequently any personal data you have disclosed may be available in the public domain indefinitely.

If our editors decide not to publish your review for any reason, the review we hold will be held on our database indefinitely (unless you ask for your review to be destroyed). In these circumstances, your personal data provided on registration will continue to be stored by us as above to enable you to be invited to review another article.

With whom will you share my data?

Your personal data held on our databases and in your review may be accessed by editors and staff of BMJ Publishing Group Limited, Clarivate Analytics (the owner of ScholarOne Manuscripts™ and very occasionally by third party researchers contracted to work on specific research projects with us. Personal data disclosed in your review will also be seen by the authors of the manuscript you are commenting on.

We will not share your data held on our databases with other parties, not specified above without your permission.

Where your review is published, any personal data disclosed in your review will be available in the public domain for all to see.

Your rights:

You have the right to ask for your data to be updated or removed at any time. Contact Sue Minns at papersadmin@bmj.com.

However once your review is published, any personal data disclosed in your review will be in the public domain and, whilst it is possible for us to retract your review from our website, once your review is in the public domain your data may no longer be within our control.

Frequently asked questions

Where can I find out more information about peer review in general?
Some additional resources:
Wikipedia article on peer review
Reviewer guidelines—Elsevier
"How to review a paper"—Benos et al 2003
Top 10 tips for peer reviewers—Guardian
A graduate student guide to peer review—American Psychological Association
How to be a peer reviewer—Political Science
Peer review – tips for junior reviewers - BMC
How to peer review - Springer Nature

How many reviews will I be expected to review?

This depends on your area of expertise and the content of the papers we receive. If we don't receive any papers that match your experience then we will not contact you. We do keep track of reviewer activity and will try to send you only a few reviews a year. It’s possible that over the course of a year you will not receive any papers for we cannot predict what will get submitted to us.

Who will see my review?

As explained above, The BMJ has an open review policy so you should be aware that if the article you review is accepted for publication that it will be publicly available on our website (bmj.com) for all to see and will include your name.

If the paper is not accepted for publication only the authors of the paper, the editor, and other reviewers of the same paper, will see your signed comments. In these circumstances you review will be stored on our database indefinitely (unless you ask us to delete it). Unfortunately, whilst we ask that our authors and reviewers keep the contents of reviews confidential, we cannot guarantee the confidentiality of your review by these third parties.

What happens if I am unable or don’t want to review an article?

If we send you an invitation to review a paper, but you are unable to review it for any reason, you can simply decline the invitation by clicking on the “decline” link in the email we send you. To avoid delays for authors, it is helpful if you decline an invitation to review quickly. If you don't decline or accept the invitation to review, you may receive an automated reminder. If you decline an invitation to review, your account will still be active on our database, and we will still invite you to review in future when we receive a suitable paper.

If you accept an invitation to review but on reading the full manuscript feel that you don’t have the expertise to review it, please let the editor who invited you know as soon as possible so they can invite another reviewer in your place.

It can be difficult for us to identify appropriate reviewers for some topics. If you have registered to review papers on general medical issues, you may receive regular invitations to review papers that you feel are inappropriate or beyond your area of expertise. You shouldn’t feel obliged to try to review these papers if you don’t feel comfortable in doing so.

Why was I sent an email saying my review was no longer needed?

We are committed to making fast decisions on articles submitted to us so that authors are not delayed. As such, there might be an occasion where you are invited to do a review, but we then withdraw the request so that the paper can be pushed on to the next stage. This is not an indication that we do not value your opinion, merely a reflection of our commitment to expedite the peer review process. To ensure we get enough reviewers for each paper we sometimes need to send out multiple invitations. Reviewers that are quick to accept the invitation will be sent the paper for review and the remaining invitations will be withdrawn.

Why do I receive invitations to review papers on topics outside of my areas of experience?

As a general medical journal we receive lots of papers on general topics of medicine eg auto immune disease, prevention medicine, diet, genetics, sexual health and maternity, to name a few. However, many patient and public reviewers have chronic diseases and have indicated that they want to review papers about these diseases/conditions. To try to meet this gap and ensure that we have a patient and public reviewer for as many relevant papers as possible, we ask our patient and public reviewers when they register with us to indicate if they are willing to also review more general topics. If you have indicted this, you may receive a number of emails to review papers that feel outside of your own experience. If you feel the papers are too far removed from your own experience or you don’t want to do the reviews, please simply decline the invitations in the emails received. Editors will recognise that the paper may be outside of your experience and that you may not feel confident reviewing it and don’t mind you declining the invitation. If you feel the paper would be challenging but you would like to try to review it, you can email Amy Price on aprice@bmj.com for guidance.

Why is the box where I write my review on the database so small?

This box expands as you write your review, so although it may appear small it should be sufficient. You might find it easier to write your review in MS Word and copy and paste it into this box.

Should I offer an opinion as to whether I think the paper should be published or not?

We do not ask any of our reviewers to comment on the paper's suitability for publication as this is an editorial decision.

Can I discuss the contents of a paper with other people?

No, papers are submitted to us in confidence, and we need you to respect this until the paper is published.

What happens if reviewers do not agree with each other?

It's not unusual for reviewers to offer conflicting advice. We will consider the opinions in all the reviews we receive and weigh them up alongside the views of other editors.

Do I need to comment on spelling mistakes and grammatical errors?

We do not need you to point out spelling mistakes or grammatical errors as we have a team of editors who correct this when they are preparing the final manuscript for publication.

Do I need to comment on the statistics?

We do not expect you to comment on the statistical approaches reported in the studies, unless you happen to have this expertise. We have a team of statistical reviewers who do this for all papers we are considering accepting, as many academic reviewers lack expertise in this area too.

Can I ask for an extension on the time to review?

If you find you need a little longer to review a paper than the standard two weeks you can email the editor who invited you to review and request an extension. We try to avoid this as we try to act quickly in the interests of our authors, but appreciate that sometimes requests come at busy times. It is also fine to just decline an invitation to review if it is not a convenient time for you. Declining to review won’t influence the chances of you being invited to review other papers in the future.

How do I access my bmj.com annual subscription?

As a thank you for completing a review, all reviewers receive an annual online subscription to bmj.com (the definitive version of the journal). You will receive a link to activate your 12 month online subscription to bmj.com once the final decision has been made on the manuscript you reviewed. Look out for an email with "BMJ Reviewer Thank You" in the email subject line. There will be a delay between you submitting your review and the final decision on the paper as it may go through several more rounds of scrutiny and the thank you email is only activated once the final decision is made on the database.

Can I be a patient and public reviewer if I already review for The BMJ as a researcher or clinician?

If you are already a peer reviewer for The BMJ, but would like to also register as a patient and public reviewer, we will need you to have two accounts on our database. This is because the database is restricted in the number of keywords we can use to match your expertise and patient experience to relevant papers. Having two accounts will avoid us confusing your role as researcher/clinician with your role as a patient and public reviewer. We suggest that you use a personal email address for your patient and public reviewer account and your work/professional email address for your academic peer reviewing. If you have any questions then please email aprice@bmj.com.

How many patient and public reviewers do you have and how often do they review?

We currently have around 800 registered patient and public reviewers. Over 85% of the research papers sent out for review between January-March 2018 included an invitation to a PPI reviewer (up from 68% in 2017) and figures from 2017 showed that PPI reviews were obtained in around a quarter of papers. We view this as a great achievement. We have found that PPI reviewers take a similar amount of time to respond to invitations and to complete their reviews as peer reviewers. In fact, they are often faster! Our hardest challenge is matching PPI reviewers’ experience with the wide range of topics we are sent and commission papers on.

What value does patient and public review add at The BMJ?

Patient and public reviews help editors decide whether to publish papers and add to the advice we give authors on how to revise and improve their papers. Feedback from authors suggests that they too value patient and public reviewers’ comments even when their papers are not accepted for publication by The BMJ, for they help refine their papers for publication in other journals.

We are often asked what value patient and public review adds at The BMJ. Our response is firstly that we seek to partner with patients and the public because we see it as an ethical imperative essential to improving the quality, safety, value, and sustainability of health systems. How patient and public review adds value is an important question, but not an easy one to answer. It’s worth noting that there is little evidence to support the effectiveness of academic peer review - despite its widely accepted role in the scientific process. We plan to do some research on the changes made to manuscripts following patient and public review and will share examples with you when we have done this.

Are there other opportunities for patients and the public to contribute to The BMJ?

Yes, there are several ways that you can do this including writing for the following two patient and public generated series.

BMJ Opinion

BMJ Opinion is a digital platform for comment and opinion written by The BMJ's international community of readers, authors, and editors and we welcome submissions from patients and carers. Most of these are published in our Patient Perspectives series (https://blogs.bmj.com/bmj/category/patient-perspectives). Guidance on writing these articles is here.

What Your Patient is Thinking series (WYPITs)

The WYPITs series is led, edited, and written by patients and their carers with the guidance of a BMJ editor. They contain messages that are thought provoking, and of important educational value for healthcare professionals. Examples of published WYPITs are here https://www.bmj.com/specialties/what-your-patient-thinking and guidance here.

Partnership in Practice

We also welcome articles co written with healthcare professionals in a new series on our digital platform which seeks to showcase good examples where productive partnerships have been forged in clinical practice, policy, education and research. [Link to rationale and guidance]