Intended for healthcare professionals

Evidence based publishing

Science is based on experimental testing and data. Yet ironically the publishing of science was until recently based on faith not study. BMJ has a vision of becoming the world's first evidence based publisher. We have been publishing around evidence based medicine for some years and hope that the same thinking can transform publishing.

We carry out research into all aspects of BMJ's operation with the objective of developing, promoting, and incorporating evidence-based and responsible publishing into our own practices. Our research projects usually originate from within BMJ, but our work has a wider focus: nearly all of the issues we research have relevance for journal editors, authors, peer reviewers and publishers working across biomedical science. We also conduct collaborative research projects with researchers in international institutions and offer a PhD program on the responsible conduct of publishing scientific research. As a publisher of numerous journals we have a rich dataset of submitted articles and privileged contact with authors and reviewers worldwide.

Here you will find useful information on research undertaken at BMJ into all aspects of publishing science including peer review, conflicts of interests, editorial decision making, open access publishing, data sharing, authorship/contributorship, presentation of research, research ethics, publication bias, and education. Projects undertaken include randomised controlled trials of open peer review and the effectiveness of peer review training, author surveys on the use of statistical expertise in medical research, preferences for presentational style of scientific research, perceptions of electronic publishing, attitudes towards open access publishing, and reader surveys to understand the effect of competing interest statements on research credibility. If you have any ideas for collaborative research in this area or wish to pursue your own PhD, please contact us.

Sara Schroter, senior researcher, The BMJ

BMJ Research: completed

Peer Review

Peer review of grant applications

Starling Group, co-chaired by Trish Groves (The BMJ) and Professor Liselotte Højgaard (European Medical Research Councils).
There is now a considerable body of evidence on the methods, outcomes, effectiveness, best practice, and ethics of editorial peer review of submissions to biomedical journals. In contrast, grant review, which is practised in a wide variety of ways, has a relatively sparse evidence base. It is possible that grant review might be informed by editorial peer review practice and evidence. The Starling Group is a newly set up international group interested in improving grant review for health and medical research. The Starling Group conducted two surveys. The first was an international survey of biomedical research funding organisations to measure typical workloads of funding organisations and current problems they face. The second was an international survey of external grant reviewers to measure typical workloads of external grant reviewers and factors that motivate them to review.

Schroter S, Groves T, Højgaard L. Surveys of current status in biomedical science grant review: funding organisations’ and grant reviewers’ perspectives. BMC Medicine 2010;8:62. [Full text PDF]


Does telling peer reviewers that their signed reviews of original research papers might be posted on The BMJ's website affect the quality of their reviews?

We conducted an RCT to see whether telling peer reviewers that their signed reviews of original research papers might be posted on The BMJ's website would affect the quality of their reviews.

We found that this had no important effect on review quality. Although the possibility of posting reviews online was associated with a high refusal rate among potential peer reviewers and an increase in the amount of time taken to write a review, we believe that the ethical arguments in favour of open peer review more than outweigh these disadvantages.

van Rooyen S, Delamothe T, Evans SJW. Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial. BMJ 2010;341:c5729 [Full Text]


Does blinding reviewers to authors' identities and/or requiring them to sign their reports affect the number of deliberately introduced errors they detect in a manuscript?

We conducted a randomised controlled trial to evaluate the effect on the quality of peer review of blinding reviewers to authors' identities and requiring reviewers to sign their reports. We found that neither blinding reviewers to the authors and origin of the paper nor requiring them to sign their reports had any effect on the rate of detection of errors. Such measures are unlikely to improve the quality of peer review reports.

Godlee F, Gale CR, Martyn CN. Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. JAMA 1998: 280: 237-240. [Abstract]]


Does blinding reviewers to authors’ identities and/or revealing the reviewers' identity to a co-reviewer affect the quality of reviews?

We conducted a randomised controlled trial to determine whether concealing authors' identities from reviewers (blinding) and/or revealing the reviewer's identity to a coreviewer (unmasking) affects the quality of reviews, the time taken to carry out reviews, and the recommendation regarding publication. We found that blinding and unmasking made no editorially significant difference to review quality, reviewers' recommendations, or time taken to review.

van Rooyen S, Godlee F, Evans S, Smith R, Black N. Effect of blinding and unmasking on the quality of peer review: A randomized trial. JAMA 1998; 280: 234 – 7. [Abstract]


Does revealing the reviewer's identity to the author affect the quality of the reviewer's opinion?

We conducted a randomised controlled trial to examine the effect on peer review of asking reviewers to have their identity revealed to the authors of the paper. We found that this had no important effect on the quality of the review, the recommendation regarding publication, or the time taken to review, but it significantly increased the likelihood of reviewers declining to review.

van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers' recommendations: A randomised trial. BMJ 1999; 318: 23 – 27. [Abstract] [Full Text]


Development and validation of an instrument to measure review quality

We developed and validated an instrument to measure the quality of reviews. The Review Quality Instrument (RQI) assesses the extent to which a reviewer has commented on five aspects of a manuscript (importance of the research question, originality of the paper, strengths and weaknesses of the method, presentation, interpretation of results) and on two aspects of the review (constructiveness and substantiation of comments). The RQI’s internal consistency was high (Cronbach's alpha 0.84). The mean total score (based on the seven items each scored on a 5-point Likert scale from 1 to 5) had good test-retest (Kw = 1.00) and inter-rater (Kw = 0.83) reliability. There was no evidence of floor or ceiling effects, construct validity was evident, and the respondent burden was acceptable (2-10 minutes). The instrument can be recommended for use in the study of peer review and we have used it in several of our studies.

van Rooyen S, Black N, Godlee F. Development of the review quality instrument (RQI) for assessing peer reviews of manuscripts. Journal of Clinical Epidemiology 1999; 52(7): 625 – 629. [Abstract]


What makes a good reviewer and a good review for a general medical journal?

We conducted a study to determine the characteristics of reviewers for a general medical journal who produce high-quality reviews and to describe the characteristics of a good review, particularly in terms of the time spent reviewing and turnaround time. In a logistic regression analysis, the only significant factor associated with higher-quality ratings by both editors and authors was reviewers trained in epidemiology or statistics. Younger age also was an independent predictor for editors' quality assessments, while reviews performed by reviewers who were members of an editorial board were rated of poorer quality by authors. Review quality increased with time spent on a review, up to 3 hours but not beyond.

Black N, van Rooyen S, Godlee F, Smith R, Evans S. What makes a good reviewer and a good review for a general medical journal? JAMA 1998; 280(3): 231 – 233. [Abstract]


RCT of the effects of training peer reviewers on the quality of review

Many studies have illustrated the inadequacies of peer review and its limitations in improving the quality of research papers. However, few studies have evaluated interventions that try to improve peer review, and there have been no previous randomised controlled trials to examine the effects of training. Training that would be feasible for reviewers to undergo and for a journal to provide would have to be short or provided at a distance. Although the effectiveness of short educational interventions is questionable, some brief interventions have been shown to be successful (depending on what is being taught and the methods used). We aimed to determine whether reviewers for The BMJ who underwent training would produce reviews of better quality than those who received no training; whether face to face training would be more beneficial than a self taught package; and whether any training effect would last at least six months. The training materials for the trial are now freely available on bmj.com.

Schroter S, Black N, Evans S, Smith R, Carpenter J, Godlee F. Effects of training on quality of peer review: A randomised controlled trial. BMJ 2004; 328: 673-5. [Abstract] [Full Text]

Schroter S, Groves T. BMJ training for peer reviewers. BMJ 2004; 328. [Full Text]

White I, Carpenter J, Evans S, Schroter S. Eliciting and using expert opinions about dropout bias in randomised controlled trials. Clinical Trials: Journal of the Society for Clinical Trials 2007;4(2):125-39. [Abstract]

Schroter S, Black N, Evans S, Godlee F, Osorio L, Smith R. What errors do peer reviewers detect, and does training improve their ability to detect them? J R Soc Med 2008:101: 507–514. [Full Text]


Comparison of author and editor suggested reviewers in terms of review quality and recommendation for publication

Many journals give authors the opportunity to suggest reviewers to review their paper, but editors’ use of these reviewers vary as some are concerned that they may favour the author. We conducted a study comparing author- and editor-suggested reviewers of ten biomedical journals in a range of specialties to investigate differences in review quality and recommendation for publication. We found no difference in review quality but author-suggested reviewers tended to make more favourable recommendations for publication.

Schroter S, Tite L, Hutchings, A, Black N. Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors. JAMA 2006;295:314-7. [Abstract]


Why do peer reviewers decline to review?

Peer reviewers are usually unpaid and their efforts are not formally acknowledged. Editors of some journals experience difficulty finding appropriate reviewers who are able to complete timely reviews, resulting in publication delay. We conducted a survey of reviewers from five BMJ Journals to determine why reviewers decline to review and their opinions of various incentives. We found that reviewers are more likely to accept to review a manuscript when it is relevant to their area of interest and lack of time is the principal factor in the decision to decline. Most respondents agreed that financial incentives would not be effective when time constraints are prohibitive

Tite L, Schroter S. Why do reviewers decline to review for journals? A survey. J Epidemiol Community Health 2007;61:9-12. [Abstract]

Back to top >

Competing interests

Does declaration of competing interests affect readers' perceptions of research? Two randomised controlled trials

Conflict of interest has been defined as a set of conditions in which professional judgment concerning a primary interest (such as patient welfare or the validity of research) can be influenced by a secondary interest (such as financial gain). Despite increasing evidence that conflict of interest influences authors' conclusions, there has been little research into the effect on readers' perceptions of research. We conducted two studies to see whether a declarations of competing interests influences readers' perceptions of the interest, importance, relevance, validity, and believability of a study.

Chaudhry S, Schroter S, Smith R, Morris J. Does declaration of competing interests affect reader perceptions: A randomised trial? BMJ 2002; 325:1391-2. [Full Text]

Schroter S, Morris J, Chaudhry S, Smith, R, Barratt H. Does the type of competing interest statement affect reader perceptions of the credibility of research? A randomised trial. BMJ 2004; 328:742-3. [Full Text]

Williams HC, Naldi L, Paul C, Vahlquist A, Schroter S, Jobling R. Conflicts of Interest in Dermatology. Acta Derm Venereol. 2006;86(6):485-97. [Abstract]


Does declaration of competing interests affect readers' perceptions of educational articles? A randomised controlled trial

Two previous RCTs showed that readers perceptions of research were influenced by financial competing interests. In this trial we investigated how different competing interest statements affect clinical readers’ confidence in the article’s conclusions (primary outcome), its importance, their level of interest in the article, and their likelihood to change practice after reading it. We tested a range of financial COI statements that are commonly reported to journals and frequently occur in medical practice.

Schroter S, Pakpoor J, Morris J, Chew M, Godlee F. The effect of different financial competing interest statements on readers’ perceptions of clinical educational articles: Study protocol for a randomised controlled trial. BMJ Open 2016;6:e012677. [Protocol]

Schroter S, Pakpoor J, Morris J, Chew M, Godlee F. Effect of different financial competing interest statements on readers’ perceptions of clinical educational articles: A randomised controlled trial. BMJ Open 2019;9:e025029. [Full text]


Editors' declaration of their own conflicts of interest: A survey

The International Committee of Medical Journal Editors (ICMJE) recognises that all persons involved in peer review and journal publishing can have conflicting interests. Although authors are expected to declare conflicts of interest at the time of publication, there is no requirement for journal editors to publicly declare their own competing interests. We conducted a survey of editors of General and Internal Medicine journals to explore their policies regarding competing interests of editors and other journal staff.

Haivas I, Schroter S, Waechter F, Smith R. Editors' declaration of their own conflicts of interest. Canadian Medical Association Journal 2004; 171(5): 475-476. [Full Text PDF]

Editorial decision making

Editorial decision making based on abstracts

We investigated whether editors at The BMJ could make a decision about manuscripts based on reading only the abstract and how this initial decision differed from when they read the whole submission. Editors were able to make an editorial decision based on reading only the abstract for 62% (229/372) of submissions. Where a decision could be made, there was no significant difference between the two assessments in the proportion of papers rejected immediately by a single editor. Neither was there a significant difference between the two assessments in the proportion of papers where it was suggested that the submission should be sent to an external reviewer.

Schroter S, Barratt H. Editorial decision-making based on abstracts. European Science Editing February 2004; 30(1): 4-5. [Abstract] [Fulltext PDF]

Back to top >

Open access publishing

Since the introduction of the world wide web, peer-reviewed journal publishers have been wrestling with the dilemma of how to survive in a market place that facilitates and encourages the free exchange of electronic content. This web-based movement is in opposition to the traditional subscriptions-based business model of scientific publishing that relies heavily on print-based pricing and distribution. One emerging alternative to the traditional journal business model is Open Access publishing where lawful free access to journal content is provided online with production funded through means other than subscription charges. Various models for funding open access have been proposed, but one in particular has generated considerable interest and has become the business model for several new open access journals. The 'author-pays' model requires that authors, or their granting agencies or institutions, pay a contribution to the cost of publishing on submission and / or acceptance of their paper. We conducted some research with authors to help understand their perspectives. The research comprised several stages:


Semi-structured interviews with authors to determine their familiarity with and range of attitudes towards open access and author-pays publishing.

We conducted semi-structured telephone interviews to explore authors' attitudes towards open access publishing and author charges, their perceptions of journals that charge authors, and whether they would be willing to submit to these journals. We found authors were more aware of the concepts of open access publishing and author pays models than previously reported. Almost all authors supported the concept of open access, but few had submitted to an open access journal, other than The BMJ. Reasons for not submitting included lack of awareness of which journals publish with open access, and journal quality taking a higher priority in decision making than the availability of open access. Authors disliked the idea of author charges without institutional support and were concerned about implications for authors from developing countries and those without research funding.

Schroter S, Tite L, Smith R. Perceptions of Open Access Publishing: Interviews with Journal Authors. BMJ 2005;330:756-9. [Full Text]


A survey of authors to quantify their knowledge and perception of open access publishing and author-pays business models.

We conducted an electronic survey of 468 journal authors to assess current knowledge and perceptions of open access and author-pays publishing. Compared with non-open access subscription-based journals, 35% agreed that open access author-pays journals have a greater capacity to publish more content making it easier to get published, 27% thought they had lower impact factors, 31% thought they had faster and more timely publications, and 46% agreed that people will think anyone can pay to get published. Over half thought they would have to make a contribution or pay the full cost of an author charge.

Schroter S, Tite L. Open access publishing and author-pays business models: A survey of authors’ knowledge and perceptions. J R Soc Med 2006;99:130-7.[Full Text PDF]


A survey to determine the proportion of research published in BMJ Journals that is externally funded and whether authors have access to funding at the time of paper acceptance.

The author-pays model (open access publishing funded through author charges) is dependent on authors having access to financial support at the time their research papers are accepted. We conducted a survey of 377 authors from three journals in different specialties to determine the availability of external funding for publication charges at different points in the research process. We found a large proportion of published research is not externally funded, and many funded researchers do not have access to financial support at the time their paper is accepted for publication.

Schroter S, Tite L., Kassem A. Financial support at the time of paper acceptance: A survey of three medical journals. Learned Publishing 2006;19(4):291-7. [Full Text PDF]


Importance of free access to research articles on decision to submit to The BMJ: An author survey

We carried out a survey to determine whether free access to research articles on bmj.com is an important factor in authors' decisions on whether to submit to the The BMJ, whether the introduction of access controls to part of the The BMJ's content has influenced authors' perceptions of the journal, and whether the introduction of further access controls would influence authors' perceptions. We found authors value free access to research articles and consider this an important factor in deciding whether to submit to The BMJ. Closing access to research articles would have a negative effect on authors' perceptions of the journal and their likeliness to submit.

Schroter S. Importance of free access to research articles on decision to submit to the BMJ: A survey of authors. BMJ 2006;332:394-6. [Full Text]

Authorship and contributorship

How statistical expertise is utilised in medical research.

We investigated the nature and frequency of statistician involvement in medical research and its relation to the final editorial decision. Authors submitting to The BMJ and Annals of Internal Medicine in 2001 were sent a questionnaire asking if they received assistance from a person with statistical expertise, the nature of any such contribution, and reasons why, if no statistical input was received. We found that statistical input to medical research is widely recommended but inconsistently obtained. Individuals providing such expertise are often not involved until the analysis of data and many go unrecognised by either authorship or acknowledgment.

Altman DG, Goodman SN, Schroter S. How statistical expertise is utilized in medical research. JAMA 2002; 287 (21):2817-20. [Abstract] [Full Text PDF]


Two postal surveys of different methods of communicating rejection to authors submitting to a general medical journal

It is important that authors understand why their work has been rejected by a journal as this may influence the publication, and whether and where they submit in future. Little research has been conducted into authors’ perceptions of editorial review. We conducted two cross-sectional surveys of The BMJ authors whose papers were rejected without external peer review. The purpose was to evaluate differences in perceptions of the review system according to method of rejection and to evaluate preferred methods of receiving rejection.

Barratt H, Schroter S, Smith R. Two postal surveys of different methods of communicating rejection to authors submitting to a general medical journal. Accountability in Research: Policies and Quality Assurance 2003;10(4):289 – 299. [Abstract]

Presentation

The role of supplementary material in biomedical journal articles: surveys of authors, reviewers and readers

Some journals allow authors to submit supplementary material for consideration when their main manuscript is going through the editorial decision-making process and for possible publication alongside the article, should it be accepted. This might include documents or other materials such as the original research protocols, data collection forms, questionnaires, additional data tables and figures, and interview transcripts. Many journals also require authors to provide some supplementary materials on submission, for example links to the study data, completed reporting guideline checklists and flow diagrams (such as CONSORT, STROBE, PRISMA, STARD, etc.). The preparation, consideration and publication of all this material requires the use of resources by the authors, peer reviewers and journal. Is this necessary? Is it wasteful? Is it useful? Who reads the various pieces of supplementary material and do they need to be accessible from the research article? In this survey we explore the value, use, and role of this material in biomedical journal articles from the perspectives of authors, peer reviewers and readers of 17 BMJ Publishing Group journals.

Schroter S, Price A, Clarke M. The role of supplementary material in journal articles: Surveys of authors, reviewers, and readers. BMJ Open 2018;8:e021753. doi: 10.1136/bmjopen-2018-021753. [Full text]


Quality of descriptions of treatments: A review of published randomised controlled trials

To be useable in clinical practise, treatments studied in trials must provide sufficient information to enable clinicians and researchers to replicate.  In this collaborative study between The Centre of Evidence Based Medicine in Oxford (CEBM) and The BMJ, we assessed the completeness of treatment descriptions in published randomised controlled trials (RCTs) using a checklist and determined the extent to which peer reviewers and editors commented on the quality of reporting of treatments.

Schroter S, Glasziou P, Heneghan C. Quality of descriptions of treatments: A review of published randomised controlled trials. BMJ Open 2012;2:e001978. doi:10.1136/bmjopen-2012-001978. [Full text]


From submission to publication: a retrospective review of the tables and figures in a cohort of randomised controlled trials submitted to The BMJ.

Tables and figures can convey details and complex relationships not easily described in text. We characterised the quantity and quality of data tables and figures in reports of randomised controlled trials (RCTs) submitted to The BMJ and published in peer-reviewed journals. We investigated how the peer review process affected table and figure quality. We found the numbers of tables and figures did not change markedly between submission and publication. Less than half the figures met their data presentation potential, with most failing to portray by-subject data and few displaying advanced features such as pairing, symbolic dimensionality, or small multiples. The BMJ external peer reviewers seldom commented on tables or figures.

Schriger DL, Sinha R, Schroter S, Liu PY, Altman DG. From Submission to Publication: A Retrospective Review of the Tables and Figures in a Cohort of Randomized Controlled Trials Submitted to the British Medical Journal. Ann Emerg Med 2006 Dec;48(6):750-6. [Abstract]


How should general medical journals present scientific papers? A survey of readers and authors

Editorial space is a scarce resource and editors struggle to meet the expectations of both authors and readers. Several scientific journals have adopted strategies to publish detailed article content on the web and a more concise presentation in the print journal. We elicited readers’ and authors’ preferences for three different presentational styles of abridged print versions: the conventional approach where the Introduction-Methods-Results-Discussion structure is maintained and the text simply shortened, a journalistic version with a very different structure, and a version with an enhanced abstract similar to those used in evidence-based medicine journals. We found authors and readers prefer a more conventional style of abridged papers.

Müllner M, Waechter F, Schroter S, Squire B. How should abridged scientific articles be presented in journals? A survey of readers and authors. CMAJ 2005; 172(2): 203-5 [Abstract] [Full Text]


Authors’ perceptions of electronic publishing: Two cross sectional surveys.

We conducted a survey to evaluate how acceptable authors find the The BMJ's current practice of publishing short versions of research articles in the paper journal and a longer version on the web (ELPS) and to determine authors’ attitudes towards publishing only abstracts in the paper journal and publishing unedited versions on bmj.com once papers have been accepted for publication.

Schroter S, Barratt H, Smith J. Authors’ perceptions of electronic publishing: Two cross-sectional surveys. BMJ 2004; 328:1350-3. [Abstract] [Full Text]

Research and publication ethics

Authors’ awareness of publication ethics: An international survey

Considerable time is expended by journals investigating ethical transgressions and misconduct that may be caused by ignorance rather than wilful deceit. Publication ethics are rarely taught. We conducted an international survey of authors submitting to 20 BMJ journals to measure the level of awareness of key ethical issues in publishing amongst a large sample of currently active researchers. We hope the results stimulate discussion within journal publishing and prompt efforts to improve knowledge and, in turn, behaviour.

Schroter S, Roberts J, Loder E, Penzien DB, Mahadeo S, Houle T. Biomedical authors’ awareness of publication ethics: an international survey. BMJ Open2018;8:e021282. doi: 10.1136/bmjopen-2017-021282.[Full text]


Can an ethics officer role reduce delays in research ethics approval? A mixed-method evaluation of an improvement project

This collaborative study with the Health Research Authority is one of the few controlled evaluations of a quality improvement intervention in research ethics review. It found that provision of an ethics officer role, as deployed in this project, does not shorten time to final decision by research ethics committees nor increase the proportion of applications that got a favourable opinion first time.

Dixon-woods M, Foy C, Hayden C, Al-Shahi Salman R, Tebbutt S, Schroter S. Can an ethics officer role reduce delays in research ethics approval? A mixed-method evaluation of an improvement project. BMJ Open 2016;6:e011973. [Full text]


Reporting of ethical committee approval and patient consent by study design in five general medical journals

Authors are required to describe in their manuscripts ethical approval from an appropriate committee and how consent was obtained from subjects when research involves human subjects. Previous studies have focused on concordance with these regulations within a single speciality or for clinical trials. We assessed reporting of these protections for several study designs published in Annals of Internal Medicine, The BMJ, JAMA, Lancet and New England Journal of Medicine. We found the reporting of ethical approval and consent has improved for RCTs but journals are less good at reporting this information for other study designs.

Schroter S, Plowman R, Hutchings A, Gonzalez A. Reporting of ethical committee approval and patient consent by study design in five general medical journals. J Med Ethics 2006;32:718-23. [Abstract] [Full text PDF]

Patient and Public Involvement in Research

Involving patients and the public in the peer review process

In 2014/15 The BMJ and Research Involvement and Engagement (RIE) became the first journals to routinely include patients and the public in the peer review process of journal articles. In this collaborative survey carried out by The BMJ and RIE, with embedded patient involvement, we explore the early experiences of our patient and public reviewers. Reviewers were asked what motivates them to review research articles, if and why they have declined to review, their confidence in reviewing, their perceptions of open review, their satisfaction with the process, how they would like to be acknowledged for their contributions and their views on how the process could be improved. This knowledge will be used to inform evidence-based guidance for patient and public reviewers.

Schroter S, Price A, Flemyng E, et al. Perspectives on involvement in the peer-review process: surveys of patient and public reviewers at two journals. BMJ Open2018;8:0:e023357. doi:10.1136/bmjopen-2018-023357.[Full text]


Frequency of reporting on patient and public involvement (PPI) in research studies

While documented plans for patient and public involvement (PPI) in research are required in many grant applications, little is known about how frequently PPI occurs in practice. We conducted a before and after comparison of PPI reported in research papers published in The BMJ before and 1 year after the introduction of a journal policy requiring authors to report if and how they involved patients and the public within their papers.

Price A, Schroter S, Snow R, Hicks M, Harmston R, Staniszewska S, Parker S, Richards T. Frequency of reporting on Patient and Public Involvement (PPI) in research studies published in a general medical journal: A descriptive study. BMJ Open 2018;8:e020452. doi: 10.1136/bmjopen-2017-020452. [Full text]

Data sharing

Early Experiences With Journal Data Sharing Policies: A Survey of Published Clinical Trial Investigators

Although the International Committee of Medical Journal Editors (ICMJE) recommendations for trial data sharing have been controversial, little is known about the attitudes and experiences of authors published in journals with existing data sharing policies. In this collaborative study with Yale University School of Medicine we conducted a self-administered online survey of the authors of clinical trials published January 1, 2012, through March 1, 2016, in 3 high-impact journals with policies either requiring all clinical trial authors to share data (PLOS Medicine) or publish a statement specifying whether they were willing to share data (The BMJ and Annals of Internal Medicine). The survey addressed sharing plans, receipt of sharing requests, and effort required to respond to sharing requests. Respondents were also asked about willingness to share data in 6 hypothetical scenarios.

Tannenbaum S, Ross J, Krumholz, H, Desai N, Ritchie J, Lehman R, Gamble G, Bachand J, Schroter S, Groves T, Gross C. Early experiences with data sharing policies: A survey of published clinical trial investigators. Ann Intern Med. [Epub ahead of print 17 July 2018] doi: 10.7326/M18-0723.

Marketing trials

Characterisation of trials where marketing purposes have been influential in study design: a descriptive study

Analysis of trial documentation has revealed that some industry-funded trials may be done more for marketing purposes than scientific endeavour. In this study we aimed to define characteristics of drug trials that appear to be influenced by marketing considerations and estimate their prevalence. We reached consensus that a fifth of drug trials published in the highest impact general medical journals in 2011 had features that were suggestive of being designed for marketing purposes. Each of the marketing trials appeared to have a unique combination of features reported in the journal publications.

Barbour V, Burch D, Godlee F, Heneghan C, Lehman R, Perera R, Ross JS, Schroter S. Characterisation of trials where marketing purposes have been influential in study design: a descriptive study. Trials 2016;17:31 (doi:10.1186/s13063-015-1107-1). [Full text]

Selective outcome reporting

Discrepancy rates in prespecified and reported outcomes in articles submitted to The BMJ

Adding, omitting or changing prespecified outcomes can result in bias because it increases the potential for unacknowledged or post hoc revisions of the planned analyses. Journals have adopted initiatives such as requiring the prospective registration of trials and the submission of study protocols to promote the transparency of reporting in clinical trials. The main objective of this feasibility study was to document the frequency and types of outcome discrepancy between prespecified outcomes in the protocol and reported outcomes in trials submitted to The BMJ.

Weston J, Dwan K, Altman D, Clarke M, Gamble C, Schroter S, Williamson PR, Kirkham JJ. A feasibility study to examine discrepancy rates in pre-specified and reported outcomes in articles submitted to The BMJ. BMJ Open 2016;6:e010075. [Full text]

Trial registration

What happened to clinical trials rejected by The BMJ for improper registration?

We evaluated the characteristics and publication fate of improperly registered clinical trials submitted to The BMJ over a 4-year period to identify common types of registration issues and their relation to publication outcomes. 123 research papers reporting apparently unregistered or retrospectively registered clinical trials were identified. 110 studies (89.4%) were retrospectively registered, nine (7.3%) were unregistered, three (2.4%) had been registered in an unapproved registry and one study originally lacking registration details was later discovered to have been prospectively registered. Of those papers submitted to The BMJ through the end of 2015, 67 of the 70 papers rejected for registration problems (96%) were subsequently published in another journal. The registration problem was disclosed in only 2 (3%). Improper registration remains a problem. Nonetheless, improperly registered trials are almost always published, suggesting that medical journal editors may not actively enforce registration requirements.

Loder E, Loder S, Cook S. Characteristics and publication fate of unregistered and retrospectively registered clinical trials submitted to The BMJ over 4 years.BMJ Open 2018;8:e020037. doi: 10.1136/bmjopen-2017-020037.[Full text]

BMJ Research: in progress

Authorship

Researchers’ awareness and use of authorship guidelines: an international survey

Sara Schroter, Ilaria Montagni, Elizabeth Loder, Matthias Eikermann, Elke Schaeffner, Tobias Kurth

We conducted a large international survey on perceptions and practice of biomedical authorship with active researchers with a wide range of research interests and experience. We sampled active researchers submitting papers to biomedical journals from a range of specialties and Impact Factors to determine: their familiarity with and use of ICMJE criteria; institutional encouragement to use authorship guidelines; how frequently they had experienced inappropriate authorship (honorary and ghost authorship); the timing of authorship decisions in relation to the last paper they coauthored; perceived fairness of authorship decisions; and awareness of their own institution’s authorship policy.

Current status: Paper in preparation. Abstract presented at the Eighth International Congress on Peer Review and Biomedical Publication, Chicago, USA, September 2017. [Abstract]

Dissemination

Frequency and format of clinical trial results disseminated to participants: a survey of trialists

Sara Schroter, Amy Price, Mario Malički, Rosamund Snow, Tessa Richards, Mike Clarke

Dissemination of research findings is a crucial part of the research process as it supports the translation of the knowledge gained in the research into practice. Dissemination is the process by which results of studies are communicated to the stakeholders of that research. Stakeholders can include health policy makers, academics, practicing clinicians and other healthcare workers, patients, the public, and research funders. However, effective communication and active dissemination of research results to patients, which is essential for them to make informed decisions and choices, is sometimes an afterthought among researchers. We surveyed researchers who had recently published a clinical trial to determine how frequently and in what format they disseminated the results of their clinical trials to trial participants (and patient groups), and to identify barriers to dissemination.

Current status: Paper in preparation. Abstract presented at the Eighth International Congress on Peer Review and Biomedical Publication, Chicago, USA, September 2017.[Abstract]

Back to top >

Publication bias

Submitted but unpublished trials: an empirical evaluation

John Ioannidis, Sara Schroter, Doug Altman

There is considerable evidence that some randomised controlled trials (RCTs) are never published or are published with great delay. Publication bias is considered to depend mostly on the fact that investigators may never submit their RCT for publication, and this may be more likely for trials with “negative” results. However, there is more debate and less evidence on whether RCTs may still remain unpublished even though their investigators make an attempt to publish them and do submit them for peer-review and publication. To answer this question, we are studying a cohort of RCTs that were submitted for publication to The BMJ between 1998 and 2001 and examining whether any of them remain unpublished after a long period of time.

Current status: Abstract accepted for presentation at the Sixth International Congress on Peer Review and Biomedical Publication, 2009.

The BMJ's research publications

Open Access

    • Schroter S, Tite L, Smith R. Perceptions of open access publishing: interviews with journal authors. BMJ2005;330:756-9. [Abstract] [Full text]
    • Schroter S, Tite L. Open access publishing and author-pays business models: a survey of authors’ knowledge and perceptions. J R Soc Med 2006;99:130-7. [Full text] [Full text PDF]
    • Schroter S, Tite L., Kassem A. Financial support at the time of paper acceptance: a survey of three medical journals. Learned Publishing 2006;19:291-7. [Abstract]
    • Schroter S. Importance of free access to research articles on decision to submit to the BMJ: A survey of authors. BMJ 2006;332:394-6. [Abstract] [Full text]

Peer review

    • van Rooyen S, Delamothe T, Evans SJW. Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial. BMJ 2010;341:c5729 [Full text ]
    • Schroter S, Groves T, Højgaard L. Surveys of current status in biomedical science grant review: funding organisations’ and grant reviewers’ perspectives. BMC Med 2010;8:62. [Full text PDF]
    • Tite L, Schroter S. Why do reviewers decline to review for journals? A survey. J Epidemiol Community Health2007;61:9-12. [Abstract]
    • Schroter S, Tite L, Hutchings A, Black N. Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors. JAMA 2006;295:314-7. [Full text]
    • Schroter S, Black N, Evans S, Smith R, Carpenter J, Godlee F. Effects of training on quality of peer review: a randomised controlled trial. BMJ 2004;328:673-5 [Full text] [Peer review training pack]
    • White I, Carpenter J, Evans S, Schroter S. Eliciting and using expert opinions about dropout bias in randomised controlled trials. Clin Trials 2007;4(2):125-39. [Abstract]
    • Schroter S, Black N, Evans S, Godlee F, Osorio L, Smith R. What errors do peer reviewers detect, and does training improve their ability to detect them? J R Soc Med 2008:101:507-14. [Full text PDF]
    • van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers' recommendations: A randomised trial. BMJ 1999;318:23 - 27 [Abstract] [Full text]
    • van Rooyen S, Black N, Godlee F. Development of the review quality instrument (RQI) for assessing peer reviews of manuscripts. J Clin Epidemiol 1999;52:625-29 [Abstract]
    • van Rooyen S, Godlee F, Evans S, Smith R, Black N. Effect of blinding and unmasking on the quality of peer review: a randomized trial. JAMA 1998;280:234-7 [Full text]
    • Black N, van Rooyen S, Godlee F, Smith R, Evans S. What makes a good reviewer and a good review for a general medical journal? JAMA 1998;280:231-3 [Full text]

Research ethics

    • Dixon-woods M, Foy C, Hayden C, Al-Shahi Salman R, Tebbutt S, Schroter S. Can an ethics officer role reduce delays in research ethics approval? A mixed-method evaluation of an improvement project. BMJ Open 2016;6:e011973.[Full text]
    • Schroter S, Plowman R, Hutchings A, Gonzalez A. Reporting of ethical committee approval and patient consent by study design in five general medical journals. J Med Ethics 2006;32:718-23. [Abstract] [Full text PDF]
    • Pace NA, Owen M, Schroter S. What happens to papers rejected by the BMJ on ethical grounds. BMJ2008;337:a1880. [Letter]

Competing interests

    • Schroter S, Pakpoor J, Morris J, Chew M, Godlee F. Effect of different financial competing interest statements on readers’ perceptions of clinical educational articles: A randomised controlled trial. BMJ Open 2019;9:e025029. [Full text]

    • Schroter S, Pakpoor J, Morris J, Chew M, Godlee F. The effect of different financial competing interest statements on readers’ perceptions of clinical educational articles: Study protocol for a randomised controlled trial. BMJ Open 2016;6:e012677. [Full text]
    • Haivas I, Schroter S, Waechter F, Smith R. Editors' declaration of their own conflicts of interest. CMAJ 2004;171:475-6 [Full text]
    • Schroter S, Morris J, Chaudhry S, Smith, R, Barratt H. Does the type of competing interest statement affect reader perceptions of the credibility of research? A randomised trial. BMJ 2004;328:742-3. [Full text]

    • Chaudhry S, Schroter S, Smith R, Morris J. Does declaration of competing interests affect reader perceptions? A randomised trial. BMJ 2002;325:1391-2. [Full text]
    • Williams HC, Naldi L, Paul C, Vahlquist A, Schroter S, Jobling R. Conflicts of interest in dermatology. Acta Derm Venereol 2006;86(6):485-97. [Full text] [Full text PDF]

Editorial decision making

    • Schroter S, Barratt H. Editorial decision-making based on abstracts. European Science Editing 2004;30:4-5.

Authorship and contributorship

    • Barratt H, Schroter S, Smith R. Two postal surveys of different methods of communicating rejection to authors submitting to a general medical journal. Accountability in Research: Policies and Quality Assurance 2003;10:289-99.
    • Altman DG, Goodman SN, Schroter S. How statistical expertise is utilized in medical research. JAMA2002;287:2817-20 [Full Text]

Medical education

    • Schroter S, Jenkins RD, Playle RA, Walsh KM, Probert C, Kellner T, et al. Evaluation of an online interactive diabetes needs assessment tool (DNAT) versus online self-directed learning: a randomised controlled trial. BMC Med Educ2011;11:35. [Full text]
    • Murray S, Lazure P, Schroter S, Leuschner PJ, Posel P, Kellner T, et al. International challenges without borders: a descriptive study of family physicians’ educational needs in the field of diabetes. BMC Fam Pract 2011;12:27. [Full text]
    • Schroter S, Jenkins D, Playle R, Walsh K, Probert C, Kellner T, Arnhofer G, Owens D. Evaluation of an online diabetes needs assessment tool (DNAT) for health professionals: a randomised controlled trial. Trials 2009;10:63. [Full text]
    • Sandars J, Schroter S. Web 2.0 technologies for undergraduate and postgraduate medical education: an online survey. Postgrad Med J 2007;83:759-62. [Abstract]

Presentation

    • Schroter S, Price A, Clarke M. The role of supplementary material in journal articles: Surveys of authors, reviewers, and readers. BMJ Open 2018;8:e021753. doi: 10.1136/bmjopen-2018-021753.[Full text]
    • Schroter S, Glasziou P, Heneghan C. Quality of descriptions of treatments: A review of published randomised controlled trials. BMJ Open 2012;2:e001978. doi:10.1136/bmjopen-2012-001978. [Full text]
    • Schriger DL, Sinha R, Schroter S, Liu PY, Altman DG. From submission to publication: a retrospective review of the tables and figures in a cohort of randomized controlled trials submitted to the British Medical Journal. Ann Emerg Med 2006;48:750-6. [Abstract]
    • Müllner M, Waechter F, Schroter S, Squire B. How should abridged scientific articles be presented in journals? A survey of readers and authors. CMAJ 2005;172:203-5 [Full text]
    • Barratt H, Schroter S, Smith J. Authors’ perceptions of electronic publishing: Two cross sectional surveys. BMJ2004;328:1350-3 [Abstract] [[Full text]

Patient and public involvement in research

    • Price A, Schroter S, Snow R, Hicks M, Harmston R, Staniszewska S, Parker S, Richards T. Frequency of reporting on Patient and Public Involvement (PPI) in research studies published in a general medical journal: A descriptive study. BMJ Open 2018;8:e020452. doi: 10.1136/bmjopen-2017-020452.  [Full text]
    • Schroter S, Price A, Flemyng E, et al. Perspectives on involvement in the peer-review process: surveys of patient and public reviewers at two journals. BMJ Open2018;8:0:e023357. doi:10.1136/bmjopen-2018-023357.[Full text]

Selective outcome reporting

    • Weston J, Dwan K, Altman D, Clarke M, Gamble C, Schroter S, Williamson PR, Kirkham JJ. A feasibility study to examine discrepancy rates in pre-specified and reported outcomes in articles submitted to The BMJ. BMJ Open 2016;6:e010075. [Full text]

Marketing trials

    • Barbour V, Burch D, Godlee F, Heneghan C, Lehman R, Perera R, Ross JS, Schroter S. Characterisation of trials where marketing purposes have been influential in study design: a descriptive study. Trials 2016;17:31 doi:10.1186/s13063-015-1107-1. [Full text]

Data sharing

    • Tannenbaum S, Ross J, Krumholz, H, Desai N, Ritchie J, Lehman R, Gamble G, Bachand J, Schroter S, Groves T, Gross C. Early experiences with data sharing policies: A survey of published clinical trial investigators. Ann Intern Med. [Epub ahead of print 17 July 2018] doi: 10.7326/M18-0723.[Abstract]

Trial registration

    • Loder E, Loder S, Cook S. Characteristics and publication fate of unregistered and retrospectively registered clinical trials submitted to The BMJ over 4 years. BMJ Open 2018;8:e020037. doi: 10.1136/bmjopen-2017-020037.[Full text]

PhD programme: Responsible conduct of publishing scientific research

The BMJ has been at the forefront of innovative thinking and practice in healthcare since 1840 and is the fourth most highly cited general medical journal in the world. Our mission is to lead the debate on health and to engage, inform, and stimulate doctors, researchers, and other health professionals in ways that will improve outcomes for patients. For over 20 years we have been conducting collaborative research into the science of publication including peer review, authorship, conflicts of interest, and publication ethics. We are now partnering with the group meta-research at Maastricht University, the Netherlands to offer a 4 year PhD program in the responsible conduct of publishing scientific research.

The PhD will be embedded within the Care and Public Health Research Institute (CAPHRI) of Maastricht University. CAPHRI is one of Maastricht University’s largest and oldest schools with a track record to high quality, multidisciplinary research in Public Health and Health Care and with a strong focus on having Societal Impact. The institute attracts top scientists from around the world and offers a comprehensive PhD training programme for young scientists.

This new PhD opportunity is for a self-funded or scholarship-based program with distance-learning. PhD candidates will be supervised by a team of research integrity researchers and BMJ staff on selected topics. Successful students will be reimbursed up to a maximum of 10k euro for the 4 years for travel (for one week annual stays at BMJ and Maastricht University), course and conference participation and consumables and free access to selected courses. There will be no University tuition fee.

Current PhD opportunities

There are only two places available for 2019. Applicants should select one of the following projects:

*** PENDING FURTHER ANNOUNCEMENT***

Requirements

Successful candidates are expected to be highly motivated and hold a Masters Degree in a relevant discipline. They will be asked to provide relevant documentation to show they:

• are fluent in English (IELTS>6.5 or equivalent);
• have the necessary time available to do the research (e.g. recommendation letter from current employer); and
• have proven scientific writing ability (e.g. at least one coauthored journal article).

Procedure

If you are interested in one of the current PhD opportunities above, please indicate which project and send your CV and letter of motivation to Sara Schroter (sschroter@bmj.com) by [date to be confirmed]. Following this you could be asked to write the research proposal, which could lead to an interview. Successful candidates will be selected on the basis of academic potential to reach the highest standards of scientific research.

Supervisory team

Click here for information about the supervisory team.