Intended for healthcare professionals

Editorials

Ninth international congress on peer review and scientific publication: call for research

BMJ 2019; 366 doi: https://doi.org/10.1136/bmj.l5475 (Published 16 September 2019) Cite this as: BMJ 2019;366:l5475
  1. John P A Ioannidis, professor1 2,
  2. Michael Berkwits, deputy editor3,
  3. Annette Flanagin, executive managing editor3,
  4. Fiona Godlee, editor in chief4,
  5. Theodora Bloom, executive editor4
  1. 1Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA
  2. 2Meta-Research Innovation Center–Berlin (METRIC-B), Berlin, Germany
  3. 3JAMA and JAMA Network, Chicago, IL, USA
  4. 4The BMJ, London, UK
  1. Correspondence to: A Flanagin Annette.Flanagin{at}jamanetwork.org

Help to improve the conduct, reporting, quality, and integrity of scientific research

The quantity and quality of scientific research have never been greater, but with unprecedented promise comes unprecedented peril. There are better scientific policies and processes, stronger standards for openness and transparency, and innovative technologies to collaborate and publish. However, the rapidly evolving scientific publication ecosystem that facilitates research dissemination also enables research waste, predation, and piracy. The challenge of distinguishing information from noise, innovation from dystopian-like disruption, and opportunity from threat has created new levels of excitement and angst for those engaged in research and its reporting, publication, and distribution.

In these historically challenging circumstances, we announce the ninth international congress on peer review and scientific publication, which will take place on 12-14 September 2021 in Chicago, Illinois. Since the first peer review congress in 1989, founded by Drummond Rennie when he was JAMA’s deputy editor,1 the aim has been to encourage research into the quality and credibility of peer review and scientific publication. The goal is to strengthen the evidence base so that all those involved in science (including but not limited to researchers, editors, publishers, funders, policy makers, academics, and representatives of universities, industry, media, and the general public) can improve the conduct, reporting, and dissemination of scientific research.2

A few core principles and practices have governed past congresses and the next one announced here.3 The congresses provide a forum for research into the processes of selection, refinement, and dissemination of scientific knowledge. They do not presume to establish rules governing publication, to settle matters by consensus, or to dictate how scientists and journal editors should conduct themselves. The congress programme is democratic, mostly determined by abstracts submitted by researchers, so it represents the work of the community, with priority given to data driven studies. There are no simultaneous breakout or parallel sessions so all attendees can hear every presentation and participate in every discussion, and the audience is given equal time to debate the presentations. Past feedback from participants identifies this democratic format as one of the keys to the success of the congresses.3

For the ninth congress we call for new studies into the processes used by researchers, authors, editors, peer reviewers, publishers, funders, universities, and any other stakeholders to improve the conduct, reporting, quality, integrity, and dissemination of scientific research and academic commentary. We encourage new ideas and rigorous evaluations of both old and new processes. Examples include evaluations of the quality of scientific reporting and publication, and developments in open and collaborative peer review, pre- and post-publication public review, open and public access, data sharing and access, reproducibility of methods and research findings, fair and reliable metrics of publication, and transparent reporting of author contributions, conflicts, and biases. Studies of bias—of all sorts—and how biases can be identified and managed are of particular interest

In addition, many problems threatening the credibility of science such as research misconduct, deviation from authorship norms, conflicts of interest (financial or other), inadequate ethical oversight of research, misuse of statistical tests and analyses, sloppy reporting of research findings, and misleading information in news and social media coverage of research are still in need of effective evidence based solutions. We also seek to shed light on looming dangers from dishonest opportunists and rogue operators who threaten the foundation of the scientific enterprise. For each of these problems and threats, potential solutions and interventions need testing.

The peer review congress began with a focus on biomedicine but in recent years has broadened its scope to include health sciences, basic sciences, and the full spectrum of social and physical sciences. We plan to continue this expansion to all sciences. Much exciting work and debate on how to improve scientific research is happening outside biomedicine. Breaking the communication barriers across disciplines may offer the greatest potential for change and tangible improvements.

The peer review congress is organised and supported by JAMA and the JAMA Network, The BMJ, and the Meta-Research Innovation Center at Stanford (METRICS). We are also supported by the members of our advisory board representing a wide variety of scientific disciplines (see the acknowledgment below).

With two years to go, we encourage you to start your research now for the ninth congress. Abstracts summarising original, high quality research on any aspect of scientific peer review and publication and the conduct, reporting, assessment, and dissemination of scientific research are welcome. Topics of interest include those listed in box 1, but we will consider any relevant novel work and welcome a range of study designs, including controlled trials and observational studies, systematic reviews and meta-analyses, bibliometric analyses, surveys, and modelling studies. As with previous congresses, well designed studies that build on previous knowledge and test or evaluate processes, policies, and interventions are preferred. We will give priority to studies with more generalisable results (eg, prospective multiyear trials and controlled studies from collaborations between researchers, journals, publishers, funders, and information disseminators) and those that report original research.

Box 1

Suggested topics of interest for research into peer review and scientific publication

Bias

  • Bias on the part of researchers, authors, reviewers, editors, funders, commentators, and consumers of scientific information

  • Publication and reporting bias

  • Efforts to manage or eliminate bias in research methods, conduct, and reporting and in interpretation of evidence

Editorial and peer review decision making and responsibilities

  • Models of peer review and editorial decision making and workflows used by journals, funders, and research disseminators

  • Evaluations of the quality, validity, and practicality of peer review and editorial decision making

  • Quality assurance for reviewers, editors, and funders

  • Editorial policies and responsibilities

  • Editorial freedom and integrity

  • Peer review of grant proposals

Research and publication ethics

  • Ethical concerns for researchers, authors, reviewers, editors, publishers, and funders

  • Authorship, contributorship, accountability, and responsibility for published material

  • Conflicts of interest

  • Research and publication misconduct

  • Ethical review and approval of studies

  • Confidentiality

  • Effects of funding and sponsorship on research and publication

  • Influence of external stakeholders: funders, journal owners, advertisers/sponsors, policy makers, legal representatives, and news media

  • Effects of opportunistic, predatory, and pirate operators

Improving quality of reporting

  • Effectiveness of guidelines and standards designed to improve the quality of scientific reporting and publication

  • Evaluations of the quality of published information

  • Data sharing, transparency, reliability, and access

  • Research reproducibility

  • Innovations to improve appropriate use of methods and statistics

  • Quality and reliability of data presentation and scientific images

  • Quality and effectiveness of new formats for scientific articles

Models for peer review and scientific publication

  • Open and public access

  • Single blind, double blind, collaborative, and open peer review

  • Preprints and prepublication posting and release of information

  • Re-analyses

  • Reproducibility checks

  • Preregistration of research

  • Embargoes

  • Post-publication review, communications, and influence

  • Effects of social media

  • Changes in readership and use of peer reviewed published content

  • Quality and effects of scientific information in multimedia and new media

  • Quality, use, and effects of publication and performance metrics and usage statistics

  • Quality and influence of advertising and sponsored publication

  • Quality and effectiveness of content tagging, mark-up, and linking

  • Use of assisted artificial intelligence and software to improve editorial review and peer review

  • Threats to scientific publication

  • The future of scientific publication

Dissemination of scientific and scholarly information

  • Methods for improving the quality, efficiency, and equitable distribution of scientific information

  • Use of novel mechanisms, formats, and platforms to disseminate science

  • New technologies that affect the quality, integrity, evaluation, dissemination, and access of scientific information

  • Funding and reward systems as they relate to science and scientific publication

  • Comparisons of and lessons from various scientific disciplines

  • Mapping of scientific methods and reporting practices and of meta-research across disciplines

RETURN TO TEXT

The deadline for submitting abstracts is January 2021. Programmes and abstracts of research presented at the previous eight congresses are available on the peer review congress website at peerreviewcongress.org.2 Additional information and future announcements will be available on the website, where you can also sign up for email alerts. We look forward to your contributions to the ninth international congress on peer review and scientific publication.

Acknowledgments

Peer review congress organisers and advisory board: Congress directors: John P A Ioannidis, METRICS, Michael Berkwits, JAMA and JAMA Network; congress executive director: Annette Flanagin, JAMA and JAMA Network; European director: Fiona Godlee, The BMJ; European coordinator: Theodora Bloom, The BMJ; associate directors: Steve Goodman, METRICS, Véronique Kiermer, PLOS, David Moher, Ottawa Hospital Research Institute, Canada; director emeritus: Drummond Rennie, University of California, San Francisco. Advisory board members: Vivienne Bachelet, Medwave, Universidad de Santiago de Chile, Chile; Albert-László Barabási, Northeastern University, Boston, MA, USA; Howard Bauchner, JAMA and JAMA Network, Chicago, IL, USA; Lisa A Bero, University of Sydney, Australia; Patrick M Bossuyt, University of Amsterdam, Netherlands; Lex Bouter, Vrije University, Amsterdam, Netherlands; Isabelle Boutron, French Cochrane Centre, Paris, France; Edward W Campion, New England Journal of Medicine, Boston, MA, USA; An-Wen Chan, University of Toronto, Canada; Kay Dickersin, Johns Hopkins Bloomberg School of Public Health, Baltimore, MA, USA; Dan Evanko, American Association for Cancer Research, Philadelphia, PA, USA; James Evans, University of Chicago, USA; Trish Groves, BMJ; R Brian Haynes, McMaster University Faculty of Health Sciences, Hamilton, Ontario, Canada; Sabine Kleinert, Lancet, London, UK; Christine Laine, Annals of Internal Medicine, Philadelphia, PA, USA; José Florencio F Lapeña Jr, University of the Philippines, Manilla; Malcolm MacLeod, University of Edinburgh, UK; Emilie Marcus, University of California at Los Angeles, USA; Ana Marušić, Journal of Global Health, Zagreb, Croatia; Frank Miedema, University Medical Center Utrecht, Netherlands; Jigisha Patel, BioMed Central, London, UK; David Schriger, University of California, Los Angeles; Victoria Stodden, University of Illinois, Champaign, IL, USA; Deborah Sweet, Cell Press, Cambridge, MA, USA; Martin R Tramèr, European Journal of Anaesthesiology, Geneva, Switzerland; Sarah Tegen, American Chemical Society, Washington, DC, USA; Valda Vinson, Science, Washington, DC, USA; Erik Von Elm, Cochrane Switzerland, Lausanne; Eric-Jan Wagenmakers, University of Amsterdam, Netherlands; Rebecca Williams, National Library of Medicine, Bethesda, MD, USA

Footnotes

  • A version of this editorial is also published in JAMA.

  • Competing interests: We have read and understood BMJ policy on declaration of interests and declare that all authors are directors or coordinators of the peer review congress. AF serves as an unpaid board member of STM: International Association of Scientific, Technical, and Medical Publishers. TB is a founder of medRxiv.

  • Provenance and peer review: Commissioned; not externally peer reviewed.

References

View Abstract