Intended for healthcare professionals

Head To Head

Will research preprints improve healthcare for patients?

BMJ 2018; 362 doi: https://doi.org/10.1136/bmj.k3628 (Published 24 September 2018) Cite this as: BMJ 2018;362:k3628

Opinion

Sharing science at today’s pace: an experience with preprints

  1. Harlan M Krumholz, Harold H Hines Jr professor of medicine and public health12,
  2. Joseph S Ross, associate professor of medicine and public health1 2,
  3. Catherine M Otto, J Ward Kennedy-Hamilton endowed chair in cardiology3
  1. 1Yale Schools of Medicine and Public Health, New Haven, CT, USA
  2. 2Center for Outcomes Research and Evaluation, Yale-New Haven Hospital, New Haven, CT, USA
  3. 3Division of Cardiology, University of Washington, Seattle, Washington, USA
  1. Correspondence to: H M Krumholz harlan.krumholz{at}yale.edu, C M Otto cmotto{at}uw.edu

Early publication of research findings without peer review could speed up knowledge dissemination and changes to clinical practice, argue Harlan M Krumholz and Joseph S Ross. But Catherine M Otto worries that publication without that quality control has the potential to confuse and cause harm

Yes—Harlan M Krumholz, Joseph S Ross

Preprinting in the clinical sciences, with appropriate safeguards, should be welcomed: it accelerates knowledge dissemination and reduces non-reporting and waste in research, while promoting transparency. Preprints could also improve the safety of clinical action based on preliminary work, by making the full data and description public and avoiding the hearsay of presentations at scientific meetings, internet postings, and media interviews. Many other disciplines already use preprints; in physics, for example, preprint posting has existed alongside formal journal publication for more than 20 years.

Peer review slows scientific communication

Progress in medicine relies on the prompt communication of research findings. the results of many clinical trials are never reported or only reported years after trial completion.123 Delays in publication can cause harm as scientists work without knowing what others have done or found.45 Moreover, for public health emergencies, when rapid communication is essential, preprint servers can immediately disseminate study findings globally, fostering scientific exchange.6

Peer review, while performing an essential function, can slow scientific communication. Peer review journals will continue to play a role in constructively scrutinising and editing clinical research, ensuring that findings are disseminated in a manner to best inform clinical care. However, peer review impedes the rapid dissemination of scientific knowledge, holding up the information until it becomes a published product—which in some cases can take years. By contrast, research shared on a preprint server enables and accelerates the sharing of pre-peer reviewed findings in a way that timestamps the contribution, makes it widely available, and provides the ability to comment while labelling it clearly as a work in progress.

Improved transparency

Preprint servers encourage transparency in the scientific process. Other researchers can build on a researcher’s initial findings and interpretation. Detailed reporting of some trials, particularly if small or with negative findings, may be thwarted by peer review and editorial priorities and never shared publicly. Yet such trials sometimes hold information that could inform researchers and other experts—and preprints enable sharing of the results at the study’s conclusion. These are just some of the reasons that major funders of research such as the US National Institutes of Health are encouraging researchers to post preprints.

Preprints can help quickly establish the provenance of ideas and scientific theories and eliminate delays between study completion and scientific dissemination. If researchers wait until formal publication to share their work, months or years after completion, further work building on their findings is in turn delayed. Such delays matter importantly to the clinical scientific enterprise, where studies already take many years to design, collect data, and disseminate.

Could acting inappropriately on information in preprints create risk for patients? Such concerns have not stymied the sharing of information from scientific meetings, which routinely promote non-peer reviewed science, albeit in a different way. And unlike dissemination of information at scientific meetings, in which preliminary, non-peer reviewed information is often promoted to the press, the preprint server archives a document that can be studied and referenced. Moreover, the public is commonly exposed to preliminary science, including animal studies; we are past a world where data can be held back.

The question is not whether the public could be shielded from such information (it cannot be), but whether preliminary information could be presented in ways that reduce risks while improving scientific communication. These concerns could be mitigated by clearly labelling the work as preliminary (with watermarks), adhering to research reporting requirements (such as EQUATOR Network guidelines), and clearly disclosing author affiliations, financial support, and research contributions, as well as proper approvals or exemptions from ethics boards.

Ultimately, science is a public good that thrives in an atmosphere of open, rapid communication. Through preprints, the scientific community will be engaged earlier, allowing the science to be examined, to inform planned or existing studies elsewhere, and perhaps identifying errors in analysis or interpretation earlier. Clinical research preprints, with appropriate safeguards, are critical to improving healthcare by accelerating knowledge dissemination and reducing non-reporting of research.

No—Catherine M Otto

Our patients deserve well designed, carefully executed, clearly presented, and appropriately analysed clinical research published expeditiously, but only once, after peer review and editing in a trustworthy journal.

Medical research preprints are unnecessary for scientific progress and dangerous for patient care. Clinicians and patients are already inundated with online medical information from various trustworthy and untrustworthy sources; adding preprints to this morass will simply add to the confusion.7 Furthermore, preprints lack the safeguards of prepublication peer review, are not presented within the curated context of a medical journal, result in duplicate publication of the same data (before and after peer review), and might harm patients if treatment is provided based on preliminary findings or superficial media coverage that may or may not turn out to be true.8

Peer review ensures trustworthy science

Prepublication peer review provides reassurance that data are presented clearly, accurately, and completely. Formal statistical review provides reassurance that the analysis is appropriate and that the conclusions are justified by the data presented. Authors often are overenthusiastic and overstate conclusions, extrapolate to different patient groups, or make unsupported clinical recommendations.

Many confuse an association between two variables with proof of a cause-effect relation. Even basic errors, such as transcription of incorrect numbers, are often present in papers initially. Medical journals minimise these problems by external peer review and in-house editing. Publishing preprints—that is, unreviewed and unedited research papers—could mislead clinicians and patients for all these reasons.

Medical journal publication adds value

Publication in a reputable journal also adds value for readers—clinicians, patients, and researchers. Journal content is curated so that readers know that the articles are relevant to their practice. The content is also trustworthy, and research advances are interpreted and contextualised. Preprints lack all these advantages.

Preprints are a danger to patients. Once information about a new treatment is online, patients will demand it, not realising that the data are preliminary. Premature adoption of treatments based on preprints will result in patients receiving therapies later found to be ineffective or harmful. Even worse, patients may not receive treatments known to be beneficial. Unless there is a formal way to link and reconcile differences between the two versions of the paper, readers may be unsure which one is correct. In addition, the same data published as a preprint and then later in a journal might be counted twice in systematic reviews and meta-analyses, corrupting the evidence base that supports clinical guidelines.9

Although safeguards have been proposed,10 preprints are unnecessary: there are better ways to exchange ideas and to speed up the publication process. We need more informed discussion of clinical research before publication at each academic institution, at scientific meetings, and online in preliminary form, but not formatted as a research paper. Preprints represent completed clinical research; ideally discussion would begin at the time of clinical study design.

It is naive to think that preprints will receive spontaneous thoughtful peer review from experts who are already overwhelmed. A journal with a dedicated editorial team can speed up publication without preprints by sending papers back to authors without delay if priority for publication is high, requesting and receiving peer reviews quickly, reducing the number of revision cycles to enough (but not too many), and publishing online promptly once the paper is accepted.

Journal “families” that share the peer review process further reduce delay by ensuring a journal suitable for the article is found quickly. Preprints do nothing to tackle the substantial delays in translating medical research into practice guidelines, and then into the room with the individual patient and doctor making a difficult clinical decision.

Let’s unclog the publication pipeline, not blow a hole in its side. Peer reviewed medical journals, particularly those affiliated with national and international professional societies, already provide rapid publication of trustworthy medical science and will continue to evolve in response to a changing research and publication environment. Preprints will have little effect on medical journals but do have the potential to harm patients and destroy public confidence in medical research.

Footnotes

  • Provenance and peer review: Commissioned; not externally peer reviewed.

  • Competing interests: All authors have read and understood BMJ policy on declaration of interests and declare the following interests: HMK and JSR are recipients of a research grant, through Yale, from Medtronic and Johnson & Johnson (Janssen) to develop methods of clinical trial data sharing, and were the recipients of a research grant, through Yale, from Medtronic and the FDA to develop methods for post-market surveillance of medical devices. HMK chairs a cardiac scientific advisory board for UnitedHealth; is a participant/participant representative of the IBM Watson Health Life Sciences Board; is a member of the advisory board for Element Science and the physician advisory board for Aetna; and is the founder of Hugo, a personal health information platform. JSR was a recipient of a research grant, through Yale, from the Blue Cross Blue Shield Association and receives research funding from the Laura and John Arnold Foundation. HMK and JSR are working with a team from BMJ and Cold Spring Harbor Laboratory to develop a “MedRxiv” preprint server for health sciences. CMO is editor in chief of Heart, published by BMJ.

References

View Abstract