How covid-19 bolstered an already perverse publishing system
BMJ 2023; 380 doi: https://doi.org/10.1136/bmj.p689 (Published 28 March 2023) Cite this as: BMJ 2023;380:p689Eric Rubin had been in charge for only three months. The editor in chief of the New England Journal of Medicine (NEJM) had come into the role with no previous editorial experience but a career in infectious disease medicine. “Little did I expect my training would be so useful,” he says.
As soon as the news of a pneumonia-like outbreak in China reached NEJM’s editors, “we made some calls to figure out what was going on, soliciting manuscripts from colleagues in Wuhan and later in Italy and the US when the epidemic spread,” says Rubin. Unsolicited submissions swelled, spiking at about 200 covid related papers a day.
Editors felt the pressure of workloads, but also responsibility. Richard Horton, editor of the Lancet, told the New York Times, “We feel very much that we are publishing research that is literally, day by day, guiding the national and global response to this virus. If we make a mistake in judgment about what we publish, that could have a dangerous impact on the course of the pandemic.”1
This was the first global pandemic that the scientific publishing industry had ever faced—while journals existed, no organised industry did when the 1918 flu pandemic occurred—and the first in a new digital age of internet communication and publishing. An estimated 1.5 million articles were added to the global literature in 2020—the largest single year increase in history, says Vincent Larivière, who studies bibliometrics at the University of Montreal, Canada. This peaked in April 2020, when many countries were deep into lockdown or applying heavy restrictions.
Some saw it as an opportunity. There were promises of more open science and publishing: a number of journals and research institutions agreed to a data sharing pledge issued by the funder the Wellcome Trust on 31 January 2020 that intended to “ensure that research findings and data relevant to this outbreak are shared rapidly and openly to inform the public health response and help save lives.”2 But it also stoked an already, some say, twisted industry—one that thrives on competitiveness—to publish the first data or to have the greatest visibility and impact. This changed the ways that papers were produced and vetted, for good and bad.
Too much, too fast, too bad?
For the 885 health and medicine journals published by Elsevier, submissions increased by more than 60%, including almost a quarter of a million submissions in the first wave of covid-19.3 At BMJ Journals they rose by almost 20% in 2020 from the year before, as nearly 4400 submissions were made to The BMJ and BMJ Open alone within five months in 2020.4
By the end of 2022 the World Health Organization’s covid-19 database contained 742 202 items. To meet such demand editors were “all hands on deck” and were often redeployed from their usual roles to rapidly appraise coronavirus papers, says Theodora Bloom, executive editor of The BMJ—a move fuelled by “the sense that we aren’t on the clinical front line but we are on a critical front line.”
Medical journals halved their turnaround times in the first half of 2020.5 Despite the unknown nature of the virus and its science, editors took far less rather than more time over decisions, a February 2023 analysis of 339 000 papers has found.6
Naomi Lee, senior executive editor for research at the Lancet during the pandemic, recalls how the usually rare practice of “fast tracking” select papers was expanded so that “practically everyone and everything was accelerated with the goal of disseminating critical knowledge.” The PubMed database shows that the five most cited articles in the Lancet since 2020—most reporting early coronavirus data—were accepted within 14 days and published within 22 days of receipt.
Alarms were raised early on about the mix of sheer volume and unprecedented speed. Commentators decried a flood of junk in the literature: observational studies, opinion articles, and duplicated efforts as researchers rushed to capitalise on covid related funding calls.7 Rubin says that NEJM’s editors had to abandon their usual full checks and balances in the acute phase of covid article triage and were publishing things they normally wouldn’t, including case reports, small trials, and uncontrolled studies. Reinforced by his own “scary” experience of treating patients with covid in intensive care at Brigham & Women’s Hospital in Boston, USA, he says that in the context of a public health emergency, publishing “some knowledge was better than none.”
More threatening to people’s trust in journals were high profile retractions, notably those of papers in the Lancet and NEJM from May 2020 that reported early efficacy of hydroxychloroquine for covid-19, subsequently found to be fraudulent.89 Yet, against expectations, retractions have not risen to a level to match the huge increase in volume and publishing speed of covid papers. Retraction Watch, a website that tracks the phenomenon, has recorded 301 retracted or withdrawn covid papers to 8 February 2023.89 It estimates that just 0.07% of covid papers have been retracted—consistent with an expected overall rate of retraction.
Not-so-open science
Proponents of open science had breathlessly heralded a revolution.10 medRxiv, a BMJ affiliated preprint server, saw a 10-fold rise in submissions within two months of the first reported covid case. But this enthusiasm receded, and submissions at medRxiv and others stabilised by mid-2020.
Analysis shows that just 5% of all peer reviewed journal articles about covid-19 published in 2020 started out as preprints.11 And, while some pivotal trials such as Recovery and Solidarity were first reported as open access preprints, none of the phase 3 covid vaccine trials supported by Oxford-AstraZeneca, Moderna, or Pfizer was, and only the Oxford-AstraZeneca phase 3 trial report was published with a gold open access licence. A 2022 evaluation by Wellcome of the data sharing commitment it initiated found that fewer than half of signatories’ covid papers contained information about where and how to access available data,12 raising concern about a lack of transparency, particularly in clinical trials.1314
Progress towards more open research has also disappointed. While the leading publishers agreed to make their covid content open and reusable,2 Wellcome’s assessment found that just 46% of signatories’ covid papers were genuinely open access, where re-use is permitted and authors retain copyright.12
Instead, most journals retained commercial rights and simply took down a paywall (“bronze” open access15), says Larivière. He adds that, while major publishers including Elsevier, Springer Nature, and Wiley continue to make covid content freely available, only about half of papers on the climate crisis are similarly available. This is despite all being signatories (along with The BMJ) to the United Nations sustainability development publishers’ compact, which commits publishers to actively promote content that advocates for themes such as sustainability, justice and safeguarding, and strengthening the environment.
“Covidisation”
Concerns are being raised that the domination of covid related papers in medical journals came at the cost of other health issues such as non-communicable conditions, violence, and mental health.
John Ioannidis, professor of medicine at Stanford University, USA, has studied citation patterns during the covid pandemic and is worried about the effect on diversity in science. His analysis of the peer reviewed scientific literature in 2020-21 (to 1 August) showed that, while covid papers represented 4% of the scientific literature, they took 20% of citations to all papers published.16
The numbers rose when drilling down to general medical literature. Of those, 17% of all articles published in the period were on covid-19, hoarding 80% of the citations.16 In other words, covid papers in a general medical journal received about five times more citations on average than non-covid papers in the same period, and the vast majority of citations to such journals were from the covid papers.
This was a huge boon to journals and their editors, as journal impact factor scores more than doubled for 2021 on the basis of pandemic publishing in 2020: the Lancet rose from 79 to 202, NEJM from 91 to 176, the Journal of the American Medical Association from 56 to 157, and The BMJ from 40 to 96. These surges in publications, article views, social media visibility, and citations have brought about a new citation elite and health leadership that will shape future funding and institutional priorities.
Ross Upshur, pandemic governance expert at the University of Toronto, Canada, who also teaches research integrity, says that all this opportunism is nothing new. To him, what happened during the pandemic reflects an already perverse system of academic reward that has little motive to change: the gold rush to publish was simply an extension of the usual “publish or perish” culture. It’s therefore unsurprising that “people had to become a covid expert to survive, or at least a self-appointed expert.”
Ivan Oransky, cofounder of Retraction Watch, agrees that the system is mutually reinforcing because the top medical journals were aggressively engaged in “an arms race for attention, eyeballs, and citations.”
Elizabeth Gadd, a UK scholarly publishing expert, says, “The pandemic just highlighted the problems of publishing. It’s expensive, slow, and reinforces journal articles being the accounting unit of scholarship. It’s not about contributing to scholarly conversation. It’s scholarly fanfaring of results that doesn’t align with the mission of science. If all authors got was feedback from reviewers and readers there would be no race to publish. As it is, the situation is a quest for glory.”
Three years on, Rubin admits to having “covid fatigue” but insists that he has “no regrets” and that NEJM is eager to move on to publish in other areas. Elizabeth Loder, head of research at The BMJ, shares the desire to “broaden the topics we are covering now to inform our audience and also to overcome the neglect of other disorders, which is just one area of fallout from the intensity of covid-19 publishing.”
Upshur is pessimistic, however: he doesn’t see reform of the traditional publishing system happening until publication is less tied to evaluation. Loder accepts that journals have a weight of responsibility in the pandemic publishing rush, but she agrees that much of the bad behaviour in the eagerness to publish, especially in top journals, is down to incentives and a lack of oversight in academic promotion systems.
“There needs to be more of a reckoning for academic medical institutions,” she says. “These institutions unduly rewarded people for pivoting their research to covid, not matching quality to volume, and ultimately contributed to harm during the pandemic.”
Footnotes
Competing interests: Jocalyn Clark is international editor of The BMJ and was executive editor at the Lancet during covid-19. She is a 2023 visiting researcher at the Brocher Foundation for a project on pandemic publishing.
Commissioning and peer review: Commissioned; not externally peer reviewed.
This article is made freely available for personal use in accordance with BMJ's website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.
https://bmj.com/coronavirus/usage