Data transparency: “Nothing has changed since Tamiflu”BMJ 2020; 369 doi: https://doi.org/10.1136/bmj.m2279 (Published 09 June 2020) Cite this as: BMJ 2020;369:m2279
Two leading medical journals retracted research papers on covid-19 last week because the authors said they could “no longer vouch for the veracity of the primary data sources,” raising serious questions about data transparency and research integrity.12
The episode shows that “no lessons have been learnt since Tamiflu,” said Tom Jefferson, an epidemiologist for the Nordic Cochrane Centre. Jefferson, along with The BMJ, campaigned for years for companies to release the clinical data for two globally stockpiled anti-influenza drugs, oseltamivir (Tamiflu) and zanamivir (Relenza) (bmj.com/open-data).3
“History is repeating itself,” he told The BMJ. “We warned people back in 2009 about these very same issues of guest authorship, reporting bias, and lack of transparency.”
On 22 May the Lancet published an observational study indicating that hospital patients with covid-19 treated with hydroxychloroquine and chloroquine were at greater risk of dying and of ventricular arrhythmia than patients not given the drugs.4 The same authors published an article in the New England Journal of Medicine on 1 May that found that angiotensin converting enzyme (ACE) inhibitors and angiotensin receptor blockers were not associated with a higher risk of harm in patients with covid-19.5
Both studies used data from a healthcare analytics company called Surgisphere. After several concerns were raised with respect to the veracity of the data, the study authors announced an independent third party peer review.67 But Surgisphere refused to transfer the full dataset and associated information, saying it would violate confidentiality requirements and agreements with clients, leading the authors to request the retraction of both studies.
The Lancet and NEJM are signatories to the Wellcome agreement on data sharing for covid-19 studies, which calls for research findings to be openly accessible and to give clear statements regarding the availability of underlying data.8 But an open letter to the study authors and the Lancet’s editor, Richard Horton, pointed out that its paper had no statement on data and code sharing availability and no ethics review.9
Robert Kiley, head of open research at Wellcome, told The BMJ, “In the case of articles which contain private and sensitive information, the data availability statement should still indicate how the data could be accessed, typically by making a request to the appropriate data access committee. We encourage all publishers to require a data availability statement for all research articles and to make this a mandatory part of the submission process.”
So is the Wellcome statement worth the paper it’s written on? Henry Drysdale, a clinical researcher with the DataLab at Oxford University, whose academic work has focused on research integrity, believes it does have some merit as it articulates an idea of best practice and provides a standard to which medical journals can be held.
“However, the statement focuses entirely on the sharing and dissemination of research and does not address information governance or research integrity standards. In the context of major concerns over the integrity of reporting and use of results for high profile covid trials, commitments to these standards are urgently needed,” he told The BMJ.
He believes that the concerns over the Lancet trial are not so much about the editorial process or reporting but about generating and collating data, with some querying the truthfulness of the data. “It’s possible that data sharing standards have been, to a greater or lesser extent, compromised through drives to produce and disseminate covid research quickly (as encouraged by the Wellcome statement). However, it’s very difficult to say with so little information about the sources of data and data collection processes for these trials,” he said.
Deluge of research
The covid-19 pandemic has created an urgent need for scientific evidence to help politicians, doctors, researchers, and the general public understand the evolving situation and know what treatments work. This has resulted in a deluge of new research, much of it published without peer review on preprint servers. “There is a headlong rush to publish in an emergency, and that is toxic,” Jefferson said.
Tracey Brown, director of the charity Sense about Science, told The BMJ, “The urgency of sharing emerging research and data on the covid-19 crisis has created huge dilemmas over quality. Rapid publication and early sharing of results is clearly warranted, but it means that the research community must also double down on communicating the status and reliability of results and their limitations.
“Open data and research evidence is useless without this information. Researchers must avoid hype and police the discussion of the research assiduously, especially as individual papers are now being seized on to push different policy responses.”
She added, “But some of the issues we are seeing are not new. Putting your name to a data analysis when you have not seen the data is research fraud, and always has been, crisis or no crisis.”
Elizabeth Loder, The BMJ’s head of research, acknowledged that it was difficult for peer reviewers or journal editors to detect deliberate, carefully orchestrated fabrication of data.
“In the case of the Surgisphere database, there was a high level of interest in the NEJM and Lancet papers because of the pandemic. This led to rapid identification and speedy retraction of the articles and underscores the value of having many people involved in evaluating and inspecting research both before and after it is published,” she said.
Loder believes that The BMJ’s open peer review process and its commitment to sharing data and the posting of signed peer review reports alongside published research papers allow for a level of public scrutiny that is valuable.
She added, “We are considering a variety of steps we could take to prevent and detect potential fraud. In addition to solutions at the level of individual journals, however, I believe that cooperation among journals and public policy initiatives also may be needed.”
This article is made freely available for use in accordance with BMJ's website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.https://bmj.com/coronavirus/usage