Data sharing: making good on promisesBMJ 2018; 360 doi: https://doi.org/10.1136/bmj.k710 (Published 15 February 2018) Cite this as: BMJ 2018;360:k710
Nothing damages trust in medical research more than the idea that clinical trial data are being hidden or manipulated. Randomised controlled trials are widely considered the best way to evaluate treatment benefits, but trials are time consuming, expensive, and complex. It is thus vital to make full use of all information collected, and it seems evident that the public interest is served by independent scrutiny and replication of trial results. These things can happen only if trial data are shared.
In 2013 The BMJ became one of the first journals to require data sharing in clinical trials of drugs or devices (doi:10.1136/bmj.e7888). In 2015 we extended that policy to require data sharing in all clinical trials (doi:10.1136/bmj.h2373). We require trialists to commit to sharing relevant data on reasonable request. We ask data requesters to submit a protocol for their reanalysis to the authors of the original trial and to post a rapid response to the original paper on bmj.com describing the aims of their reanalysis. We have committed to ask authors who refuse to supply data to explain their reasons.
Florian Naudet and colleagues (doi:10.1136/bmj.k400) have undertaken an important practical test of this policy. They find that many promises are only that: on request, just 46% of trial authors actually provided complete data with clear labelling. Better news is that the primary outcome was fully reproduced in 14 of the 17 trials on which data were provided. Small discrepancies were found in two studies that did not change conclusions, and one lacked sufficient detail in the methods section to carry out the analysis.
The authors acknowledge that some trialists may have refused to provide data because they did not judge the request reasonable, perhaps because the project lay outside the purpose for which the trial was done and participants gave consent. After all, data sharing is not a cost-free endeavour. The BMJ statistician who evaluated this paper runs a clinical trial unit; he estimated that it takes a week of work to prepare a trial dataset and dictionary for sharing.
Several parts of our policy were not tested in this study. The researchers did not post a rapid response on bmj.com describing their reasons for wanting the study data, and they did not appeal to The BMJ to intercede with authors who did not agree to share data. Pressure from The BMJ would possibly have persuaded more authors to provide data. Still, it is disappointing that we cannot rely on BMJ authors to make good on their promises.
“Trust is the crux of the matter,” says Milton Packer in a linked editorial (doi:10.1136/bmj.k510). In his view, researchers have “implicitly agreed to a social contract” that includes making raw data accessible. “It has always been delusional,” he suggests, “for researchers to imagine that clinicians and the public would believe their findings and accept their conclusions without access to supporting data.”
And yet that delusion apparently persists among some, if not most, researchers. Drawing on the lessons of this study, we are considering asking authors to deposit their data at the time of publication. We will also outline more clearly the procedures that data requesters should follow. Data sharing is an important part of ensuring trust in research, and it should be the norm. We pledge to do better, and we hope that trialists will too.