Difficulty accessing data from randomised trials of drugs for heart failure: a call for actionBMJ 2015; 351 doi: https://doi.org/10.1136/bmj.h5002 (Published 21 October 2015) Cite this as: BMJ 2015;351:h5002
- Robert Fleetcroft, clinical lecturer in general practice1,
- John Ford, NIHR doctoral research fellow1,
- Nicholas D Gollop, MRC doctoral research fellow in cardiology2,
- Pieter MacKeith, academic clinical fellow in primary care1,
- Kosala Perera, general practitioner3,
- Ashia A Shafi, specialist trainee, obstetrics and gynaecology4,
- Jhenaan Sorefan, medical student1,
- Caroline Thurlow, academic clinical fellow in primary care1,
- Rachel Wakelin, academic clinical fellow in primary care1,
- Martin R Cowie, professor of cardiology5,
- Nicholas Steel, reader in primary care1
- 1Norwich Medical School, Norfolk NR4 7TJ, UK
- 2Norfolk and Norwich University Hospital, Norfolk, UK
- 3JS Medical Practice, London N15 4JR, UK
- 4North Central and East London Deanery, London, UK
- 5National Heart and Lung Institute, Imperial College London, UK
- Correspondence to: R Fleetcroft
- Accepted 11 August 2015
Access to individual patient data from published research has the potential to improve patient outcomes by enabling researchers to answer new questions with existing data, to validate findings, and to combine the power from individual studies, for example in meta-analyses.1 It may also deter selective reporting and analysis, and research fraud. Difficulty accessing data from clinical trials means that only part of the evidence is available, which may lead to erroneous clinical decisions.
A survey of corresponding authors of clinical trials found that, although in principle they agreed with sharing their data, they also had concerns about appropriate data use, investigator or funder interests, and protection of trial participants.2 Sometimes data are not available because the trial was not published. For example, the Carcinoembryonic Antigen Second Look Trial in colorectal cancer was not published until over 20 years after completion and reported no benefit for the common use of carcinoembryonic antigen to detect and resect disseminated cancer.3 Barriers to accessing data have led to delays in determining the effectiveness or adverse effects of commonly used drugs such as oseltamivir, rofecoxib, and rosiglitazone.4 5 6
To make research data more widely available several journals and funders have published statements supporting data sharing for all published articles.1 7 8 9 10 The Association of Medical Research Charities (AMRC) in the United Kingdom, the European Medicines Agency, and the Institute of Medicine in the United States all recommend data sharing between researchers.11 12 13
Here, we describe our experiences trying to access patient data for a systematic review of drugs for heart failure—although the necessary studies had been published, we were unable to answer our review question owing to poor reporting and non-disclosure of data. We discuss possible solutions.
Unanswered questions in treating heart failure
Heart failure is a common condition, affecting 0.7% of the English population in 2013-14.14 Most patients with heart failure have only minor symptoms; one study has reported that 83% of patients with heart failure in primary care in the UK have mild symptoms, classified as class I (no dyspnoea) or class II (slight to moderate dyspnoea on exercise) on the New York Heart Association (NYHA) scale.15 However, the National Institute for Health and Care Excellence (NICE) guidelines for the management of cardiac failure in primary care rely heavily on evidence from trials of patients in secondary care, who tend to have more severe symptoms.16 Thus, drugs for cardiac failure may be less effective in patients in primary care.17
We conducted a systematic review to determine the effectiveness of heart failure drugs for patients with NYHA class I and II symptoms. We included randomised controlled trials of beta blockers, angiotensin converting enzyme inhibitors, or angiotensin receptor blockers. We used standard systematic review methodology. Two independent reviewers screened the titles and abstracts for results. A total of 30 studies met the inclusion criteria.
Searching for evidence
Twenty eight of the 30 studies reported NYHA class in baseline characteristics, but none included enough data to calculate outcomes for patients in different NYHA classes.18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 We contacted the corresponding authors (or the first authors where these details were absent) of each study by email on 28 July 2014, asking for additional summary outcome data stratified by NYHA class (fig 1⇓).
Twelve studies had no email contact details for the corresponding or first author, but an internet search found contact details for six of these. In total 24 authors were contacted. Six emails were returned with a “non-delivery report.” One of these authors (who was the author of two papers) had died, and three had left their institutions. A further internet search found alternative contact details for two of these authors, and data were again requested leaving four non-delivery reports that we were unable to follow up. We followed up requests with a second email on 13 September 2014 if there had been no reply to the first email (fig 2⇓; see appendix on thebmj.com for further details of the search results).
Inadequate responses to requests for data
We received six replies in total. Three authors said that data were not available; one said that only one class of heart failure patient had been included in their study; one author refused to supply data stating that “such subgroup analyses are not appropriate and likely to generate spurious findings;” and one author recommended that we approach a co-author for additional data, which we are now following.
Of our 30 requests we received only one response with the potential for further data to inform our research. We anticipated that the numbers of patients included in the 30 studies (80 450) would have been more than sufficient to provide an answer to our research question. The barriers we experienced were poor reporting of results in the original publications; inadequate or outdated contact information for the lead author; no response to requests for data; and one refusal to release data. Lack of contact details is partly understandable because 11 of the studies were published in the 1990s.
Difficulties accessing data
Our findings are surprising in the current climate of increasing access to information. Outcome data were usually presented for all classes of heart failure combined, leading to incomplete reporting of results. Many of the trials were published long ago—11 in the 1990s, 15 in the 2000s, and four published since 2010. One study was retracted in 2013.33 Email addresses for the corresponding author were missing for 12 studies, and a further six were invalid. One corresponding author had died, and three reported that the data were no longer available. We encountered one refusal owing to concerns about misleading secondary analysis of their data.
We were surprised by the extent of the difficulties we had, given that a cross sectional survey of corresponding authors on the sharing of their clinical trial data found that 77% had granted at least one data sharing request.2 Some authors may refuse to release data because they think that they have intellectual property rights.48 or that the data will be misinterpreted and cause harm.2 We acknowledge that intellectual property concerns may be potentially a big obstacle, but we do not think it valid to withold data when there is a public interest and where the results are expected to influence healthcare.
Ethical approval for future randomised controlled trials of drugs for heart failure in patients with mild symptoms may be difficult to obtain as guidelines have already been published.49 50 51 And they may be unnecessary because substantial numbers of these patients have already been included in these 30 published studies.
Plans to improve access to data from clinical trials are underway, and we agree with the suggestions from the AMRC and the Institute of Medicine. The AMRC calls for guidelines to govern the handling of personal health information and for standardised models for gathering and sharing data. It recommends that the UK government engages with Europe to ensure that updated data protection legislation supports secure access to patient data for research. However it does not make detailed recommendations on how data sharing might happen.
The Institute of Medicine recommends the sharing of full analysable datasets for clinical trials within 18 months of study completion.13 It says that funders and sponsors should develop a mechanism for data sharing in accordance with the terms and conditions of grants and contracts; that disease advocacy organisations should require data sharing plans as part of protocol reviews and criteria for funding grants; that regulatory and research bodies should work internationally to support the responsible sharing of clinical trial data; that research ethics committees should consider data sharing plans when assessing the benefits and risks of clinical trials; and that research institutions and universities should ensure data sharing.
The next stage is for mechanisms of data sharing to be developed. We need central national repositories for trial data based in the country of the trial sponsor, accessible to researchers after 18 months. Possible repositories in the UK include the UK data archive, which already provides access to research data from many observational studies, or the Health Research Authority, as recommended by the AMRC. This will require coordinated actions from funders, journals, ethics committees, and national guideline developing bodies.
Guideline developers such as NICE have an important role. We think it is unacceptable for NICE to make decisions about new drugs based on clinical effectiveness data that are not in the public domain. Such transparency is essential to bolster trust in the process of evaluation of new treatments and technologies. Research ethics and funding committees should make future access to the raw data a mandatory requirement of their approvals. Access to raw data is mentioned in section 52 of the UK Integrated Research Application System ethics application, but only in terms of ensuring the anonymity of patient identifiable data when publishing the results.
Funders could, for example, withhold the final part of grant funding until individual data are made publicly accessible, in the same way that some funders withhold final payment until they have receipt of a satisfactory final report. They should also provide adequate funding to cover the costs of archiving data and should insist that arrangements for archiving are discussed in the protocol. Journals could require evidence of archived data as a condition of publication of clinical trials, in the same way that they currently require evidence of ethical committee approval.
However, these recommendations will not tackle the problem of accessing historical data. We encountered difficulties identifying corresponding authors from older clinical trials, authors refusing to release data, and datasets that were no longer available. Others have had difficulties with older data formats, coding changes in computer software, and use of stored paper data.52 Authors may not agree on who owns the dataset and have concerns about intellectual property rights and other legal issues.53 This is contentious—it has been suggested that the data really belong to the patients, and the researcher is merely the custodian.54
Lack of informed consent can be a concern if data are not anonymous or where historical consent was used. The Health Research Authority standard consent form now includes consent for anonymous data sharing with other researchers. Considerable time and cost may be spent trying to access historical data, especially if they are in paper format. These barriers prevent an easy solution to accessing historical data, but progress can be made. Guideline committees could insist on data sharing agreements for the research they use, which would encourage authors of historical and important studies to share their data, but this is a complex issue. Guidelines should include all relevant data but the research quality of a particular guideline could be downgraded or given a “health warning” if those data are not shared, so that this information is transparent to the users of the guideline.
Although it would not be feasible for all data from historical trials to be placed in a repository, the Wellcome Trust has suggested the storage of historical data from priority research areas such as cardiovascular disease and cancer.55 Until data are openly shared and researchers can undertake secondary analyses, unnecessary trials will continue to take place and the evidence base supporting best clinical practice will be harder to identify.
The evidence base for the benefits of drugs for patients with mild heart failure is not clear, so we tried to perform a systematic review based on published trials
Access to requested data on the outcomes of treatment for patients with mild heart failure was not possible for 29 of the 30 studies we identified
Difficulties with access to patient data from published studies are well known and several organisations have recommended improvements
Mechanisms for data sharing need to be developed, such as central national repositories for trial data
Cite this as: BMJ 2015;351:h5002
Contributors and sources: RF conceived the original hypothesis for the systematic review, RF and JF designed the study and all authors commented on the design. JF oversaw the systematic review. Data were extracted by RF, NG, PM, KP, AS, JS, CT, and RW. RF, JF, and NG analysed the findings. RF wrote the first draft of the analysis and all authors contributed to revisions and approved the final draft. RF is the guarantor.
Competing interests: We have read and understood BMJ policy on declaration of interests and declare the following: MRC reports grants and personal fees from ResMed, grants and personal fees from Boston Scientific, personal fees from Novartis, personal fees from Sorin, personal fees from Servier, personal fees from Pfizer, grants and personal fees from Bayer, and personal fees from St Jude Medical, outside the submitted work. His salary is supported by the NIHR Cardiovascular Biomedical Research Unit at the Royal Brompton Hospital, London. The remaining authors have no relevant interests to declare.
Transparency declaration: RF affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
Data sharing statement: no additional data have been obtained beyond the published studies identified in the systematic review and cited in the references.