How should clinical trial data be shared?

Full access models

Two recent initiatives, the Yale Open Data Access (YODA) project and Restoring Invisible and Abandoned Trials (RIAT), involve the full release of all patient level data.

In YODA’s approach to solving the problem of missing or misreported data, companies voluntarily provide all relevant product data, which are then subjected to independent review by two separate research groups and subsequently made publicly available.3

The first results from YODA, published in the Annals of Internal Medicine and the BMJ4 in mid-June, represented an unprecedented step by industry. Medtronic turned over the entirety of its data about its controversial product recombinant human bone morphogenetic protein 2 to Yale, with no strings attached.

Harlan Krumholz, who directs the YODA effort, told the BMJ: “The contract that Medtronic signed with us is astounding. They gave us complete authority and jurisdiction over the data, over who would do the reviews, what would be produced, and how we would release the data. If I wanted to put it on the web I could have.”

And, he noted, the fact that the findings of the two independent research groups differed in some respects underscores the importance of opening access to data for wider scrutiny. “What does that speak to the reproducibility? It says there’s wiggle room,” said Krumholz, the Harold H Hines Jr professor of medicine and professor of epidemiology and public health at Yale University School of Medicine.

The YODA group is now in discussions with other companies whose products are not necessarily controversial.

“Under the current rules, people who own data derive benefit from the exclusivity. But there is a compelling societal interest in making sure we get the fullest return on investment in research dollars. I say, for products being sold, that privilege comes with the responsibility to share everything you know about the risks and benefits,” Krumholz told the BMJ.

The RIAT initiative was announced in mid-June in the BMJ.5 Johns Hopkins University doctoral fellow Peter Doshi and colleagues called on all investigators of unpublished trials to publish them and for those who published papers containing errors or who misrepresented their findings to formally correct the errors and republish within a year, along with electronic data at the patient level.

If the original study sponsors do not step up to the plate, the RIAT proposal calls for the trial data—obtained via avenues such as litigation or Freedom of Information requests—to be considered “public access” for others to analyze and publish.

The current proprietary system, Doshi and colleagues say, is not only unethical but also leads to an incomplete and misleading body of medical literature from which systematic reviews and meta-analyses are based, and from which clinical decision making is derived.

“We live in a world that’s based on journals. So unless we make sure that journals are an accurate and complete reflection of the reality of what happened in trials, we remain at risk of drawing inappropriate and incorrect conclusions about the effects of healthcare interventions,” Doshi told the BMJ.

He hopes RIAT will set a precedent. “It may be that when people start to do this voluntarily, what emerges is a new standard in which research that is published along with datasets is generally treated as more reproducible, and therefore of higher integrity and quality.”

But Robert M Califf, director of the Duke Translational Medicine Institute, disagrees. He believes that access to patient level data should be restricted to individuals or groups with proven track records in conducting clinical trials and who can show that they understand how data should be interpreted.

Califf—whose institution was hired by GlaxoSmithKline to conduct the re-adjudication of the RECORD study of the controversial diabetes drug Avandia6—supports the model recently adopted by the company, in which researchers must submit an application, and if accepted, query the database that remains behind a firewall.7

“I really do feel strongly that people who are going to manipulate other people’s data that they worked years to collect and organize in a particular way ought to [have done] a few clinical trials . . . Analyzing complex datasets is not for rookies and amateurs,” Califf told the BMJ.

“I’m completely in favor of liberating data, but it needs to be done carefully and we have a lot to learn about the most responsible way to do it,” said Califf, who is also professor of medicine and vice chancellor for clinical and translational research at Duke.

Krumholz told the BMJ that although he supports the open data aspect of RIAT, he’s not in favor of the ultimatum that it sets. Rather, he prefers to work collaboratively with industry, as YODA aims to do. “To make the kind of progress we need, I’m looking for forward-looking companies that are willing to take leadership and I’m finding that there are some.”

The FDA’s role

The extent to which the US Food and Drug Administration (FDA) should release clinical trials data—for which it is the world’s largest repository—has been a focus of debate.

In early June, the FDA issued a request for comment8 regarding the release of “de-identified and masked data” derived from medical product applications—a far more cautious step than the European Medicine Agency’s recent proposal to proactively publish clinical trials data supporting the authorization of medicines.9

The FDA’s proposal, for which comments are being accepted until 5 August, would remove specific product and company information, releasing only class level information. “Release of commercial confidential and trade secret information is not being considered under this proposal,” FDA spokeswoman Patricia El-Hinnawy told the BMJ.

Krumholz sees the FDA’s shielding of the individual drugs’ identities as restrictive. “I don’t see how that will help. I think they’re just trying to tippy-toe into it . . . If you make a half hearted attempt that’s not authentic, it can slow progress because the pressure is relieved and yet the progress isn’t made.”

Geetter said that although she could not speak on what may have informed FDA’s specific approach, “In the US, there is a strong culture around intellectual property that contributes to the complexities and debate around data sharing.”

Just as with biospecimens such as genes, she said, there is legal uncertainty surrounding data as well. Current laws governing privacy and informed consent do not apply universally to data sharing, and can conflict in some cases.

“The lack of a unified, comprehensive legal framework can, in some cases, facilitate data sharing because some entities may have more flexibility to share data, but there may also be fewer protections in some cases,” said Geetter, a partner at the law firm McDermott Will and Emery in Washington, DC.

Referring to data and biospecimens together as “biomedical assets,” she added: “Given the centrality of biomedical assets to biomedical innovation, we can expect that legal and public stakeholders will continue to debate and refine the rules for how biomedical assets are accrued, used, and shared.”

“I would hypothesize that the FDA is moving slowly because it’s not moving into a well boundaried, well established framework . . . there is a lot of uncharted water.”

Nonetheless, El-Hinnawy told the BMJ that the FDA has been actively engaged in data sharing activities as part of its transparency initiative,10 launched in 2009. “The agency has focused over the past several years on advancing the regulatory science to improve medical product development and evaluation. This request for comments is intended to further this dialogue.”

The PhRMA view

The Pharmaceutical Research and Manufacturers of America (PhRMA) opposes the release of patient level data by its member organizations.

In a statement issued to the BMJ, PhRMA senior vice president Matt Bennett said: “Biopharmaceutical companies support responsible data sharing that protects patient privacy, maintains the integrity of the regulatory review process, and preserves incentives for biomedical research.”

He endorsed the work of multi-stakeholder groups such as the Institute of Medicine, which held a workshop on data sharing last autumn11 and is now designing a study aimed at informing new guidelines, and Harvard University’s Multi-Regional Clinical Trials Center, for which PhRMA is among the funders.12

Bennett also cited concerns that anonymized patient data could be re-identifed if released publicly, and that regulatory decisions could be “second-guessed, thereby undermining patient trust and confidence in the safety and effectiveness of approved medicines.”

Moreover, the “public release of companies’ clinical and pre-clinical raw data and reports as proposed in Europe will harm incentives to invest in biomedical research and likely delay European regulatory filings, while providing little or no incremental value to patients and healthcare professionals.”

“A step-wise approach”

One notable project that several PhRMA member companies help to fund is the soon-to-be launched Project DataSphere (www.projectdatasphere.org/), an initiative of the CEO Roundtable on Cancer (http://ceo-lsc.org/). It involves a platform for sharing and manipulating historical data from clinical trials on cancer, with the aim of pooling datasets to accelerate research.

“Our feeling all along has been that a platform across a single therapeutic area is the most valuable for research . . . You don’t want to have to go to 15 different databases to pull out information,” Charles Hugh-Jones, vice president of medical affairs in North America, Sanofi Oncology, told the BMJ.

In DataSphere, he said that there is a “fairly good degree of balance between the intellectual property [IP] protection from the data provider but also incentive for the user in that you can gain new IP from the data you’re using, provided it doesn’t impinge on the existing providers’ IP.”

Initially, the platform will include only control arm data, but more sensitive data will be added in subsequent iterations, Robin Jenkins, head of strategic planning of North American medical affairs, Sanofi Oncology, told the BMJ.

“One of the biggest hurdles we have to overcome is this siloed mentality and the culture around not sharing data . . . It’s a step-wise approach,” she said.

Identifying best practices

Of course, Califf pointed out, no amount of data sharing will be helpful if trials themselves are not well conducted and the data are not good enough quality to begin with.

Such was the case with the RECORD study, he said. “The real message from the Avandia hearing6 is that we’re 14 years out and no one on earth can tell us what the benefit/risk balance is because there wasn’t enough good evidence generated in the whole 14 years to know.”

Better models, he said, include that of the National Institutes of Health’s Pediatric Oncology Branch, in which nearly all oncology centers in the country participate and most children diagnosed with cancer are enrolled in clinical drug trials.13 In this model, “you don’t really have to combine the data, because it all resides in a national network,” he commented.

Other novel research models that hold promise, Califf said, include the I-SPY 2 TRIAL (Investigation of Serial Studies to Predict Your Therapeutic Response with Imaging And moLecular Analysis 2), which tests new drugs as add-ons to standard chemotherapy for breast cancer14; and Patients Like Me, in which patients self enroll and share data.15

Geetter noted that the process of containing costs in healthcare reform in the US lends urgency to the search for best practices in data sharing. She said: “One common theme is trying to unplug the power of data to address whatever fiscal constraints stakeholders are experiencing in light of health reform . . . The way our health insurance system works, data is king. I don’t think the US can solve the challenges it faces in the health industry without figuring out how to use data as a more effective tool.”