FDA to begin releasing clinical study reports in pilot programmeBMJ 2018; 360 doi: https://doi.org/10.1136/bmj.k294 (Published 23 January 2018) Cite this as: BMJ 2018;360:k294
All rapid responses
Mathew Herder and others’ (Herder Letter) January 23, 2019 Rapid Response in the British Medical Journal (BMJ) supporting the US Food and Drug Administration’s (US FDA) pilot project advocates that releasing new drug Clinical Study Reports (CSRs) will allow members of the research community:
“(1) to learn more about safety and efficacy evidence supporting FDA’s approval decisions;
(2) to include additional data from CSRs in systematic reviews along with other regulatory data;
(3) to conduct methodological research to better characterize how to use CSRs in evidence synthesis;
(4) to ensure the reports of trials in the biomedical literature (the building blocks for the majority of systematic reviews and clinical practice guidelines) are reported fully and accurately, with mechanisms available for public access to underlying data.” (1)
The Herder Letter relates that mandatory disclosures of CSRs are already in place in Europe and is being implemented in Canada. CSRs are contained in International Council for Harmonisation (ICH) harmonized applications and it would appear that access is already available to CSRs for drugs approved in both Europe and the US.
The Herder authors maintain that CSRs have an enormous potential to improve public health and cite a September 2015 reanalysis of Study 329 concluding, “… no advantage of paroxetine or imipramine over placebo in adolescents with symptoms of depression on any of the prespecified variables.” In contrast the
July 2001 Study 329 found: “Paroxetine is generally well tolerated and effective for major depression in adolescents”. (2,3)
The reanalysis of Study 329 actually provided a strong argument that patients and prescribers should be directed to free publicly available US FDA approved professional product labels and agency detailed analyses of data submitted by manufacturers in support of new drug approvals. The information that paroxetine had not been shown to be safe and effective in pediatric patients was made publicly available on the US FDA website in 1998, three years before the publication of Study 329 and 17 years before the publication of the Study 329 reanalysis. Additionally, a black box warning highlighting suicidality and pediatric use has been required in the US FDA professional product labels for paroxetine for at least a decade before the release of the reanalysis to Study 329.(4)
US FDA detailed analyses of new drug marketing submissions and other publicly available agency documents may be more useful than manufacturers unabridged applications to alert patients and prescribers about safety and efficacy issues in a timely manner.
CSRs should be made publicly available but who owns the data and can the US FDA legally release this information rather than their detailed public analyses of data submitted by manufactures?
A key question that the Herder Letter authors should address is: What has been the impact of the Study 329 reanalysis, and CSRs, on the numbers of paroxetine prescriptions written for pediatric and adolescent patients?
2. Le Noury J, Nardo JM, Healy D, et al. Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. BMJ 2015; 351:h4320.
3. Keller MB, Ryan ND, Strober M, et al. Efficacy of paroxetine in the treatment of adolescent major depression: a randomized, controlled trial. J Am Acad Child Adolesc Psychiatry 2001;40:762-72.
4. Sasich LD. Paroxetine and Study 329: what we already knew and when. BMJ.com/content/351/bmj.h4320/rr-0.(Letter)
Competing interests: No competing interests
An Open Letter in Support of FDA’s Clinical Study Report Pilot Project
In January 2018 the US Food and Drug Administration (FDA) launched a pilot project designed to enhance transparency around drug approval and assess the value of publicly disclosing Clinical Study Reports (CSRs). CSRs contain company-generated summaries of clinical trial methods and results, including more detailed information than the published literature. In March, one CSR underpinning the approval of a new drug was made available by the FDA with the consent of the sponsor, and posted on the FDA’s website. At the time of the pilot’s launch, up to eight additional CSRs were expected to follow.
The pilot project is an important step forward by the FDA. It marked a renewal of the FDA’s longstanding authority to disclose regulatory data that is generally not publicly available. CSRs are, in fact, unabridged reports of clinical studies of pharmaceutical interventions. The reports exist for all clinical studies sponsors undertake (irrespective of whether such studies are eventually published in the biomedical literature) and are submitted to regulators as part of the licensing application. While FDA is furnished with additional data (such as electronic, participant-level datasets) and does not rely solely on CSRs to arrive at its decisions, disclosure of CSRs can help better explain the basis for FDA’s decisions, and in turn, inform prescribers and patients about the evidence underlying a given drug.
Further, CSRs have enormous potential to improve public health by identifying unpublished, underreported, and misreported trials in the published medical literature. Consider the case of paroxetine (Paxil). On the strength of a 2001 article reporting the results of “study 329,” paroxetine was widely prescribed for treatment of depression, including amongst teenagers. In 2002, however, an FDA reviewer (with access to the study 329 CSR) reported that the trial had shown the drug was not more effective than placebo—a report that was buried, even though publicly available, on the FDA’s website. It was not until 2015, when independent researchers finally gained access to the full CSR, that the 2001 journal article results were effectively called into question through the republication of the trial. This is one of several cases in which the disclosure of CSRs can amplify important findings made by the FDA to the benefit of public health.
Despite the value of disclosing CSRs, a year after its launch the pilot project appears to have stalled. No additional CSRs have been posted on FDA’s website.
We, the undersigned, therefore write to express our support for FDA’s CSR Pilot Project. We urge the FDA to not only see the Pilot Project through to completion, but to make public disclosure of CSRs mandatory. The fact that only one company has volunteered to participate in the Pilot Project to date indicates that a policy of mandatory disclosure—a policy that is already in place in Europe and is being implemented in Canada—is necessary to improve transparency of CSRs submitted to the FDA.
If FDA were to publicly disclose all CSRs following a decision to approve a new drug or biologic or a new indication for an existing drug or biologic, we will—as members of a research community dedicated to evidence-based medicine—use CSRs in the following ways for the purpose of improving public health: (1) to learn more about safety and efficacy evidence supporting FDA’s approval decisions; (2) to include additional data from CSRs in systematic reviews along with other regulatory data; (3) to conduct methodological research to better characterize how to use CSRs in evidence synthesis; (4) to ensure the reports of trials in the biomedical literature (the building blocks for the majority of systematic reviews and clinical practice guidelines) are reported fully and accurately, with mechanisms available for public access to underlying data.
Competing interests: Matthew Herder: Matthew Herder has received grants from the Canadian Institutes of Health Research (2012-2019, 2018-2022) and he is a member of the Patented Medicine Prices Review Board (PMPRB), Canada’s national drug price regulator. He receives honoraria for his work as a member of the PMPRB. Peter Doshi: Peter Doshi has received travel funds from the European Respiratory Society (2012) and Uppsala Monitoring Center (2018); grants from the Laura and John Arnold Foundation (2017-20), American Association of Colleges of Pharmacy (2015), Patient-Centered Outcomes Research Institute (2014-16), Cochrane Methods Innovations Fund (2016-18), and UK National Institute for Health Research (2011-14); and is an editor at The BMJ and unpaid member of the Reagan-Udall Foundation for the FDA. Joshua D. Wallach: In the past 36 months, Joshua Wallach has received research support through the Meta Research Innovation Center at Stanford (METRICS) and the Collaboration for Research Integrity and Transparency from the Laura and John Arnold Foundation. Janice E. Graham: No competing interests to declare. Joel Lexchin: In 2015-2018, Joel Lexchin was a paid consultant on three projects: one looking at indication-based prescribing (United States Agency for Healthcare Research and Quality), a second to develop principles for conservative diagnosis (Gordon and Betty Moore Foundation) and a third deciding what drugs should be provided free of charge by general practitioners (Government of Canada, Ontario Supporting Patient Oriented Research Support Unit and the St Michael’s Hospital Foundation). He also received payment for being on a panel that discussed a pharmacare plan for Canada (Canadian Institute, a for-profit organization), a panel at the American Diabetes Association, for a talk at the Toronto Reference Library and for writing a brief for a law firm. He is currently a member of research groups that are receiving money from the Canadian Institutes of Health Research and the Australian National Health and Medical Research Council. He is member of the Foundation Board of Health Action International and the Board of Canadian Doctors for Medicare. Trudo Lemmens: Has received research support from the Canadian Institutes of Health Research (co-PI), the Office of the Canadian Federal Privacy Commissioner (PI), the Canadian Social Sciences and Humanities Research Council, Genome Canada, and the Stem Cell Network for research related to pharmaceutical regulation including clinical trial transparency. Has supported litigation against Health Canada in relation to sharing of regulatory data. Received travel funds to present on the regulation of research ethics committees at a conference organized by La Asociación de Médicos Especializados en la Industria Farmacéutica (AMEIFAC) (a Mexican Rx&D organization) in conjunction with the National Autonomous University of Mexico. Speaker fee was donated directly to a non-governmental organization. Joseph S. Ross: In the past 36 months, Dr. Ross has received research support through Yale University from Medtronic, Inc. and the Food and Drug Administration (FDA) to develop methods for postmarket surveillance of medical devices (U01FD004585), from the Centers of Medicare and Medicaid Services (CMS) to develop and maintain performance measures that are used for public reporting (HHSM-500-2013-13018I), and from the Blue Cross Blue Shield Association to better understand medical technology evaluation, and he currently receives research support through Yale University from Johnson and Johnson to develop methods of clinical trial data sharing, from the Food and Drug Administration to establish Yale-Mayo Clinic Center for Excellence in Regulatory Science and Innovation (CERSI) program (U01FD005938), from the Agency for Healthcare Research and Quality (R01HS022882), from the National Heart, Lung and Blood Institute of the National Institutes of Health (NIH) (R01HS025164), and from the Laura and John Arnold Foundation to establish the Good Pharma Scorecard at Bioethics International and to establish the Collaboration for Research Integrity and Transparency (CRIT) at Yale. Gregg Gonsalves: Dr. Gonsalves is an Assistant Professor at Yale School of Public Health. He has received support from the Laura and John Arnold Foundation to establish the Collaboration for Research Integrity and Transparency (CRIT) at Yale. Nav Persaud: Dr Persaud has received research funding from the Canadians Institutes of Health Research, Health Canada, Ontario Supporting Patient Oriented Research Support Unit and the St Michael’s Hospital Foundation. Peter Lurie: Dr. Lurie is President of the Center for Science in the Public Interest and a former Associate FDA Commissioner. He has no competing interests to declare. Huseyin Naci: No competing interests to declare. Mark Jones: was a co-investigator on a UK National Institute for Health Research grant for a Cochrane review of neuraminidase inhibitors for influenza; was a co-recipient of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane reviews; and is a paid consultant on a John and Laura Arnold Foundation grant for development of a RIAT Support Center (2017-2020). Tom Jefferson: TJ was a recipient of a UK National Institute for Health Research grant for a Cochrane review of neuraminidase inhibitors for influenza. In addition, TJ receives royalties from his books published by Il Pensiero Scientifico Editore, Rome and Blackwells. TJ is occasionally interviewed by market research companies about phase I or II pharmaceutical products. In 2011-13, TJ acted as an expert witness in litigation related to the antiviral oseltamivir, in two litigation cases on potential vaccine-related damage (including the vaccine Pandemrix (2015-2017) and in a labour case on influenza vaccines in healthcare workers in Canada. He has acted as a consultant for Roche (1997-99), GSK (2001-2), Sanofi-Synthelabo (2003), and IMS Health (2013).In 2014 he was retained as a scientific adviser to a legal team acting on oseltamivir. TJ has a potential financial conflict of interest in the drug oseltamivir. In 2014-16, TJ was a member of three advisory boards for Boerhinger Ingelheim. TJ was holder of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane reviews. TJ was a member of an independent data monitoring committee for a Sanofi Pasteur clinical trial on an influenza vaccine. Between 1994 and 2013, TJ was the coordinator of the Cochrane Vaccines Field. TJ was a co-signatory of the Nordic Cochrane Centre Complaint to the European Medicines Agency (EMA) over maladministration at the EMA in relation to the investigation of alleged harms of HPV vaccines and consequent complaints to the European Ombudsman. TJ is co-holder of a John and Laura Arnold Foundation grant for development of a RIAT support centre (2017-2020) and Jean Monnet Network Grant, 2017-2020 for The Jean Monnet Health Law and Policy Network. TJ is an unpaid collaborator to the project Beyond Transparency in Pharmaceutical Research and Regulation led by Dalhousie University and funded by the Canadian Institutes of Health Research (2018-2022). G. Caleb Alexander: Dr. Alexander is Chair of FDA’s Peripheral and Central Nervous System Advisory Committee; serves as a paid advisor to IQVIA; is a consultant and holds equity in Monument Analytics, a health care consultancy whose clients include the life sciences industry as well as plaintiffs in opioid litigation; and is a member of OptumRx’s National P&T Committee. This arrangement has been reviewed and approved by Johns Hopkins University in accordance with its conflict of interest policies. Erick H. Turner: During the last 36 months, Dr Turner has served as a member of the FDA Psychotropic Drugs Advisory Committee (Oct 2017-present). He is a member of the Board of Directors for the Alliance for Benzodiazepine Best Practices (pro bono work). He was US partner for OpenTrialsFDA, whose team was awarded $80,000 as runner-up for NIH Open Science Prize. In 2016, he consulted (2 x 1 hr) to venture capital firms on the efficacy of a drug in development based on data the sponsor had presented at a scientific conference. He received protected time for research as a co-investigator in AHRQ’s Scientific Resource Center. From 1998-2001, he was employed as a Medical Officer for the Food and Drug Administration. Evan Mayo-Wilson receives support from the U.S. Food and Drug Administration (FDA), National Institutes of Health (NIH), Laura and John Arnold Foundation, and Robert Wood Johnson Foundation. He previously received support from the Patient Centered Outcomes Research Institute (PCORI) and from a fund at Johns Hopkins Bloomberg School of Public Health established by Kay Dickersin using a donation from Greene LLP, the plaintiffs' lawyers in litigation against Pfizer in which Kay Dickersin served as an unpaid expert witness for the plaintiffs. Jennifer E. Miller has received a grant from the Laura and John Arnold Foundation (2015-19) to support the Good Pharma Scorecard and is a co-founder of the nonprofit Bioethics International (2005-present). Lars G. Hemkens: In the past 36 months, Dr. Hemkens received funding for research projects related to approval evidence from the Swiss Cancer League and the Mach-Gaensselen Stiftung, Switzerland. David Healy has been an expert witness on behalf of plaintiffs in treatment induced injury cases. Jon Jureidini has provided expert reports where actions have been taken for misrepresentation of outcomes of RCTs. Tianjing Li received grant support from the Food and Drug Administration through the Johns Hopkins Center of Excellence in Regulatory Science and Innovation. Grant number U01FD005942 (PI: Caleb Alexander). Kay Dickersin: Kay Dickersin (emeritus professor) is retired from Johns Hopkins Bloomberg School for Public Health. She currently is principal investigator on one conference grant from the Agency for Healthcare Research and Quality for Consumers United for Evidence-based Healthcare (5R13HS024461-03). Aaron S. Kesselheim: Dr. Kesselheim’s research is funded by the Laura and John Arnold Foundation, Harvard-MIT Center for Regulatory Science, and the Engelberg Foundation. Till Bruckner: No competing interests to declare. Tim Reed: No competing interests to declare. Courtney Davis: No competing interests to declare.
It has been six months since FDA announced its pilot program to release clinical study reports. But just one clinical study report for one drug has been released. The pilot program—voluntary for sponsors and limited to clinical study reports of key “pivotal” trials—is supposed to include “up to nine recently submitted new drug applications”.[1,2] At this rate, the pilot will take 4.5 years to complete.
In contrast to the FDA, the European Medicines Agency has released millions of pages of clinical study reports since late 2010 and now publishes these and other clinical data on its website for public consumption. The Canadian regulator has published draft regulations proposing to do something similar, and last week a Canadian federal judge ordered Health Canada to immediately release clinical trial data to me, without requiring a confidentiality agreement, possibly paving the way for others.
FDA says it is “committed to improving efficiency and transparency in the drug approval process.” Already faster than its peers at approving drugs, it’s not clear that efficiency is a problem. It’s the agency’s transparency on clinical trial data that needs an efficiency boost.
 Doshi P. FDA to begin releasing clinical study reports in pilot programme. BMJ 2018;360:k294 https://www.bmj.com/content/360/bmj.k294.full
 Iacobucci G. Canadian government ordered to release unpublished Tamiflu data in landmark ruling. BMJ 2018;362:k3112 https://www.bmj.com/content/362/bmj.k3112
 Downing N, Zhang A, Ross J. Regulatory Review of New Therapeutic Agents — FDA versus EMA, 2011–2015. N Engl J Med 2017; 376:1386-1387 https://www.nejm.org/doi/full/10.1056/NEJMc1700103
Competing interests: In addition to those printed in original article (https://doi.org/10.1136/bmj.k294), I sued the Canadian government - and won - in a case regarding access to clinical trial data, as described in this article and reference 5. I am also an associate editor at The BMJ.
On 19 March, the FDA announced it had published its first clinical study report (CSR) to its website under its new pilot programme. Viewers can access the CSR body, protocol, and statistical analysis plan of the pivotal trial of the drug, Erleada (apalutamide), through the FDA's Clinical Data Summary Pilot Program webpage  or by looking up the drug's "approval package" (medical officer review, statistical officer review, etc.) directly on FDA's Drugs@FDA database . Other sections of the CSR, such as blank case report forms, certificates of analysis, or informed consent forms, are not available as they are outside the scope of FDA’s pilot.[1,4]
True to the FDA commissioner’s remarks in January, the drug's labeling and approval review memos list the trial's ClinicalTrials.gov identifier (i.e. NCT number), NCT01946204, allowing for better cross referencing between sources.
While the CSR body was otherwise not publicly available prior to publication on FDA’s website, the protocol and statistical analysis plan were both already available on the website of the New England Journal of Medicine (NEJM), which published the trial. According to the NEJM publication, the trial ran from 14 October 2013 to 15 December 2016. The FDA-released CSR body is dated 25 September 2017, the protocol (amendment 8) 15 March 2017, and statistical analysis plan (v.7.0) 26 June 2017. Therefore FDA did not make available any a priori versions (i.e. versions before the study commenced) of the protocol or statistical analysis plan, however both documents do include an amendment history, with reasons given for the amendments. In contrast, a priori versions of the protocol and statistical analysis plan (dated 5 November 2012) were published as supplemental material in the NEJM.
 Woodcock J. FDA’s New Pilot Program Aims for More Transparency about New Drug Approvals. March 19, 2018. https://blogs.fda.gov/fdavoice/index.php/2018/03/fdas-new-pilot-program-...
 Food and Drug Administration. Clinical data summary pilot program. 2018. http://www.fda.gov/Drugs/DevelopmentApprovalProcess/ucm589210.htm
 Doshi P. FDA to begin releasing clinical study reports in pilot programme. BMJ. 2018 Jan 23;360:k294. doi: 10.1136/bmj.k294.
 Smith MR, Saad F, Chowdhury S, Oudard S, Hadaschik BA, Graff JN, Olmos D, Mainwaring PN, Lee JY, Uemura H, Lopez-Gitlitz A, Trudel GC, Espina BM, Shu Y, Park YC, Rackoff WR, Yu MK, Small EJ; SPARTAN Investigators. Apalutamide Treatment and Metastasis-free Survival in Prostate Cancer. N Engl J Med. 2018 Feb 8. doi: 10.1056/NEJMoa1715546
Competing interests: PD: no change from those printed in original article (https://doi.org/10.1136/bmj.k294); RO: no competing interests; KH: no competing interests; OS: Received the Maryland CERSI Scholar award from the Food and Drug Administration (grant #1U01FD005946).