Postmarket studies required by the US Food and Drug Administration for new drugs and biologics approved between 2009 and 2012: cross sectional analysisBMJ 2018; 361 doi: https://doi.org/10.1136/bmj.k2031 (Published 24 May 2018) Cite this as: BMJ 2018;361:k2031
- Joshua D Wallach, postdoctoral research fellow1 2,
- Alexander C Egilman, research associate1 2,
- Sanket S Dhruva, postdoctoral research fellow3 4,
- Margaret E McCarthy, executive director2,
- Jennifer E Miller, assistant professor5,
- Steven Woloshin, professor of medicine6,
- Lisa M Schwartz, professor of medicine6,
- Joseph S Ross, associate professor of medicine and public health1 3 7 8
- 1Center for Outcomes Research and Evaluation (CORE), Yale-New Haven Hospital, New Haven CT 06510, USA
- 2Collaboration for Research Integrity and Transparency (CRIT), Yale Law School, New Haven CT, USA
- 3National Clinician Scholars Program, Department of Internal Medicine, Yale School of Medicine, New Haven CT, USA
- 4Veterans Affairs Connecticut Healthcare System, New Haven CT, USA
- 5Division of Medical Ethics, Department of Population Health, NYU School of Medicine and Bioethics International, New York NY, USA
- 6Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon NH, USA
- 7Section of General Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven CT, USA
- 8Department of Health Policy and Management, Yale School of Public Health, New Haven CT, USA
- Correspondence to: J D Wallach @JoshuaDWallach on Twitter) (or
- Accepted 16 April 2018
Objectives To characterize postmarketing requirements for new drugs and biologics approved by the US Food and Drug Administration (FDA), and to examine rates and timeliness of registration, results reporting, and publication of required prospective cohort studies, registries, and clinical trials.
Design Cross sectional analysis.
Setting Postmarketing requirements for all new drugs and biologics approved by the FDA between 1 January 2009 and 31 December 2012, with follow-up up to 15 November 2017.
Main outcome measures Postmarketing requirements and their characteristics known at the time of FDA approval, including FDA authority, study design, and study characteristics. Rates and timeliness of registration and results reporting on ClinicalTrials.gov and publication in peer reviewed journals of required prospective cohort studies, registries, and clinical trials.
Results Between 2009 and 12, the FDA approved 97 new drugs and biologics for 106 indications with at least one postmarketing requirement at the time of first approval, for a total of 437 postmarketing requirements. Postmarket study descriptions were short (median word count 44 (interquartile range 29-71)) and often lacked information to determine an up to date progress (131 (30%)). 220 (50.3%) postmarketing requirements were for new animal or other studies (including pharmacokinetic studies); 134 (30.7%) were for prospective cohort studies, registries, and clinical trials; and 83 (19.0%) were for secondary analyses or follow-up studies. Of 110 clinical trials, 38 (34.5%), 44 (40.0%), 62 (56.4%), 66 (60.0%), and 98 (89.1%) did not report enough information to establish use of randomization, comparator type, allocation, outcome, and number of patients to be enrolled, respectively. Of 134 required prospective cohort studies, registries, and clinical trials, 102 (76.1%) were registered on ClinicalTrials.gov; of 50 registered and completed studies, 36 (72.0%) had reported results on ClinicalTrials.gov. Among 65 completed studies, 47 (72.3%) had either reported results or were published a median of 47 months (interquartile range 32-67) after FDA approval. 32 (68.1%) of these 47 studies did not report results publicly by the time of their original FDA report submission deadline.
Conclusions Postmarketing requirements for new drugs and biologics were often briefly described and did not contain enough information to characterize study designs. Approximately three quarters of postmarketing requirements for prospective cohort studies, registries, and clinical trials were registered on ClinicalTrials.gov, and nearly three quarters of completed studies reported results or were published, suggesting that at least a quarter of these required studies are not being publicly disseminated.
In the United States, the Food and Drug Administration (FDA) requires all new drugs and biologics to undergo clinical testing to demonstrate that they are safe and effective. However, over the past decade, the FDA has increasingly approved new drugs and biologics on the basis of shorter, smaller, and fewer trials.1 This shift corresponds with the FDA’s adoption of a lifecycle evaluation process, which emphasizes the importance of continued evaluation and monitoring of safety and effectiveness in the postmarket period.2345 Reflecting this emphasis, the FDA can use four separate authorities to require drug sponsors to conduct studies after approval to answer important questions about the benefits, harms, and optimal uses of new drugs and biologics (that is, postmarketing requirements, table 1).78
Postmarket studies required by the FDA can have important public health implications. Their findings can provide new evidence on the safety and efficacy of approved drugs and biologics, which can lead to regulatory actions and help guide decisions made by payers, physicians, and patients.9 However, over the past few years, there have been growing concerns about the fulfilment of postmarketing requirements.10111213 For instance, an analysis of all new drugs and biologics granted accelerated approval between 2009 and 2013 found that at a minimum of three years of follow-up, only half of the required confirmatory studies were completed.12 However, postmarket studies required under the accelerated approval pathway represent less than 4% of all postmarketing requirements issued by the FDA between 2008 and 2014.9
Furthermore, it is not sufficient for postmarket studies to be completed; successful translation of clinical trial evidence into practice requires timely dissemination of their results. In 2007, the US FDA Amendments Act (FDAAA) was enacted, which mandated registration and results reporting on a publicly accessible clinical trial registry established by the National Institutes of Health, ClinicalTrials.gov, for all ongoing and forthcoming “applicable clinical trials” of FDA regulated products.1415 According to a recent internal evaluation by the FDA, over one third of fulfilled postmarket interventional clinical trials and other trials were not published in either the scientific literature or on the ClinicalTrials.gov website.16 However, the authors were able to rely on internal agency information, as opposed to information that is available to the public, and did not examine the proportion of all clinical study postmarketing requirements that were fulfilled, the rigor of the evidence generated, or the timeliness of results reporting.
With the increasing reliance on FDA postmarketing requirements for new drugs and biologics as part of lifecycle evaluation efforts, we sought to characterize these requirements for all new drugs and biologics approved between 2009 and 2012. We focused on the different types of required studies under the four separate FDA authorities; the status of these postmarket studies using publicly available data sources; and the study characteristics and rates and timeliness of registration and results reporting on ClinicalTrials.gov and publication in peer review journals of postmarketing requirements for prospective cohort studies, registries, and clinical trials.
The methods were specified in advanced and were documented in a study protocol (supplementary appendix protocol 1).
Study design and sample
We used the publicly available Drugs@FDA database to identify and categorize all new drug and biologic licensing applications for drugs and biologics first approved between 1 January 2009 and 31 December 2012, excluding generic drugs, reformulations, and combination treatments of non-novel therapeutic agents, using a previously described approach.117 We selected 2012 as a cutoff date to allow for at least four years for completion and publication of required postmarket studies. New drugs and biologics were classified by orphan status, by use of a previously described approach.1
FDA approval letters were used to determine the first-approved indication for each new drug and biologic and FDA priority review status (that is, review required to be completed within six instead of 10 months).18 The fast track designation, which provides enhanced communication with the FDA during the development process, and the breakthrough therapy designation, which was not implemented until 2014, were not assessed in this study.1 We used the World Health Organization’s anatomic therapeutic classification system to categorize each indication.19 Indications were then grouped into one of six treatment areas: cancer; infectious disease; cardiovascular disease and diabetes mellitus; autoimmune, musculoskeletal, and dermatology; neurology and psychiatry; and other.19
Identifying postmarketing requirements and postmarketing requirements features
One reviewer (JDW) identified all postmarket studies that the FDA required (that is, postmarketing requirements) from the approval letters hyperlinked in the Drugs@FDA database. These letters include a brief description of the study type and regulatory authority (accelerated approval, Pediatric Research Equity Act (PREA), animal efficacy rule, or the FDAAA (table 1).6 We also recorded the dates that the FDA sets for important milestones: final protocol submission, trial completion, and final report submission, when available. We then classified each postmarketing requirement into six study categories (box 1).
Postmarketing requirement categorization
New prospective cohort studies, registries, and clinical trials*
Postmarketing requirements that outline new randomized controlled trials or other clinical trials evaluating safety and efficacy; prospective cohort studies and registries
Complete or submit results from ongoing prospective cohort studies, registries, and clinical trials
Instead of requesting a new prospective study or trial, these postmarketing requirements call for the completion and submission of the results from ongoing prospective cohort studies or trials
New retrospective observational studies
Postmarketing requirements that outline new case-control, cross sectional, and retrospective cohort studies; analyses of spontaneous adverse event reporting data
New animal or “other” studies required
Postmarketing requirements that outline new animal trials; pharmacokinetic or pharmacodynamics trials; in vitro or in vivo, drug transport, drug-drug or drug-therapeutic, prenatal and postnatal development, antidrug antibody response, mass balance, dosing, lactation, or QT/QTc studies
Analyze/follow-up from observational studies, registries, or clinical trials (and other flexible requirements)†
Postmarketing requirements that outline longer follow-up or new analyses of data from existing trials or studies; submission of a final report for ongoing case-control, cross sectional, or retrospective cohort studies; studies or trials that can be done as expansions of the previous observational studies; and postmarketing requirements that require the enrollment of additional patients in an existing registry
Analyze/follow-up from an existing animal or “other” studies (and other flexible requirements)†
Instead of requesting a new animal or “other” study, these postmarketing requirements call for the submission of a final report for an ongoing “other” or animal study; planned “other” studies that have already been outlined or proposed
*Generally includes “controlled clinical investigation(s), other than phase I clinical investigation, of a drug subject to section 505 of the Federal Food, Drug, and Cosmetic Act or section 351 of this Act.”14 Under section 801 of the FDA Amendments Act, only applicable clinical trials are required to submit information to ClinicalTrials.gov. Trials that must be registered “either were initiated after September 27, 2007, or initiated on or before that date and were still ongoing as of December 26, 2007,” and meet one of three conditions: have one or more sites in the USA, is conducted under an FDA investigational new drug application (IND), or involves a drug or biologic that is manufactured in the USA or its territories and is exported for research.15 Because registration and results reporting of pharmacokinetic and phase I trial data on ClinicalTrials.gov can be inconsistent, we did not classify trials that only evaluated pharmacokinetics under the first two categories.
†Some postmarketing requirements are flexible and can be satisfied in more than one way. A flexible postmarketing requirement could outline that drug manufacturers have the option of collecting safety data from an open label extension of a clinical trial that the manufacturer has already committed to perform, from separate longer term, open label safety trials, or from long term controlled safety and efficacy trials.
If only limited information was available, we used strict classification criteria. For example, a one sentence postmarketing requirement for a pharmacokinetic study, without study duration or outcomes, would be classified as a “new animal or ‘other’ study required,” because there could be inconsistent registration and results reporting of pharmacokinetic and phase I trial data on ClinicalTrials.gov.14 This category would include all new animal trials; pharmacokinetic or pharmacodynamics trials; and in vitro or in vivo, drug transport, drug-drug or drug-therapeutic, prenatal and postnatal development, antidrug antibody response, mass balance, dosing, lactation, or QT/QTc studies (box 1). However, a postmarketing requirement evaluating “PK [pharmacokinetics], safety, and efficacy” would be classified as a “new prospective cohort study, registry, or clinical trial.” If a ClinicalTrials.gov registration or a corresponding publication had more information, we incorporated that information to improve the fidelity of our categorization.
Using only the information from FDA approval letters hyperlinked in the Drugs@FDA database, we calculated the length of each postmarket study description (word count) and abstracted whether there was information provided about the use of randomization; whether patient allocation was double blind, single blind, open label, or unclear; whether there was a comparator; whether the comparator was placebo, active control, both, or unclear; and study duration. As a non-prespecified abstraction, we also recorded whether information was provided about the estimated number of patients to be enrolled (exact or approximate number provided, minimum number provided, minimum number in the treatment arm only provided, or no information provided).
Status of postmarket studies
The Postmarketing Study and Clinical Trial Requirements and Commitments Database File (available from 31 October 2017 and downloaded on 10 November 2017), which is publicly accessible through the FDA website and includes descriptions, schedules for completing, and characterizations of the current status of postmarketing requirements,20 was used to determine the status for each postmarket study. The file is updated every three months, at the end of January, April, July, and October, based on a FDA review of annual status reports sponsors submit.
The FDA assigns each postmarketing requirement to one of seven status categories (supplementary appendix box 1). Because fulfilled and released requirements are only displayed on the online database for one year after the date of fulfillment or release, the FDA.gov archive was used to locate previous postmarketing study and clinical trial requirements and commitments database files. When archived databases with the final statuses were unavailable, we recorded the most recent status and date for each postmarketing requirement (eg, last available status: pending, 31 October 2010). We then performed additional Google searches using the terms “postmarketing” or “PMR” in combination with manufacturers’ names to determine whether manufacturers were publicly sharing their own information about postmarketing requirements (eg, “Pfizer PMRs” or “Pfizer postmarketing requirements”).
Lastly, we reviewed the supplemental letters on the Drugs@FDA database to determine whether they included information regarding the fulfillment of postmarketing requirements. When there were status inconsistencies between the FDA and drug sponsor data, we selected the study status that was the furthest along (eg, submitted instead of ongoing) or the most definitive (fulfilled instead of unclear). The abstractions were performed by one reviewer (JDW). Consistency and accuracy were verified through a 10% random sample validation performed by a second reviewer (ACE).
Trial registration and results reporting on ClinicalTrials.gov and peer reviewed publication
For all new prospective cohort studies, registries, and clinical trials and all requirements that call for the completion and submission of the results from “ongoing” prospective cohort studies and trials (hereafter “prospective cohort studies, registries, and clinical trials”), we determined study registration and results reporting on ClinicalTrials.gov. These study designs are likely to be the most important to physicians and patients. However, we also evaluated registration and results reporting rates separately for clinical trials, because only ongoing and forthcoming “applicable clinical trials” (which excludes non-interventional studies) of FDA regulated products are subject to mandated clinical trial registration and results reporting on ClinicalTrials.gov according to the final rule for section 801 of the FDAAA in 2016 (supplementary appendix box 2).1415 One requirement for FDAAA coverage requires manufacturing data. In particular, the FDAAA states that a trial must have a drug manufacturer in the USA for export or be conducted in the USA in order to be covered.14152122 This information is difficult to determine with public sources. Therefore, our sample of postmarket clinical trials were “highly likely” to be “applicable clinical trials.”
One reviewer (JDW) entered new drug or biologic names and study characteristics (eg, indication, comparator, outcome, and population) based on the information available in the postmarket study descriptions into the advanced search feature of ClinicalTrials.gov. Nine criteria were used to match trial registrations with the postmarket study descriptions:
Similar years for ClinicalTrials.gov registration and for postmarketing requirement protocol submission outlined in the FDA approval letters
Trial identification name/number provided in the postmarketing requirement descriptions
Industry sponsor funding source (yes, no)
At a minimum, matches were required to fulfill criteria 1-3 or 4. A third author (SSD) repeated all searches for trials that were determined to be unregistered. Potential matches not fulfilling criteria 1-4 were discussed with the senior investigator (JSR).
Once identified, for each registered prospective cohort study, registry, and clinical trial, one reviewer (JDW) abstracted study characteristics from the ClinicalTrials.gov registration, including National Clinical Trial (NCT) number; ClinicalTrials.gov status (eg, currently recruiting, completed, terminated, and withdrawn)23; first submission, first results reporting, study start, and primary completion dates; estimated overall population; use of randomization; whether participant allocation was double or triple blind, single blind, or none/open label; and whether there was a placebo, active comparator, or no comparator. When postmarketing requirements did not specify a primary endpoint, we recorded the primary endpoint and corresponding duration provided in the ClinicalTrials.gov registration. Each primary endpoint was then classified as either a clinical outcome, clinical scale, surrogate outcome, or safety and tolerability outcome based on conventions used in previous research.13
For all postmarket prospective cohort studies, registries, and clinical trials with a completed or terminated status on ClinicalTrials.gov, for which results reporting would be expected, we recorded whether any study results were reported or corresponding articles were published. For all prospective cohort studies, registries, and clinical trials without publications listed on ClinicalTrials.gov and all unregistered prospective cohort studies, registries, and clinical trials classified as submitted, fulfilled, released, or unclear (eg, last available status: pending, 31 October 2010) according to the FDA or drug sponsor data, one author (JDW) used a systematic two step search strategy to locate publications, as has been done elsewhere.24 Firstly, Google and the Scopus (Elsevier) and PubMed databases were searched using the NCT number. If a matching publication was not found, we searched for original research articles in the Scopus database using the terms “[intervention name]” and “clinical trial” in the “article title, abstract, keywords” field. For prospective cohort studies and registries, we searched for “[intervention]” and “registry” as well as “[intervention]” and “cohort.” If necessary, we added “[indication]” to the search. We used five criteria to identify matching publications: study design, indication, intervention, primary outcome(s), and intention to treat enrollment, as has been done elsewhere.2425 If there were multiple publications, we used the date of the earliest publication that reported the primary results of the trial. A third author (SSD) repeated all searches for postmarketing requirements that were determined to be unpublished. Lastly, we extracted the date of first publication in a peer reviewed journal and the 2015 journal impact factor according to InCites Journal Citation Reports. Consistency and accuracy were verified through a 10% random sample validation performed by a second reviewer (ACE). All uncertainties and disagreements were resolved by consensus with input from the senior investigator (JSR).
Using descriptive statistics, we characterized the new drugs and biologics and postmarket study characteristics. Fisher’s exact and Kruskal-Wallis tests were used, as appropriate, to examine differences among postmarket study characteristics, including treatment area, orphan status, and postmarketing requirement category. To estimate time to first results reported (either on ClinicalTrials.gov or in a peer reviewed publication), we generated Kaplan-Meier plots. Analyses were performed by R (version 3.2.3; R Project for Statistical Computing). All statistical tests were two tailed.
No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for design or implementation of the study. No patients were asked to advise on interpretation or writing up of results. There are no plans to disseminate the results of the research to study participants or the relevant patient community.
Characteristics of new drugs and biologics
Between 2009 and 2012, the FDA approved 110 new drugs and biologics for 120 indications, 13 (11.8%) of which did not have any postmarketing requirements at the time of first approval. Of 97 novel drugs and biologics for 106 indications in the final study sample (table 2), 75 (77.3%) were drugs and 22 (22.7%) were biologics. Drugs and biologics indicated for the treatment of cancer and hematology (26 (26.8%)) were the most common. Nearly one third of the drugs (28 (28.9%)) received priority review, and about one tenth (9/97 (9.3%)) received accelerated approval. Fifteen (15.5%) novel drugs and biologics that were designated as orphan products.
Postmarketing requirements from 2009 to 2012
We found 437 postmarketing requirements associated with these 97 new drugs and biologics. The median number of requirements per approval letter for each new drug or biologic was four (interquartile range 2-6), which was consistent between 2009 and 2012. Half the postmarket studies required (220 (50.3%)) were for “new animal or ‘other’ studies” (table 3), and nearly one third were for prospective cohort studies, registries, and clinical trials (134 (30.7%)). More than three quarters of postmarket studies were issued under the FDAAA authority (344 (78.7%)), more than one sixth under the PREA authority (77 (17.6%)).
Individual postmarket study descriptions were often short and difficult to categorize (supplementary appendix box 3), with a median word count of 44 (interquartile range 29-71). Among the 110 clinical trials, there was not enough information to establish use of randomization, comparator type, allocation, outcome, and number of patients to be enrolled for 38 (34.5%), 44 (40.0%), 62 (56.4%), 66 (60.0%), and 98 (89.1%), respectively (supplementary appendix table 1).
Of the 437 postmarket studies overall, 166 (38.0%) were classified as fulfilled according to FDA or drug sponsor data (supplementary appendix table 2). Of 134 postmarket prospective cohort studies, registries, and clinical trials, one third (44 (32.8%)) were classified as either submitted or fulfilled. Fifty postmarket studies did not have an up to date status in any of the available postmarketing study and clinical trial requirements and commitments database files, and were classified as fulfilled according to supplemental letters on Drugs@FDA. Drug sponsor data was available for 106 postmarketing requirements. After exclusion of postmarket studies with an unclear status based on FDA data, the FDA and drug sponsors provided a different status for nine postmarket studies. Most of these studies (8/9 (88.9%)) were classified as further along according to drug sponsor data. Overall, 131 (30.0%) postmarket studies did not have enough information in any publicly available source to determine a recent, up to date status (supplementary appendix data 1).
Prospective cohort studies, registries, and clinical trials: registration and study characteristics
Among the 134 postmarket prospective cohort studies, registries, and clinical trials, 102 (76.1%) were registered on ClinicalTrials.gov (table 4); among the 110 studies explicitly described as clinical trials, 84 (76.4%) were registered. Nearly all accelerated approval pathway (9/10 (90.0%)) and FDAAA authority (60/71 (84.5%)) studies were registered. All studies for autoimmune, musculoskeletal, or dermatological indications were registered (19/19 (100.0%)).
Of the 102 registered prospective cohort studies, registries, and clinical trials, most were randomized (67 (65.7%)), with open label allocation (56 (54.9%)); fewer than half were placebo controlled (41 (40.2%); table 5). Although safety and tolerability endpoints were used in nearly half of these studies (50 (49.0%)), only 15 (14.7%) focused on clinical outcomes. According to the ClinicalTrial.gov registrations, median study duration and estimated sample size were 12 months (interquartile range 2.8-31.0) and 265.0 (83.5-690.5), respectively. All required postmarketing trials of agents approved through the accelerated approval pathway were randomized (9/9 (100.0%)), unlike those with FDAAA and PREA postmarketing requirements (34/60 (56.7%) and 24/33 (72.7%), respectively, P=0.01). Over half of the studies used for cancer were for surrogate endpoints (11/19 (57.9%)), whereas most of those required under the FDAAA authority had safety and tolerability outcomes (40/60 (66.7%)).
Prospective cohort studies, registries, and clinical trials: results reporting and publication
Among the 50 prospective cohort studies, registries, and clinical trials classified as completed or terminated on ClinicalTrials.gov, 36 (72.0%) had reported results (table 4). Among the 65 registered or unregistered studies for which publication would be expected based on the most recent status provided by the FDA, drug sponsors, or on ClinicalTrials.gov, 37 (56.9%) were published in the peer reviewed literature and 47 (72.3%) had either reported results or were published. The publication journals’ median impact factor was 12.8 (interquartile range 2.8-33.9). Reporting and publication rates did not differ according to postmarketing requirement authority, treatment area, and orphan status. Reporting and publication rates stratified by ClinicalTrials.gov and the FDA statuses are reported in supplementary appendix tables 4 and 5. The results reporting and publication rates were consistent when studies with an unclear status according to the FDA or ClinicalTrials.gov were excluded. Furthermore, the dissemination rates were only slightly lower when limited to studies classified as released or with unclear statuses according to the FDA or ClinicalTrials.gov (10/16 (62.5%) and 8/16 (50.0%), respectively). Finally, among the seven studies classified as terminated according to ClinicalTrials.gov, five (71.4%) had either reported results or were published.
The median time from FDA approval to reported results or publication of postmarket studies was 47 months (interquartile range 32-67). Although one third of postmarket studies (15/47 (31.9%)) reported public results ahead of schedule (median 19 (10-23) months before the FDA report submission deadline), two thirds (32/47 (68.1%)) reported results behind schedule (14 (7-14) months after the deadline). Approximately half (69 (51.5%)) of all 134 required prospective cohort studies, registries, or clinical trials reported results on ClinicalTrials.gov or were published (final follow-up 15 November 2017, fig 1). Among the 47 postmarket studies registered on ClinicalTrials.gov that reported results or were published, the median time from estimated study completion according to ClinicalTrials.gov to results reporting or publication was 15 (12-23) months.
Among 14 studies that were expected to report results but did not, only two had an explanation in the FDA’s postmarketing study and clinical trial requirements and commitments database files, outlining why the studies were delayed. Time from the primary study completion dates on ClinicalTrials.gov to final follow-up (15 November 2017) were a median 26 months (interquartile range 8-43) for studies classified as completed and 39 (14-68) for studies classified as terminated (supplementary appendix table 6). There were 39 postmarket studies that were not classified as fulfilled by the FDA with a status explanation in the FDA’s postmarketing study and clinical trial requirements and commitments database files. Most explanations either provided an enrollment update (eg, “272 patients have been screen; 110 have been randomized”) or outlined that a deferral extension had been granted (eg, “Original Final Report Due Date: 03/30/2018; Deferral Extension granted per FDA letter dated 07/13/2017”; supplementary appendix table 7).
Of the 46 clinical trials, which are highly likely to be subject to mandatory registration and results reporting under the FDAAA, classified as completed or terminated on ClinicalTrials.gov, 35 (76.1%) had reported results. Of 61 registered or unregistered clinical trials, 37 (60.7%) were published in the peer reviewed literature and 46 (75.4%) had either reported results or were published. Among the 35 required clinical trials with reported results on ClinicalTrials.gov, 15 (42.9%) reported results ahead of schedule (median 16 months (interquartile range 6-23) before the FDA report submission deadline), and 20 (57.1%) reported results behind schedule (median 15 (10-22) months after the deadline).
Consistency and accuracy of abstractions
A total of 1764 excel cells were abstracted independently by two reviewers (JDW and ACE), and consensus was reached for all 26 (1.5%) differences.
Among 97 new drugs and biologics approved by the FDA between 2009 and 2012, we identified 437 associated postmarketing requirements issued by the FDA at the time of approval. Many of these postmarketing requirements were only briefly described and often did not contain enough public information to understand the purpose of the requirement or characterize the required study designs. Furthermore, we were unable to find up to date information on the progress of about one third of the postmarketing requirements. Among prospective cohort studies, registries, and clinical trials, which are likely to be of most clinical importance to physicians and patients, we found evidence of successful dissemination of research findings: three quarters were registered on ClinicalTrials.gov, and nearly three quarters had either reported results or were published. However, two thirds of the postmarket studies reported results publicly after their original FDA report submission deadline, potentially limiting their application to clinical practice. Furthermore, we observed similar dissemination rates when focusing on clinical trials—which are highly likely to be subject to mandatory registration and results reporting under the FDAAA, US legislation that mandates clinical trial registration and outcome reporting on ClinicalTrials.gov. In view of this, opportunities exist to increase transparency and dissemination of research findings, mitigating selective registration and results reporting.
The brief descriptions of many postmarket clinical trials often did not contain enough information to establish use of randomization, comparator type, number of patients to be enrolled, and allocation. Moreover, over half of all postmarketing requirements did not specify an endpoint, an essential feature to understand how the study might inform clinical practice. Detailed descriptions are also necessary to determine corresponding ClinicalTrials.gov registrations and journal publications. Our findings are consistent with a recent report published by the Office of Inspector General, which discussed difficulties classifying 37 postmarket studies and emphasized previous concerns related to the classification of postmarket study statuses.9 Similar to the Office of Inspector General, we found that about one quarter of postmarketing requirements were for new clinical trials, whereas half were for new animal or “other” studies, including pharmacokinetic trials and in vitro or in vivo studies. Overall, our findings may suggest that manufacturers are given substantial flexibility in designing studies, and that most postmarket studies may not answer clinical questions that are of greatest interest to physicians and patients.
Our study also found that about three quarters of postmarket clinical trials were registered on ClinicalTrials.gov, which is less than previously reported registration rates for clinical trials supporting New Drug Applications.2226 Our finding that about three quarters of the postmarket clinical trials had either reported results or were published is consistent with a recent study by the FDA, which showed that nearly two thirds of postmarket drug interventional clinical trials and other trials designated as “fulfilled” were published in either the scientific literature or on the ClinicalTrials.gov website.16 However, only 55.5% of postmarket studies for which publication would be expected were published in peer-reviewed medical journals. This contrasts to prior research on trials supporting FDA approval of new drugs and biologics showing that nearly 90% were published in peer reviewed journals.25 Given the increasing importance of postmarketing requirements to evaluate new drug and biologic safety and effectiveness as part of lifecycle evaluation efforts, even greater emphasis must be placed on registration, results reporting, and publication of all required postmarket studies.
Furthermore, the majority of postmarket studies reported public results after their original FDA report submission deadline. Although drug sponsors may be meeting FDA reporting deadlines, our work supports previous claims of the slow pace of postmarket studies.13 More timely results reporting across all postmarketing requirement authorities is necessary to ensure that the findings from postmarket studies can inform clinical practice.
Implications and recommendations
In the USA, expedited review pathways are being increasingly used for the approval of new drugs and biologics,27 which can provide market authorization on the basis of small and shorter clinical trials focused on surrogate markers for trial endpoints. As FDA regulatory paradigms shift toward lifecycle evaluation, there will be an increasing reliance on data generated by postmarket studies. Although more detailed postmarket study descriptions and increased FDA transparency are necessary, it is promising that the majority of postmarket prospective cohort studies, registries, and clinical trials are registered and have reported results or were published.
Our findings support a recent proposal for FDA reform, which outlined opportunities to enhance transparency at the FDA and suggested that the FDA release the final reports that fulfill postmarketing requirements.28 The FDA already has high standards for reviewing and publishing information on pediatric studies conducted under PREA and the Best Pharmaceutical for Children’s Act. This information includes publicly available medical, statistical, and clinical pharmacology reviews and information regarding the types of studies conducted (eg, trial design, number of pediatric patients). To further strengthen postmarketing requirement transparency, postmarket drug study descriptions should include a clear study design (eg, animal trial, prospective cohort study), trial endpoints, potential comparator arms, study populations, follow-up duration, and a target sample size. Recently, the FDA announced a plan to add ClinicalTrials.gov NCT numbers to materials for future drug approvals.29 The FDA should consider expanding this initiative to add NCT numbers to postmarketing requirement descriptions and the postmarketing study and clinical trial requirements and commitments database files to make it easier for patients, physicians, and researchers to link clinical trial listings to FDA documents for postmarket studies.
After reviewing publicly available FDA and drug sponsor data, we were only able to locate up to date statuses for two thirds of the postmarket drug studies. The FDA should consider making certain components of their Document Archiving, Reporting, and Regulatory Tracking System (DARRTS), a non-publicly available database that includes information for prescription drug postmarketing requirements, publicly available. In particular, it appears as if DARRTS includes annual status reports, which are the detailed reports that drug sponsors must submit annually to the FDA on the status of each open postmarketing requirement.9
At a minimum, the FDA should not remove “fulfilled” and “released” requirements from the postmarketing study and clinical trial requirements and commitments database files, rendering them no longer publicly identifiable. Currently, these requirements are displayed on the website for “not more than 1 year from the date of fulfillment or release.”7 The FDA already provides more extensive information, including an extensive reporting schedule, for both active and inactive postapproval studies for medical devices.30 Similar reporting standards could be adopted for drug and biologic postmarketing requirements. These, and other suggestions to promote transparency and improve the oversight of postmarketing requirements,91322 may be key to ensuring the successful translation of results from postmarket studies into clinical practice.
Although we found relatively high rates of registration, results reporting, and publication of required clinical trials, registration and results reporting is required by law for ongoing and forthcoming non-interventional “applicable clinical trials” of FDA regulated products.14 Our findings suggest that clearer and more consistent regulatory standards and FDA oversight might be necessary to ensure universal registration and result reporting on ClinicalTrials.gov for applicable postmarket studies.2226 In particular, the FDA might need to provide additional clarity to sponsors about which trials need to be registered and when results need to be reported. Furthermore, new regulations might be needed to ensure that the results from postmarket studies that are of most interest to the clinical community, including prospective cohort studies and registries, are publicly disseminated.16 Alternatively, sponsors can also voluntarily take on part of the responsibility and commit to greater registration and results dissemination.
Limitations of this study
This study had limitations. Firstly, our study relied on publicly available data sources. The brief postmarketing requirement descriptions provided in the FDA approval letters made categorizing postmarket drug studies and determining ClinicalTrials.gov registrations and peer reviewed publications difficult. However, it is unlikely that we misclassified trials as other study designs; about one quarter of the postmarket studies in our sample were classified as new clinical trials, consistent with an estimate of 28% reported in a previous evaluation by the Office of Inspector General.9 Furthermore, we used various versions of the FDA’s postmarketing study and clinical trial requirements and commitments database files to determine the final statuses of the postmarketing requirements. “Fulfilled” and “released” requirements are only displayed on the online database for one year after the date of fulfillment or release, and we were unable to locate archived databases for all years of follow-up. To account for this limitation, we searched for postmarketing requirement statuses provided in supplementary applications or online by drug sponsors. Furthermore, since the data in DARRTS are not available to the public, the statuses of certain postmarketing requirements and our estimates regarding the timeliness of results dissemination might not be based on up to date data.31
Secondly, while we focused on postmarketing requirements that were imposed between 2009 and 2012, potentially allowing for at least four years for completion and publication, our study did not account for the time that it might take to prepare and disseminate research findings.32 In particular, once a postmarket study has been completed, authors need to prepare manuscripts, submit them to journals, revise, and potentially resubmit to multiple journals before acceptance. We looked for publications for all registered studies classified as completed or terminated on ClinicalTrials.gov or unregistered studies classified as submitted, fulfilled, released, or unclear according to FDA or drug sponsor data. This decision was based on previous work by our group and others that used ClinicalTrials.gov to characterize results reporting and publication of registered trials.243334353637 We acknowledge that additional studies could get published, but were not published at the time of our search.
Thirdly, we did not determine whether the results from required “ongoing” prospective cohort studies, registries, or clinical trials were reported or published. Although some of these “ongoing” studies could have reported or published results, they are less likely to have done so. Lastly, our sample of clinical trials could contain studies that were not “applicable clinical trials” according to the FDAAA. The FDAAA states that a trial must have a drug manufacturer in the USA for export or be conducted in the USA to be covered.14152122 We could not verify these characteristics because this information is difficult to determine on the basis of public information on postmarketing requirements.
In the present analysis, postmarketing requirements for new drugs and biologics were often briefly described, difficult to categorize, and frequently did not contain enough information to characterize the required study designs. Nearly three quarters of postmarket prospective cohort studies, registries, and clinical trials (which are often of most interest to physicians and patients) were registered on ClinicalTrials.gov or had either reported results or were published, suggesting that at least one quarter of these required studies are not being publicly disseminated.
Furthermore, two thirds of the postmarket studies reported public results after their original FDA report submission deadline. Similar registration and reporting rates were observed when focused exclusively on clinical trials, which are highly likely to be subject to mandatory registration and results reporting on ClinicalTrials.gov under the FDAAA. These findings highlight the need for more detailed postmarketing requirement study descriptions, increased FDA transparency, and clearer and more consistent registration and results reporting standards for these critical FDA required studies.
What is already known on this topic
The US Food and Drug Administration (FDA) can require drug sponsors to conduct studies after approval to answer important questions about the safety and efficacy of new drugs and biologics
There have been growing concerns about the fulfillment of postmarketing requirements, and when fulfilled, the rigor of the evidence generated
One third of required postmarket clinical trials that are classified as fulfilled are not published in either the scientific literature or reported on ClinicalTrials.gov
What this study adds
Many postmarketing requirements issued by the FDA at approval are only briefly described and often do not contain enough public information to understand their study designs or purpose
Among required clinical studies, about three quarters were registered on ClinicalTrials.gov; however, at least one quarter of studies for which results reporting or publication would be expected have not been publicly disseminated
Among required clinical studies that either reported results or were published, two thirds reported results publicly after their original FDA report submission deadline, potentially limiting their application to clinical practice
Contributors: JDW, ACE, and JSR conceived and designed this study. JDW, ACE, and SSD acquired the data. JDW conducted the statistical analysis and drafted the manuscript. All authors participated in the interpretation of the data and critically revised the manuscript for important intellectual content. JDW and JSR had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. JSR provided supervision. JDW and JSR are guarantors. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding: This project was conducted as part of the Collaboration for Research Integrity and Transparency (CRIT) at Yale, funded by the Laura and John Arnold Foundation, which supports JDW, ACE, MEM, and JSR. SSD is supported by the Robert Wood Johnson Foundation Clinical Scholars Program and the Department of Veterans Affairs. JEM receives funding from the Laura and John Arnold Foundation. These funders played no role in the design of the study, analysis or interpretation of findings, or drafting the manuscript and did not review or approval the manuscript before submission. The authors assume full responsibility for the accuracy and completeness of the ideas presented.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Laura and John Arnold Foundation for the submitted work; in the past 36 months, JDW received research support through the Meta Research Innovation Center at Stanford (METRICS) from the Laura and John Arnold Foundation; JSR received research support through Yale from Johnson and Johnson to develop methods of clinical trial data sharing, from Medtronic and the US Food and Drug Administration (FDA) to develop methods for postmarket surveillance of medical devices (U01FD004585), from the Centers of Medicare and Medicaid Services (CMS) to develop and maintain performance measures that are used for public reporting, from the FDA to establish a Center for Excellence in Regulatory Science and Innovation (CERSI) at Yale University and the Mayo Clinic (U01FD005938), from the Blue Cross Blue Shield Association to better understand medical technology evaluation, and from the Agency for Healthcare Research and Quality (R01HS022882); SSD receives support as a scholar in the Yale University/Mayo Clinic FDA CERSI; LMS and SW were cofounders of Informulary, a company that proved data and the benefits, harms, and uncertainties of prescription drugs, and has since ceased operations; LMS and SW report personal fees from Ross Feller Case outside the submitted work; JEM receives funding from the Laura and John Arnold Foundation.
Ethical approval: As a cohort study using publicly available data, this study does not require research ethics approval or patient consent.
Data sharing: Requests for the dataset can be made to the corresponding author at firstname.lastname@example.org. The dataset will be made available via a publicly accessible repository after publication.
The lead authors affirm that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant registered) have been explained.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.