Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study

BMJ 2012; 344 doi: http://dx.doi.org/10.1136/bmj.d7373 (Published 3 January 2012)
Cite this as: BMJ 2012;344:d7373

Recent rapid responses

Rapid responses are electronic letters to the editor. They enable our users to debate issues raised in articles published on bmj.com. Although a selection of rapid responses will be included as edited readers' letters in the weekly print issue of the BMJ, their first appearance online means that they are published articles. If you need the url (web address) of an individual response, perhaps for citation purposes, simply click on the response headline and copy the url from the browser window.

Displaying 1-5 out of 5 published

Dear Editor,

It is interesting to note that the spirit of FDAAA is to disseminate scientific knowledge, both for society and scientific community.

It was an important step towards mitigating the deleterious effects of selective publication of clinical trials.

The limited effects of FDAAA, as demonstrated by Prayle et al, should be better investigated.

However, it is noteworthy that many clinical trials are conducted outside the United States by non-American researchers. This means that FDAAA simply does not apply to these studies and these people. We could call this nationality bias.

Therefore, this discussion goes beyond the U.S. borders, as it addresses universal human rights: the right to health and the right to be informed, as set forth in the Universal Declaration of Human Rights (Article XIX).

Thus, we must rethink the FDAAA and discuss the topic under a global scope, proposing alternatives that may affect and compel researchers worldwide and not just in the U.S..

A viable alternative - and consistent with the status of fundamental right associated with health information - would be discussing and signing an international treaty addressing the issue and effect beyond U.S. borders. That way, disputes about the disclosure of information would be elevated to an unprecedented global level, subject to further discussion and possibly able to capture all the complexity involved in the process of dissemination of science its ethical and legal implications.

DH Marin dos Santos
dhmsantos@unifesp.br

Competing interests: None declared

Douglas H Marin dos Santos, Bioethics Researcher

Federal University of São Paulo (UNIFESP), RUA 34 SUL LT 10 12 BL A ap 602 Brasília DF 71930500 Brazil

Click to like:

Dear Editor,

We thank Huser for his interest in our work, and aim to clarify some important points which he raises.

In our paper, we present an analysis of ClinicalTrials.gov which aimed to identify which clinical trials were compliant with the FDA Amendment Act 2007 (FDAAA 2007).[1, 2] In order to do this, we identified trials which completed in 2009, which were conducted with one site in the USA, and which were of drugs previously approved by the FDA. We conducted our search on the 19th Jan 2011. We found a low overall reporting rate of 22%. Huser questions our use of the field “Primary Completion Date”, and the type of results available.

We did not look at which results were available for each individual study. We note that the FDAAA 2007 requires information on demographics, primary and secondary outcomes, and information regarding agreements between the sponsor and chief investigator restricting discussion of results from the trials. In our study, we investigated if any results were available, and did not analyse the results for particular trials.

As we were assessing compliance with the FDAAA 2007, we used the definition of completion date from the Act: “the date that the final subject was examined or received an intervention for the purposes of final collection of data for the primary outcome, whether the clinical trial concluded according to the prespecified protocol or was terminated.” This definition maps onto the field “Primary Completion Date” within the ClinicalTrials.gov record. Whilst it is true the study may well continue data collection after this date for secondary outcomes (and so the completion date would be later than the “Primary Completion Date”), the regulations requiring reporting results clearly apply to the “Primary Completion Date”. We therefore used the “Primary completion date” to identify when a trial had completed from the point of view of FDAAA reporting requirements. However, where this was missing, we used “completion date”, as stated in the methods. This affected 19/738 (2.5%) of the “mandatory reporting required” records. If a trial did not give either, we could not include it in our study, as we did not know when it had completed by either definition.

Huser asserts that had we used “completion date” (when the trial actually ended), our findings would have been significantly different. We argue that the “completion date” is not referenced in the legislation, and therefore we did not use it in the analysis which we present in the paper. However, if we restrict our analysis to the trials which we categorized as subject to mandatory reporting, and which list a date in 2009 in the “Completion date” field, 138/527 (26.2%) of the mandatory reporting required trials had reported results by the date of our search in early 2011. This is not a large difference.

It would appear to us that however you look at it, neither the letter nor the spirit of the law is being followed.

1. Prayle, A.P., M.N. Hurley, and A.R. Smyth, Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ, 2012. 344: p. d7373.
2. Food and Drug Administration Amendments Act of 2007.

Competing interests: We wrote the original article.

Andrew P Prayle, Clinical Research Fellow

Matthew N Hurley, Alan R Smyth

University of Nottingham, University Park, Nottingham, NG7 2UH

Click to like:

The authors present significant findings about reporting results of clinical trials.

We followed their methodology and obtained the R script which extracts data from ClinicalTrials.gov (CT.gov). We would like to point out that the key result of 22% of trials reporting mandatory results is dependent on (1) accurately identifying trial completion date and also on (2) what results are required: outcomes limited to primary study outcome or all study outcomes (including secondary outcomes specified during trial registration).

As for the date, CT.gov provides two dates related to study completion: ‘completion date’ and ‘primary completion date’. The authors used ‘primary completion date’ as the key date for computing the fact that one year has elapsed since the study completion. CT.gov data specification available at http://prsinfo.clinicaltrials.gov/definitions.html define ‘study completion date’ as ‘final date on which data was (or is expected to be) collected.’ The second date of ‘primary completion date’ is defined as ‘the date that the final subject was examined or received an intervention for the purposes of final collection of data for the primary outcome’. Of note is that the primary completion date definition implies that the trial principal investigator (PI) may need additional time to collect data for secondary outcomes. In our analyses of CT.gov data, the ‘completion date’ is often stated to be somewhat later that the ‘primary completion date’. Moreover, many PI are motivated to specify a later official trial completion date. The main reason for this are IRB regulations which require a PI to keep the study active (not completed) if the team wants to carry on any follow-up or secondary analyses of the collected trial data.

From the point of view of required data fields (by the trial registry), it is significant to note that ‘completion date’ is an optional parameter and could be missing. The data field of ‘primary completion date’ is required parameter by the FDAA law; however, neither this date is formally required by CT.gov and could also be missing. In the study we respond to, however, the authors used such enrollment trial criteria that the ‘primary completion date’ cannot be missing in their final set of 1465 studies (per Figure 1, step 2: completed trials with primary completion date in 2009).

Given the issues related to the accurately specifying study completion date, we argue that the key results of 22% result-reporting compliance could be significantly different if the authors would have used ‘completion date’ instead of ‘primary completion date’ to compute the mandatory reporting deadline.

We downloaded the study data from the provided source and we used them to investigate how many trials would be affected by the fact that a later completion date is specified in the second completion date field. In the dataset of ‘FDA_table_with_sens.csv’, we found that this would move the reporting deadline boundary in 28.3% of analyzed trials (414 out of 1465 trials). The difference in those two dates ranges from 28 days to 669 days with mean of 118 days and median of 62 days.

We generally agree with their analysis and importance of their findings; however, we would like to point to data source issues in obtaining accurate trial completion date and highlight possible regulatory framework factors which affect completion date accurate reporting.

Data analysis steps in R:

#obtain data

d<-read.csv(file='http://datadryad.org/bitstream/handle/10255/dryad.36679/FDA_table_with_sens.csv')

#add column with time difference in days

#(if 0, then dates are the same, if positive, then completion date is x days later then primary completion date)

d<-transform(d,days_difference=as.integer(as.Date(d$Completion.Date, format = "%d/%m/%Y") - as.Date(d$Primary.Completion.Date, format = "%d/%m/%Y"),units='days'))
#proportion of trials which have positive days diference (28.3%)
nrow(d[which(d$days_difference>0), ])/nrow(d)

#time difference characteristics

summary(d[which(d$days_difference>0),'days_difference'])

Competing interests: None declared

Vojtech Huser, Assistant Clinical Investigator

National Institute of Health, Clinical Center, 10 Center Dr, Bethesda, MD, USA

Click to like:

All research funders have a long way to go before meeting "mandatory" reporting requirements, but some funders (eg, government and academica) have much further to go than others (eg, industry).

The results from Prayle et al are consistent with another analysis presented at the last Annual Meeting of the International Society for Medical Publication Professionals (industry-funded trials were 4-6 times more compliant with reporting requirements than government- or academia-funded trials; Giel JL).

PS I wonder whether the mainstream media will be running with this story?

Competing interests: I conduct and publish research on ethical medical writing practices. I am actively involved in not-for-profit associations that educate members on ethical publication practices. I am paid to provide ethical medical writing training courses and services for not-for-profit and for-profit clients.

Karen L. Woolley, Professional medical writer (NOT a ghostwriter)

University of Queensland; University of the Sunshine Coast; ProScribe Medical Communications , 18 Shipyard Cct Noosaville Qld Australia 4566

Click to like:

Prayle et al tell us they have investigated the proportion of studies that reported results within 1 year of study completion, and found that the proportion was alarmingly low.

However, this raises an obvious question: if the trial hadn't reported results, how do you know when it was completed? Was this done via the "estimated completion date" field that's posted to clinicaltrials.gov at the time the study is registered?

If so, then the results are hard to interpret without also knowing what proportion of clinical trials actually meet their estimated completion date. Experience teaches that clinical trials don't always finish on schedule.

Competing interests: None declared

Adam Jacobs, Director

Dianthus Medical Limited, London SW19 2RL

Click to like:

THIS WEEK'S POLL