Re: Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study
The authors present significant findings about reporting results of clinical trials.
We followed their methodology and obtained the R script which extracts data from ClinicalTrials.gov (CT.gov). We would like to point out that the key result of 22% of trials reporting mandatory results is dependent on (1) accurately identifying trial completion date and also on (2) what results are required: outcomes limited to primary study outcome or all study outcomes (including secondary outcomes specified during trial registration).
As for the date, CT.gov provides two dates related to study completion: ‘completion date’ and ‘primary completion date’. The authors used ‘primary completion date’ as the key date for computing the fact that one year has elapsed since the study completion. CT.gov data specification available at http://prsinfo.clinicaltrials.gov/definitions.html define ‘study completion date’ as ‘final date on which data was (or is expected to be) collected.’ The second date of ‘primary completion date’ is defined as ‘the date that the final subject was examined or received an intervention for the purposes of final collection of data for the primary outcome’. Of note is that the primary completion date definition implies that the trial principal investigator (PI) may need additional time to collect data for secondary outcomes. In our analyses of CT.gov data, the ‘completion date’ is often stated to be somewhat later that the ‘primary completion date’. Moreover, many PI are motivated to specify a later official trial completion date. The main reason for this are IRB regulations which require a PI to keep the study active (not completed) if the team wants to carry on any follow-up or secondary analyses of the collected trial data.
From the point of view of required data fields (by the trial registry), it is significant to note that ‘completion date’ is an optional parameter and could be missing. The data field of ‘primary completion date’ is required parameter by the FDAA law; however, neither this date is formally required by CT.gov and could also be missing. In the study we respond to, however, the authors used such enrollment trial criteria that the ‘primary completion date’ cannot be missing in their final set of 1465 studies (per Figure 1, step 2: completed trials with primary completion date in 2009).
Given the issues related to the accurately specifying study completion date, we argue that the key results of 22% result-reporting compliance could be significantly different if the authors would have used ‘completion date’ instead of ‘primary completion date’ to compute the mandatory reporting deadline.
We downloaded the study data from the provided source and we used them to investigate how many trials would be affected by the fact that a later completion date is specified in the second completion date field. In the dataset of ‘FDA_table_with_sens.csv’, we found that this would move the reporting deadline boundary in 28.3% of analyzed trials (414 out of 1465 trials). The difference in those two dates ranges from 28 days to 669 days with mean of 118 days and median of 62 days.
We generally agree with their analysis and importance of their findings; however, we would like to point to data source issues in obtaining accurate trial completion date and highlight possible regulatory framework factors which affect completion date accurate reporting.
Data analysis steps in R:
#add column with time difference in days
#(if 0, then dates are the same, if positive, then completion date is x days later then primary completion date)
d<-transform(d,days_difference=as.integer(as.Date(d$Completion.Date, format = "%d/%m/%Y") - as.Date(d$Primary.Completion.Date, format = "%d/%m/%Y"),units='days'))
#proportion of trials which have positive days diference (28.3%)
#time difference characteristics