Rethinking credible evidence synthesisBMJ 2012; 344 doi: https://doi.org/10.1136/bmj.d7898 (Published 17 January 2012) Cite this as: BMJ 2012;344:d7898
All rapid responses
Congratulations to Doshi and colleagues (1) on their persistence in gaining access to data on Tamiflu and highlighting the problem of unpublished evidence.
Our organisation (Institute for Quality and Efficiency in Health Care, IQWiG) routinely requests clinical study reports (CSRs) from drug manufacturers as a data source for HTA reports. We can confirm the findings of Doshi and colleagues that, compared with journal publications, CSRs include a substantial amount of additional relevant information for the evaluation of a clinical trial. This was also shown in our recent analysis of documents reporting clinical trials (2) in which we investigated, among other things, to what extent journal publications and CSRs submitted to regulatory authorities provide sufficient information for trial evaluation, focusing on key data such as primary outcomes and overall adverse event rates. In our experience, CSRs are even more important as a source of information for secondary outcomes, characteristics of the study population, and detailed information on study methods.
The announcement by the European Medicines Agency (EMA) that “it plans to start publishing reports for all drugs submitted for approval in the next few years” (3) will be a major contribution towards improving the evidence base on new drugs. However, like other researchers (4,5), we criticise the limited availability of data on older drugs and suggest making the corresponding CSRs available, especially as older trials are not covered by mandatory registration, even though older drugs are widely prescribed in current clinical practice. This is all the more necessary due to the discrepancies between CSRs and the corresponding published papers. Both IQWiG and Doshi and colleagues have identified such discrepancies, which raise further doubts about the trustworthiness of published evidence.
CSRs are not generally available to authors of systematic reviews or HTA bodies. As stated, IQWiG requests CSRs from drug manufacturers, but there is no obligation to provide them. In our cited analysis, CSRs were only available for 38% of studies included in HTAs on drugs published by our Institute.
We have published the CSRs used in one of our assessments on the IQWiG website (6). Although these reports mostly do not include full appendices, they might be helpful to obtain an impression of the information available in this type of document.
1) Doshi P, Jones M, Jefferson T. Rethinking credible evidence synthesis. BMJ 2012;344:d7898 doi: 10.1136/bmj.d7898. http://www.bmj.com/highwire/filestream/559451/field_highwire_article_pdf...
2) Wieseler B, Kerekes MF, Vervoelgyi V, McGauran N, Kaiser T. Impact of document type on reporting quality of clinical drug trials: a comparison of registry reports, clinical study reports, and journal publications. BMJ. 2012 Jan 3;344:d8141. doi: 10.1136/bmj.d8141. http://www.bmj.com/highwire/filestream/554662/field_highwire_article_pdf...
3) BMJ Press Release. Effects of Tamiflu still uncertain, warn experts, as Roche continues to withhold key trial data. http://www.bmj.com/press-releases/2012/01/17/effects-tamiflu-still-uncer...
4) Turner EH. Closing a loophole in the FDA Amendments Act. Science. 2008 Oct 3;322(5898):44-6.
5) Wood AJJ. Progress and deficiencies in the registration of clinical trials. N Engl J Med. 2009;360:824-30
Competing interests: All authors are employees of IQWiG. In order to produce unbiased health technology assessment reports, the institute depends on access to all of the relevant data on the topic under investigation. The authors therefore support public access to clinical study reports submitted to regulatory authorities.
Clinical trial participants cannot be guaranteed a clinical benefit from their participation. Approval by a research ethics committee takes into account the benefits that will accrue to science, society and future patients.
Therefore, I have a proposal to force companies to make their trial reports available. Make the agreement to post the trial reports on a public website a condition of ethics committee approval. Without full disclosure, it does not appear that there is any real guarantee of any benefit to anyone.
Can all of the research ethics committees band together to achieve this goal? That will be the challenge.
Competing interests: No competing interests