Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications
BMJ 2009; 339 doi: https://doi.org/10.1136/bmj.b2981 (Published 07 August 2009) Cite this as: BMJ 2009;339:b2981
- Santiago G Moreno, research student1,
- Alex J Sutton, professor of medical statistics1,
- Erick H Turner, assistant professor2,
- Keith R Abrams, professor of medical statistics1,
- Nicola J Cooper, senior research fellow1,
- Tom M Palmer, research associate3,
- A E Ades, professor of public health science4
- 1Department of Health Sciences, University of Leicester, Leicester LE1 7RH
- 2Department of Psychiatry, Oregon Health and Science University, Portland Veterans Affairs Medical Center, Portland, Oregon, USA
- 3MRC Centre for Causal Analyses in Translational Epidemiology, Department of Social Medicine, University of Bristol
- 4Department of Community Based Medicine, University of Bristol
- Correspondence to: S G Moreno sgm8{at}le.ac.uk
- Accepted 10 May 2009
Abstract
Objective To assess the performance of novel contour enhanced funnel plots and a regression based adjustment method to detect and adjust for publication biases.
Design Secondary analysis of a published systematic literature review.
Data sources Placebo controlled trials of antidepressants previously submitted to the US Food and Drug Administration (FDA) and matching journal publications.
Methods Publication biases were identified using novel contour enhanced funnel plots, a regression based adjustment method, Egger’s test, and the trim and fill method. Results were compared with a meta-analysis of the gold standard data submitted to the FDA.
Results Severe asymmetry was observed in the contour enhanced funnel plot that appeared to be heavily influenced by the statistical significance of results, suggesting publication biases as the cause of the asymmetry. Applying the regression based adjustment method to the journal data produced a similar pooled effect to that observed by a meta-analysis of the FDA data. Contrasting journal and FDA results suggested that, in addition to other deviations from study protocol, switching from an intention to treat analysis to a per protocol one would contribute to the observed discrepancies between the journal and FDA results.
Conclusion Novel contour enhanced funnel plots and a regression based adjustment method worked convincingly and might have an important part to play in combating publication biases.
Footnotes
Contributors: AJS conceived the project and led the research together with SGM. AJS and SGM carried out the statistical analyses and interpretation of the data. EHT, AEA, KRA, and NJC participated in data analysis and interpretation. TMP made a substantial contribution by designing and developing the plots. SGM and AJS drafted the paper, which was revised by all coauthors through substantial contributions to the contents of the paper. All authors approved the final version of the paper for publication. SGM is the guarantor.
Funding: SGM was supported by a Medical Research Council Health Services Research Collaboration studentship in the UK. AEA was funded by the Medical Research Council Health Services Research Collaboration. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, and writing and publishing the report. The corresponding author as well as the other authors had access to all the data and take responsibility for the integrity of the data and the accuracy of the data analysis.
Competing interests: None declared.
Ethical approval: Not required.
Data sharing: Data are available on request from the first author.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.