Published 7 August 2009, doi:10.1136/bmj.b2981
Cite this as: BMJ 2009;339:b2981

Research

Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications

Santiago G Moreno, research student1, Alex J Sutton, professor of medical statistics1, Erick H Turner, assistant professor2, Keith R Abrams, professor of medical statistics1, Nicola J Cooper, senior research fellow1, Tom M Palmer, research associate3, A E Ades, professor of public health science4

1 Department of Health Sciences, University of Leicester, Leicester LE1 7RH, 2 Department of Psychiatry, Oregon Health and Science University, Portland Veterans Affairs Medical Center, Portland, Oregon, USA, 3 MRC Centre for Causal Analyses in Translational Epidemiology, Department of Social Medicine, University of Bristol, 4 Department of Community Based Medicine, University of Bristol

Correspondence to: S G Moreno sgm8{at}le.ac.uk

Objective To assess the performance of novel contour enhanced funnel plots and a regression based adjustment method to detect and adjust for publication biases.

Design Secondary analysis of a published systematic literature review.

Data sources Placebo controlled trials of antidepressants previously submitted to the US Food and Drug Administration (FDA) and matching journal publications.

Methods Publication biases were identified using novel contour enhanced funnel plots, a regression based adjustment method, Egger’s test, and the trim and fill method. Results were compared with a meta-analysis of the gold standard data submitted to the FDA.

Results Severe asymmetry was observed in the contour enhanced funnel plot that appeared to be heavily influenced by the statistical significance of results, suggesting publication biases as the cause of the asymmetry. Applying the regression based adjustment method to the journal data produced a similar pooled effect to that observed by a meta-analysis of the FDA data. Contrasting journal and FDA results suggested that, in addition to other deviations from study protocol, switching from an intention to treat analysis to a per protocol one would contribute to the observed discrepancies between the journal and FDA results.

Conclusion Novel contour enhanced funnel plots and a regression based adjustment method worked convincingly and might have an important part to play in combating publication biases.


This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to StumbleUpon StumbleUpon   Add to Technorati Technorati    What's this?

Relevant Articles

New methods to deal with publication bias
Hans-Hermann Dubben
BMJ 2009 339: b3272. [Extract] [Full Text]

Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study
Lesley Wood, Matthias Egger, Lise Lotte Gluud, Kenneth F Schulz, Peter Jüni, Douglas G Altman, Christian Gluud, Richard M Martin, Anthony J G Wood, and Jonathan A C Sterne
BMJ 2008 336: 601-605. [Abstract] [Full Text] [PDF]

The case of the misleading funnel plot
Joseph Lau, John P A Ioannidis, Norma Terrin, Christopher H Schmid, and Ingram Olkin
BMJ 2006 333: 597-600. [Full Text] [PDF]

Measuring inconsistency in meta-analyses
Julian P T Higgins, Simon G Thompson, Jonathan J Deeks, and Douglas G Altman
BMJ 2003 327: 557-560. [Extract] [Full Text] [PDF]

Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications
Hans Melander, Jane Ahlqvist-Rastad, Gertie Meijer, and Björn Beermann
BMJ 2003 326: 1171-1173. [Abstract] [Full Text] [PDF]

Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis
Dean Fergusson, Shawn D Aaron, Gordon Guyatt, and Paul Hébert
BMJ 2002 325: 652-654. [Extract] [Full Text] [PDF]

Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis
Jonathan A C Sterne, Matthias Egger, and George Davey Smith
BMJ 2001 323: 101-105. [Extract] [Full Text] [PDF]

Empirical assessment of effect of publication bias on meta-analyses
A J Sutton, S J Duval, R L Tweedie, K R Abrams, and D R Jones
BMJ 2000 320: 1574-1577. [Abstract] [Full Text] [PDF]

What is meant by intention to treat analysis? Survey of published randomised controlled trials
Sally Hollis and Fiona Campbell
BMJ 1999 319: 670-674. [Abstract] [Full Text] [PDF]

Meta-analysis: Principles and procedures
Matthias Egger, George Davey Smith, and Andrew N Phillips
BMJ 1997 315: 1533-1537. [Extract] [Full Text]

Bias in meta-analysis detected by a simple, graphical test
Matthias Egger, George Davey Smith, Martin Schneider, and Christoph Minder
BMJ 1997 315: 629-634. [Abstract] [Full Text]

Misleading meta-analysis
Matthias Egger and George Davey Smith
BMJ 1995 310: 752-754. [Extract] [Full Text]

This article has been cited by other articles:

  • Barbui, C., Cipriani, A., Furukawa, T. A, Salanti, G., Higgins, J. P T, Churchill, R., Watanabe, N., Nakagawa, A., Omori, I. M, Geddes, J. R (2009). Making the best use of available evidence: the case of new generation antidepressants: A response to: Are all antidepressants equal?. Evid. Based Ment. Health 12: 101-104 [Full text]  
  • Dubben, H.-H. (2009). New methods to deal with publication bias. BMJ 339: b3272-b3272 [Full text]  
Access jobs at BMJ Careers
Whats new online at Student BMJ