Relation of study quality, concordance, take home message, funding, and impact in studies of influenza vaccines: systematic review

The degree to which source of funding plays a role in the publication
of scientific research is now well-established. This was elegantly
demonstrated in the context of nicotine replacement therapy for smoking
cessation by Etter and colleagues (1). They showed evidence of possible
publication bias among studies funded by the pharmaceutical industry, but
not among studies funded from other sources, and that the former tended to
report larger effect size estimates than the latter. Moreover, when the
pooled effect size estimate was corrected for this potential bias, the
pooled effect size across the two groups of studies became identical.

We were motivated by the study of Jefferson and colleagues to re-
analyse the data reported by Etter and colleagues to investigate the role
of journal Impact Factor. Of the 90 studies included in the original
report (1), we were able to extract Impact Factor information for k = 79
studies. Studies funded by the pharmaceutical industry were published in
journals with a higher Impact Factor (median 4) than those funded by other
sources (median 3), and a Mann-Whitney U-test indicated that this
difference was statistically significant (p = 0.007). Moreover, the effect
size reported in an individual study was positively correlated with the
Impact Factor of the journal within which it was published (Spearman's r =
+0.32, p = 0.004).

It is worth noting, however, that the relationship between effect
size and Impact Factor may extend beyond studies funded by the
pharmaceutical industry. We recently reported evidence from genetic
association studies that the degree to which an individual study over-
estimated the likely true effect size correlated with the Impact Factor of
the journal in which it was published (2), as well as the geographical
region from which the study originated (3). Our conclusion is that certain
features of the scientific environment may influence the apparent strength
of evidence for reported relationships, and this itself is an important
area for future research.

References

1. Etter JF, Burri M, Stapleton J. The impact of pharmaceutical
company funding on results of randomized trials of nicotine replacement
therapy for smoking cessation: a meta-analysis. Addiction 2007;102(5):815-
22.

2. Munafo MR, Stothart G, Flint J. Bias in genetic association
studies and impact factor. Mol Psychiatry 2009;14(2):119-20.

3. Munafo MR, Attwood AS, Flint J. Bias in genetic association
studies: effects of research location and resources. Psychol Med
2008;38(8):1213-4.

Competing interests:
MM has received nicotine replacement products from GlaxoSmithKline and Pfizer for distribution to study participants. He has received consulting fees from the European Commission, the National Audit Office, the American Institutes for Research and G-Nostics Ltd, and honoraria for invited lectures from GlaxoSmithKline, Novartis and Sepracor. RS has no conflicts of interest to declare.