Intended for healthcare professionals

Papers

Association between competing interests and authors' conclusions: epidemiological study of randomised clinical trials published in the BMJ

BMJ 2002; 325 doi: https://doi.org/10.1136/bmj.325.7358.249 (Published 03 August 2002) Cite this as: BMJ 2002;325:249

Authors' response

As previously described, we used a six-point scale to assess authors'
conclusions in trials with and without competing interests.[1] The scale
we used for assessing authors' conclusions was originally developed by
Gilbert JP et al in 1977.[2] A modified version of the scale was used by
Colditz GA et al in 1989[3] and Djulbegovic B et al in 2000.[4] In our
paper, we include a figure showing authors' conclusions (means and
confidence intervals) in trials stratified by competing interests. The
figure presents the original data without logarithmic transformation. The
confidence intervals are therefore symmetrical. We used analysis of
variance (ANOVA[5]) to estimate the association between competing
interests and authors' conclusions. In the ANOVA, authors' conclusions
were log transformed. The results were described by the r2, mean
differences, standard error of mean differences, and level of
significance. To comply with William P Plummers’ suggestions, we here also
present the results by the F statistics, degrees of freedom, and P values
adjusted for multiple comparisons. With four independent comparison
groups, the level of significance can be set to P=0.0125 with the
Bonferroni method.[6] Accordingly, our analyses show a significant
association between competing interests and authors' conclusions without
adjustments (F=5.78 with 3 degrees of freedom; P=0.001) and with
adjustments for potential confounding factors (F=5.17 with 3 degrees of
freedom; P=0.002). Kruskal Wallis test also shows that authors'
conclusions are significantly different in trials stratified by competing
interests (H=20.59 with 3 degrees of freedom; P<_0.001. accordingly="accordingly" the="the" robustness="robustness" of="of" results="results" anova="anova" can="can" be="be" confirmed="confirmed" in="in" non-="non-" parametric="parametric" testing.="testing." p="p"/> William P Plummer is correct in pointing out that we have misspelled
the intraclass correlation coefficient.[1]

We assessed authors' conclusions without and with blinding of
competing interests.[1] Consensus was achieved in all cases before the
analyses and blinding was maintained until after completion of the
analyses. Reliability analyses showed god agreement between the unblinded
and blinded assessment of authors' conclusions. Unlike the assessment of
authors' conclusions, the extraction of competing interests was not based
on subjective assessment. In (65 of 159) trials, authors reported
financial or other competing interests. Incorrect registration of this
information could have occurred by mistake or intentionally. We estimated
that unintentional mistakes would not be affected by blinding and that we
would not make intentional mistakes. Accordingly, we did not blind the
extraction of competing interests. To comply with William P Plummers’
critique, we have analysed the association between competing interests and
authors' conclusions assessed in a blinded manner. This analysis confirmed
that competing interests were significantly associated with authors'
conclusions (Kruskal-Wallis H=17.25 with 3 degrees of freedom; P=0.001).
Accordingly, we are not convinced that the risk of bias is large enough to
dismiss the results of our study. We do, however, acknowledge that
additional larger studies are needed to confirm our results and to
estimate the generalisability of our findings.

Adam Jacobs is correct in pointing out that we did not include the
quantitative results of the trials in our analyses. Accordingly, it is
unclear whether the association between competing interests and authors'
conclusions reflects the quantitative results. We did not perform this
analysis due to the diversity of the included trials. For example, the
quantitative results of trials on interventions for cardiovascular and
allergic diseases cannot be compared. We agree that preferential funding
of trials with a high possibility of positive results may exist. However,
we were unable to address this question, as only published trials were
included. Additional studies are needed to estimate whether competing
interests are associated with the results, interpretation of results,
selective funding, or selective publication. We recognise that the
statistical power of trials depends on the number of included patients and
events. From each of the 159 trials, we extracted the number of included
patients and authors' sample size calculations (reflecting the expected
event rate). We also extracted whether the required sample size was
reached. The required sample size was not reached in 20 (13%) trials [1].
None of these trials were terminated preliminary due to differences
between experimental and control groups. These potential confounding
factors did not explain the association between competing interests and
authors' conclusions. Accordingly, the association between competing
interests and authors' conclusions does not seem to reflect differences in
statistical power.

The association between funding and authors' conclusions in published
trials has previously been addressed.[4,7,8] We agree with Benjamin
Djulbegovic that it would also be interesting to explore the relation
between competing interests and authors' conclusions in trials that were
rejected by the BMJ. However, it is not clear to us how this can form
basis for an assessment of authors' tendency to submit their work to high
impact journals. We suggest that such an assessment would also require
information about trials that were not submitted for publication.
Furthermore, we are not convinced that ‘self pervasive bias’ or ‘editors
bias’ can explain our results. First, it is not clear to us why competing
interests would affect authors' tendency to submit their best work to high
impact journals. Second, it is also not clear to us why editors would be
less likely to publish trials funded by for profit organisations if the
conclusions were negative. Publication bias can occur at the pre-
publication as well as the publication stage. We therefore disagree that
information about rejected trials is the only way we can assess which
factors affect the published body of knowledge. As mentioned by Benjamin
Djulbegovic we have previously had an interesting exchange of materials on
these matters. As always, we appreciate his suggestions and ideas although
we disagree on the best way to assess the question of the denominator
problem.

As pointed out by Ron Law, we acknowledge that the risk of
publication bias in schizophrenia as well as other areas should be
considered.

1. Kjaergard LL, Als-Nielsen B. Association between competing
interests and authors' conclusions: epidemiological study of randomised
clinical trials published in the BMJ. BMJ 2002;325:249

2. Altman DG, Bland JM. Comparing several groups using analysis of
variance. BMJ 1996;312:1472-3.

3. Bland JM, Altman DG. Multiple significance tests: the Bonferroni
method. BMJ 1995;310:170.

4. Gilbert JP, McPeek B, Mosteller F. Statistica and ethics in surgery and
anaesthesia. Science 1977; 198:684-9.

5. Colditz GA, Miller JN, Mosteller F. How study design affects outcomes
in comparisons of therapy. I: Medical. Statistics in Medicine 1989;8:441-
54.

6. Djulbegovic B, Lacevic M, Cantor A, Fields KK, Bennett CL, Adams JR, et
al. The uncertainty principle and industry-sponsored research. Lancet
2000;356:635-8.

7. Wahlbeck K, Adams C. Beyond conflict of interest. Sponsored drug trials
show more-favourable outcomes. BMJ 1999;318:465

8. Thomas PS, Tan K-S, Yates DH. Sponsorship, authorship and
accountability. Lancet 2002; 359: 351 (letter).

Competing interests: No competing interests

09 September 2002
Lise L Kjaergard
consultant physician
Lise L Kjaergard, Bodil Als-Nielsen
Cochrane Hepato-Biliary Group, Copenhagen Trial Unit, Rigshospitalet 2100-DK, Denmark