Kjaergard and Als-Nielsen present interesting data on the
relationship between competing interests and conclusions, but I think they
have missed an opportunity to tell us something even more interesting
about the reasons for that relationship.
The observed association could be because authors of industry
sponsored publications are more likely to draw inappropriately positive
conclusions, or it could be because industry sponsored research is more
likely to reach positive results. This could be for many reasons, perhaps
because pharmaceutical companies are more likely to fund trials if they
believe the results are likely to be positive, or perhaps because industry
sponsored studies, usually being better funded than independent studies,
are more likely to be adequately powered and therefore less likely to
reach a false negative conclusion. The difference between these potential
causes of the relationship between funding and conclusions is important.
The authors' attempts to address this by statistical correction for
sample size seems very crude. What is important is not the sample size of
the study, but its power to detect a difference between treatments. A
study with 200 patients that only needed 150 to have 90% power to detect a
clinically important difference between treatments is clearly a more
highly powered study than one with 500 patients that should have had 1000,
but this would not be captured by the analysis used in the paper.
The study would have given us much more interesting information if we
had been told not only the authors conclusions, but how positive the
results of the studies really were, as assessed independently by someone
blinded to the authors' conclusions and the source of funding. Perhaps
Kjaergard and Als-Nielsen could consider doing this for a future
publication.
Rapid Response:
Separating results and conclusions
Kjaergard and Als-Nielsen present interesting data on the
relationship between competing interests and conclusions, but I think they
have missed an opportunity to tell us something even more interesting
about the reasons for that relationship.
The observed association could be because authors of industry
sponsored publications are more likely to draw inappropriately positive
conclusions, or it could be because industry sponsored research is more
likely to reach positive results. This could be for many reasons, perhaps
because pharmaceutical companies are more likely to fund trials if they
believe the results are likely to be positive, or perhaps because industry
sponsored studies, usually being better funded than independent studies,
are more likely to be adequately powered and therefore less likely to
reach a false negative conclusion. The difference between these potential
causes of the relationship between funding and conclusions is important.
The authors' attempts to address this by statistical correction for
sample size seems very crude. What is important is not the sample size of
the study, but its power to detect a difference between treatments. A
study with 200 patients that only needed 150 to have 90% power to detect a
clinically important difference between treatments is clearly a more
highly powered study than one with 500 patients that should have had 1000,
but this would not be captured by the analysis used in the paper.
The study would have given us much more interesting information if we
had been told not only the authors conclusions, but how positive the
results of the studies really were, as assessed independently by someone
blinded to the authors' conclusions and the source of funding. Perhaps
Kjaergard and Als-Nielsen could consider doing this for a future
publication.
Competing interests: No competing interests