Publication bias skewed results of anxiety drug treatment trials, study finds

Publication bias, outcome reporting bias, and spin led to an “overly positive representation” of results from trials of second generation antidepressants for treating anxiety disorders, a study in JAMA Psychiatry has found.1

In the trial Annelieke M Roest and colleagues, of University Medical Center Groningen in the Netherlands, studied the results of phase II and III double blind, placebo controlled clinical trials that were submitted for review by the US Food and Drug Administration in pursuit of marketing approval for second generation antidepressants for five anxiety disorders, comparing them with results that appeared in journal articles.

Nine drugs were evaluated in the trials: seven selective serotonin reuptake inhibitors (paroxetine hydrochloride, paroxetine controlled release, sertraline hydrochloride, fluoxetine hydrochloride, fluvoxamine maleate, fluvoxamine CR, and escitalopram oxalate) and two serotonin noradrenaline (norepinephrine) reuptake inhibitors (venlafaxine hydrochloride extended release and duloxetine hydrochloride). The five anxiety disorders were generalised anxiety disorder, panic disorder, social anxiety disorder, post-traumatic stress disorder, and obsessive-compulsive disorder.

The researchers looked for evidence of: publication bias, defined as when studies with positive findings are more likely to be published than are those with negative findings; outcome reporting bias, defined as when a study’s “positive” findings are published without the study’s “negative” outcomes or when primary and secondary outcomes are switched to the advantage of the intervention; and spin, defined as when treatments are described by investigators as beneficial even though published results of the primary outcomes are non-significant.

The researchers found a significant difference between trial outcomes reviewed by the FDA and the results published in scientific journals. The findings in only 41 of 57 trials (72%) registered with the FDA were positive, but 43 of the 45 published article conclusions (96%) were positive (P<0.001). Of the 16 trials whose results were not positive, only three (19%) were published with results that agreed with their FDA registered results. The researchers wrote, “Overall, trials that the FDA judged as positive were five times more likely to be published in agreement with that decision than were those determined by the FDA to be not-positive trials (risk ratio, 5.20; 95% CI, 1.87-14.45; P<0.001).”.

In the area of study publication bias the researchers found that, from 16 of the 57 trials (28%) that were not positive, seven (44%) were not published, while only 1 of the 41 positive trials (2%) was not published—a difference that was statistically significant (Fisher’s test of exact probability, P<0.001).

In the area of outcome reporting bias the researchers found that, for 3 of the 16 not-positive trials (19%) the published conclusions differed from those in the FDA review and the drug effects were changed from non-significant to statistically significant. “By contrast, outcome reporting bias was found in none of the 41 FDA positive trials. The difference in proportions was statistically significant (Fisher exact test, P=0.02),” the researchers wrote.

Finally, spin was present in articles reporting 3 of the 16 (19%) of the not-positive trials. “Each of these three articles reported that the primary end point was nonsignificant in the results section but, in the abstract, concluded that the trial was positive,” the researchers wrote.

When researchers conducted meta-analyses to compare the results of the FDA data with those appearing in the scientific literature they found that the effect size in the published literature represented a 15% increase in the effect size when compared with the value derived from the FDA data. However, this difference was not statistically significant by meta-regression (β=0.04 (95% confidence interval −0.02 to 0.10); t=1.36; P=0.18).

The researchers concluded that, although most trials for the FDA approved second generation antidepressants for anxiety disorders were positive, reporting bias gave a more positive representation of the trials’ findings in the scientific literature.

• Download figure
• Open in new tab
• Download powerpoint