Studies included in meta-analysis had heterogenous, not homogenous, results

EDITOR—Jaakkimainen et al's meta-analysis concludes that an improvement in dyspeptic symptoms occurred among patients with non-ulcer dyspepsia in whom Helicobacter pylori was eradicated.1 Unfortunately, there is a small but crucial problem at the heart of the analysis. The authors report that the summary estimates are statistically homogenous, but this is incorrect. In the observational studies the P value of <0.001 indicates massive heterogeneity between the results of the studies included. In the therapeutic trials the P value of 0.046 also indicates heterogeneity.

Meta-analysts faced with such heterogeneity have three choices: they may ignore the heterogeneity and pool the results with a fixed effects model; they may use a random effects model, which takes the heterogeneity into account; or they may decide not to pool the results. In this instance the authors chose to use a fixed effects model despite the heterogeneity. In consequence the confidence intervals of the pooled estimates are narrow and significance is imputed. A random model might well lead to a summary estimate that does not reach significance. The data extracted from the individual trials have not been published in bmj.com so it is not possible to check this.

The authors have carried out a sensitivity analysis of the therapeutic trials and noted that the single regimen trials measured only short term outcomes and were of lower methodological quality. In contrast, the two trials of triple treatment measured outcomes at 12 months and were of higher quality; these trials showed a much smaller summary estimate of eradication, which barely reaches significance (odds ratio 1.4, 95% confidence interval 1.0 to 2.3). There is clinical as well as statistical heterogeneity between these two groups of studies, so a summary estimate that combines both is of doubtful meaning. Thus in my opinion the best option is not to pool …