EDITOR—The meta-analysis of inhaled salmeterol compared with inhaled steroids by Shrewsbury et al1 is a fine counter example to recent controversial claims that the standard of industry sponsored meta-analyses in asthma is inferior to that of other meta-analyses.2 Nevertheless, the paper raises one general difficulty with the reporting of meta-analyses.

In referring to the quality of individual studies the authors write: “In all studies, appropriate methods were used for summarising and comparing treatments, and methods for handling missing data were preplanned” (p 1368). If, as was almost certainly the case, the studies were carried out to the pharmaceutical industry's standards, the analyses will also have been preplanned as required by the guidelines of the International Conference on Harmonisation.3 It does not follow, however, that this is necessarily the analysis that found its way into print, or even the one that was used for this meta-analysis.

The version published by investigators is often inferior, as regards statistical analysis, to that presented to the regulator. This is because editors are less exacting and readers are less tolerant when it comes to statistics than are regulators; sponsors cannot control publications to the same extent as they do trial reports.

It seems unlikely that the scale on which Shrewsbury et al chose to summarise outcomes will have been the same as that in the original trials, which may have varied from trial to trial. This raises the question of prespecification of methods for meta-analysis. For example, the authors chose the binary probability scale for the outcome “exacerbation.” But the original scale was “ordered categorical,” which suggests logistic regression. Furthermore, a summary on the log-odds scale (the usual default) for the data presented in their paper would disagree with their finding that salmeterol is significantly better than steroid. For example, a fixed …