We read with interest the systematic review by Brian Hutton et al [1], but there are several statistical issues on network meta-analysis we are concerned about.

Basically, network meta-analysis holds three methodological assumptions [2]: 1) like standard meta-analysis, it is assumed that different trials in every pair-wise comparison in the network are sufficiently homogeneous; 2) the similarity of the relative effects of treatment should be held across the entire set of trials; and 3) the relative effects of direct and adjusted indirect comparisons should be consistent when direct and indirect data are combined together. These three issues of comparability concern different levels of decisions for a research synthesis of clinical trials. The trial similarity assumption for adjusted indirect comparison requires the prerequisite of homogeneity assumption; and the consistency assumption is relevant only if both the homogeneity assumption and the similarity assumption are valid [2]. In this systematic review, the consistency was tested by splitting the nodes but the homogeneity and the similarity were neglected. It is unclear whether these assumptions are fulfilled or not in this study.

This systematic review set a very broad inclusion criteria. No restrictions were set for many essential factors like surgical history, surgical urgency, surgical type, and drug dose. By checking the included studies, we find that the patient baseline characteristics vary greatly among different studies, especially for drug dose. This makes it highly possible that the data analysis results would change when stratify the data by the above factors. We kindly suggest the authors 1) testing the homogeneity by traditional chi-square test or I-square test [3]; 2) assessing the similarity using subgroup analysis or meta-regression; and 3) splitting the nodes in the network model according to dosage to provide a more detailed safety profile of the treatments compared.

Reference
1. Brian Hutton, Lawrence Joseph, Dean Fergusson, C David Mazer, Stan Shapiro, Alan Tinmouth. Risks of harms using antifibrinolytics in cardiac surgery: systematic review and network meta-analysis of randomised and observational studies. BMJ 2012;345:e5798
2. Song F, Loke YK, Walsh T, Glenny AM, Eastwood AJ, Altman DG. Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ 2009;338:b1147.
3. Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011): The Cochrane Collaboration; 2011: www.cochrane-handbook.org

Competing interests: None declared