Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviewsBMJ 2009; 338 doi: https://doi.org/10.1136/bmj.b1147 (Published 03 April 2009) Cite this as: BMJ 2009;338:b1147
All rapid responses
Important under-recognised issue in the use of Bayesian mixed-treatment comparisons for evaluating healthcare interventions: prior sensitivity
In the discussion in the article by Song et al, the authors
appropriately touch upon some methodological problems with mixed-treatment
comparisons (MTC i.e., the combination direct and indirect evidence). We
believe that MTC is an important and valuable form of analysis when
properly conducted, and wish to extend this discussion by pointing out
another important under-recognised issue in Bayesian MTC which, when
neglected, is likely to distort the inferences from MTC.
Non-naïve MTC have predominantly been carried out in the Bayesian
framework. Eighteen out of the 20 non-naive MTC publications identified by
Song et al. used Bayesian methods. Bayesian analysis is well known for
incorporating prior beliefs (priors) in the analysis. Bayesian inferences
are typically sensitive to the choice of priors , yet, only 3 of the 18
Bayesian MTC identified by Song et al. employed prior sensitivity
analysis. We are concerned that this inappropriate practice will continue
in future Bayesian MTC unless awareness is raised among authors.
Prior sensitivity is particularly an issue when data are sparse.
Although diagrams of treatment networks of several treatments and
available head-to-head comparisons often appear impressively informative,
the limitations are often hidden from the non-experienced user. First,
head-to-head treatment comparisons in MTC are typically only informed by a
limited number of trials, and several may not have occurred. Second, in
any evidence network of three treatments, indirect evidence from four
trials is approximately as precise as direct evidence from one trial .
Therefore, MTC evidence will typically be sparse.
Authors of Bayesian MTC typically try to avoid prior sensitivity by
employing ‘vague’ or 'non-informative' priors, which are priors that are
believed to have an ignorable influence on the analysis or represent lack
of prior information on the part of the practitioner. However, in
conventional head-to-head meta-analysis, simulations have demonstrated
that ‘vague’ priors do not always have this property . In MTC, the
situation is likely to be worse because the evidence is often sparse.
Gelman provides a general discussion of non-informative and weakly
informative priors for hierarchical models as alternatives to the priors
currently used in the MTC literature , and these alternatives could
readily be explored as part of any sensitivity analysis.
The body of randomised evidence on medical therapeutic interventions
has grown exponentially and treatment networks which are subjected to meta
-analytic evaluation are becoming increasingly complex. It therefore seems
reasonable to believe that Bayesian MTC methods will play an increasingly
important role in medical decision-making. Collectively, the above
arguments suggest that prior sensitivity analysis should be mandated for
Bayesian MTC to ensure that the interpretations and decisions based on
these analyses are appropriate.
 Spiegelhalter DJ, Abrams KJ, Myles JP. Prior distributions.
Bayesian Approaches to Clinical Trials in
Health-Care Evaluation. Chicester: Wiley; 2004. 139-176.
 Glenny AM, Altman DG, Song F, Sakarovitch C, Deeks J, D'Amico R et al.
Indirect comparisons for
competing interventions. Health Technology Assessment 2005; 9:1-148.
 Lampert PC, Sutton AJ, Burton PR, Abrams KR, Jones DR. How vague is
vague? A simulation study of the impact of the use of vague prior
distributions in MCMC using WinBUGS. Statistics in Medicine 2005;
 Gelman, A. Prior distributions for variance parameters in hierarchical
models. Bayesian Analysis 2006; 1(3):515-533.
Competing interests: No competing interests