Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples

Organisations such as the National Institute for Health and Care Excellence require the synthesis of evidence from existing studies to inform their decisions—for example, about the best available treatments with respect to multiple efficacy and safety outcomes. However, relevant studies may not provide direct evidence about all the treatments or outcomes of interest. Multivariate and network meta-analysis methods provide a framework to address this, using correlated or indirect evidence from such studies alongside any direct evidence. In this article, the authors describe the key concepts and assumptions of these methods, outline how correlated and indirect evidence arises, and illustrate the contribution of such evidence in real clinical examples involving multiple outcomes and multiple treatments

Supplementary material 1: Technical details of multivariate and network random effects metaanalysis models

Standard (univariate) random effects meta-analysis
A standard random effects meta-analysis combines the study estimates of a single parameter of interest (e.g. a treatment effect) in order to estimate the average parameter value (denoted by ) and the standard deviation of the parameter values (denoted by ) across studies. The term 'random' denotes that the true parameter value in an individual study can vary randomly about the average value due to unexplained heterogeneity. If Y i and var(Y i ) denote the parameter estimate and its variance in study i, then assuming within-study and between-study normal distributions, a random effects meta-analysis is: Y i N( ,var(Y i ) + ).

Multivariate random effects meta-analysis
A multivariate random effects meta-analysis generalises the standard approach, by allowing for multiple correlated parameter estimates to be combined per study, typically by assuming withinstudy and between-study multivariate normal distributions. For example, in the fibrinogen example a bivariate random effects meta-analysis will jointly synthesise 'fully' adjusted log hazard ratio estimates (Y iF ) and partially adjusted estimates (Y iP ), whilst accounting for within-study and between-study correlations, to give a summary result for each ( and , say) and corresponding estimates of between-study standard deviation ( and ). Within-study correlation refers to the association between two parameter estimates in the same study, and is caused by the same individuals contributing related data toward each outcome. Between-study correlation indicates the strength of association between true parameter values across studies, and is caused by differences across studies in patient and study characteristics modifying the true values in a related way. In the fibrinogen example, high within-study correlation arises due to the same patient data contributing to both 'fully' and partially adjusted estimates, whilst high betweenstudy correlation arises because studies with a higher than average 'fully' adjusted effect will also have a higher than average partially adjusted effect. Between-study correlations are estimated in the meta-analysis, but within-study correlations need to be obtained for each study prior to fitting the meta-analysis. Options have been proposed for dealing with missing within-study correlations, including an alternative approach to multivariate meta-analysis that models an 'overall' correlation, which is an amalgamation of the within and between-study correlations. 1-3

Network meta-analysis
Network meta-analysis of multiple treatment comparisons can be expressed using various approaches, depending on the data available. If there are only two treatments per trial (i.e. one treatment comparison), then the simplest approach is a standard meta-regression, which regresses the treatment effect estimates against indicator variables that represent particular treatment effects in relation to a chosen reference treatment. This can be extended to a multivariate meta-regression to accommodate trials with multiple treatment comparisons. 4 5 . When the number of events and patients per treatment group are available per study, a hierarchical logistic regression with random effects can be used to model the binomial data directly within trials. Similarly a hierarchical linear regression or Poisson regression with random effects could be used to model continuous outcomes or rates directly. For all approaches, indicator variables are required to model treatment comparisons relative to a particular reference group, and the model framework must preserve the clustering of patients within trials; approaches that ignore this are inappropriate, as they break the original randomisation within each trial. 6-8 7 9 For example, in the logistic regression approach, a separate intercept per trial is needed.

Estimation of the models
We consider the random effects approach more plausible than assuming a fixed (common) effect, and so use random effects models for all our examples. For all the above models, there are many different methods to estimate each and , and subsequent confidence intervals, including frequentist and Bayesian approaches. In the examples within the paper, we used restricted maximum likelihood (REML) within a frequentist (classical) statistical framework, with confidence intervals accounting for the uncertainty of variance estimates, via the 'mvmeta' and 'network' packages within Stata. 10 11 7 8 12-15 Further, mainly for pragmatic reasons, our network metaanalyses assume a common for each treatment comparison. As in a standard meta-analysis, the relative magnitude of heterogeneity can be expressed using multivariate extensions of the Isquared statistic, to give the percentage of the total variability that is due to between-study heterogeneity, 16 and estimates of between-study standard deviation ( ).
An alternative option is a Bayesian approach, which is a natural way to account for all parameter uncertainty, to make predictions and to derive (joint) probabilistic statements regarding the multiple effects of interest. The Bayesian approach combines the likelihood of the observed data with prior probability distributions for the unknown parameters (e.g. treatment effects and between-study variances), to obtain a joint posterior probability distribution of the parameters from which inferences are made. The prior distributions should reflect prior beliefs about possible values of the unknown parameters and can be used to incorporate various sources of uncertainty. It is common for "non-informative" or "flat" prior distributions to be used for unknown parameters, aiming that posterior results should not be influenced by the prior distribution (only the data). However, assessing sensitivity to the choice of such prior distributions is recommended. 17 18 Informative prior distributions based on external (empirical) evidence are also available. 19

Software
Software for multivariate meta-analysis is available in various statistical packages, including mvmeta in Stata 10 and mvmeta in R 20 , and via PROC MIXED in SAS. 21 Network meta-analysis is often implemented in WinBUGS using, for example, code available at http://www.nicedsu.org.uk/. Dedicated software is also available, such as network and network_graphs in Stata 11 22 and netmeta in R. 23 Supplementary material 2: Forest plot for the meta-analyses of the 'fully' adjusted effect of fibrinogen.
Each solid point denotes the hazard ratio estimate for that study, with its size proportional to the number of patients in that study, and the corresponding horizontal line denotes the confidence interval. The centre of the diamond denotes the summary hazard ratio, and the width of the diamond provides its 95% confidence interval.