You are here

  • Home
  • Archive
  • Volume 3, Issue 7
  • Methods used to conduct and report Bayesian mixed treatment comparisons published in the medical literature: a systematic review

Article Text

Article menu
XML
Evidence based practice
Research
Methods used to conduct and report Bayesian mixed treatment comparisons published in the medical literature: a systematic review
  1. Diana M Sobieraj1,
  2. Joseph C Cappelleri2,
  3. William L Baker1,
  4. Olivia J Phung1,
  5. C Michael White1,
  6. Craig I Coleman1
  1. 1University of Connecticut/Hartford Hospital Evidence-based Practice Center, Hartford, Connecticut, USA
  2. 2Department of Biostatistics, Pfizer, Groton, Connecticut, USA
  1. Correspondence to Dr Craig I Coleman; ccolema{at}harthosp.org

Abstract

Objectives To identify published closed-loop Bayesian mixed treatment comparisons (MTCs) and to summarise characteristics regarding their conduct and reporting.

Design Systematic review.

Methods We searched multiple bibliographic databases (January 2006–31 July 2011) for full-text, English language publications of Bayesian MTCs comparing the effectiveness or safety of ≥3 interventions based on randomised controlled trials and having at least one closed loop. Methodological and reporting characteristics of MTCs were extracted in duplicate and summarised descriptively.

Results We identified 34 Bayesian MTCs spanning 13 clinical areas. Publication of MTCs increased over the 5-year period; with 76.5% published during or after 2009. MTCs included a mean (±SD) of 35.9±30.1 trials (n=33 459±71 233 participants) and 8.5±4.3 interventions (85.7% pharmacological). Non-informative and informative prior distributions were reported to be used in 44.1% and 8.8% of MTCs, respectively, with the remainder failing to specify the prior used. A random-effects model was used to analyse the networks of trials in 58.5% of MTCs, all using WinBUGS; however, code was infrequently provided (20.6%). More than two-thirds of MTCs (76.5%) also conducted traditional meta-analysis. Methods used to evaluate convergence, heterogeneity and inconsistency were infrequently reported, but from those providing detail, methods appeared varied. MTCs most often used a binary effect measure (85.3%) and ranking of interventions based on probability was common (61.8%), although rarely displayed in a figure (8.8% of MTCs). MTCs were published in 24 different journals with a mean impact factor of 9.20±8.71. While 70.8% of journals imposed limits on word counts and 45.8% limits on the number of tables/figures, online supplements/appendices were allowed in 79.2% of journals. Publication of closed-loop Bayesian MTCs is increasing in frequency, but details regarding their methodology are often poorly described. Efforts in clarifying the appropriate methods and reporting of Bayesian MTCs should be of priority.

  • Statistics & Research Methods
  • mixed treatment comparison
  • network meta-analysis

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/

https://doi.org/10.1136/bmjopen-2013-003111

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Article summary

    Article focus

    • To identify published closed-loop Bayesian mixed treatment comparisons (MTCs) and to summarise characteristics regarding their conduct and reporting.

    Key messages

    • We identified 34 closed-loop Bayesian MTCs spanning 13 clinical areas, published in 24 different journals.

    • Closed-loop Bayesian MTCs are increasing in frequency, but details regarding their methodology are often poorly described. Efforts in clarifying the appropriate methods and reporting of Bayesian MTCs should be of priority.

    Strengths and limitations of this study

    • Our systematic review adds to this existing literature by updating results and adding new information as prior reviews only included literature through 2007/2008. Unlike prior publications, our systematic review focused only on Bayesian MTCs of networks with at least one closed loop.

    • Unlike prior reviews, we evaluated reporting of additional model characteristics in depth including testing for model fit, evaluation of convergence, adjustment for covariates or multiarm trials, the specific priors used and availability of the code and aggregated study-level data.

    • An important limitation of our review is that we cannot say with certainty that a lack of reporting means a given method or analysis was not undertaken (ie, the testing for convergence or inconsistency need not be described in a paper for it to have been performed by the investigators) or that the reporting of a piece of data or statistical code was not considered.

    Introduction

    Clinicians and decision-makers often need to select from multiple available interventions when determining the optimal treatment for a disease. Ideally, high-quality randomised controlled trials (RCTs) that estimate the effectiveness of all possible interventions directly against one another would be available to guide decision-making.1–4 However, interventions are commonly compared with placebo or non-active control in RCTs rather than another active intervention. When direct comparative trials are completed, they typically include only two interventions from a larger group of possible treatments. As such, decision-makers are faced with a lack of adequate direct comparative data with which to make their judgements.

    In the absence of head-to-head trials, indirect comparisons may provide valuable information. For example, if two different interventions have been evaluated against a common comparator, the comparative effects of the two interventions versus each other can be estimated indirectly.1 ,2 Even in the presence of head-to-head data, indirect comparisons may add value by improving precision of treatment effect estimates.

    Methodologies exist to indirectly compare interventions, as do modes to implement such methodologies.1 ,5–8 In the simplest form, interventions that are evaluated against a common comparator in separate trials can be compared using an anchored-indirect treatment comparison approach.5 As a generalisation of indirect comparisons, when more than two treatments are being compared indirectly, and at least one pair of treatments is being compared directly as well as indirectly (a closed loop is present), both direct and indirect types of data can be used to estimate effects in a mixed treatment comparison (MTC) meta-analysis using a Bayesian or frequentist framework.1–8 Prior research has attempted to categorise the use of indirect comparisons in the medical literature, but either did not included Bayesian MTCs or collected limited data on this approach.9 ,10 The Agency for Healthcare Research and Quality commissioned us to evaluate how MTCs in published systematic reviews are conducted and reported.11 We present the findings of our systematic review identifying closed-loop MTCs using a Bayesian framework and descriptively summarise their methodological and reporting characteristics.

    Methods

    A systematic literature search was conducted in MEDLINE, the Centre for Reviews and Dissemination Databases (including the Database of Abstracts and Reviews of Effects, Health Technology Assessment and the National Institute for Health Research Economic Evaluation Database), The Cochrane Library and the American College of Physicians Journal Club from 1 January 2006 through 31 July 2011. The search strategy in online supplementary appendix S1 was used. Manual additions were permitted based on the citations identified by the literature search.

    Two independent investigators assessed citations for inclusion in a parallel manner based on a priori defined criteria. Specifically, we included meta-analyses that compared the clinical effectiveness or safety of interventions (any pharmacological (including placebo and different doses), behavioural or procedural interventions) based on RCTs, utilised a Bayesian approach to conduct MTC, had at least one closed loop (see online supplementary appendix S2) and were published in full-text and in the English language. There has been inconsistency in what constitutes a MTC in the medical literature12; therefore, for the purposes of this systematic review a MTC was defined as the comparison of three or more interventions in which direct as well as indirect evidence was used. Methodological publications that presented MTCs for illustrative purposes and cost-effectiveness analyses were not considered in this systematic review, nor were individual patient data meta-analyses.

    Two reviewers independently extracted data with disagreements resolved through discussion. For each included closed-loop Bayesian MTC, all published material including the manuscript, supplements, appendices or external websites which the reader of the article was referred to for additional information were used during data extraction. Therefore, the extraction of data was predicated on the reporting of the information by the authors within these sources. When extracting data, we recorded what the authors reported without ourselves judging whether the methods were appropriate or not. If there was insufficient data from all available sources, we indicated ‘not reported’ for that criterion on data extraction.

    General characteristics of each MTC were extracted including author and funding information, if a methodologist was an author, the number and type of intervention comparisons made, number of printed pages and use of supplement or appendix, the number of trials and patients in the analyses, clinical area (eg, cardiology and endocrinology) and the network pattern. For the purpose of this project, we defined a methodologist as an individual having an affiliation with a department of statistics, biostatistics, epidemiology, clinical epidemiology or public health services, as determined by author information and affiliations listed in the publication.13 The country in which a review was conducted was determined by the corresponding author's affiliation.

    The network pattern3 ,4 ,11 ,14 was determined by figures presented within the identified publication. If a figure was not available, we determined the pattern based on text descriptions of included trials.

    We also extracted information regarding the methodology used to conduct the closed-loop Bayesian MTC including the models applied (eg, fixed vs random effects), description of model parameters (eg, choices of prior distributions), methods for assessment of model fit, potential bias, inconsistency and heterogeneity, use of covariate adjustment in models, whether the model accommodated multiarm trials, software utilised and availability of code.

    Finally, we extracted data concerning the reporting of results including the type of endpoint (eg, binary vs continuous), effect size and measure of variance, use of other methods to report results (eg, probability of treatment being best, claims of equivalence or non-inferiority) and the format/presentation of results (eg, text, tables and figures). Characteristics of the journals in which included MTCs were published were collected, including journal name, impact factor, allowance of supplements or appendices, and limitations on word, table and figure counts.

    The characteristics of the closed-loop Bayesian MTCs and journals were summarised descriptively. Categorical data are presented using frequencies and continuous data as means±SDs.

    Results

    A total of 626 citations were identified through the database searches with an additional five MTCs identified through manual review (figure 1). After full text review, 35 articles representing 34 unique closed-loop Bayesian MTCs were included.15–49 The publication by Orme et al25 analysed two distinct networks of RCTs.

    Figure 1
    Figure 1

    Flow diagram of citation inclusion and exclusion.

    The rate of publication of closed-loop Bayesian MTCs increased over the 5-year search period, with 26 (76.5%) of the MTCs published between 2009 and 2011 compared with only 8 (23.5%) published prior to 2009. On average, 6.1±4.8 authors were listed per publication and less than half of publications (47.1%) included a methodologist as an author (table 1). The most common country from which authors published MTCs was the UK (35.3%), followed by the USA (11.8%) and Greece (11.8%).

    View this table:
    Table 1

    General characteristics of Bayesian mixed treatment comparisons

    Funding sources for the MTCs included governmental/foundation (29.4%), industry (26.5%) and unfunded (17.6%) with 23.6% not making a statement regarding funding source(s). Only two publications identified an organisational affiliation, one each with the Health Technology Assessment Program and The Cochrane Collaboration. The mean number of printed pages per publication was 16.6±36.3 (range 4–221) and over half published a supplement or appendix. From those that did not publish a supplement or appendix, one publication did not have the option to do so, given journal (or report) specifications.

    There were 13 different categories of disease states evaluated among included MTCs. The mean number of interventions included within the analyses was 8.5±4.3, of which most were pharmacological (85.7%) in nature. The mean number of trials included in the MTCs was 35.9±30.1 and the mean number of patients included was 33 459±71 233 (range 594–324 168).

    The most common model used in closed-loop Bayesian MTCs was a random-effects model (58.5%; table 2). Very few analyses reported information about whether there was adjustment for covariates (25.6%). Of the 28 MTCs that included trials with three or more arms, 10 (35.7%) reported use of an adjustment for multiarm trials. Less than half of all analyses reported testing model fit. Of the 15 analyses that reported testing model fit in some manner, the most common method was residual deviance (40%). More than two-thirds of the MTCs (76.5%) also included a traditional meta-analysis.

    View this table:
    Table 2

    Methods characteristics in Bayesian MTCs

    Closed-loop Bayesian MTCs used WinBUGS software, and two also specified the use of additional software including the BUGS XLA Wrapper and S-Plus. The statistical WinBUGS code was made available to the reader in only 20.6% of cases, most often in an online supplement/appendix (71.4%). Aggregated study-level patient data used in the MTC was frequently made available to the reader and of these 21 analyses (61.8%) it was most commonly published within the manuscript itself (85.7%). Evaluation of convergence was found in 35.3% of analyses, most commonly using the Gelman-Rubin statistic (58.3%).

    Utilised priors were reported as either non-informative (vague or flat) or informative in 44.1% and 8.8% of analyses, respectively. The remaining analyses (47.1%) did not specify the nature of the prior distributions used. It was also uncommon for the actual prior distribution to be reported for the population treatment effect (d) and the between-study SD of population treatment differences across studies (σ); with only 32.4% and 29.4% of MTCs, respectively, reporting this information. Sensitivity analyses based on priors were conducted in 11.8% of MTCs.

    Accompanying traditional meta-analyses were common (61.5%). The most common method used to assess heterogeneity was the I2 statistic (81.3%) followed by the Cochrane Q-statistic (43.8%). Evaluation of heterogeneity within the MTC was less common, reported in only 32.4% of publications. Of these 11 analyses, τ2 (among-study variance of true effects) was used in 54.5% of analyses followed by between-study SD (45.5%) and several other less frequent methods (some MTCs reported multiple means to test for heterogeneity and therefore are counted twice in the numerator).

    Inconsistency between indirect and direct estimates was evaluated in 24 (70.6%) studies. One review reported being unable to evaluate inconsistency due to lack of direct data while the remaining MTCs simply did not comment on inconsistency. The most common method used to evaluate inconsistency was comparing results of the MTC to those of a traditional meta-analysis conducted by the authors simultaneously or a previously published traditional meta-analysis.

    Most analyses (85.3%) reported outcomes that were binary (table 3). Of these 29 analyses, ORs were the most commonly reported effect measure (62.1%), followed by relative risks (17.2%) and HRs (13.8%), among other less frequent measures. Of the 10 (29.4%) analyses that reported continuous outcomes, the weighted-mean difference was the most common effect measure (80%). All analyses reported variance with 95% credible intervals and one also reported SEs. Most analyses did not report if the posterior distribution was the mean or median value (85.3%). Presentation of results varied, although most analyses used multiple media including tables, figures and text.

    View this table:
    Table 3

    Outcomes and results reporting in Bayesian mixed treatment comparisons

    Few analyses (8.8%) presented graphical representations of the posterior distributions of outcomes. Rank-ordering of interventions based on probability statements (including rankograms with the probability of a treatment being best, second best and so on) for a given outcome was reported in 21 (61.8%) of the MTCs. Only one MTC made claims of equivalence and two made claims of non-inferiority, of which two defined the minimally important difference required to make these statements.

    Complete details of each journal in which at least one MTC was published can be found in tables 4 and 5. The 34 MTCs were published in 24 different journals, with a mean impact factor of 9.20±8.71. BMJ published the most MTCs (6 of the 34, 17.6%) followed by Current Medical Research and Opinion (4 of the 34, 11.8%). The majority of journals (70.8%) imposed word count limits and 45.8% imposed table/figure limitations; however, 79.2% of journals allowed online supplements or appendices.

    View this table:
    Table 4

    Aggregate journal characteristics

    View this table:
    Table 5

    Individual journal characteristics

    Discussion

    Meta-analysis has been regarded as the most highly cited study design in health science.50 However, a drawback of the traditional meta-analysis is its ability to compare only two interventions, without the ability to simultaneously evaluate other comparators. This is inconsistent with clinical practice as in many instances there are a variety of interventions that exist and one must decide which is best. The use of statistical methods (including simple approaches as well as MTC meta-analysis) to compare greater than two interventions simultaneously is on the rise within the peer-reviewed literature. As recent as 2005, a search of the medical literature yielded four publications that utilised such methods; while in 2011, the number increased to 57.12 The results of our systematic review also suggest that indirect comparisons, specifically closed-loop Bayesian MTC, have become more prevalent. A recent study found that a median of three studies (IQR 2–6) were included per meta-analysis, with close to 75% of meta-analyses including five or less trials.51 Our results suggest that compared to traditional meta-analyses, closed-loop Bayesian MTCs are larger and more comprehensive. Moreover, identified MTCs were published in a wide variety of journals covering a range of disease states and thus likely to reach a large readership given their collective mean impact factor. However, we found a variety of reporting strategies or a lack of reporting of characteristics that are important to the conduct of closed-loop Bayesian MTC. This may be related to the limited guidance as to how to conduct and report an MTC, a topic which has been extensively reviewed and summarised elsewhere.11

    Prior research by Donegan et al9 has attempted to categorise published indirect comparisons and evaluate their quality, although advanced methods including Bayesian (and frequentist) MTCs were not included. Of the 43 included comparisons, 23 used an anchored indirect approach while others used hypothesis testing, CI overlap and meta-regression methods to draw indirect comparisons. The authors concluded that quality of published indirect comparisons, in particular the assessment of model assumptions and the methods used to do so, were suboptimal. A set of quality criteria were proposed by the authors to be used in future indirect comparisons, specifically evaluating if the method of indirect comparison applied was appropriate, if methods to assess similarity, homogeneity and consistency were stated and if such methods were appropriate, and details of overall interpretation and reporting of results.

    Song et al10 also have systematically reviewed previously published indirect comparisons and, of the 88 identified, found only 18 using ‘network or Bayesian approaches’. Their findings are similar to that of Donegan and colleagues, suggesting that the main methodological problems included unclear understanding of assumptions, incomplete inclusion of relevant studies, flawed or inappropriate methods, lack of similarity assessment and inappropriate combination of direct and indirect evidence.

    Our systematic review adds to this existing literature by updating results and adding new information. First, the aforementioned prior reviews only included literature through 2007/2008, making ours the most up-to-date review available. Unlike prior publications, our systematic review focused only on Bayesian MTCs of networks with at least one closed loop, perhaps the most common method utilised of late to analyse complex networks of RCTs. While prior publications focused on the evaluation and reporting of assumptions made within the models, we evaluated additional model characteristics in depth including testing for model fit, evaluation of convergence, adjustment for covariates or multiarm trials, the specific priors used and availability of the code and aggregated study-level data. Despite these differences, however, our findings are consistent with prior research and with the opinion of experts regarding the challenges and concerns around implementing and reporting these more complex statistical methods.10 ,12 ,52 Perhaps clearer guidance as to how to conduct and report these types of meta-analyses will lead to a more optimal and consistent approach.

    While we only characterised the methods and reporting of closed-loop Bayesian MTC in this report, our search strategy was designed to capture MTCs regardless of methodological approach (including frequentist MTC). Of note, only a handful (n=9) of frequentist MTCs were identified in our search, three of which specifically reference using the methods for MTC proposed by Lumley and colleagues, while the others more generically referenced mixed-model approaches.49 ,53–60 This suggests that meta-analysts at present seem to favour a Bayesian approach to MTC, since investigators could have chosen to use either a Bayesian or frequentist method for any of the MTC identified in our search (given all analysed networks with at least one closed loop). Given the relative paucity of frequentist models, we do not describe the characteristics of their methods and reporting in this paper but they can be found elsewhere.11

    An important limitation of our review is that we cannot say with certainty that a lack of reporting means a given method or analysis was not undertaken (ie, the testing for convergence or inconsistency need not be described in a paper for it to have been performed by the investigators) or that the reporting of a piece of data or statistical code was not considered. However, we evaluated word, table and figure limits imposed by journals in which these MTCs were published and our findings do not suggest journal space should be an obstacle to complete reporting. Another limitation is the definition used to describe a methodologist. While this definition has been used by previous researchers in a similar topic area,13 to our knowledge it has not been validated and therefore may not accurately depict the true involvement of an individual who considered themselves a methodologist.

    With the growing publication of Bayesian MTCs in the peer-reviewed literature and the recognised challenges of such methods, its appropriate use and interpretation becomes imperative. Efforts in clarifying the appropriate use and reporting of Bayesian MTC should be of priority.

    References

    1. ↵
      1. Bucher HC,
      2. Guyatt GH,
      3. Griffith LE,
      4. et al
      . The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol 1997;50:683–91.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Lumley T
      . Network meta-analysis for indirect treatment comparisons. Stats Med 2002;21:2313–24.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Jansen JP,
      2. Fleurence R,
      3. Devine B,
      4. et al
      . Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. Value Health 2011;14:417–28.
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Hoaglin DC,
      2. Hawkins N,
      3. Jansen JP,
      4. et al
      . Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. Value Health 2011;14:429–37.
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Sutton A,
      2. Ades AE,
      3. Cooper N,
      4. et al
      . Use of indirect and mixed treatment comparisons for technology assessments. Pharmacoeconomics 2008;26:753–67.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Lu G,
      2. Ades AE
      . Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004;23:3105–24.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Caldwell DM,
      2. Ades AE,
      3. Higgins PT
      . Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ 2005;331:897–900.
      OpenUrlFREE Full Text
    8. ↵
      Health Information and Quality Authority. Guidelines for evaluating the clinical effectiveness of health technologies in Ireland. Dublin: Health Information and Quality Authority, 2011. http://www.hiqa.ie (accessed 28 Dec 2011).
    9. ↵
      1. Donegan S,
      2. Williamson P,
      3. Gamble C,
      4. et al
      . Indirect comparisons: a review of reporting and methodological quality. PLoS ONE 2010;5:e11054.
      OpenUrlCrossRefPubMed
    10. ↵
      1. Song F,
      2. Loke YK,
      3. Walsh T,
      4. et al
      . Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ 2009;338:b1147.
      OpenUrlAbstract/FREE Full Text
    11. ↵
      1. Coleman CI,
      2. Phung OJ,
      3. Cappelleri JC,
      4. et al
      . Use of mixed treatment comparisons in systematic reviews. Methods Research Report. (Prepared by the University of Connecticut/Hartford Hospital Evidence-based Practice Center under Contract No. 290 2007 10067I.) AHRQ Publication No. 12-EHC119-EF. Rockville, MD: Agency for Healthcare Research and Quality. August 2012.
    12. ↵
      1. Salanti G
      . Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Method 2012;3:80–97.
      OpenUrl
    13. ↵
      1. Sung L,
      2. Hayden J,
      3. Greenberg ML,
      4. et al
      . Seven items were identified for inclusion when reporting a Bayesian analysis of a clinical study. J Clin Epidemiol 2005;58:261–8.
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      Canadian Agency for Drugs and technologies in Health. Guidelines for the economic evaluation of health technologies: Canada. 3rd edn. Ottawa: Canadian Agency for Drugs and Technologies in Health, 2006. http://www.cadth.ca (accessed 28 Dec 2011).
    15. ↵
      1. Bangalore S,
      2. Kumar S,
      3. Kjeldsen E,
      4. et al
      . Antihypertensive drugs and risk of cancer: network meta-analyses and trial sequential analysis of 324168 participants from randomized trials. Lancet Oncol 2011;12:65–82.
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Gross JL,
      2. Kramer CK,
      3. Leitao CB,
      4. et al
      . Effect of antihyperglycemics agents added to metformin and a sulfonylurea on glycemic control and weight gain in type 2 diabetes: a network meta-analysis. Ann Intern Med 2011;154:672–9.
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Hartling L,
      2. Fernandes RM,
      3. Bialy L,
      4. et al
      . Steroids and bronchodilators for acute bronchiolitis in the first two years of life: systematic review and meta-analysis. BMJ 2011;342:1714.
      OpenUrl
    18. ↵
      1. Maund E,
      2. McDaid C,
      3. Rice S,
      4. et al
      . Paracetamol and selective and non-selective non-steroidal anti-inflammatory drugs for the reduction in morphine-related side-effects after major surgery: a systematic review. Br J Anaesth 2011;106:292–7.
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. McDaid C,
      2. Maund E,
      3. Rice S,
      4. et al
      . Paracetamol and selective and non-selective non-steroidal anti-inflammatory drugs (NSAIDs) for the reduction of morphine-related side effects after major surgery: a systematic review. Health Technol Assess 2010;14:1–153.
      OpenUrl
    20. ↵
      1. Sciarretta S,
      2. Palano F,
      3. Tocci G,
      4. et al
      . Antihypertensive treatment and development of heart failure in hypertension: a Bayesian network meta-analysis of studies in patients with hypertension and high cardiovascular risk. Arch Intern Med 2011;171:384–94.
      OpenUrlCrossRefPubMed
    21. ↵
      1. Trelle S,
      2. Reichenback S,
      3. Wandel S,
      4. et al
      . Cardiovascular safety of non-steroidal anti-inflammatory drugs: network meta-analysis. BMJ 2011;342:c7086.
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. van de Kerkhof P,
      2. de Peuter R,
      3. Ryttov J,
      4. et al
      . Mixed treatment comparison of a two-compound formulation (TCF) product containing calcipotriol and betamethasone dipropionate with other topical treatments in psoriasis vulgaris. Curr Med Res Opin 2011;27:225–38.
      OpenUrlPubMed
    23. ↵
      1. Van den Bruel A,
      2. Gailly J,
      3. Devriese S,
      4. et al
      . The protective effect of ophthalmic viscoelastic devices on endothelial cell loss during cataract surgery: a meta-analysis using mixed treatment comparisons. Br J Ophthalmol 2011;95:5–10.
      OpenUrlAbstract/FREE Full Text
    24. ↵
      1. Dakin H,
      2. Fidler C,
      3. Harper C
      . Mixed treatment comparison meta-analysis evaluating the relative efficacy of nucleos(t)ides for treatment of nucleos(t)ide-naive patients with chronic hepatitis B. Value Health 2010;13:934–45.
      OpenUrlPubMedWeb of Science
    25. ↵
      1. Orme M,
      2. Collins S,
      3. Dakin H,
      4. et al
      . Mixed treatment comparison and meta-regression of the efficacy and safety of prostaglandin analogues and comparators for primary open-angle glaucoma and ocular hypertension. Curr Med Res Opin 2010;26:511–28.
      OpenUrlCrossRefPubMed
    26. ↵
      1. Phung OJ,
      2. Scholle JM,
      3. Talwar M,
      4. et al
      . Effect of noninsulin antidiabetic drugs added to metformin therapy on glycemic control, weight gain, and hypoglycemia in type 2 diabetes. JAMA 2010;303:1410–18.
      OpenUrlCrossRefPubMed
    27. ↵
      1. Uthman OA,
      2. Abdulmalik J
      . Comparative efficacy and acceptability of pharmacotherapeutic agents for anxiety disorders in children and adolescents: a mixed treatment comparison meta-analysis. Curr Med Res Opin 2010;26:53–9.
      OpenUrlCrossRefPubMed
    28. ↵
      1. Vissers D,
      2. Stam W,
      3. Nolte T,
      4. et al
      . Efficacy of intranasal fentanyl spray versus other opioids for breakthrough pain in cancer. Curr Med Res Opin 2010;26:1037–45.
      OpenUrlCrossRefPubMed
    29. ↵
      1. Walsh T,
      2. Worthington HV,
      3. Glenn A,
      4. et al
      . Fluoride toothpastes of different concentrations for preventing dental caries in children and adolescents. Cochrane Database Syst Rev 2011;(1):CD007868.
    30. ↵
      1. Wandel S,
      2. Juni P,
      3. Tendal B,
      4. et al
      . Effects of glucosamine, chondroitin, or placebo in patients with osteoarthritis of hip or knee: network meta-analysis. BMJ 2010;341:4675.
      OpenUrl
    31. ↵
      1. Wang H,
      2. Huang T,
      3. Jing J,
      4. et al
      . Effectiveness of different central venous catheters for catheter-related infections: a network meta-analysis. J Hosp Infect 2010;76:1–11.
      OpenUrlCrossRefPubMed
    32. ↵
      1. Woo G,
      2. Tomlinson G,
      3. Nishikawa Y,
      4. et al
      . Tenofovir and entecavir are the most effective antiviral agents for chronic hepatitis B: a systematic review and Bayesian meta-analyses. Gastroenterology 2010;139:1218–29.
      OpenUrlPubMedWeb of Science
    33. ↵
      1. Baker WL,
      2. Baker EL,
      3. Coleman CI
      . Pharmacologic treatments for chronic obstructive pulmonary disease: a mixed-treatment comparison meta-analysis. Pharmacotherapy 2009;29:891–905.
      OpenUrlPubMedWeb of Science
    34. ↵
      1. Bansback N,
      2. Sizto S,
      3. Sun H,
      4. et al
      . Efficacy of systemic treatments for moderate to severe plaque psoriasis: systematic review and meta-analysis. Dermatology 2009;219:209–18.
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      1. Cipriani A,
      2. Furukawa TA,
      3. Salanti G,
      4. et al
      . Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis. Lancet 2009;373:746–58.
      OpenUrlCrossRefPubMedWeb of Science
    36. ↵
      1. Edwards SJ,
      2. Lind T,
      3. Lundell L,
      4. et al
      . Systematic review: standard- and double-dose proton pump inhibitors for the healing of severe erosive oesophagitis—a mixed treatment comparison of randomized controlled trials. Aliment Pharmacol Ther 2009;30:547–56.
      OpenUrlPubMed
    37. ↵
      1. Edwards SJ,
      2. Smith CJ
      . Tolerability of atypical antipsychotics in the treatment of adults with schizophrenia or bipolar disorder: a mixed treatment comparison of randomized controlled trials. Clin Ther 2009;31(Pt 1):1345–59.
      OpenUrlPubMed
    38. ↵
      1. Golfinopoulos V,
      2. Pentheroudakis G,
      3. Salanti G,
      4. et al
      . Comparative survival with diverse chemotherapy regimens for cancer of unknown primary site: multiple-treatments meta-analysis. Cancer Treat Rev 2009;35:570–3.
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      1. Manzoli L,
      2. Georgia S,
      3. De Vito C,
      4. et al
      . Immunogenicity and adverse events of avian influenza A H5N1 vaccine in healthy adults: multiple-treatments meta-analysis. Lancet Infect Dis 2009;9:481–92.
      OpenUrl
    40. ↵
      1. Meader N
      . A comparison of methadone, buprenorphine and alpha2 adrenergic agonists for opiod detoxification: a mixed treatment comparison meta-analysis. Drug Alcohol Depen 2010;108:110–14.
      OpenUrlCrossRefPubMedWeb of Science
    41. ↵
      1. Coleman CI,
      2. Baker WL,
      3. Kluger J,
      4. et al
      . Antihypertensive medication and their impact on cancer incidence: a mixed treatment comparison meta-analysis of randomized controlled trials. J Hypertens 2008;26:622–9.
      OpenUrlCrossRefPubMedWeb of Science
    42. ↵
      1. Mauri D,
      2. Polyzos NP,
      3. Salanti G,
      4. et al
      . Multiple-treatments meta-analysis of chemotherapy and targeted therapies in advanced breast cancer. J Natl Cancer Inst 2008;100:1780–91.
      OpenUrlAbstract/FREE Full Text
    43. ↵
      1. Stettler C,
      2. Allemann S,
      3. Wandel S,
      4. et al
      . Drug eluting and bare metal stents in people with and without diabetes: collaborative network meta-analysis. BMJ 2008;337:a1331.
      OpenUrlAbstract/FREE Full Text
    44. ↵
      1. Golfinopoulos V,
      2. Salanti G,
      3. Pavlidis N,
      4. et al
      . Survival and disease-progression benefits with treatment regimens for advanced colorectal cancer: a meta-analysis. Lancet Oncol 2007;8:898–911.
      OpenUrlCrossRefPubMedWeb of Science
    45. ↵
      1. Lam SK,
      2. Owen A
      . Combined resynchronisation and implantable defibrillator therapy in left ventricular dysfunction: Bayesian network meta-analysis of randomised controlled trials. BMJ 2007;335:925.
      OpenUrlAbstract/FREE Full Text
    46. ↵
      1. Nixon R,
      2. Bansback N,
      3. Brennan A
      . The efficacy of inhibiting tumour necrosis factor alpha and interleukin 1 in patients with rheumatoid arthritis: a meta-analysis and adjusted indirect comparisons. Rheumatology 2007;46:1140–7.
      OpenUrlAbstract/FREE Full Text
    47. ↵
      1. Cooper NJ,
      2. Sutton AJ,
      3. Lu G,
      4. et al
      . Mixed comparison of stroke prevention treatments in individuals with nonrheumatic atrial fibrillation. Arch Intern Med 2006;166:1269–75.
      OpenUrlCrossRefPubMedWeb of Science
    48. ↵
      1. Kyrgiou M,
      2. Salanti G,
      3. Pavlidis N,
      4. et al
      . Survival benefits with diverse chemotherapy regimens for ovarian cancer: meta-analysis of multiple treatments. J Natl Cancer Inst 2006;98:1655–63.
      OpenUrlAbstract/FREE Full Text
    49. ↵
      1. Baldwin D,
      2. Woods R,
      3. Lawson R,
      4. et al
      . Efficacy of drug treatments for generalized anxiety disorder: systematic review and meta-analysis. BMJ 2011;342:1199.
      OpenUrl
    50. ↵
      1. Patsopoulos NA,
      2. Analatos AA,
      3. Ioannidis JPA
      . Relative citation impact of various study designs in the health sciences. JAMA 2005;293:2362–6.
      OpenUrlCrossRefPubMedWeb of Science
    51. ↵
      1. Davey J,
      2. Turner RM,
      3. Clarke MJ,
      4. et al
      . Characteristics of meta-analyses and their component studies in the Cochrane Database of Systematic Reviews: a cross-sectional, descriptive analysis. BMC Med Res Methodol 2011;11:160.
      OpenUrlCrossRefPubMed
    52. ↵
      1. Li T,
      2. Puhan MA,
      3. Vedula SS,
      4. et al
      . Ad Hoc Network Meta-analysis Methods Meeting Working Group. Network meta-analysis-highly attractive but more methodological research is needed. BMC Med 2011;9:79.
      OpenUrlCrossRefPubMed
    53. ↵
      1. Freemantle N,
      2. Lafuente-Lafuente C,
      3. Mitchell S,
      4. et al
      . Mixed treatment comparison of dronedarone, amiodarone, sotalol, flecainide, and propafenone, for the management of atrial fibrillation. Europace 2011;13:329–45.
      OpenUrlAbstract/FREE Full Text
    54. ↵
      1. Anothaisintawee T,
      2. Attia J,
      3. Nickel JC,
      4. et al
      . Management of chronic prostatitis/chronic pelvic pain syndrome: a systematic review and network meta-analysis. JAMA 2011;305:78–86.
      OpenUrlCrossRefPubMed
    55. ↵
      1. Hansen RA,
      2. Gaynes BN,
      3. Gartlehner G,
      4. et al
      . Efficacy and tolerability of second-generation antidepressants in social anxiety disorder. Int Clin Psychopharmacol 2008;23:170–9.
      OpenUrlPubMed
    56. ↵
      1. Jalota L,
      2. Kalira V,
      3. George E,
      4. et al
      . Prevention of pain on injection of propofol: systematic review and meta-analysis. BMJ 2011;342:1110.
      OpenUrl
    57. ↵
      1. Trikalinos TA,
      2. Alsheikh-Ali AA,
      3. Tatsioni A,
      4. et al
      . Percutaneous coronary interventions for non-acute coronary artery disease: a quantitative 20-year synopsis and a network meta-analysis. Lancet 2009;373:911–18.
      OpenUrlCrossRefPubMedWeb of Science
    58. ↵
      1. Elliott WJ,
      2. Meyer PM
      . Incident diabetes in clinical trials of antihypertensive drugs: a network meta-analysis. Lancet 2007;369:201–7.
      OpenUrlCrossRefPubMedWeb of Science
    59. ↵
      1. Singh JA,
      2. Wells GA,
      3. Christensen R,
      4. et al
      . Adverse effects of biologics: a network meta-analysis and Cochrane overview (review). Cochrane Database Syst Rev 2011;2:CD008794.
    60. ↵
      1. Roskell NS,
      2. Lip GYH,
      3. Noack H,
      4. et al
      . Treatments for stroke prevention in atrial fibrillation: a network meta-analysis and indirect comparison versus dabigatran etexilate. Thromb Haemost 2010;104:1106–15.
      OpenUrlCrossRefPubMedWeb of Science

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors DMS, JCC, CIC, WLB, OJP and CMW were responsible for study design. DMS, WLB and OJP were responsible for data collection. DMS, CIC and JCC were responsible for data analysis and interpretation. All authors contributed in drafting the manuscript, revising the manuscript and approved the final manuscript. CIC is responsible for the overall content as the corresponding author.

    • Funding This work was supported by the Agency for Healthcare Research and Quality contract number HHSA 290 2007 10067 I.

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Individual study data that has been extracted can be found by accessing the full report on the AHRQ EHC website.