We should be cautious about several potential methodological pitfalls
We read the recent article by Feng Q et al. in The BMJ and greatly appreciate their efforts to prove that effects of quadruple combination antiretroviral therapy were not better than triple combination antiretroviral therapy in treatment naive people with HIV. Nevertheless, we should be cautious about several potential methodological pitfalls.
Firstly, the authors stated that the meta-analysis was reported following the PRISMA statement. But we found that this pre-defined protocol was still not registered in any platforms (such as International Prospective Register of Systematic Reviews (PROSPERO), the Cochrane Library and BMJ Open), which may influence its potential transparency and reproducibility in the process and increase the risk of selective reporting bias.[2, 3]
Secondly, the recognition and the processing of publication bias should be an imperative step in the meta-analysis. But the authors claimed, “We planned to assess potential publication bias by funnel plots and Egger’s test but actually did not do so, because the number of studies included in every meta-analysis was fewer than 10”. Actually, Harbord test, not Egger’s test, may better be routinely performed to assess the publication bias for binary outcomes.
Thirdly, it is obvious that “I2=0” is shown in the authors’ Fig 5 and Fig 6, so why not apply the fix-effects models to these meta-analyses in principle?
Last but not least, trial sequential analysis (TSA) was strongly advocated to be performed to assess uncertainty and imprecision in meta-analyses with sparse data..[6, 7] For this meta-analysis based on a limited number of studies, TSA would be an excellent choice to monitor potential random error, false positives, and false negatives.
1 Feng Q, Zhou A, Zou H, et al. Quadruple versus triple combination antiretroviral therapies for treatment naive people with HIV: systematic review and meta-analysis of randomised controlled trials. BMJ 2019;366:l4179. doi:10.1136/bmj.l4179
2 Lefebvre C, Eisinga A, McDonald S, Paul N. Enhancing access to reports of randomized trials published world-wide--the contribution of EMBASE records to the Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library. Emerg Themes Epidemiol 2008;5:13. doi:10.1186/1742-7622-5-13
3 Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009;339:b2700. doi:10.1136/bmj.b2700
4 Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. www.cochrane-handbook.org.
5 Harbord RM, Egger M, Sterne JAC. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Statistics in Medicine 2006;25:3443-3457. doi:10.1002/sim.2380
6 Holst LB, Petersen MW, Haase N, Perner A, Wetterslev J. Restrictive versus liberal transfusion strategy for red blood cell transfusion: systematic review of randomised trials with meta-analysis and trial sequential analysis. BMJ 2015;350:h1354. doi:10.1136/bmj.h1354
7 Shah A, Smith AF. Trial sequential analysis: adding a new dimension to meta‐analysis. Anaesthesia 2019. Published online May 20. doi:10.1111/anae.14705. [Epub ahead of print]
Competing interests: No competing interests