The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study
BMJ 2009; 339 doi: https://doi.org/10.1136/bmj.b3244 (Published 07 September 2009) Cite this as: BMJ 2009;339:b3244
- Eveline Nüesch, research fellow12,
- Sven Trelle, associate director12,
- Stephan Reichenbach, senior research fellow13,
- Anne W S Rutjes, senior research fellow14,
- Elizabeth Bürgi, research fellow5,
- Martin Scherer, professor of health services research67,
- Douglas G Altman, professor of statistics in medicine8,
- Peter Jüni, head of division12
- 1Institute of Social and Preventive Medicine, University of Bern, Switzerland
- 2CTU Bern, Bern University Hospital, Switzerland
- 3Department of Rheumatology, Immunology and Allergology, Bern University Hospital, Switzerland
- 4Department of Clinical Pharmacology and Epidemiology, Consorzio Mario Negri Sud, Santa Maria Imbaro, Chieti, Italy
- 5Department of Internal Medicine, Bern University Hospital, Switzerland
- 6Department of General Practice, University of Göttingen, Germany
- 7Institute of Social Medicine, University of Luebeck, Germany
- 8Centre for Statistics in Medicine, University of Oxford, Oxford
- Correspondence to: P Jüni juni{at}ispm.unibe.ch
- Accepted 23 March 2009
Abstract
Objective To examine whether excluding patients from the analysis of randomised trials are associated with biased estimates of treatment effects and higher heterogeneity between trials.
Design Meta-epidemiological study based on a collection of meta-analyses of randomised trials.
Data sources 14 meta-analyses including 167 trials that compared therapeutic interventions with placebo or non-intervention control in patients with osteoarthritis of the hip or knee and used patient reported pain as an outcome.
Methods Effect sizes were calculated from differences in means of pain intensity between groups at the end of follow-up, divided by the pooled standard deviation. Trials were combined by using random effects meta-analysis. Estimates of treatment effects were compared between trials with and trials without exclusions from the analysis, and the impact of restricting meta-analyses to trials without exclusions was assessed.
Results 39 trials (23%) had included all patients in the analysis. In 128 trials (77%) some patients were excluded from the analysis. Effect sizes from trials with exclusions tended to be more beneficial than those from trials without exclusions (difference −0.13, 95% confidence interval −0.29 to 0.04). However, estimates of bias between individual meta-analyses varied considerably (τ2=0.07). Tests of interaction between exclusions from the analysis and estimates of treatment effects were positive in five meta-analyses. Stratified analyses indicated that differences in effect sizes between trials with and trials without exclusions were more pronounced in meta-analyses with high between trial heterogeneity, in meta-analyses with large estimated treatment benefits, and in meta-analyses of complementary medicine. Restriction of meta-analyses to trials without exclusions resulted in smaller estimated treatment benefits, larger P values, and considerable decreases in between trial heterogeneity.
Conclusion Excluding patients from the analysis in randomised trials often results in biased estimates of treatment effects, but the extent and direction of bias is unpredictable. Results from intention to treat analyses should always be described in reports of randomised trials. In systematic reviews, the influence of exclusions from the analysis on estimated treatment effects should routinely be assessed.
Footnotes
We thank Sacha Blank, Liz King, Katharina Liewald, Linda Nartey, Rebekka Sterchi, and Beatrice Tschannen for help with data extraction; the authors who provided study data; Malcolm Sturdy for the development and maintenance of the database; and Fitore Sallahaj for data entry.
Contributors: PJ conceived the study and developed the protocol. EN, ST, SR, AWSR, EB, and MS collected the data. EN, ST, and PJ did the analysis and interpreted the analysis in collaboration with SR, AWSR, EB, MS, and DGA. EN, ST, and PJ drafted the manuscript. All authors critically revised the manuscript for important intellectual content and approved the final version of the manuscript. PJ and SR obtained public funding. PJ provided administrative, technical, and logistic support. EN and PJ are the guarantors.
Funding: Swiss National Science Foundation (grant Nos 4053-40-104762/3 and 3200-066378) to PJ and SR, and the Swiss Society of Internal Medicine to PJ. The study was part of the Swiss National Science Foundation’s National Research Programme 53 on musculoskeletal health. SR was a recipient of a research fellowship funded by the Swiss National Science Foundation (grant No PBBEB-115067). MS was supported by a young investigators’ award of the German Ministry of Education and Research (grant No 01 GK 0516). PJ was a PROSPER (programme for social medicine, preventive and epidemiological research) fellow funded by the Swiss National Science Foundation (grant No 3233-066377). The funders had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all data of the study and had final responsibility for the decision to submit for publication. None of the authors is affiliated with or funded by any manufacturer of any intervention used for osteoarthritis.
Competing interests: None declared.
Ethical approval: Not required.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.