Published 7 September 2009, doi:10.1136/bmj.b3244
Cite this as: BMJ 2009;339:b3244

Research

The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study

¹ Institute of Social and Preventive Medicine, University of Bern, Switzerland, ² CTU Bern, Bern University Hospital, Switzerland, ³ Department of Rheumatology, Immunology and Allergology, Bern University Hospital, Switzerland, ⁴ Department of Clinical Pharmacology and Epidemiology, Consorzio Mario Negri Sud, Santa Maria Imbaro, Chieti, Italy, ⁵ Department of Internal Medicine, Bern University Hospital, Switzerland, ⁶ Department of General Practice, University of Göttingen, Germany, ⁷ Institute of Social Medicine, University of Luebeck, Germany, ⁸ Centre for Statistics in Medicine, University of Oxford, Oxford

Objective To examine whether excluding patients from the analysis of randomised trials are associated with biased estimates of treatment effects and higher heterogeneity between trials.

Design Meta-epidemiological study based on a collection of meta-analyses of randomised trials.

Data sources 14 meta-analyses including 167 trials that compared therapeutic interventions with placebo or non-intervention control in patients with osteoarthritis of the hip or knee and used patient reported pain as an outcome.

Methods Effect sizes were calculated from differences in means of pain intensity between groups at the end of follow-up, divided by the pooled standard deviation. Trials were combined by using random effects meta-analysis. Estimates of treatment effects were compared between trials with and trials without exclusions from the analysis, and the impact of restricting meta-analyses to trials without exclusions was assessed.

Results 39 trials (23%) had included all patients in the analysis. In 128 trials (77%) some patients were excluded from the analysis. Effect sizes from trials with exclusions tended to be more beneficial than those from trials without exclusions (difference –0.13, 95% confidence interval –0.29 to 0.04). However, estimates of bias between individual meta-analyses varied considerably (²=0.07). Tests of interaction between exclusions from the analysis and estimates of treatment effects were positive in five meta-analyses. Stratified analyses indicated that differences in effect sizes between trials with and trials without exclusions were more pronounced in meta-analyses with high between trial heterogeneity, in meta-analyses with large estimated treatment benefits, and in meta-analyses of complementary medicine. Restriction of meta-analyses to trials without exclusions resulted in smaller estimated treatment benefits, larger P values, and considerable decreases in between trial heterogeneity.

Conclusion Excluding patients from the analysis in randomised trials often results in biased estimates of treatment effects, but the extent and direction of bias is unpredictable. Results from intention to treat analyses should always be described in reports of randomised trials. In systematic reviews, the influence of exclusions from the analysis on estimated treatment effects should routinely be assessed.

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    What's this?

Relevant Articles

Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study

Lesley Wood, Matthias Egger, Lise Lotte Gluud, Kenneth F Schulz, Peter Jüni, Douglas G Altman, Christian Gluud, Richard M Martin, Anthony J G Wood, and Jonathan A C Sterne
BMJ 2008 336: 601-605. [Abstract] [Full Text] [PDF]

Optimal search strategies for retrieving systematic reviews from Medline: analytical survey

Victor M Montori, Nancy L Wilczynski, Douglas Morgan, R Brian Haynes for the Hedges Team
BMJ 2005 330: 68. [Abstract] [Full Text] [PDF]

Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group

Heloisa P Soares, Stephanie Daniels, Ambuj Kumar, Mike Clarke, Charles Scott, Suzanne Swann, and Benjamin Djulbegovic
BMJ 2004 328: 22-24. [Abstract] [Full Text] [PDF]

Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications

Hans Melander, Jane Ahlqvist-Rastad, Gertie Meijer, and Björn Beermann
BMJ 2003 326: 1171-1173. [Abstract] [Full Text] [PDF]

Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis

Dean Fergusson, Shawn D Aaron, Gordon Guyatt, and Paul Hébert
BMJ 2002 325: 652-654. [Extract] [Full Text] [PDF]

Systematic reviews in health care: Assessing the quality of controlled clinical trials

Peter Jüni, Douglas G Altman, and Matthias Egger
BMJ 2001 323: 42-46. [Extract] [Full Text] [PDF]

What is meant by intention to treat analysis? Survey of published randomised controlled trials

Sally Hollis and Fiona Campbell
BMJ 1999 319: 670-674. [Abstract] [Full Text] [PDF]

Online poll

Find out more on doc2doc >>