Influence of trial sample size on treatment effect estimates: meta-epidemiological studyBMJ 2013; 346 doi: https://doi.org/10.1136/bmj.f2304 (Published 24 April 2013) Cite this as: BMJ 2013;346:f2304
- Agnes Dechartres, assistant professor of epidemiology123,
- Ludovic Trinquart, senior statistician14,
- Isabelle Boutron, professor of epidemiology1234,
- Philippe Ravaud, professor of epidemiology and director12345
- 1INSERM, U738, Paris, France
- 2AP-HP (Assistance Publique des Hôpitaux de Paris), Hôpital Hôtel Dieu, Centre d’Épidémiologie Clinique, 75004 Paris, France
- 3Univ Paris Descartes, Sorbonne Paris Cité, Faculté de Médecine, Paris, France
- 4French Cochrane Centre, Paris, France
- 5Columbia University, Mailman School of Public Health, New York, NY, USA
- Correspondence to: A Dechartres
- Accepted 2 April 2013
Objective To assess the influence of trial sample size on treatment effect estimates within meta-analyses.
Design Meta-epidemiological study.
Data sources 93 meta-analyses (735 randomised controlled trials) assessing therapeutic interventions with binary outcomes, published in the 10 leading journals of each medical subject category of the Journal Citation Reports or in the Cochrane Database of Systematic Reviews.
Data extraction Sample size, outcome data, and risk of bias extracted from each trial.
Data synthesis Trials within each meta-analysis were sorted by their sample size: using quarters within each meta-analysis (from quarter 1 with 25% of the smallest trials, to quarter 4 with 25% of the largest trials), and using size groups across meta-analyses (ranging from <50 to ≥1000 patients). Treatment effects were compared within each meta-analysis between quarters or between size groups by average ratios of odds ratios (where a ratio of odds ratios less than 1 indicates larger effects in smaller trials).
Results Treatment effect estimates were significantly larger in smaller trials, regardless of sample size. Compared with quarter 4 (which included the largest trials), treatment effects were, on average, 32% larger in trials in quarter 1 (which included the smallest trials; ratio of odds ratios 0.68, 95% confidence interval 0.57 to 0.82), 17% larger in trials in quarter 2 (0.83, 0.75 to 0.91), and 12% larger in trials in quarter 3 (0.88, 0.82 to 0.95). Similar results were obtained when comparing treatment effect estimates between different size groups. Compared with trials of 1000 patients or more, treatment effects were, on average, 48% larger in trials with fewer than 50 patients (0.52, 0.41 to 0.66) and 10% larger in trials with 500-999 patients (0.90, 0.82 to 1.00).
Conclusions Treatment effect estimates differed within meta-analyses solely based on trial sample size, with stronger effect estimates seen in small to moderately sized trials than in the largest trials.
We thank Mickaël Randrianandrasana for data extraction from the Cochrane reports for the second collection of meta-analyses.
Contributors: AD is the study guarantor, and was involved in the study conception, selection of trials, data extraction, data analysis, interpretation of results, and drafting of the manuscript; LT was involved in the study conception, data analysis, interpretation of results, drafting of the manuscript; IB and PR were involved in the study conception, interpretation of results, and drafting of the manuscript. All authors, external and internal, had full access to all the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.
Funding: This study was funded by an academic grant from the Programme Hospitalier de recherche Clinique Régional (AOR10017). Our team is supported by an academic grant (DEQ20101221475) for the programme “Equipe espoir de la Recherche,” from the Fondation pour la Recherche Médicale. The researchers declare that they are independent from the funders. The funders did not have any role in the study.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Programme Hospitalier de recherche Clinique Régional and the Fondation pour la Recherche Médicale for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: Not required.
Data sharing: No additional data available.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.