Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publicationsBMJ 2014; 349 doi: https://doi.org/10.1136/bmj.g4539 (Published 16 July 2014) Cite this as: BMJ 2014;349:g4539
- Benjamin Kasenda, researcher1,
- Stefan Schandelmaier, researcher12,
- Xin Sun, assistant professor34,
- Erik von Elm, senior researcher5,
- John You, associate professor36,
- Anette Blümle, researcher7,
- Yuki Tomonaga, researcher8,
- Ramon Saccilotto, researcher1,
- Alain Amstutz, medical student1,
- Theresa Bengough, researcher5,
- Joerg J Meerpohl, researcher7,
- Mihaela Stegert, researcher1,
- Kelechi K Olu, researcher1,
- Kari A O Tikkinen, adjunct professor3910,
- Ignacio Neumann, researcher311,
- Alonso Carrasco-Labra, researcher312,
- Markus Faulhaber, researcher313,
- Sohail M Mulla, research fellow3,
- Dominik Mertz, assistant professor3614,
- Elie A Akl, associate professor31516,
- Dirk Bassler, professor17,
- Jason W Busse, assistant professor31819,
- Ignacio Ferreira-González, assistant professor20,
- Francois Lamontagne, assistant professor21,
- Alain Nordmann, associate professor1,
- Viktoria Gloy, researcher122,
- Heike Raatz, researcher1,
- Lorenzo Moja, assistant professor23,
- Rachel Rosenthal, associate professor24,
- Shanil Ebrahim, assistant professor31825,
- Per O Vandvik, assistant professor26,
- Bradley C Johnston, assistant professor32728,
- Martin A Walter, assistant professor22,
- Bernard Burnand, professor5,
- Matthias Schwenkglenks, associate professor8,
- Lars G Hemkens, researcher1,
- Heiner C Bucher, professor1,
- Gordon H Guyatt, professor3,
- Matthias Briel, assistant professor1329
- 1Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital of Basel, Basel, Switzerland
- 2Academy of Swiss Insurance Medicine, University Hospital of Basel, Basel, Switzerland
- 3Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
- 4Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, Chengdu, China
- 5Cochrane Switzerland, Institute of Social and Preventive Medicine, Lausanne University Hospital, Lausanne, Switzerland
- 6Department of Medicine, McMaster University, Hamilton, Ontario, Canada
- 7German Cochrane Centre, Institute of Medical Biometry and Medical Informatics, Freiburg University Medical Centre, Freiburg, Germany
- 8Institute of Social and Preventive Medicine, University of Zurich, Zurich, Switzerland
- 9Department of Urology, Helsinki University Central Hospital and University of Helsinki, Helsinki, Finland
- 10Department of Public Health, University of Helsinki, Helsinki, Finland
- 11Department of Internal Medicine, Pontificia Universidad Catolica de Chile, Santiago, Chile
- 12Evidence-Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Santiago, Chile
- 13Department of Family Medicine, McMaster University, Hamilton, Ontario, Canada
- 14Michael G DeGroote Institute for Infectious Diseases Research, McMaster University, Hamilton, Ontario, Canada
- 15Department of Internal Medicine, American University of Beirut, Beirut, Lebanon
- 16Department of Medicine, State University of New York at Buffalo, Buffalo, NY, USA
- 17Department of Neonatolgy, University Hospital Zurich, Zurich, Switzerland
- 18Department of Anesthesia, McMaster University, Hamilton, Ontario, Canada
- 19Michael G DeGroote Institute for Pain Research and Care, McMaster University, Hamilton, Ontario, Canada
- 20Epidemiology Unit, Department of Cardiology, Vall d’Hebron Hospital and CIBER de Epidemiología y Salud Publica, Barcelona, Spain
- 21Centre de Recherche Clinique Étienne-Le Bel and Department of Medicine, Université de Sherbrooke, Sherbrooke, Canada
- 22Institute of Nuclear Medicine, University Hospital Bern, Bern, Switzerland
- 23IRCCS Orthopedic Institute Galeazzi, Milano, Italy
- 24Department of Surgery, University Hospital Basel, Basel, Switzerland
- 25Stanford Prevention Research Center, Stanford University, Stanford, CA, USA
- 26Department of Medicine, Innlandet Hospital Trust-Division Gjøvik, Oppland, Norway
- 27Department of Anesthesia and Pain Medicine, Hospital for Sick Children, Toronto, Ontario, Canada
- 28Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
- 29Department of Clinical Research, University of Basel, Basel, Switzerland
- Correspondence to: M Briel Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, Hebelstrasse 10, 4031 Basel, Switzerland
- Accepted 29 June 2014
Objective To investigate the planning of subgroup analyses in protocols of randomised controlled trials and the agreement with corresponding full journal publications.
Design Cohort of protocols of randomised controlled trial and subsequent full journal publications.
Setting Six research ethics committees in Switzerland, Germany, and Canada.
Data sources 894 protocols of randomised controlled trial involving patients approved by participating research ethics committees between 2000 and 2003 and 515 subsequent full journal publications.
Results Of 894 protocols of randomised controlled trials, 252 (28.2%) included one or more planned subgroup analyses. Of those, 17 (6.7%) provided a clear hypothesis for at least one subgroup analysis, 10 (4.0%) anticipated the direction of a subgroup effect, and 87 (34.5%) planned a statistical test for interaction. Industry sponsored trials more often planned subgroup analyses compared with investigator sponsored trials (195/551 (35.4%) v 57/343 (16.6%), P<0.001). Of 515 identified journal publications, 246 (47.8%) reported at least one subgroup analysis. In 81 (32.9%) of the 246 publications reporting subgroup analyses, authors stated that subgroup analyses were prespecified, but this was not supported by 28 (34.6%) corresponding protocols. In 86 publications, authors claimed a subgroup effect, but only 36 (41.9%) corresponding protocols reported a planned subgroup analysis.
Conclusions Subgroup analyses are insufficiently described in the protocols of randomised controlled trials submitted to research ethics committees, and investigators rarely specify the anticipated direction of subgroup effects. More than one third of statements in publications of randomised controlled trials about subgroup prespecification had no documentation in the corresponding protocols. Definitive judgments regarding credibility of claimed subgroup effects are not possible without access to protocols and analysis plans of randomised controlled trials.
We thank the presidents and staff of participating research ethics committees from Switzerland (Basel, Lausanne, Zurich, Lucerne), Germany (Freiburg), and Canada (Hamilton) for their continuous support and cooperation.
The following are members of the DISCO study group: Benjamin Kasenda (Basel, Switzerland), Stefan Schandelmaier (Basel, Switzerland), Xin Sun (Hamilton, Canada; Chengdu, China), Erik von Elm (Lausanne, Switzerland), John You (Hamilton, Canada), Anette Blümle (Freiburg, German), Yuki Tomonaga (Zurich, Switzerland), Ramon Saccilotto (Basel, Switzerland), Alain Amstutz (Basel Switzerland), Theresa Bengough (Lausanne, Switzerland), Joerg J Meerpohl (Freiburg, Germany), Mihaela Stegert (Basel, Switzerland), Kelechi K Olu (Basel, Switzerland), Kari A O Tikkinen (Hamilton, Canada; Helsinki, Finland), Ignacio Neumann (Hamilton, Canada; Santiago, Chile), Alonso Carrasco-Labra (Hamilton, Canada; Santiago, Chile), Markus Faulhaber (Hamilton, Canada), Sohail M Mulla (Hamilton, Canada), Dominik Mertz (Hamilton, Canada), Elie A Akl (Hamilton, Canada; Beirut, Lebabon; Buffalo, NY), Dirk Bassler (Zurich, Switzerland), Jason W Busse (Hamilton, Canada), Ignacio Ferreira-González (Barcelona, Spain), Francois Lamontagne (Sherbrooke, Canada), Alain Nordmann (Basel, Switzerland), Viktoria Gloy (Basel and Bern, Switzerland), Heike Raatz (Basel, Switzerland), Lorenzo Moja (Milan, Italy), Rachel Rosenthal (Basel, Switzerland), Shanil Ebrahim (Hamilton, Canada, Stanford, CA), Per O Vandvik (Oppland, Norway), Bradley C Johnston (Hamilton and Toronto, Canada), Martin A Walter (Bern, Switzerland), Bernard Burnand (Lausanne, Switzerland), Matthias Schwenkglenks (Zurich, Switzerland), Lars G Hemkens (Basel, Switzerland), Heiner C Bucher (Basel, Switzerland), Gordon H Guyatt (Hamilton, Canada), Matthias Briel (Basel, Switzerland; Hamilton, Canada)
Contributors: BK, EvE, and MB designed the study, collected data, interpreted the results, and wrote the manuscript. BK and SS managed the database and conducted all analyses, which were checked by MB. SS, JY, AB, YK, RS, AA, TB, JJM, MS, KKO, KAOT, IN, AC, MF, SMM, DM, EAA, DB, JWB, IF, FL, AN, VG, HR, LM, RR, SE, XS, POV, BCJ, MAW, MS, and LGH contributed to the data collection. BB, HCB, and GHG provided methodological and logistical support. All authors critically revised the manuscript and approved the final version before submission. BK, SS, EvE, and MB are guarantors.
Funding: This study was funded by the Swiss National Science Foundation (grant 320030_133540/1) and the German Research Foundation (grant EL 544/1-2). MB, AN, VG, HR, LGH, and HCB were supported by Santésuisse and the Gottfried and Julia Bangerter-Rhyner-Foundation. XS was supported by a young investigators award (2013SCU04A37) from Sichuan University, China. During study preparation, EvE was supported by the Brocher Foundation. JWB was funded by a new investigator award from the Canadian Institutes of Health Research and Canadian Chiropractic Research Foundation. DM was a recipient of a research early career award from Hamilton Health Sciences Foundation (Jack Hirsh Fellowship). KAOT was funded by unrestricted grants from the Finnish Cultural Foundation, Finnish Medical Foundation, Jane and Aatos Erkko Foundation, and Sigrid Jusélius Foundation. JY was supported by a research early career award from Hamilton Health Sciences.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: study funding by the Swiss National Science Foundation (grant 320030_133540/1) and the German Research Foundation (grant EL 544/1-2), no financial relationships with any organisations that might have an interest in the submitted work in the previous three years, and no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: This study was approved by the participating research ethics committees, or if no ethical approval was required this was explicitly stated.
Data sharing: No additional data available.
Transparency: The lead author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.