Intended for healthcare professionals

  1. Research
  2. Subgroup analyses in...
  3. Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications
CCBYNC Open access
Research

Subgroup analyses in randomised controlled trials: cohort study on trial protocols and journal publications

BMJ 2014; 349 doi: https://doi.org/10.1136/bmj.g4539 (Published 16 July 2014) Cite this as: BMJ 2014;349:g4539

This article has a correction. Please see:

  1. Benjamin Kasenda, researcher1,
  2. Stefan Schandelmaier, researcher12,
  3. Xin Sun, assistant professor34,
  4. Erik von Elm, senior researcher5,
  5. John You, associate professor36,
  6. Anette Blümle, researcher7,
  7. Yuki Tomonaga, researcher8,
  8. Ramon Saccilotto, researcher1,
  9. Alain Amstutz, medical student1,
  10. Theresa Bengough, researcher5,
  11. Joerg J Meerpohl, researcher7,
  12. Mihaela Stegert, researcher1,
  13. Kelechi K Olu, researcher1,
  14. Kari A O Tikkinen, adjunct professor3910,
  15. Ignacio Neumann, researcher311,
  16. Alonso Carrasco-Labra, researcher312,
  17. Markus Faulhaber, researcher313,
  18. Sohail M Mulla, research fellow3,
  19. Dominik Mertz, assistant professor3614,
  20. Elie A Akl, associate professor31516,
  21. Dirk Bassler, professor17,
  22. Jason W Busse, assistant professor31819,
  23. Ignacio Ferreira-González, assistant professor20,
  24. Francois Lamontagne, assistant professor21,
  25. Alain Nordmann, associate professor1,
  26. Viktoria Gloy, researcher122,
  27. Heike Raatz, researcher1,
  28. Lorenzo Moja, assistant professor23,
  29. Rachel Rosenthal, associate professor24,
  30. Shanil Ebrahim, assistant professor31825,
  31. Per O Vandvik, assistant professor26,
  32. Bradley C Johnston, assistant professor32728,
  33. Martin A Walter, assistant professor22,
  34. Bernard Burnand, professor5,
  35. Matthias Schwenkglenks, associate professor8,
  36. Lars G Hemkens, researcher1,
  37. Heiner C Bucher, professor1,
  38. Gordon H Guyatt, professor3,
  39. Matthias Briel, assistant professor1329
  1. 1Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital of Basel, Basel, Switzerland
  2. 2Academy of Swiss Insurance Medicine, University Hospital of Basel, Basel, Switzerland
  3. 3Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada
  4. 4Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, Chengdu, China
  5. 5Cochrane Switzerland, Institute of Social and Preventive Medicine, Lausanne University Hospital, Lausanne, Switzerland
  6. 6Department of Medicine, McMaster University, Hamilton, Ontario, Canada
  7. 7German Cochrane Centre, Institute of Medical Biometry and Medical Informatics, Freiburg University Medical Centre, Freiburg, Germany
  8. 8Institute of Social and Preventive Medicine, University of Zurich, Zurich, Switzerland
  9. 9Department of Urology, Helsinki University Central Hospital and University of Helsinki, Helsinki, Finland
  10. 10Department of Public Health, University of Helsinki, Helsinki, Finland
  11. 11Department of Internal Medicine, Pontificia Universidad Catolica de Chile, Santiago, Chile
  12. 12Evidence-Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Santiago, Chile
  13. 13Department of Family Medicine, McMaster University, Hamilton, Ontario, Canada
  14. 14Michael G DeGroote Institute for Infectious Diseases Research, McMaster University, Hamilton, Ontario, Canada
  15. 15Department of Internal Medicine, American University of Beirut, Beirut, Lebanon
  16. 16Department of Medicine, State University of New York at Buffalo, Buffalo, NY, USA
  17. 17Department of Neonatolgy, University Hospital Zurich, Zurich, Switzerland
  18. 18Department of Anesthesia, McMaster University, Hamilton, Ontario, Canada
  19. 19Michael G DeGroote Institute for Pain Research and Care, McMaster University, Hamilton, Ontario, Canada
  20. 20Epidemiology Unit, Department of Cardiology, Vall d’Hebron Hospital and CIBER de Epidemiología y Salud Publica, Barcelona, Spain
  21. 21Centre de Recherche Clinique Étienne-Le Bel and Department of Medicine, Université de Sherbrooke, Sherbrooke, Canada
  22. 22Institute of Nuclear Medicine, University Hospital Bern, Bern, Switzerland
  23. 23IRCCS Orthopedic Institute Galeazzi, Milano, Italy
  24. 24Department of Surgery, University Hospital Basel, Basel, Switzerland
  25. 25Stanford Prevention Research Center, Stanford University, Stanford, CA, USA
  26. 26Department of Medicine, Innlandet Hospital Trust-Division Gjøvik, Oppland, Norway
  27. 27Department of Anesthesia and Pain Medicine, Hospital for Sick Children, Toronto, Ontario, Canada
  28. 28Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
  29. 29Department of Clinical Research, University of Basel, Basel, Switzerland
  1. Correspondence to: M Briel Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital Basel, Hebelstrasse 10, 4031 Basel, Switzerland Matthias.Briel{at}usb.ch
  • Accepted 29 June 2014

Abstract

Objective To investigate the planning of subgroup analyses in protocols of randomised controlled trials and the agreement with corresponding full journal publications.

Design Cohort of protocols of randomised controlled trial and subsequent full journal publications.

Setting Six research ethics committees in Switzerland, Germany, and Canada.

Data sources 894 protocols of randomised controlled trial involving patients approved by participating research ethics committees between 2000 and 2003 and 515 subsequent full journal publications.

Results Of 894 protocols of randomised controlled trials, 252 (28.2%) included one or more planned subgroup analyses. Of those, 17 (6.7%) provided a clear hypothesis for at least one subgroup analysis, 10 (4.0%) anticipated the direction of a subgroup effect, and 87 (34.5%) planned a statistical test for interaction. Industry sponsored trials more often planned subgroup analyses compared with investigator sponsored trials (195/551 (35.4%) v 57/343 (16.6%), P<0.001). Of 515 identified journal publications, 246 (47.8%) reported at least one subgroup analysis. In 81 (32.9%) of the 246 publications reporting subgroup analyses, authors stated that subgroup analyses were prespecified, but this was not supported by 28 (34.6%) corresponding protocols. In 86 publications, authors claimed a subgroup effect, but only 36 (41.9%) corresponding protocols reported a planned subgroup analysis.

Conclusions Subgroup analyses are insufficiently described in the protocols of randomised controlled trials submitted to research ethics committees, and investigators rarely specify the anticipated direction of subgroup effects. More than one third of statements in publications of randomised controlled trials about subgroup prespecification had no documentation in the corresponding protocols. Definitive judgments regarding credibility of claimed subgroup effects are not possible without access to protocols and analysis plans of randomised controlled trials.

Introduction

The primary goal of a randomised controlled trial is to determine the benefits and harms of an intervention. However, trial populations are typically heterogeneous for individual patient characteristics such as age, sex, disease severity, or comorbidity. The question therefore arises as to whether effects of an intervention vary across these patient characteristics. Randomised controlled trials commonly report exploration of such possible subgroup effects1 2 3 4 5 and, if conducted appropriately, such exploration can lead to more targeted clinical recommendations, better informed clinical decision making, and improved patient care.6 7 More often, their results are misleading and can have detrimental consequences.8 9

Because subgroup analyses may be either informative or misleading, healthcare providers and policymakers need criteria to differentiate credible from spurious subgroup effects.8 10

Clinical epidemiologists have suggested criteria8 9 11 12 that allow readers to gauge the likelihood that a subgroup effect is real, on a continuum from highly plausible to extremely unlikely.13 All available criteria include the prespecification of subgroup analyses; some additionally include the anticipated direction of the subgroup effect and the use of a statistical test tackling the likelihood that apparent subgroup effects may be explained by chance.8 9 11 12 13

Judging the credibility of a reported subgroup effect relies on the information provided in published articles, because trial protocols are usually not freely accessible. Little is known about the planning of subgroup analyses in trial protocols and the extent to which they are reported in subsequent publications, and, in particular, which claims of prespecification correspond to these descriptions.14 15 Pioneer work by Chan and colleagues16 suggested large discrepancies between protocols and publications, but their sample was limited to 70 protocols of randomised controlled trials from a single centre.

We investigated subgroup planning and reporting based on protocols of randomised controlled trials from six international centres and the corresponding publications. We focused specifically on the agreement between statements about subgroup prespecification in the publication and corresponding statements in the protocols.

Methods

Study design

We used protocols of randomised controlled trials and corresponding publications included in a retrospective cohort study; the rationale and design have been described elsewhere.17 In short, the study examined protocols approved between 2000 and 2003 by six research ethics committees in Switzerland (Basel, Lucerne, Zurich, and Lausanne), Germany (Freiburg), and Canada (Hamilton). We focused on protocols that had been approved 10 or more years ago to ensure that the number of ongoing randomised controlled trials would be limited.18

Eligibility criteria for protocols and subsequent publications

In the present study, we included protocols regardless of publication status. We excluded those of trials that compared different doses or routes of administering the same drug (early dose finding studies), enrolled only healthy volunteers, were never started, or were still ongoing as of April 2013. We included only full (peer reviewed) journal publications from corresponding protocols of randomised controlled trials; we excluded research letters, letters to the editor, or conference abstracts.

Definitions

We defined a subgroup as a subset of all trial participants with distinct characteristics at randomisation (for example, age, sex, stage of disease). We defined a subgroup analysis as an analysis that explored whether intervention effects (experimental versus control) differed according to these characteristics. For protocols, we considered a subgroup analysis as planned if at least one of the following was reported: any statement in the protocol analogous to the definition above (for example, “intervention effects will be investigated according to patient baseline characteristics”); a stratified analysis (for example, “patients will be stratified according to sex and analysed separately”); a test for interaction (that is, interaction between intervention and patient characteristic); or an investigation of effect modifying factors. For publications, we considered a subgroup analysis as reported if the article included at least one of the following: an effect estimate and an associated confidence interval or a P value for one or more subgroups; a difference between effect estimates of different patient subgroups; investigation of potential effect modifiers, or the results from a test for interaction; or an explicit statement that a subgroup analysis had been undertaken. We assessed protocols for industry sponsorship or investigator sponsorship using the following criteria: the protocol clearly named the sponsor, displayed a company or institution logo prominently, mentioned affiliations of authors of the protocol, included statements about data ownership or publication rights, or included statements about full funding by industry or public funding agencies.18

Data extraction process and search for publications

Twelve investigators trained in clinical research methodology independently extracted data from eligible trial protocols and correspondence between the research ethics committees and the local investigators. Thirty per cent of the extractions were done in duplicate as an initial calibration process to maximise the consistency of data extraction across reviewers. If the files of the ethics committee provided no information about the publication status of a trial, we conducted comprehensive searches of electronic databases to find any associated publications; previous publications present details of the searches and data extraction process.17 18 When randomised controlled trials that mentioned any prespecified subgroup analyses in their publications did not mention any subgroup analyses in corresponding protocols, we searched for additional versions of the protocol published in journals, any available analysis plans (from journals, filed documents at research ethics committees, or websites), and information published in trial registries (clinicaltrials.gov, WHO International Clinical Trials Registry Platform). Twenty two investigators trained in clinical research methodology extracted data from all corresponding publications, independently and in duplicate; disagreements were resolved by consensus or by third party adjudication. Protocols and corresponding publications were not extracted by the same person.

Information collected about subgroup analyses

We recorded the number of subgroup analyses planned in protocols and reported in publications. We asked the several questions, guided by criteria for the credibility of subgroup analyses.19 For protocols: any subgroup analyses mentioned? If yes: Any clear hypothesis for the planned subgroup analyses mentioned? Any anticipated direction of a subgroup effect mentioned? Any test for interaction mentioned? How many subgroup analyses were planned?

For publications: does the publication report any subgroup analysis? If yes: Does the publication report that subgroup analyses were prespecified? Does the publication report that subgroup analyses were done post hoc? Does the publication provide a rationale for any subgroup analysis? Does the publication report an anticipated direction of any subgroup effect? Does the publication report any separate power calculation for subgroup analyses? Does the publication report any test for interaction? How many subgroup analyses are reported? Does the publication report any claim about a subgroup effect? We considered a subgroup effect as claimed if the investigators explicitly stated in the abstract or discussion/conclusion that the effect of an intervention was different between subgroups or a clear benefit or harm was seen in one or more subgroups.

Statistical analysis

For binary data we summarised results as frequencies and proportions and for continuous data as medians and interquartile ranges. We considered three analysis sets: a dataset based on all protocols (protocol set), a dataset based on corresponding publications (publication set), and a dataset of publications and matched corresponding protocols (publication-protocol set). We prespecified stratification of our descriptive analyses by sponsorship and hypothesised, based on results reported by Sun and colleagues, that industry sponsored trials more often planned subgroup analyses.1 We examined the difference between these proportions using the χ2 test. We used the statistical programmes R version 2.15.3 (www.r-project.org) and STATA version 13.0 (Stata, College Station, TX, USA) for our analyses.

Results

Planning of subgroup analyses—the protocol set

Of 894 eligible protocols of randomised controlled trials involving patients (figure⇓), 252 (28.2%) planned at least one subgroup analysis. Those trials planning subgroup analysis had on average a larger sample size, were more often multicentre trials, and were from the specialty of cardiovascular medicine (table 1⇓). Industry sponsored trials more often planned subgroup analyses than investigator sponsored trials (195/551 (35.4%) v 57/343 (16.6%), P <0.001). Of the 252 protocols planning at least one subgroup analysis, 17 (6.7%) provided a hypothesis and 10 (4.0%) provided an anticipated direction of a potential subgroup effect (table 2⇓).

Figure1

Study flow of protocols of randomised controlled trials and publications. *Only protocols from two subsidiary research ethics committees responsible for paediatric and surgical randomised controlled trials were screened. †No results from randomised comparison published

Table 1

Trial characteristics based on protocols. Values are numbers (percentages) unless stated otherwise

View this table:
Table 2

Subgroup credibility criteria based on trial protocols that planned at least one subgroup analysis. Values are numbers (percentages) unless stated otherwise

View this table:

Reporting of subgroup analyses—the publication set

For 515 protocols we identified corresponding full journal publications (publication set, figure). Of those, 246 (47.8%) publications reported subgroup analyses. These trials were, on average, larger and more often published in high impact journals than published randomised controlled trials without subgroup reporting (see supplementary table 1). Table 3⇓ summarises the reporting of subgroup credibility criteria and characteristics of subgroup analyses in these full journal publications. Similar to the protocol set, subgroup hypotheses or anticipated directions of subgroup effects were rarely provided. Of 86 publications claiming a subgroup effect, 39 (45.3%) reported the use of an interaction test, 9 (10.5%) provided a subgroup hypothesis, and 5 (5.8%) provided an anticipated direction of effect.

Table 3

Reported subgroup credibility criteria and interpretation of subgroup analyses based on publications that reported at least one subgroup analysis. Values are numbers (percentages) unless stated otherwise

View this table:

Agreement between subgroup reporting in publications and corresponding protocols—the publication-protocol set

Of 515 publications of randomised controlled trials, 132 (25.6%) reported the conduct of subgroup analyses that were not mentioned in the corresponding protocols; 64 (12.4%) publications did not report subgroup analyses that were planned in the corresponding protocols.

Of those 246 publications that reported subgroup analyses, overall 114 (46.3%) corresponding protocols planned at least one subgroup analysis (for industry sponsored trials 86/160 (53.8%), for investigator sponsored trials 28/86 (32.6%)). In those 114 trials, the reported number of subgroup analyses matched the planned number in the protocol in 11 (9.6%) instances. Table 4⇓ summarises the agreements of subgroup credibility criteria for those 246 trials reporting at least one subgroup analysis. In 81 of 246 (32.9%) publications reporting subgroups, authors stated for at least one of their reported subgroup analyses that it was prespecified, but 28 (34.6%) corresponding protocols had not mentioned any planned subgroup analysis. For 12 of these 28 randomised controlled trials, the authors mentioned a separate analysis plan in the publication or the protocol without mentioning subgroup analyses. However, these analysis plans were not made available to readers. We found registered information for 9 (32.1%) of the 28 randomised controlled trials but without any evidence of planned subgroup analyses. Of the 86 publications claiming a subgroup effect, 36 (41.8%) corresponding protocols reported a planned subgroup analysis.

Table 4

Agreement of planning and reporting of subgroup credibility criteria based on those 246 publications reporting at least one subgroup analysis. Numbers are protocols/publications reporting or not reporting subgroup credibility criteria (percentages)

View this table:

Discussion

Our study provides empirical evidence documenting the planning and reporting of subgroup analyses in a sample of 894 randomised controlled trials involving patients, which were approved by six research ethics committees in three countries. About half of the published trials reported the conduct of subgroup analyses, of which only 46% had mentioned any planned subgroup analyses in the corresponding protocols. Industry sponsored randomised controlled trials planned subgroup analyses more often than investigator sponsored trials, but still only half of industry sponsored trials reporting results for subgroups explicitly stated such planned analyses in the protocol. In trials with subgroup analyses mentioned in both the protocol and the publication, the number of subgroup analyses reported in publications matched the number in protocols in only 10%. Investigators rarely provided a rationale for or indicated the anticipated direction of potential subgroup effects in either protocols or reports of randomised controlled trials. Of the journal publications stating that at least one subgroup analysis was preplanned, a third failed to mention any subgroup analysis in the corresponding protocol.

Strengths and limitations of this study

The data for the present study were collected as part of a large international cohort involving six research ethics committees that allowed full access to trial protocols and filed correspondence.17 18 As outlined previously,20 unrestricted access is absolutely necessary (but not always granted) to maintain scientific rigor: asking trialists and sponsors for permission to access their protocols would very likely introduce bias, because those with substandard reporting practices may be less likely to allow additional scrutiny. As further strengths we involved only trained methodologists in data abstraction and performed all data extractions from identified publications independently and in duplicate. Finally, our sample included randomised controlled trials from various disciplines of clinical medicine, thus enhancing generalisability of our results.

Our study has limitations. Firstly, we did not have access to statistical analysis plans that may have had prespecified subgroup analyses not mentioned in the protocol. However, we exhaustively checked all available evidence (published protocols, trial websites, filed documents at research ethics committees, trial registries) for prespecification of subgroup analyses. Nevertheless, our results fail to take into account changes in the protocol that occurred before examination of the data and that were not recorded in any of the above documents. Secondly, we did not systematically extract information from protocols about separate power calculations for subgroup analysis. However, since only 4% of protocols that planned subgroup analysis provided an anticipated direction of a subgroup effect, appropriate power calculations (additionally including an estimate for the magnitude of the subgroup effect) were likely to be even less common. Only 2.4% of publications that mentioned a subgroup analysis reported a corresponding power calculation. Thirdly, we used a convenience sample of six research ethics committees, which were, to our knowledge, not in any way particular. Still, we cannot say whether they are representative of other research ethics committees in their own or other countries. Fourthly, owing to limited resources we used single data extraction for 70% of protocols, thereby potentially increasing errors in extraction. However, we used pre-piloted extraction forms with detailed written instructions, conducted formal calibration exercises with all data extractors, and checked extractions from a random sample of protocols at several points during the process. Agreement was good, with no more than two discrepancies in 30 extracted key variables.18 Fifthly, instead of a formal protocol for the current substudy, we previously published a protocol only of the overall project, mentioning this study without details.17 Therefore we limited hypothesis testing in this study to one prespecified subgroup analysis and we make our data extraction forms reflecting all collected variables available to readers on request. Sixthly, included protocols were approved 10-13 years ago; the planning of subgroup analyses in protocols may have improved since that time.

Comparison with other studies

In an earlier systematic review of 469 randomised controlled trials19 we found that 44% of full text publications reported subgroup analyses, which is consistent with our present finding of 48%. In the previous study, we found that most claimed subgroup effects in randomised controlled trials had low credibility and prespecification was seldom reported. The present study not only confirms this finding, but reveals that, often, the claim of prespecification of subgroups in publications is not supported by the corresponding protocols.

Many previous empirical studies mentioned that justification of subgroup analysis and the statistical methods used were rarely reported.2 3 4 5 14 16 21 22 Of those, only some smaller studies compared grant applications22 or protocols of randomised controlled trials14 16 with publications for information about subgroup analyses and identified considerable discrepancies: Boonacker and colleagues noted that only 11 of 47 (23%) grant proposals for randomised controlled trials were in agreement with publications22; Chan and colleagues found that 25 of 70 (36%) randomised controlled trials reported subgroup analyses in the protocol or in the publication and that there were discrepancies between the two documents for all 25 randomised controlled trials16; and Al-Marzouki and colleagues documented that only 8 of 19 (42%) protocols of randomised controlled trials not mentioning subgroup analyses and 7 of 18 (39%) protocols planning subgroup analyses were consistent with corresponding publications.14 In our sample, numbers of subgroup analyses in protocols and publications were identical in less than 5% (11/246) of randomised controlled trials reporting subgroups. Only Chan and colleagues examined whether reported prespecification of subgroup analyses in publications (7/20, 35%) was supported by planned subgroup analyses in protocols. Four of 7 (57%) randomised controlled trials with reported prespecifications lacked evidence of prespecification in the corresponding protocols.16

Implications for reporting and interpreting subgroup analyses

Current recommendations aim to help readers when judging the credibility of subgroup analyses based on information provided in the publication.9 13 Empirical evidence from comparisons of protocols of randomised controlled trials and publications has been limited.14 16 Our results challenge a key criterion of all previous recommendations—that is, the a priori specification of the subgroup analysis. Given that in one out of three studies’ protocols do not corroborate reported claims of prespecification of subgroup analyses, gains in credibility from this criterion are limited.

The following steps could help to improve the trustworthiness of reported subgroup analyses. Firstly, planned subgroup analyses should be documented in trial registries. To date, however, possibilities to enter such information in trial registries are insufficiently developed. For example, there is a non-mandatory “Group/Cohort” field in the registry clinicaltrials.gov that could be used for subgroup prespecification, but the corresponding data element description remains unclear.23 The WHO International Clinical Trials Registry platform24 and the registry Controlled Clinical Trials25 currently do not enable entry of information about subgroups. Secondly, clinical investigators should adhere to guidelines for protocols of randomised controlled trials such as the SPIRIT statement.26 27 Research ethics committees and other review boards should promote the use of such guidance documents.

Thirdly, journals should request access to protocols or statistical analysis plans for their review process and make these documents accessible to readers. In addition, journals could enforce adherence to guidelines for the reporting of randomised controlled trials (for example, the CONSORT statement)28 to reduce the prevalent incomplete reporting of subgroup analyses. Unless a reliable source such as a comprehensive trial protocol is available, readers of trial reports should consider statements about subgroup prespecifications with scepticism. When judging the credibility of a subgroup effect, readers may look for similar studies instead and consider whether subgroup findings are consistent.

Conclusion

Large discrepancies exist between the planning and reporting of subgroup analyses in randomised controlled trials. Published statements about subgroup prespecification were not supported by study protocols in about a third of cases. Our results highlight the importance of enhancing the completeness and accuracy of protocols of randomised controlled trials and their accessibility to journal editors, reviewers, and readers.

What is already known on this topic

  • Claims of subgroup effects in randomised trials have little credibility

  • Prespecification of subgroup analyses is an important criterion to assess the credibility of subgroup effects in randomised trials

What this study adds

  • Large discrepancies exist between planning of subgroup analyses in protocols and their reporting in publications of randomised trials

  • Statements about subgroup prespecification in journal publications are of low credibility if access to trial protocols and analysis plans are not provided

  • Protocol and registry information of randomised trials should include a statement whether subgroup analyses are planned or not, and if so, should specify them

Notes

Cite this as: BMJ 2014;349:g4539

Footnotes

  • We thank the presidents and staff of participating research ethics committees from Switzerland (Basel, Lausanne, Zurich, Lucerne), Germany (Freiburg), and Canada (Hamilton) for their continuous support and cooperation.

  • The following are members of the DISCO study group: Benjamin Kasenda (Basel, Switzerland), Stefan Schandelmaier (Basel, Switzerland), Xin Sun (Hamilton, Canada; Chengdu, China), Erik von Elm (Lausanne, Switzerland), John You (Hamilton, Canada), Anette Blümle (Freiburg, German), Yuki Tomonaga (Zurich, Switzerland), Ramon Saccilotto (Basel, Switzerland), Alain Amstutz (Basel Switzerland), Theresa Bengough (Lausanne, Switzerland), Joerg J Meerpohl (Freiburg, Germany), Mihaela Stegert (Basel, Switzerland), Kelechi K Olu (Basel, Switzerland), Kari A O Tikkinen (Hamilton, Canada; Helsinki, Finland), Ignacio Neumann (Hamilton, Canada; Santiago, Chile), Alonso Carrasco-Labra (Hamilton, Canada; Santiago, Chile), Markus Faulhaber (Hamilton, Canada), Sohail M Mulla (Hamilton, Canada), Dominik Mertz (Hamilton, Canada), Elie A Akl (Hamilton, Canada; Beirut, Lebabon; Buffalo, NY), Dirk Bassler (Zurich, Switzerland), Jason W Busse (Hamilton, Canada), Ignacio Ferreira-González (Barcelona, Spain), Francois Lamontagne (Sherbrooke, Canada), Alain Nordmann (Basel, Switzerland), Viktoria Gloy (Basel and Bern, Switzerland), Heike Raatz (Basel, Switzerland), Lorenzo Moja (Milan, Italy), Rachel Rosenthal (Basel, Switzerland), Shanil Ebrahim (Hamilton, Canada, Stanford, CA), Per O Vandvik (Oppland, Norway), Bradley C Johnston (Hamilton and Toronto, Canada), Martin A Walter (Bern, Switzerland), Bernard Burnand (Lausanne, Switzerland), Matthias Schwenkglenks (Zurich, Switzerland), Lars G Hemkens (Basel, Switzerland), Heiner C Bucher (Basel, Switzerland), Gordon H Guyatt (Hamilton, Canada), Matthias Briel (Basel, Switzerland; Hamilton, Canada)

  • Contributors: BK, EvE, and MB designed the study, collected data, interpreted the results, and wrote the manuscript. BK and SS managed the database and conducted all analyses, which were checked by MB. SS, JY, AB, YK, RS, AA, TB, JJM, MS, KKO, KAOT, IN, AC, MF, SMM, DM, EAA, DB, JWB, IF, FL, AN, VG, HR, LM, RR, SE, XS, POV, BCJ, MAW, MS, and LGH contributed to the data collection. BB, HCB, and GHG provided methodological and logistical support. All authors critically revised the manuscript and approved the final version before submission. BK, SS, EvE, and MB are guarantors.

  • Funding: This study was funded by the Swiss National Science Foundation (grant 320030_133540/1) and the German Research Foundation (grant EL 544/1-2). MB, AN, VG, HR, LGH, and HCB were supported by Santésuisse and the Gottfried and Julia Bangerter-Rhyner-Foundation. XS was supported by a young investigators award (2013SCU04A37) from Sichuan University, China. During study preparation, EvE was supported by the Brocher Foundation. JWB was funded by a new investigator award from the Canadian Institutes of Health Research and Canadian Chiropractic Research Foundation. DM was a recipient of a research early career award from Hamilton Health Sciences Foundation (Jack Hirsh Fellowship). KAOT was funded by unrestricted grants from the Finnish Cultural Foundation, Finnish Medical Foundation, Jane and Aatos Erkko Foundation, and Sigrid Jusélius Foundation. JY was supported by a research early career award from Hamilton Health Sciences.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: study funding by the Swiss National Science Foundation (grant 320030_133540/1) and the German Research Foundation (grant EL 544/1-2), no financial relationships with any organisations that might have an interest in the submitted work in the previous three years, and no other relationships or activities that could appear to have influenced the submitted work.

  • Ethical approval: This study was approved by the participating research ethics committees, or if no ethical approval was required this was explicitly stated.

  • Data sharing: No additional data available.

  • Transparency: The lead author (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.

References

  1. ↵
    Sun X, Briel M, Busse JW, You JJ, Akl EA, Mejza F, et al. The influence of study characteristics on reporting of subgroup analyses in randomised controlled trials: systematic review. BMJ2011;342:d1569.
    OpenUrlAbstract/FREE Full Text
  2. ↵
    Hernandez AV, Boersma E, Murray GD, Habbema JD, Steyerberg EW. Subgroup analyses in therapeutic cardiovascular clinical trials: are most of them misleading? Am Heart J2006;151:257-64.
    OpenUrlCrossRefPubMedWeb of Science
  3. ↵
    Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM. Statistics in medicine--reporting of subgroup analyses in clinical trials. N Engl J Med2007;357:2189-94.
    OpenUrlCrossRefPubMedWeb of Science
  4. ↵
    Gabler NB, Duan N, Liao D, Elmore JG, Ganiats TG, Kravitz RL. Dealing with heterogeneity of treatment effects: is the literature up to the challenge? Trials2009;10:43.
    OpenUrlCrossRefPubMed
  5. ↵
    Assmann SF, Pocock SJ, Enos LE, Kasten LE. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet2000;355:1064-9.
    OpenUrlCrossRefPubMedWeb of Science
  6. ↵
    Szczech LA, Berlin JA, Feldman HI. The effect of antilymphocyte induction therapy on renal allograft survival. A meta-analysis of individual patient-level data. Anti-Lymphocyte Antibody Induction Therapy Study Group. Ann Intern Med1998;128:817-26.
    OpenUrlCrossRefPubMedWeb of Science
  7. ↵
    Rothwell PM, Eliasziw M, Gutnikov SA, Fox AJ, Taylor DW, Mayberg MR, et al. Analysis of pooled data from the randomised controlled trials of endarterectomy for symptomatic carotid stenosis. Lancet2003;361:107-16.
    OpenUrlCrossRefPubMedWeb of Science
  8. ↵
    Oxman AD, Guyatt GH. A consumer’s guide to subgroup analyses. Ann Intern Med1992;116:78-84.
    OpenUrlCrossRefPubMedWeb of Science
  9. ↵
    Sun X, Ioannidis JP, Agoritsas T, Alba AC, Guyatt G. How to use a subgroup analysis: users’ guide to the medical literature. JAMA2014;311:405-11.
    OpenUrlCrossRefPubMedWeb of Science
  10. ↵
    Pocock SJ, Assmann SE, Enos LE, Kasten LE. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med2002;21:2917-30.
    OpenUrlCrossRefPubMedWeb of Science
  11. ↵
    Rothwell PM. Treating individuals 2. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet2005;365:176-86.
    OpenUrlCrossRefPubMedWeb of Science
  12. ↵
    Yusuf S, Wittes J, Probstfield J, Tyroler HA. Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA1991; 266: 93-8.
  13. ↵
    Sun X, Briel M, Walter SD, Guyatt GH. Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses. BMJ2010;340:c117.
    OpenUrlFREE Full Text
  14. ↵
    Al-Marzouki S, Roberts I, Evans S, Marshall T. Selective reporting in clinical trials: analysis of trial protocols accepted by The Lancet. Lancet2008;372:201.
    OpenUrlPubMedWeb of Science
  15. ↵
    Chan AW. Bias, spin, and misreporting: time for full access to trial protocols and results. PLoS Med2008;5:e230.
    OpenUrlCrossRefPubMed
  16. ↵
    Chan AW, Hrobjartsson A, Jorgensen KJ, Gotzsche PC, Altman DG. Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols. BMJ2008;337:a2299.
    OpenUrlAbstract/FREE Full Text
  17. ↵
    Kasenda B, von Elm EB, You J, Blumle A, Tomonaga Y, Saccilotto R, et al. Learning from Failure - Rationale and Design for a Study about Discontinuation of Randomized Trials (DISCO study). BMC Med Res Methodol2012;12:131.
    OpenUrlCrossRefPubMed
  18. ↵
    Kasenda B, von Elm E, You J, Blumle A, Tomonaga Y, Saccilotto R, et al. Prevalence, characteristics, and publication of discontinued randomized trials. JAMA2014;311:1045-51.
    OpenUrlCrossRefPubMedWeb of Science
  19. ↵
    Sun X, Briel M, Busse JW, You JJ, Akl EA, Mejza F, et al. Credibility of claims of subgroup effects in randomised controlled trials: systematic review. BMJ2012; 344: e1553.
  20. ↵
    Chan AW, Upshur R, Singh JA, Ghersi D, Chapuis F, Altman DG. Research protocols: waiving confidentiality for the greater good. BMJ2006;332:1086-9.
    OpenUrlFREE Full Text
  21. ↵
    Bhandari M, Devereaux PJ, Li P, Mah D, Lim K, Schunemann HJ, et al. Misuse of baseline comparison tests and subgroup analyses in surgical trials. Clin Orthop Relat Res2006;447:247-51.
    OpenUrlCrossRefPubMed
  22. ↵
    Boonacker CW, Hoes AW, van Liere-Visser K, Schilder AG, Rovers MM. A comparison of subgroup analyses in grant applications and publications. Am J Epidemiol2011;174:219-25.
    OpenUrlAbstract/FREE Full Text
  23. ↵
    ClinicalTrials.gov. Protocol Data Element Definitions (DRAFT). 2013. http://prsinfo.clinicaltrials.gov/definitions.html (accessed March 2014).
  24. ↵
    ICTRP. Search Portal. 2014. http://www.who.int/ictrp/search/en/ (accessed March 2014).
  25. ↵
    ISRCTN. Data Set. 2012. http://www.controlled-trials.com/isrctn/isrctn_dataset (accessed March 2014).
  26. ↵
    Chan AW, Tetzlaff JM, Altman DG, Laupacis A, Gotzsche PC, Krleza-Jeric K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med2013;158:200-7.
    OpenUrlCrossRefPubMedWeb of Science
  27. ↵
    Chan AW, Tetzlaff JM, Gotzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ2013;346:e7586.
    OpenUrlAbstract/FREE Full Text
  28. ↵
    Schulz KF, Altman DG, Moher D, Group C. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ2010;340:c332.
    OpenUrlFREE Full Text
View Abstract
Back to top