Intended for healthcare professionals

CCBYNC Open access
Research

Reporting and analysis of trials using stratified randomisation in leading medical journals: review and reanalysis

BMJ 2012; 345 doi: https://doi.org/10.1136/bmj.e5840 (Published 14 September 2012) Cite this as: BMJ 2012;345:e5840
  1. Brennan C Kahan, medical statistician,
  2. Tim P Morris, medical statistician
  1. 1MRC Clinical Trials Unit, Aviation House, 125 Kingsway, London WC2B 6NH, UK
  1. Correspondence to: B C Kahan brk{at}ctu.mrc.ac.uk
  • Accepted 20 August 2012

Abstract

Objectives To assess how often stratified randomisation is used, whether analysis adjusted for all balancing variables, and whether the method of randomisation was adequately reported, and to reanalyse a previously reported trial to assess the impact of ignoring balancing factors in the analysis.

Design Review of published trials and reanalysis of a previously reported trial.

Setting Four leading general medical journals (BMJ, Journal of the American Medical Association, Lancet, and New England Journal of Medicine) and the second Multicenter Intrapleural Sepsis Trial (MIST2).

Participants 258 trials published in 2010 in the four journals. Cluster randomised, crossover, non-randomised, single arm, and phase I or II trials were excluded, as were trials reporting secondary analyses, interim analyses, or results that had been previously published in 2010.

Main outcome measures Whether the method of randomisation was adequately reported, how often balanced randomisation was used, and whether balancing factors were adjusted for in the analysis.

Results Reanalysis of MIST2 showed that an unadjusted analysis led to larger P values and a loss of power. The review of published trials showed that balanced randomisation was common, with 163 trials (63%) using at least one balancing variable. The most common methods of balancing were stratified permuted blocks (n=85) and minimisation (n=27). The method of randomisation was unclear in 37% of trials. Most trials that balanced on centre or prognostic factors were not adequately analysed; only 26% of trials adjusted for all balancing factors in their primary analysis. Trials that did not adjust for balancing factors in their analysis were less likely to show a statistically significant result (unadjusted 57% v adjusted 78%, P=0.02).

Conclusion Balancing on centre or prognostic factors is common in trials but often poorly described, and the implications of balancing are poorly understood. Trialists should adjust their primary analysis for balancing factors to obtain correct P values and confidence intervals and to avoid an unnecessary loss in power.

Footnotes

  • We thank Daniel Bratton, Ben Sear, and Najib Rahman for their comments on the manuscript; the MIST2 trial team for the use of their data; and the journal editors and reviewers whose comments improved the manuscript.

  • Contributors: BCK devised the study, designed the data extraction forms, extracted data from all trial reports, tabulated the results, and wrote the first draft of the manuscript. He is guarantor. TPM contributed to the design of the data extraction forms, extracted data from 20 trial reports, and contributed to the writing of the manuscript.

  • Funding: Both authors are employed by the MRC Clinical Trials Unit. TPM is funded by an MRC studentship (MC-US-A737-0012).

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.

  • Ethical approval: Not required.

  • Data sharing: A full list of the trials reviewed and data abstracted for each trial is available from the corresponding author at brk{at}ctu.mrc.ac.uk.

This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

View Full Text