Intended for healthcare professionals

  1. Research
  2. Three simple rules to...
  3. Three simple rules to ensure reasonably credible subgroup analyses
CCBYNC Open access
Research Methods & Reporting

Three simple rules to ensure reasonably credible subgroup analyses

BMJ 2015; 351 doi: https://doi.org/10.1136/bmj.h5651 (Published 04 November 2015) Cite this as: BMJ 2015;351:h5651
  1. James F Burke, assistant professor12,
  2. Jeremy B Sussman, assistant professor23,
  3. David M Kent, professor of medicine45,
  4. Rodney A Hayward, professor of medicine23
  1. 1Department of Neurology, University of Michigan School of Medicine, Ann Arbor, MI 48109-2800, USA
  2. 2VA Center for Clinical Management and Research, Ann Arbor
  3. 3Department of Internal Medicine, University of Michigan School of Medicine
  4. 4Department of Internal Medicine, Tufts University School of Medicine, Boston, MA, USA
  5. 5Predictive Analytics and Comparative Effectiveness Center, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, Tufts University School of Medicine
  1. Correspondence to: J F Burke jamesbur{at}umich.edu
  • Accepted 2 October 2015

The limitations of subgroup analyses are well established—false positives due to multiple comparisons, false negatives due to inadequate power, and limited ability to inform individual treatment decisions because patients have multiple characteristics that vary simultaneously. In this article, we apply Bayes’s rule to determine the probability that a positive subgroup analysis is a true positive. From this framework, we derive simple rules to determine when subgroup analyses can be performed as hypothesis testing analyses and thus inform when subgroup analyses should influence how we practice medicine.

Summary points

  • Limitations of subgroup analyses are well established—false positives due to multiple comparisons, false negatives due to inadequate power, and limited ability to inform individual treatment decisions because patients have multiple characteristics that vary simultaneously. It remains uncertain when subgroup analyses should influence clinical practice

  • Categorical subgroup analyses should not be part of a typical clinical trial’s hypothesis testing analysis unless the prior probability for a subgroup effect being present is at least 20% and preferably higher than 50%

  • Rarely should more than one to two primary categorical subgroup analyses be performed

  • In trials with exceptional power to identify subgroup effects, hypothesis testing analyses of subgroups should be justified a priori

A table or figure reporting about a dozen subgroup analyses is a near ubiquitous feature of major clinical trial publications.1 2 The motivation behind these analyses is clear and compelling—to determine which patients most benefit from treatment, based on specific risk factors. However, the limitations of these analyses are well established—false positives due to multiple comparisons, false negatives due to inadequate power, and limited ability to inform individual treatment decisions because patients have multiple characteristics that vary simultaneously.3 When, if ever, should subgroup analyses, tested using subgroup treatment interactions, influence how we practice medicine?

Contrary to common belief, the well documented unreliability …

View Full Text
Back to top