Research Methods & Reporting

Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses

BMJ 2010; 340 doi: (Published 30 March 2010) Cite this as: BMJ 2010;340:c117
  1. Xin Sun, research fellow12,
  2. Matthias Briel, senior researcher13,
  3. Stephen D Walter, professor1,
  4. Gordon H Guyatt, professor14
  1. 1Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada
  2. 2Center for Clinical Epidemiology and Evidence-Based Medicine, West China Hospital, Sichuan University, Chengdu, China
  3. 3Basel Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Basel, Switzerland
  4. 4Department of Medicine, McMaster University, Hamilton, Canada
  1. Correspondence to: Gordon H Guyatt, 1200 Main Street West, Rm 2C12, Hamilton, Ontario, Canada, L8N 3Z5 guyatt{at}
  • Accepted 29 December 2009

How can we tell the difference between spurious and real subgroup effects? This article identifies new criteria and proposes a checklist for judging the credibility of subgroup analyses


Subgroup analyses in randomised controlled trials (RCTs) or in meta-analyses of RCTs examine whether treatment effects vary according to patient group, way of giving an intervention, or approach to measuring an outcome. Subgroup analyses are common and often associated with claims of difference of treatment effects between subgroups—termed “subgroup effect”, “effect modification”, or “interaction between a subgroup variable and treatment”.1 2 3 A difference in effect between subgroups, if true, is likely to have important implications for clinical practice and policy making. Many subgroup claims are, however, subsequently shown to be false.4 Thus, investigators, clinicians, and policy makers face the challenge of whether or not to believe apparent differences in effect.

Debates about subgroup effects may be framed in terms of absolute acceptance or rejection. For instance, in an intense academic debate,5 6 7 8 9 10 11 one camp maintained that effects of propranolol on death differed in two groups of study centres, whereas the other remained highly sceptical. This “yes” versus “no” polarised approach is undesirable and destructive, mainly because it ignores the uncertainty that is inevitably part of such judgments. An approach that is more productive and more realistic is to place the likelihood that a subgroup effect is real on a continuum from “highly plausible” to “extremely unlikely”, possibly by using a visual analogue scale. The question is then a decision of where on this continuum a putative subgroup effect lies.

In 1991, Yusuf et al12 discussed principles of analysing and interpreting subgroup effects, and stated that qualitative interactions (that is, when treatment is beneficial in one subgroup but harmful in another) are rare. …

View Full Text

Sign in

Log in through your institution