Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses

I am troubled by the suggestion by Sun et al that a subgroup effect
consistent with a pre-specified direction will increase the credibility of
a subgroup analysis, and that getting the direction wrong weakens the case
for a real underlying subgroup effect. 1

Freedman has suggested that the concept of clinical equipoise in the
conduct of a study requires that there is uncertainty, not necessarily on
the part of the individual investigator, but within the clinical
community. 2

Greenland argues that the design and analysis of studies may be
biased towards results desired by an investigator. Investigator bias may
arise from many sources and can be a major source of uncertainty about
study effects. 3

While it may be appropriate for an investigator to reveal a bias in
one direction or another, it should not be necessary to accurately predict
what is going to be discovered in any sub-group analysis. If a bias is
revealed or a prediction made, it is important that neither influence the
conduct of the study, nor the validity or credibility of the results.

1. Sun X, Briel M, Walter SD, Guyatt GH. Is a subgroup effect
believable? Updating criteria to evaluate the credibility of subgroup
analyses BMJ 2010; 340: c117

2. Freedman, B. Equipoise and the ethics of clinical research. N Engl
J Med. 1987; 317:141-145.

3. Greenland, S. Accounting for uncertainty about investigator bias:
disclosure is informative: How could disclosure of interests work better
in medicine, epidemiology and public health? J Epidemiol Community Health
2009; 63:593-598

Competing interests:
None declared