Other method for adjustment of multiple testing exists

BMJ 1999;318:127 ( 9 January )

Letters

Other method for adjustment of multiple testing exists

The first 150 words of the full text of this article appear below.

EDITOR $---$ Perneger's paper on Bonferroni adjustments consists almost entirely of errors.¹ He states that the Bonferroni adjustmentsare concerned with the wrong hypothesis and that the two groupsare identical on all 20 variables (the universal null hypothesis).This misses the main point of multiple testadjustments.

Similarly he says, "If one or more of the 20 P values is less than 0.00256 ... we can say that the two groups are not equalfor all 20 variables, but we cannot say which, or even how many,variables differ." Researchers who adjust P values almost alwayspresent them for their individual hypotheses. With n hypotheseseach tested at level $alpha$ , Perneger claims that "the formula forthe error rate across the study is 1 $-$ (1 $-$ $alpha$ )ⁿ." This formula assumes independence of the test statistics; theactual bound on the error probability is n $alpha$ .