Sample size and power

Answers

Answer b is true, while a, c, and d are false.

The researchers reported that limited evidence existed to inform clinical decision making in the treatment of plantar warts. Therefore, the randomised controlled trial was carried out to compare the effectiveness of cryotherapy with that of salicylic acid. Designed as a superiority trial,2 the study aimed to establish whether cryotherapy was superior to salicyclic acid or vice versa. For one treatment to be considered superior to the other, a difference between treatments in cure rates of 15% or more needed to be observed. This difference is termed the smallest effect of clinical interest and is based on prior clinical expertise. If this difference in cure rates between treatments was seen, the null hypothesis would be rejected in favour of the alternative hypothesis at the 5% level of significance. The alternative hypothesis is two sided, stating that in the population the cure rate for salicylic acid is less than, or greater than, that for cryotherapy by 15% or more (a is false).

It was not known if the smallest effect of clinical interest actually existed in the population. It was essential that the sample was representative of the population so that if the actual difference between treatments in the population was 15% or more, it was then found in the trial. The only way to ensure that the sample estimate was a truly accurate estimate of the population difference was for the entire population to be included in the trial. However, this was obviously not possible, and so a sample was taken from the population. The sample needed to be large enough for the sample estimate to be similar in magnitude to the population treatment difference. Therefore, if the smallest effect of clinical interest actually existed, it would be observed.

To calculate the required sample size, some idea of the expected cure rate for one of the treatments was needed. Published evidence indicated that the cure rate for salicylic acid was 70% at 12 weeks. The sample size was derived by using the theoretical concept of repeated sampling from the population. If samples of size of 240 (120 in each treatment group) were taken repeatedly at random from the population, then 80% of these samples would show a treatment difference of 15% or more—but obviously only if that difference actually existed in the population. This is called the power of the study: that is, the probability of observing the smallest effect of clinical interest, if it exists (b is true). We wish the power to be as high as possible and close as possible to 1.0 (that is, 100%). The only way the power could be 100% is if the entire population was included in the trial. Therefore, increases in power result in a greater sample size (c is false). Typically in clinical trials, power is set to be a minimum of 0.8 (80%).

No statistically significant difference between treatments was observed. However, it cannot be concluded that such a difference does not exist in the population (d is false), only that there was no evidence of one. The trial participants were a single sample from the population that may not have been representative. It is possible that another sample may give different results. This concept has led to the phrase, “Absence of evidence is not evidence of absence.”3 4 The sample size was calculated with a power of 80%, and therefore there was a probability of 0.2 that the calculated sample size would not demonstrate the smallest effect of clinical interest even if it existed in the population.