Confidence intervals: predicting uncertainty

Answers

Statement c best describes the information provided by the 95% confidence interval for mean weight loss at four months for the standardised consultation group.

At four months, the standardised consultation group showed a mean weight loss of 1.11 kg. The mean weight loss is the sample estimate, sometimes referred to as a point estimate, of the population parameter. The population parameter is the mean weight loss that would be seen in the population if all members received the standardised consultation. The population would be those adults with osteoarthritis of the knee who fulfilled the trial’s inclusion criteria. The population parameter is in effect constant yet unknown, and this is why a sample is taken from the population to estimate the population parameter. Because not all of the population was studied, sampling error may have been introduced—the sample estimate may not equal the population parameter exactly. It is therefore essential that some indication of the precision of the sample estimate is derived. The 95% confidence interval is an interval estimate for the population parameter of mean weight loss, and it represents the uncertainty of the sample in estimating the population parameter as a result of sampling error. The confidence interval does not represent uncertainty in the sample mean of weight loss; the sample estimate of mean weight loss is a single value that is known exactly (statement d).

The information provided by the 95% confidence interval for the population mean weight loss for the standardised consultation is that, with a probability of 0.95, the population parameter of mean weight loss is contained by the interval 0.7 kg to 1.52 kg (statement c). Therefore, with a probability of 0.95 the population mean weight loss after standardised consultation could be as small as 0.70 kg or as great as 1.52 kg. The confidence interval is derived from the standard error of mean weight loss for the standardised consultation group. Standard error, described in a previous question,2 represents the precision of the sample estimate of the population parameter and is derived from the sample data. The confidence interval extends either side of the sample mean of weight loss by a multiple of the standard error. Because the population parameter is unknown, deriving a confidence interval that contains the population parameter with a fixed probability is often described as estimating uncertainty.

The 95% confidence interval is quoted as standard. Confidence intervals can be derived with a different percentage—for example, 90% and 99%. A 99% confidence interval for the population mean difference in weight loss would be wider than the 95% confidence interval because it would reflect increased certainty about the possible values of the population parameter. A 90% confidence interval would be narrower, reflecting reduced certainty. The 95% confidence interval is chosen as a trade off—a smaller probability would not provide an interval estimate with enough certainty, whereas a greater probability would present an interval estimate that was too wide to be of practical benefit.

The 95% confidence interval is an interval estimate for the population parameter of mean weight loss. The confidence interval does not describe a range of values for which 95% of the sample members achieved a weight loss (statement a). A previous question described how the sample standard deviation of weight loss could be used to derive a series of ranges in weight loss that would contain the weight loss achieved by certain percentages of the sample members.2 Equally, the 95% confidence interval does not describe a range of values in which 95% of the population members would show weight loss if they received the standardised consultation (statement b).

The 95% confidence intervals for the population mean weight loss for standardised consultation and usual care overlapped. However, it cannot be inferred that the mean weight loss between treatments was not significant at the 5% level. It is a common misinterpretation that overlapping 95% confidence intervals between groups imply a lack of significance at the 5% level.3 When comparing the two interventions in mean weight loss, it would have been good practice and more informative to present a confidence interval for the difference in mean weight loss between treatments rather than confidence intervals for the mean weight loss in each group. When compared with usual care, the standardised consultation achieved an average of 0.74 kg (0.20 to 1.28) greater weight loss. There is a unique relation between the 95% confidence interval and 5% level of significance when hypothesis testing. Because the 95% confidence interval for the difference in mean weight loss does not include zero, it can be inferred that the difference in weight loss between treatment groups was significant at the 5% level. The P value for the statistical test of the difference in mean weight loss between groups was 0.007.