Statistical significance and confidence intervalsBMJ 2009; 339 doi: https://doi.org/10.1136/bmj.b3401 (Published 02 September 2009) Cite this as: BMJ 2009;339:b3401
A prospective population based cohort study investigated the association between occupation and pregnancy outcome for mothers who had missed antenatal visits. The mean birth weight was 3371 g (95% confidence interval 3274 to 3469) and 3227 g (3140 to 3315 g) for babies born to mothers in non-manual and manual jobs, respectively.
On the basis of the above information, which of the following can be deduced for mothers who missed antenatal visits?
a) In the population, 95% of mothers with a non-manual occupation have a baby with a birth weight between 3274 g and 3469 g
b) Because the 95% confidence intervals overlap, the mean difference in birth weight between mothers in non-manual and manual jobs is not statistically significant at the 5% level
c) On average, mothers with a non-manual job had heavier babies than those with a manual job
d) If a mother has a manual job, this will cause her baby to have a lower birth weight than if she had a non-manual job
c—In this sample, babies born to mothers with a non-manual job were heavier than those born to mothers with a manual job by an average of 144 g.
Answer a) is incorrect because the 95% confidence interval does not provide information about the distribution of birth weights in the population. It represents the uncertainty of the sample mean in estimating the population mean.
Answer b) is false because, although the 95% confidence intervals overlap, we cannot infer that the mean difference in birth weight between mothers in non-manual and manual jobs is not statistically significant at the 5% level. For the data presented, the mean difference in birth weight (non-manual minus manual) was 144 g (95% confidence interval 14 to 274; P=0.03). We observe that the mean difference in birth weight is statistically significant at the 5% level. More generally, when 95% confidence intervals for two sample means overlap, it is not possible to make any inference about statistical significance, either present or absent, at the 5% level. However, if the 95% confidence intervals for two means do not overlap, then the mean difference will be statistically significant at the 5% level. When comparing two group means, it is good practice and more informative to present a confidence interval for the mean difference rather than the separate group means.
Answer d) is false because we cannot conclude causation from an observational study. Mothers with a manual job may have had poorer diets, smoked more, and so on, and these factors may have produced the observed difference in birth weight.
Cite this as: BMJ 2009;339:b3401
Thanks to Janet Peacock and Martin Bland for providing the above data.