Intended for healthcare professionals

Endgames Statistical Question

Internal and external validity

BMJ 2010; 340 doi: https://doi.org/10.1136/bmj.c1705 (Published 31 March 2010) Cite this as: BMJ 2010;340:c1705

Author's Reply

Random allocation will have produced two groups similar in baseline
characteristics (answer c). It was not claimed that the two groups would
have been exactly the same, only similar in baseline characteristics.
Therefore, we can conclude that answer c is true. Providing the sample
size is large enough randomisation will ensure that there are no
systematic differences between groups due to possible confounding
variables. Confounding variables are those that may influence patient
outcome. The obvious question is how large a sample needs to be before
systematic differences are sufficiently minimised not to affect patient
outcome. However, there is no obvious answer to that question. As I always
tell my students – the larger the better.

Lowe uses the unfamiliar term of “samplings”. Presumably a “sampling”
refers to the statistical hypothesis test of a difference between
treatment groups in a single baseline characteristic. If there were no
differences between treatment groups in baseline characteristics, then by
definition we would still expect 1 in 20 of any such statistical
comparisons to be significant at the 0.05 level of significance (and not
the 0.0005 level as Lowe suggests). Statistical hypothesis testing will be
discussed in future Endgame Questions.

What is Lowe’s justification for suggesting that baseline
characteristics with wide confidence intervals for either treatment group
should be treated with suspicion? What should we be suspicious of? Lowe
seems to imply that groups can be compared in baseline characteristics
based on whether the confidence intervals overlap or not. Researchers
should be cautious of this approach.[1] When comparing two group means, it
is good practice and more informative to present a confidence interval for
the mean difference rather than a confidence interval for each of the two
group means.

1. Sedgwick P. Statistical significance and confidence intervals. BMJ
2009;339:b3401

Competing interests:
None declared

Competing interests: No competing interests

15 April 2010
Philip M Sedgwick
Senior Lecturer in Medical Statistics
Centre for Medical and Healthcare Education, St. George's, Iniversity of London, London SW17 0RE