Intended for healthcare professionals

Letters

Confidence intervals illuminate absence of evidence

BMJ 2004; 328 doi: https://doi.org/10.1136/bmj.328.7446.1016-b (Published 22 April 2004) Cite this as: BMJ 2004;328:1016
  1. Doug Altman, director (doug.altman{at}cancer.org.uk),
  2. J Martin Bland, professor of health statistics
  1. Cancer Research UK Medical Statistics Group, Centre for Statistics in Medicine, Oxford OX3 7LF
  2. Department of Health Sciences, University of York, York YO10 5DD

    EDITOR—We agree with Alderson that authors should recognise that non-significant results are compatible with a range of possible findings.1 Papers in the same issue of the BMJ do not adhere to this good advice.

    Koivunen et al concluded that adenoidectomy is not effective and cannot be recommended, yet the 95% confidence interval for further episodes of otitis media is compatible with an 18% absolute risk reduction.2 The clinically important difference sought was a 25% reduction.

    Kariminia et al said that hands and knees exercise with pelvic rocking did not reduce the incidence of persistent occiput posterior position at birth3; the 95% confidence interval was from 1.8% reduction to 2.5% increased risk. This trial sought a risk reduction of 2.5%.

    Marre et al concluded that “low dose ramipril has no effect on cardiovascular and renal outcomes”4—the 95% confidence interval was from 15% reduction to 11% increased risk. A 20% reduction was considered clinically important.

    None of these non-significant trials ruled out some treatment benefit. Others may judge that a smaller benefit would be clinically useful. Even when a clinically useful effect has been ruled out, phrases such as “is not effective,” “did not reduce,” and “has no effect” are not justified.

    Also, confidence intervals reflect only uncertainty owing to random allocation, not that owing to failure to follow the protocol, non-random loss to follow up, and so on. True uncertainty is greater, therefore, than indicated by confidence intervals.

    Lastly, we cannot claim priority with the title “Absence of evidence is not evidence of absence”: a paper with this title was published in 1983.5

    Footnotes

    • Competing interests None declared.

    References

    View Abstract