Published 19 November 2009, doi:10.1136/bmj.b4809
Cite this as: BMJ 2009;339:b4809

Head to Head

Should the NHS strive to eradicate all unexplained variation? No

Variation exists in all aspects of health care. Stephen Richards (doi:10.1136/bmj.b4811) argues that it is damaging to both quality of care and finances, but Richard Lilford believes that imposing uniformity risks stifling medical progress

I assume that most people would agree that variation is acceptable when patients make different personal choices—for example, whether to have screening for Down’s syndrome or prostate cancer. Likewise, I take it that variation is not acceptable when compelling evidence of cost effectiveness is available. These points I take as "self-evident truths" and turn to the interesting question that relates to the acceptability of eclectic practice when the evidence is not compelling.

When I started out in practice, some clinicians recommended bed rest for threatened miscarriage while others advised women to continue with normal activities. Some mothers were encouraged to nurse their babies prone, others supine. Should mandatory compliance with one or the other practice have been imposed in each case? If so, how could the decision have been made? At the time little salient empirical evidence was available so the opinion of the majority would have prevailed. In that case bed rest would have been recommended for threatened miscarriage and the prone position advised (or perhaps even mandated) for infants. Yet we now have strong evidence that bed rest does not reduce the probability of miscarriage and that risk of infant death is increased in the prone position.

In both scenarios premature enforcement of a standard that was poorly supported by the prevailing evidence would have thwarted assembly of the necessary data. In the event, lack of an enforced policy on sleeping positions generated two sufficiently large groups of infants that enabled rates of unexpected infant death to be compared with a high degree of precision. Likewise, Everett and colleagues perturbed previously entrenched beliefs concerning management of threatened miscarriage by showing non-uniform practice and opinion,¹ thereby generating sufficient equipoise to mount a clinical trial.^{2 3}

Value of variation

The cost effectiveness of new treatments should be established before they are widely adopted in health services, but no matter how many randomised trials are done observational studies will still be needed .⁴ There will always be more questions than answers, in part because for every piece of new knowledge, a string of secondary questions arises. For example, the National Institute for Health and Clinical Excellence (NICE) appraisal committee had plenty of evidence to recommend clopidrogel for prevention of recurrent stroke, but subsidiary questions, such as recommended duration of treatment and its use in patients with various comorbidities, were harder to answer.

Historically controlled studies are more biased than concurrently controlled observational studies,⁵ but concurrent studies can contribute data on effectiveness and safety only if treatment is allowed to vary. The epistemic advantages of non-uniform practice do not entail any loss for patients. Different clinician-patient pairs are likely to reach different judgments concerning the effects of treatment. More than one treatment option is therefore acceptable when evidence is weak. Prior opinions about the effects of treatment can be captured in the form of prior probability estimates under a bayesian model. However, prior beliefs are updated as evidence accrues so that an individual becomes progressively less uncertain while the beliefs of different individuals converge.⁶ Mothers are no longer sent routinely to bed for threatened miscarriage, and a much higher proportion of the world’s babies are now nursed supine. It has been shown that variation in practice diminishes as uncertainty is reduced.^{6 7} However, in the absence of compelling evidence different prior beliefs are rational and differences in practice do not offer a disservice to patients. They are therefore not inequitable.

A Boston physician was censured in the 19th century for not using blood letting to treat pneumonia.⁸ Semmelweis was ridiculed for insisting on antisepsis in maternity wards.⁹ There is a real risk that practice might ossify if uniformity is imposed when the evidence is insufficient to generate consensus (that is, before convergence of probability estimates has occurred). Eclectic practice uninformed by evidence is obviously undesirable. But different decisions in a given scenario are entirely justifiable and indeed desirable when the jury is still out.

People with the power to impose their beliefs on others will always be tempted to do so.¹⁰ However, post-enlightenment societies have succeeded because they allow the freedom to use individual judgment when there is no good reason to impose conformity. Competition between ideas can be resolved empirically when more than one option can be experienced. A randomised trial is but one method under which treatment may vary so that safety and effectiveness can be assessed.

Cite this as: BMJ 2009;339:b4809

---

Competing interests: None declared.

References

1. Everett C, Ashurst H, Chalmers I. Reported management of threatened miscarriage by general practitioners in Wessex. BMJ 1987;295:583-6.[Abstract/Free Full Text]
2. Djulbegovic B. The paradox of equipoise: the principle that drives and limits therapeutic discoveries in clinical research. Cancer Control 2009;16:342-7.[Medline]
3. Edwards SJ, Lilford RJ, Braunholtz DA, Jackson JC, Hewison J, Thornton J. Ethical issues in the design and conduct of randomised controlled trials. Health Technol Assess 1998;2(15).
4. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996;312:1215-8.[Free Full Text]
5. Sacks H, Chalmers TC, Smith H Jr. Randomized versus historical controls for clinical trials. Am J Med 1982;72:233-40.[CrossRef][Web of Science][Medline]
6. Jones P, Johanson R, Baldwin KJ, Lilford R, Jones P. Changing belief in obstetrics: impact of two multicentre randomised controlled trials. Lancet 1998;352:1988-9.[CrossRef][Web of Science][Medline]
7. McPherson K, Wennberg JE, Hovind OB, Clifford P. Small-area variations in the use of common surgical procedures: an international comparison of New England, England, and Norway. N Engl J Med 1982;307:1310-4.[Abstract]
8. Silverman W. In: Chalmers I, McIlwaine G, eds. Perinatal audit and surveillance. London: Royal College of Obstetricians and Gynaecologists, 1980:110.
9. Gillies D. Hempelian and Kuhnian approaches in the philosophy of medicine: the Semmelweis case. Stud Hist Philos Biol Biomed Sci 2005;36:159-81.[CrossRef][Medline]
10. Gore A. Assault on reason. New York, NY: Penguin Press, 2007.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    StumbleUpon    Technorati    Twitter    What's this?

Relevant Articles

On the origin of variation

Nigel Hawkes
BMJ 2009 339: b5071. [Extract] [Full Text]

Should the NHS strive to eradicate all unexplained variation? Yes

Stephen Richards
BMJ 2009 339: b4811. [Extract] [Full Text]

Why we need observational studies to evaluate the effectiveness of health care

Nick Black
BMJ 1996 312: 1215-1218. [Extract] [Full Text]

Reported management of threatened miscarriage by general practitioners in Wessex.

C Everett, H Ashurst, and I Chalmers
Br Med J (Clin Res Ed) 1987 295: 583-586. [Abstract] [PDF]

Online poll

Find out more>>