Intended for healthcare professionals

Rapid response to:

Information In Practice

Performance league tables: the NHS deserves better

BMJ 2002; 324 doi: (Published 12 January 2002) Cite this as: BMJ 2002;324:95

Rapid Response:

Sensitive and specific measures of performance

Adab et al1 draw attention to the difference between sensitive and
specific measures of performance. They assert that control charts are more
specific but but less sensitive than league tables, because they use wider
confidence intervaaals (CIs). We clinical auditors do not consider this
when designing our audit criteria; perhaps we should. A contingency table
may help to illustrate the difference. Take as an example an audit of
missing outpatient films, such as I have recently done for The Royal
College of Radiologists. We can regard the 'missing film rate' (which may
be 'high' or 'low') as a diagnostic test for a department's 'overall
performance', (which may be 'bad' or 'good'); let a, b, c and d be the
number of departments in the four possible categories:

Overall Performance = bad	Overall Performance = good
Missing Film Rate = high	a	b
Missing Film Rate = low	c	d

Sensitivity = a/a + c and specificity = d/b + d. The rule is2: 'High
sensitivity means a negative results rules a diagnosis out whereas high
specificity means a positive result rules a diagnosis in'. So the valid
inferences are: using a league table implies a narrow CI (+/- 2 or 1.64
SDs) and high sensitivity; (c/a + c) is low; so a low missing film score
implies overall performance is not bad; but we cannot infer that
performance is good. Using a control chart implies a wide confidence
interval (+/- 3 SDs) and high specificity; (b/b + d) is low; so a high
missing film score implies bad overall performance.

In point of fact one could construct league tables with wide CIs and
control charts with narrow CIs, so the difference is customary rather than
absolute. The real issue is the choice of underlying philosophy. Before
they write an audit protocol, auditors should ask themselves which of
these two statements better represents their atttitude and intent: 'our
healthcare providers are dissimilar and have dissimilar performance: we
will use narrow confidence intervals so that our audits identify those
whose performance is not bad' or 'our healthcare providers are similar and
have similar performance: we will use wide confidence intervals so that
our audits will identify those whose performance is bad.'.

Christopher Squire

1 Adab P, Pouse AM, Mohammad MA, Marshall T. Performaaance league
tables: the nHS deserves better. BMJ 2002;324:95-8.

2 Sackett D L et al. Evidence-based medicine (Section 3b1: Is this
evidence about a diagnostic test important?). 1st edn. London: Churchill
Livinngstone, 1997.

Competing interest: none.

Competing interests: Overall Performance = bad Overall Performance = goodMissing Film Rate = high a bMissing Film Rate = low c d

14 February 2002
Christopher J Squire
Clinical audit officer
The Royal College of Radiologists