Assessing the value of diagnostic tests: a framework for designing and evaluating trials

BMJ 2012; 344 doi: http://dx.doi.org/10.1136/bmj.e686 (Published 21 February 2012)
Cite this as: BMJ 2012;344:e686

Recent rapid responses

Rapid responses are electronic letters to the editor. They enable our users to debate issues raised in articles published on bmj.com. Although a selection of rapid responses will be included as edited readers' letters in the weekly print issue of the BMJ, their first appearance online means that they are published articles. If you need the url (web address) of an individual response, perhaps for citation purposes, simply click on the response headline and copy the url from the browser window.

Displaying 1-2 out of 2 published

Lavinia Ferrante di Ruffano et al are absolutely right in calling for studies which determine a diagnostic test’s true clinical value.

As the authors point out, such studies can be difficult to execute and evaluate. However the few such studies available in the field of clinical microbiology are robust enough to guide and direct the microbiology service as a whole. Doern et al in a prospective controlled trial, (J. Clin. Microbiol. 1994;32:1757-1762), showed that a group of hospitalised patients assigned to receive rapidly reported bacterial identification and antimicrobial susceptibility tests had a lower mortality rate, underwent significantly fewer procedures and incurred significantly lower laboratory and pharmacy costs than a control group assigned to receive routine overnight reports. Numerous complementary studies show that early use of the appropriate antibiotic improves survival following bacteraemia. If these studies were taken into consideration, microbiologists would seek to implement this rapid reporting protocol as a priority. However, laboratory costs of rapid reporting are relatively high and are only affordable if non-laboratory savings are reinvested in the laboratory, and if inappropriate tests are rigorously excluded- all within a reasonably estimated and flexible capitation type budget. A microbiology service procured on a cost per test basis cannot pursue this clinical agenda. Consolidation, with the loss of on-site microbiology facilities further hampers rapid reporting.Demonstration of improved patient outcomes should be included in all laboratory service specifications.

Competing interests: None declared

Malila Noone, consultant microbiologist

retired, 41 Cleveland Terrace

Click to like:

Clearly, a test has to be requested and interpreted in the context of other findings [1]. However, it is also important to ensure that the test’s results are interpreted appropriately and in an ‘evidence-based’ way. For example, using a single cut-off point that gives ‘positive’ and ‘negative’ results may not be good enough as the number needed to treat (NNT) and thus the risk/benefit balance often changes with each actual value of the test used to select patients for treatment [2]. The NNT for each test result can be worked out by designing test-treatment randomised trials to study the effect of using different selection criteria and their resulting values on treatment outcome (and diagnostic criteria).

The specificity and likelihood ratio (equal to the sensitivity divided by ‘one minus the specificity) is useful for assessing the accuracy of tests for population screening. Such likelihood ratios are also advocated for calculating ‘posterior probabilities’ from ‘prior probabilities’ in diagnostic reasoning but this approach is of dubious validity [3]. In practice, clinicians use a ‘prior’ finding to suggest a differential diagnosis and then look for other findings that occur commonly in at least one of those differential diagnoses and rarely in at least one other. This gives a ‘sensitivity ratio’ or ‘differential likelihood ratio’ (not a standard likelihood ratio). There is a differential diagnosis ‘theorem’ that can be used to calculate diagnostic probabilities during such reasoning [4], including in decision analysis.

In order to practice evidence-based medicine properly, the diagnostic process leading to sensible selection of patients likely to benefit from treatment has to be understood more clearly, and especially if value based pricing of drugs [5] is to be used in the NHS.

References

1. di Ruffano LF, Hyde CJ, McCaffery JK, Bossuyt PMM, Deeks JJ. Assessing the value of diagnostic tests: a framework for designing and evaluating trials. BMJ 2012; 344: e686

2. Llewelyn D E H, Garcia-Puig, J. How different urinary albumin excretion rates can predict progression to nephropathy and the effect of treatment in hypertensive diabetics. JRAAS 2004, 5; 141-5.

3. Llewelyn H, Ang AH, Lewis K, Abdullah A. The Oxford Handbook of Clinical Diagnosis, 2nd edition. Oxford University Press, Oxford 2009, p760.

4. Llewelyn H, Ang AH, Lewis K, Abdullah A. The Oxford Handbook of Clinical Diagnosis, 2nd edition. Oxford University Press, Oxford 2009, pp754 – 772.

5. Hawkes N. UK government pushes ahead with value based pricing of drugs. BMJ 2011; 343: d4632

Competing interests: None declared

Huw Llewelyn, General Physician and Endocrinologist

Nevill Hall Hospital, Brecon Road, Abergavenny NP7 7EG

Click to like:

THIS WEEK'S POLL