Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Selecting diagnostic tests for evaluation
- Nicholas Summerton (14 March 2008)
-
Genomic Horoscopes.
- Saty Satya-Murti (15 March 2008)
-
Evaluating diagnostic tests – gaps in understanding
- Huw Llewelyn (15 March 2008)
|
|
|||
|
Nicholas Summerton, Clinical & Public Health Advisor, NICE NICE, MidCity Place, 71 High Holborn, London. WC1V 6NA
Send response to journal:
|
In his editorial Tom Walley outlines some of the problems encountered in evaluating laboratory diagnostic tests. One of the great difficulties faced by organisations such as NICE is to find a means to prioritise diagnostic technologies for rapid evaluation. It is also always important to appreciate that diagnostics is much broader than a single laboratory test. The ACCE framework mentioned by Walley emphasises the requirement to focus on clinical outcomes in the context of a diagnostic processing pathway rather than the simple performance characteristics of the specific test. Furthermore comparative diagnostic technologies are not only found in laboratories, endoscopy suites or imaging departments; the medical history and the clinical examination remain powerful diagnostic tools (even if not CE marked!). Finally the clinical setting will not only impact on the discriminant ability of a test but also on the level of precision required by the clinician. I have recently developed the following prioritisation criteria for diagnostic technology evaluation by organisations such as NICE. The disease: 1. Can the disease be clearly defined? 2. Is the condition an important problem in terms of prevalence/incidence or morbidity/mortality? 3. Is it a policy priority? 4. Does the condition present a diagnostic problem (i.e. inaccuracy or inefficiency) and would it be useful to have better diagnostic tools? 5. Is there evidence of current variation in diagnostic practice (or inappropriate variations in treatment/morbidity/mortality resulting from diagnostic variability)? 6. Could the diagnostic processing pathway for the disease be improved by obtaining information in a less risky fashion or in a manner more acceptable to patients? The diagnostic technology (e.g. history/examination, physiological measurement, imaging, endoscopy, pathology): 1. Is there clarity about the purposes and the costs of this technology in the context of a diagnostic processing pathway? 2. Has the safety and analytical validity been established and is it CE marked? 3. Is there evidence of clinical validity in the appropriate setting? 4. Are there opportunities for enhanced efficiency or cost-savings in relation to the current diagnostic processing pathway if this technology was more widely employed? 5. In the absence of an appraisal is there any likelihood of `drift` with overuse or inappropriate use of this technology? 6. Is it feasible to change current practice to incorporate this technology (e.g. considering additional requirements for training, infrastructure and quality control)? The impact of the diagnostic technology: 1. Is there an effective treatment for the target condition and could greater diagnostic precision using the technology lead to better targeted treatment delivery? 2. Is there an effective treatment for the target condition and could more rapid diagnosis using the technology lead to shorter treatment delays? 3. Would better diagnosis result in lowered morbidity/mortality both from the disease and from the diagnostic process? Dr Nick Summerton. Clinical & Public Health Advisor, NICE Competing interests: I am currently undertaking a review for NICE on their approach to diagnostic technology evaluation |
|||
|
|
|||
|
Saty Satya-Murti, Neurologist, Health Policy Consultant Topeka, Kansas 66610
Send response to journal:
|
The Analysis by Professor Melzer and colleagues on “Genetic tests for common diseases: new insights, old concerns ” wisely compares genetic testing to astrologic predictions. This analogy reminded me of the ancient system of astrological consultations practiced, even to this day, in India. Usually a worried member seeks astrological prediction and protective advice. The sagacious astrologer consults the member’s horoscope and declares that the transiting planets and stars, or their inconvenient conjunctions, are casting malevolent vibes. They forecast relief usually in months, or at times in years. They go on to suggest propitiatory prayers, rites, fasts or diets. With passage of time, regression to mean and adaptation provide some relief. Hard times pass and some improvement ensues. The believer attributes the improvement to the sought after predictions and recommendations. The let-down skeptic may avoid future consultations. Astrology has thus been a respectable, enduring and generally innocuous salve, but not always so. Melzer et al provide excellent guidelines to ensure that genome-wide tests be used appropriately. What should one do after receiving a direct- to-consumer genomic analysis report – the modern day genomic horoscope? Much like the astrologer our best advice, although unattractive, would be to wait. We could always emphasize, in the interim, exercise, prudent diet and behavior alteration. Distress over “Snips”, has much in common with anxiety over astral sojourns. As the Editorial by Walley points out genetic tests also must prove their “Clinical usefulness” with no less rigor than any currently extant diagnostic test. Competing interests: None declared |
|||
|
|
|||
|
Huw Llewelyn, Consultant Physician Kettering General Hospital, NN16 8UZ
Send response to journal:
|
The difficulties with evaluating diagnostic tests may also be due to gaps in understanding of some aspects of the diagnostic process. The commonest uses of tests - in differential diagnosis and for deciding whom to treat, appear to be given little attention in studies to assess the performance of tests and during teaching of the diagnostic process. Symptoms, signs and test results play a major role as ‘diagnostic leads’: to narrow the number of differential diagnoses [1]. Laboratory tests and other technologies are used in this way by experienced doctors (for example as reported in grand rounds [2]). Learning this approach should not be left to chance or ‘osmosis’ but taught in a systematic way; tests should also be assessed properly for their usefulness in the process. This assessment can be done during normal care, in a similar way to audit. ‘Differentiators’ are also important: these are findings that differentiate between two of the differential diagnoses of a ‘lead’ because they occur very often in one diagnosis and less often in the other. The resulting ‘differential likelihood ratio’ is different to the overall ‘likelihood ratio’ based on dividing the frequency of the test result in one diagnosis by its frequency in all others, including healthy individuals. This better known ‘overall likelihood ratio’ is helpful in population screening (including of the worried well) but not in differential diagnosis in clinics and hospitals. However, if a screening result is positive, its differential diagnosis then has to be considered. Another important role of laboratory tests is to identify patients who respond best to a treatment compared to control. It is also important to establish the optimum cut-off points for such tests in an ‘evidence-based’ way. For example, the treatment cut-off point for the albumin excretion rate (AER) is assumed to be 20mcg/min simply because this is the ‘upper limit of normal’. However, in a typical trial, there was no difference in outcome in the treatment and control limbs unless the AER was greater than 40mcg/min. So, about 30% of the patients in the trial had started with an ‘unresponsive’ AER between 20 and 40mcg/min [3]. Assessments of this kind to optimise cut-off points and to compare the effect of different ‘treatment criteria’ (e.g. the AER versus the albumin- creatinine ratio as candidate ‘gold standards’) can be built into ordinary clinical trials. If tests are assessed for use in differential diagnosis and as treatment indications, then the production of relevant primary evidence need not be expensive. However, the cost to the taxpayer and to patients will remain high if the use of tests for these main clinical purposes are not assessed and not taught properly. Unless these major gaps in understanding are put right, international collaboration or a new national system [4] will be of limited help. References: 1. Llewelyn H, Ang H, Lewis K, Al-Abdullah A. The Oxford handbook of clinical diagnosis. Oxford: Oxford University Press, 2006. 2. Eddy DM, Clanton. The art of diagnosis. NEJM 1982, 306: 1263-8. 3. Llewelyn DEH, Garcia-Puig J. How different urinary albumin excretion rates can predict progression to nephropathy and the effect of treatment in hypertensive diabetics. JRAAS 2004; 5: 141-5. 4. Mayor S. UK experts call for system to evaluate diagnostic tests. BMJ 2008; 336:575 Competing interests: None declared |
|||