Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Analytical problems (not only) with BNP determining
- Rudolf Gasko (26 March 2005)
-
Licensing diagnostic tests may benefit everyone
- Eric S Kilpatrick (26 March 2005)
-
Use established methods for evaluating diagnostic methods
- Sten Öhman (29 March 2005)
-
Diagnostics is a sea: it’s time to navigate into it
- Giuseppe Giocoli (1 April 2005)
-
Multivariable approaches should not be left out in evidence based diagnostics
- Gerben ter Riet, Lucas M. Bachmann, Karel G.M. Moons, Patrick J. Bindels, and Alfons G.H. Kessels (4 April 2005)
-
Evidence based diagnostics: what about customary clinical tests
- Alireza Moayyeri, Akbar Soltani (11 April 2005)
-
Evidence based diagnostics and genetic testing
- Simon P Sanderson, Mark Kroese, Ron Zimmern (15 April 2005)
-
Authors' reply
- Christian Gluud, Lise Lotte Gluud (24 May 2005)
-
Cost-effectiveness should be included in diagnostic research
- Joanna D Schaafsma, Erik Buskens, Gabriel J. Rinkel, and Yolanda van der Graaf (3 June 2005)
|
|
|||
|
Rudolf Gasko, clinical biochemist Health Insurance Comp. Dovera, Murgasova 3, SK-043 25 Kosice, Slovakia
Send response to journal:
|
No international consensus exists yet on the methods for assessing diagnostic tests and it may be a serious problem. As I recently stated here (1), analytical problems with BNP determination are discussed in the literature. Analytical problems are hot topics in all new diagnostics. The more different diagnostics, the more analytical bias (2), and the more difficult establishing the normal range and determining the diagnostic accuracy. Better assay harmonisation could improve practice. Phase I in architecture of diagnostic research thus should be the key phase. 1. Gasko R. Analytical problems with BNP determining. Rapid response to: Doust JA, Pietrzak E, Dobson A, Glasziou P. How well does B-type natriuretic peptide predict death and cardiac events in patients with heart failure: systematic review. BMJ 2005;330:625-634. 2. Steele BW, Wang E, Klee GG, Thienpont LM, Soldin SJ, Sokoll LJ et al. Analytic Bias of Thyroid Function Tests: Analysis of a College of American Pathologists Fresh Frozen Serum Pool by 3900 Clinical Laboratories. Arch Pathol Lab Med 2005;129:310-317. Competing interests: None declared |
|||
|
|
|||
|
Eric S Kilpatrick, Consultant in Chemical Pathology Hull Royal Infirmary, Anlaby Road, Hull, UK, HU3 2JZ
Send response to journal:
|
The paper by Gluud rightly compares the introduction of new diagnostic tests to those of a new drug. In contrast to the steps required to be taken by the pharmaceutical industry before a drug can be used on patients, laboratory diagnostic tests only need to prove that they are able to measure the analyte in question. There is no requirement to show that measurement of this analyte is of any use in managing patients. Indeed, an inappropriately used test may have ‘side effects’ which lead to either a delay in diagnosis or further investigation for a condition which the test result has wrongly suggested. Conversely, when a new test is proven to be of value, its introduction in the UK can often be piecemeal, being subject to the vagaries of particular clinician demand and the priorities of local funding. These two issues may be able to be addressed together. Gluud mentions using the pharmaceutical model to effectively ‘license’ a clinical test for use. In the UK this would require a joint approach from the diagnostics industry and a Department of Health agency to acquire sufficient information on the clinical utility of a new test before it becomes routinely available. There is certainly no shortage of laboratories that would be willing to participate together as a network to carry out these ‘diagnostic trials’. After a test has been licensed for use and is of proven benefit the challenge is then for there to be no ‘postcode lottery’ in its use. Here, its widespread introduction within the NHS could then be facilitated by an organisation equivalent to the role that NICE has in recommending a particular drug’s use. This way forward could prove beneficial to all parties involved. For the diagnostics industry, currently under financial pressures in only manufacturing ‘generic’ tests, it could allow a widespread deployment of a new diagnostic test. For academic medicine in the UK, the ‘diagnostic trials’ could represent a new direction which is also free from many drug related EU directives. Lastly, for the patient, it means they can be more confident that they will not become the victim of an unproven or misused diagnostic investigation. Competing interests: None declared |
|||
|
|
|||
|
Sten Öhman, director Antivenena AB, Evastigen 9, S-590 71 Ljungsbro, Sweden
Send response to journal:
|
I agree with Gluud & Gluud that all diagnostic tests must be evaluated as good as the therapeutic methods. However, the history of such evalutations is long and a lot of references could be cited. Especially the approved recommendations (1986) from the IFCC should be followed.(1) I also agree that "phase III" and "phase IV" studies should be done, but such studies usually meet a considerable conservatism among not only clinicians but also administrators. When a new diagnostic method is developed the aim is to improve the diagnosis by using a different technique. The new method is assumed to rely on a more specific and reliable biochemical and physiological basis than established methods and hence it is supposed to improve the diagnostic situation, e.g. by establishing a diagnosis earlier and hence in a stage where it can be treated more successfully than now. "Phase III" and "Phase IV" studies must therefore include not only the aim of the method but also the biochemical basis. According to the IFCC rules all diagnostic methods in the test should be standardized to the same specificity, i.e. 97,5 %. Then the sensitivity is evaluated according to the results of the tests. For each method the result can be positive or negative. Reject all cases where the two methods yield concordant results (positive or negative) and make a binomial test on P=0.5 for discordant results. The method yielding significantly more positive results is better than the other(s) (2). References: 1. Solberg HE. (Part 6: Dybkjaer R, Solberg HE):Approved recommendations (1986) on the theory of reference values. Part 1-6. J Clin Chem Clin Biochem 1987;25:337-662 2. Öhman S et al: Comparisono of seven formulae and isoelectrofocusing for determination of intrathecally produced IgG in neurological diseases. Ann Clin Biochem 1992;29:405-10 Competing interests: None declared |
|||
|
|
|||
|
Giuseppe Giocoli, GdL EBM Associazione Microbiologi Clinici Italiani V.Farini, 81 20159 Milano (Italia)
Send response to journal:
|
Sir, I fully agree with Christian and Lise Lotte Gluud about the need of ensuring by phase I-IV studies that the harms and benefits of new tests are fully understood. These concepts have been introduced by Alvan Feinstein (1), then underlined by several scientists (2). Nevertheless, I think this model should be applied differently, according to the fields of diagnostics, e.g. biochemistry, microbiology, imaging, pathology, etc. About microbiology tests, I would like to quote this sentence from a paper by Moons and Grobbee (3): “We believe that evaluation of diagnostic tests on patient outcome is not always necessary. In general, we think that follow up studies are not necessary and the (beneficial) effect of a diagnostic test for patient outcome may be considered as established if (a) diagnostic (cross sectional) studies have shown the test’s ability to detect a particular disease and (b) therapeutic studies provided evidence on efficacy of the management of this disease”. The above statement could often be applied to testing in infectious diseases, where an effective therapy is prompted by the identification of an aetiological agent. Sometimes situations are more complicated, e.g. nucleic acid-based tests generally target only a single organism, whereas cultures generally are more broad in coverage, from which the necessity of an impact study on molecular detection of M.tuberculosis (4). In another case, the long term effect of a test and treat strategy for H.pylori has been compared with endoscopy by a randomised trial, for management of dyspeptic patients in primary care (5). Moreover, triggering receptor expressed on myeloid cells (TREM-1) in samples of bronchoalveolar-lavage fluid has been studied as a marker of pneumonia in patients receiving mechanical ventilation, following negative results of impact studies on quantitative analysis of cultures of respiratory secretions (6). And so on. As a convinced student of evidence-based medicine application to clinical laboratory practice, I would like to see an implementation of correct principles of test evaluation, but also more adaptive effort to different branches of diagnostics. Thanks, Giuseppe Giocoli MD (retired) 1. Misguided efforts and future challenges for research on “diagnostic tests”. AR Feinstein. J Epidemiol Community Health 2002;56:330–332 2. The evidence base of clinical diagnosis. Knottnerus JA ed. BMJ Books, 2002:1-226 3. Diagnostic studies as multivariable, prediction research. KGM Moons, DE Grobbee. J Epidemiol Community Health 2002;56:337–338 4. Molecular Detection of Mycobacterium tuberculosis: Impact on Patient Care. KL Kaul. Clinical Chemistry 2001;47:1553–8 5. Helicobacter pylori test and eradicate versus prompt endoscopy for management of dyspeptic patients: 6.7 year follow up of a randomised trial. AT Lassen, et al Gut 2004;53:1758–63. 6.Diagnosing Ventilator-Associated Pneumonia. A Torres, and S Ewig, NEJM 2004;350: 433-5 Competing interests: None declared |
|||
|
|
|||
|
Gerben ter Riet, Clinical Epidemiologist Department of General Practice, Academic Medical Center, 1105 AZ Amsterdam, The Netherlands, Lucas M. Bachmann, Karel G.M. Moons, Patrick J. Bindels, and Alfons G.H. Kessels
Send response to journal:
|
We congratulate Gluud and Gluud with their paper on a phased approach for
research into diagnostic tests.[1] However, a phased approach (loosely)
reflecting the approach for drug research has been covered in the
literature.[see e.g. 2-4] We agree with their proposal for phases I and IIa but
have major difficulties with phases III and IV. Competing interests: None declared |
|||
|
|
|||
|
Alireza Moayyeri, Research Fellow EBM Working Team, EMRC, Shariati Hospital, Tehran University of Medical Sciences, Tehran, Iran, Akbar Soltani
Send response to journal:
|
Christian and Lise Lotte Gluud have proposed an interesting framework for research into diagnostic tests that parallels the conventional phases in drug research [1]. This concept is really appreciable and beneficial to all the researchers and clinicians. Predictably, the presented framework has some shortcomings and needs more developments. For instance, defining normal range based on phase I studies might be problematic in some cases as several other items should be considered for diagnostic and therapeutic thresholds (like determination of normal range for blood sugar or cholesterol). Moreover, especially for phase III and IV studies, some other issues are really important for evaluation of the results and estimation of the level of evidence, such as the setting of the study (primary or tertiary care), pretest probability of the disease, length of follow-up, and response rate to follow-up. Considering all these factors, as well as cost, availability, and invasiveness of the test, we think that a Grade of Recommendation should be defined for any diagnostic test. However, our major concern about the quality of this framework is the group of tests targeted in it. It appears that the authors have developed the framework just for newly introduced paraclinical tests (including laboratory techniques, imaging, pathology, electrodiagnostic tests, and endoscopy). Nevertheless, another major group of tests not considered in this framework are traditional clinical tests including signs, symptoms and clinical scores. Although no one can expect a newly found or defined sign or symptom these days, the clinical utility and accuracy of most of the costumary clinical tests have not yet defined. We think that a new framework (or a modified version of the presented framework) is also required on the methods for assessing these tests. Clinicians on their practice firstly rely on the pointers derived from clinical presentation of patients and paraclinical tests are usually considered as supplements to these signs and symptoms. Not surprisingly, in most of the cases, diagnostic accuracy of the clinical signs and symptoms have not been quantified in methodologically strong studies and physicians just trust in their own experience or some low level evidences when using these signs and symptoms in their decision-making. We think that the major problem in the way to performing high-quality studies for clinical tests is to find funding sources. In the case of paraclinical tests, the designer or producer company is required to perform several studies to approve the test for clinical use. Using the framework proposed by the authors can standardize this process. In the case of clinical signs and symptoms, however, there is no definite foundation to fund and support such investigations. An international consensus in this regard is deeply needed. We propose that international agencies such as Cochrane Collaborations develop a similar framework and go ahead for diagnostic studies to reveal suitability of different clinical tests. 1. Gluud C, Gluud LL. Evidence based diagnostics. BMJ 2005;330:724-6 Competing interests: None declared |
|||
|
|
|||
|
Simon P Sanderson, Clinical Lecturer Primary Care Genetics Public Health Genetics Unit and University of Cambridge, Mark Kroese, Ron Zimmern
Send response to journal:
|
Gluud and Gluud have written an excellent article that emphasises the need for rigour in evaluating diagnostics, which we support(1). However, we disagree with their statement that “in some cases, the value of a diagnostic test is self-evident – for example, in genetic testing”. The term “genetic test” is shorthand to describe a test to detect (i) a particular genetic variant (or set of variants), (ii) for a particular disease, (iii) in a particular population and (iv) for a particular purpose(2). Genetic tests can be used for diagnosis, risk prediction, susceptibility testing or the detection of carrier status. A genetic test is therefore a complex intervention that is one component of an overall intervention. It is fundamental that evidence of a genetic test’s clinical utility should be obtained before its introduction into clinical practice; this should include clinical, economic and psychological measures of benefit and especially potential harms from testing. Genetic tests may have certain perceived advantages compared to other types of diagnostic test, but the assumption that their value is self- evident is flawed. For example, the recent UK Government's White Paper on Genetics used the example of testing for gene variants that determine susceptibility to the adverse effects of warfarin(3). It was suggested that general practitioners might use genetic testing to identify individuals at high risk of bleeding before treating them with warfarin. Whilst this sounds like a good idea, there is no evidence showing that genetic testing improves clinical outcomes compared with routine management. Indeed, with the careful assessment of patients prior to treatment, the use of carefully controlled dosing regimens at initiation, regular monitoring of the INR, and computerised decision support systems controlling dose recommendations, patients on warfarin can be satisfactorily managed without a genetic test(4). There are already other genetic variants that could be tested for but which have doubtful utility (including ApoE and Alzheimer disease, HFE and haemochromatosis, and Factor V Leiden and venous thrombo-embolism)(5). A test that does not improve or alter clinical management should not be introduced into clinical practice. It is our view that genetic tests must be evaluated as rigorously as possible in terms of their analytical validity (how well tests perform in the laboratory), clinical validity (how well they perform in patients), clinical utility (whether testing confers any net benefit to those tested), along with due consideration of their ethical, legal and social implications. These principles have been developed in the United States as part of the Centers for Disease Control’s ACCE programme and adopted by the United Kingdom’s Genetic Testing Network to produce a system for evaluating emerging genetic tests for use in the NHS(6;7). Although we believe that all genetic tests should be rigorously evaluated, we are not implying that a bureaucratic statutory system of regulation should be established because arguments that genetic tests require more statutory regulation than other diagnostics are generally weak(8). Reference List (1) Gluud C, Gluud LL. Evidence based diagnostics. BMJ. 2005;330:724 -26. (2) Kroese M, Zimmern RL, and Sanderson SP. Genetic Tests and their Evaluation: Can we answer the key questions? Genet.Med. 6(6), 475-480. 2004. (3) Department of Health. Our inheritance, our future - realising the potential of genetics in the NHS. 2004. (4) Sanderson S, Emery J, Higgins J. CYP2C9 gene variants, drug dose, and bleeding risk in warfarin-treated patients: a HuGEnet systematic review and meta-analysis. Genet Med. 2005;7:97-104. (5) Evans JP, Skrzynia C, Burke W. The complexities of predictive genetic testing. BMJ. 2001;322:1052-56. (6) Haddow J, Palomaki G. ACCE: A Model Process for Evaluating Data on Emerging Genetic Tests. In: Khoury M, Little J, Burke W, eds. Human Genome Epidemiology. First ed. Oxford University Press; 2004: 217-33. (7) Department of Health. United Kingdom Genetic Testing Network. www.ukgtn.org . 2004. (8) Burke W, Zimmern RL. Ensuring the appropriate use of genetic tests. Nat Rev Genet. 2004;5:955-59. Competing interests: None declared |
|||
|
|
|||
|
Christian Gluud, head of department H:S Rigshospitalet, Copenhagen University Hospital, DK-2100 Copenhagen, Denmark, Lise Lotte Gluud
Send response to journal:
|
Setting sail for the sea of diagnostic research EDITOR—A number of people have shown a very kind interest in our article on the architecture of evidence based diagnostic research. We are greatly thankful for that. Gasko is right in pointing out that phase I of diagnostic research is very important. By underlining the following phases II to IV, we aimed at getting the ‘key’ into the lock and open the door to evidence based diagnostics. This is the point Öhman and Giocoli may have missed. To us it is not a question about getting significantly more positive results, but of getting results that are significantly more meaningful to patients. We agree with Kilpatrick, that widespread introduction of diagnostic interventions that do more good than harm may be facilitated through NICE or its equivalent. However, diagnostic trials should not become a safe haven from EU directives. Diagnostic trials should be conducted with the same rigor as therapeutic trials. We do find that it is high time to improve the quality of all trials irrespective of topic (1,2,3). Ter Riet and coworkers refer to literature on phased approaches to diagnostic research. We agree that the approaches have been too loosely connected to evidence based diagnostics. We think one can only learn about the improvement of clinical outcomes by studying the clinical course of patients. Therefore we need randomised clinical trials, even though they may get large and difficult to handle. Moayyeri and Soltani correctly point out that more development is needed. Not only new tests, but also traditional tests including signs, symptoms, or clinical scores certainly need evaluation. Sanderson and coworkers rightly argue that genetic tests should not be seen as something special. When pointing out that some diagnostic tests may be self evident, e.g. the confirmatory test for trisomy 21 in a person with Down's syndrome, we may have oversimplified matters unnecessarily in our article. Christian Gluud
Cochrane Hepato-Biliary Group, Copenhagen Trial Unit, Centre for Clinical Intervention Research, H:S Rigshospitalet, Copenhagen University Hospital, DK-2100 Copenhagen, Denmark 1. Kjaergard LL, Villumsen J, Gluud C. Reported methodological quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 2001;135:982-989. 2. Kjaergard LL, Frederiksen S, Gluud C. Validity of randomized clinical trials in GASTROENTEROLOGY from 1964-2000. Gastroenterology 2002;122:1157-1160. 3. Kjaergard LL, Gluud C. Funding, disease area, and internal validity of hepato-biliary randomized clinical trials. Am J Gastroenterol 2002;97:2708-2713. Competing interests: None declared |
|||
|
|
|||
|
Joanna D Schaafsma, MD Department of Neurology, University Medical Centre, Utrecht 3508 GA, The Netherlands, Erik Buskens, Gabriel J. Rinkel, and Yolanda van der Graaf
Send response to journal:
|
We read with great interest the paper by Gluud on the need for a more structured methodology to assess diagnostic tests. [1] As already pointed out in other comments, the proposed architecture for diagnostic research into phase I to IV trials as in drug research, may lead to a few problems. First, the added value of a diagnostic test should be evaluated in the context of other test results. [2,3,4] Second, a phased approach should not be considered a blueprint for all different types of diagnostic studies. [5] In fact several others have already pointed out that a randomised controlled trial is not always the best approach nor necessary for many types of diagnostic research. [2,3,6] Other important issues not raised so far are resource use and efficiency. Clearly, implementation of a new diagnostic strategy can have a pronounced effect on medical expenses. These expenses pertain not only to the costs of the diagnostic tool itself, but also to its subsequent events, i.e., therapeutic decisions following the test results and health outcome of the patients. [7] We would like to emphasise that the balance between costs and effects should be determined and compared for the different diagnostic strategies including the new strategy under investigation. Only then a considered policy decision can be reached regarding the diagnostic strategy that results in an optimal trade-off between costs and effects. [8] In conclusion, we fully agree that there is a need for a more structured methodology in diagnostic research. However, this methodology should reflect the natural hierarchy of tests, the different types of diagnostic settings, and last but not least the health-economic aspects. References: 1.Gluud C, Gluud LL. Evidence based diagnostics. BMJ 2005;330:724-6. 2.Moons KGM, Grobbee DE. Diagnostic studies as multivariable, prediction research. J Epidemiol Community Health 2002;56:337-8. 3.Ter Riet G, Bachmann LM, Moons KGM, Bindels PJ, Kessels AGH. Multivariable approaches should not be left out in evidence based medicine. BMJ Rapid Responses[1] 2005 April 4th. 4.Moayyeri A, Soltani A. Evidence based diagnostics: what about customary clinical tests. BMJ Rapid Responses[1] 2005 April 11th. 5.Giocoli G. Diagnostics is a sea: it's time to navigate into it. BMJ Rapid Responses[1] 2005 April 1st. 6.Bossuyt PMM, Lijmer JG, Mol BWJ. Randomised comparisons of medical tests: sometimes invalid, not always efficient. Lancet 2000;356:1844-7. 7.McMahon PM, Araki SS, Sandberg EA, Neumann PJ, Gazelle GS. Cost- effectiveness of PET in the diagnosis of Alzheimer's Disease. Radiology 2003;228:515-22. 8.Buskens E, Nederkoorn PJ, Buijs-Van der Woude T, Mali WPTM, Kappelle LJ, Eikelboom BC, Van der Graaf Y, Hunink MGM. Imaging of carotid arteries in symptomatic patients: Cost-effectiveness of diagnostic strategies. Radiology 2004;233:101-12. Competing interests: None declared |
|||