Communicating accuracy of tests to general practitioners: a controlled study
BMJ 2002; 324 doi: https://doi.org/10.1136/bmj.324.7341.824 (Published 06 April 2002) Cite this as: BMJ 2002;324:824
- Johann Steurer (johann.steurer{at}dim.usz.ch)⇑, director,
- Joachim E Fischer, senior research fellow,
- Lucas M Bachmann, research fellow,
- Michael Koller, research fellow,
- Gerben ter Riet, senior research fellow
- Horten-Zentrum für praxisorientierte Forschung und Wissenstransfer, Universitätsspital Zürich, Bolleystrasse 40, Postfach Nord, CH-8091 Zurich, Switzerland
- Correspondence to: J Steurer
- Accepted 22 January 2002
Abstract
Objective: To assess the extent to which different forms of summarising diagnostic test information influence general practitioners' ability to estimate disease probabilities.
Design: Controlled questionnaire study.
Setting: Three Swiss conferences in continuous medical education.
Participants: 263 general practitioners.
Intervention: Questionnaire with multiple choice questions about terms of test accuracy and a clinical vignette with the results of a diagnostic test described in three different ways (test result only, test result plus test sensitivity and specificity, test result plus the positive likelihood ratio presented in plain language).
Main outcome measures: Doctors' knowledge and application of terms of test accuracy and estimation of disease probability in the clinical vignette.
Results: The correct definitions for sensitivity and predictive value were chosen by 76% and 61% of the doctors respectively, but only 22% chose the correct answer for the post-test probability of a positive screening test. In the clinical vignette doctors given the test result only overestimated its diagnostic value (median attributed likelihood ratio (aLR)=9.0, against 2.54 reported in the literature). Providing the scan's sensitivity and specificity reduced the overestimation (median aLR=6.0) but to a lesser extent than simple wording of the likelihood ratio (median aLR=3.0).
Conclusion: Most general practitioners recognised the correct definitions for sensitivity and positive predictive value but did not apply them correctly. Conveying test accuracy information in simple, non-technical language improved their ability to estimate disease probabilities accurately.
What is already known on this topic
What is already known on this topic Many doctors confuse the sensitivity of clinical tests and their positive predictive value
Doctors tend to overestimate information derived from such tests and underestimate information from a patient's clinical history
Most primary research on diagnostic accuracy is reported using sensitivity and specificity or likelihood ratios
What this study adds
What this study adds In a cohort of experienced Swiss general practitioners most were unable to interpret correctly numerical information on the diagnostic accuracy of a screening test
When presented with a positive result alone they grossly overestimated its value
Adding information on the test's sensitivity and specificity moderated these overestimates, and expressing the same numerical information as a positive likelihood ratio in simple, non-technical language brought the estimates still closer to their true values
Footnotes
Contributors JS, GtR, and LMB initiated the study. GtR and JEF contributed to study design and data analysis. All authors helped to interpret data and write the paper. JS and GtR are guarantors for the study.
Funding None declared.
Competing interests None declared.
The study questionnaire appears on bmj.com