Intended for healthcare professionals

Rapid response to:

Information In Practice

Communicating accuracy of tests to general practitioners: a controlled study

BMJ 2002; 324 doi: https://doi.org/10.1136/bmj.324.7341.824 (Published 06 April 2002) Cite this as: BMJ 2002;324:824

Rapid Response:

Understanding Probability Information

We congratulate Steurer and colleagues(1) on an excellent and timely study. It has long been established in the cognitive psychology literature that people have difficulties understanding probability information(2). More recently, research has taken a more positive angle, looking at how probabilities can be communicated more effectively and how people can be taught to interpret them(3)(4). Unfortunately these findings have rarely been applied to actual health screening as done by Steurer et al.

Our recent study of medical undergraduates demonstrated the difficulties of integrating information on test sensitivity and specificity, and the base rates of conditions. 194 students were presented with one of four scenarios giving the same numerical information in slightly different ways. The situation described was either health- related (prenatal serum screening for Down’s syndrome) or neutral (screening engine parts for faults), and the information was either presented as percentages (e.g. 90%) or frequencies (e.g. 9 out of 10). Participants were asked to estimate the probability that a positive test result indicated the presence of an abnormality, given the sensitivity of the test (90%), the false positive rate (i.e. 1-specificity) (1%), and the base rate of abnormalities (1%) (adapted from(2)). (Participants were informed that numbers were for the purpose of the study only).

It was hypothesised that presenting the information as a health scenario in percentages would result in more pronounced and consistent errors. Only 5.2% of participants made correct estimates (10 out of 194), too few to allow valid chi-square analysis of differences between scenarios. The incorrect responses clustered around 1% and 90%, apparently a consequence of participants paying either too little or too much attention to the base rate (i.e. 1%, the prior probability of any individual having the abnormality). Base rate neglect was the most common error when information was presented as percentages, and over-attention to the base rate was most common in the frequency conditions. A smaller study of midwifery students showed similar patterns of errors. These tendencies towards certain answers suggest that the information is not interpreted in the same way if it is presented as frequencies rather than percentages, however neither presentation makes the reader more likely to interpret it correctly.

Work continues on the responses of healthcare professionals, and pregnant women attending a booking-in appointment. If informed uptake of screening tests is to be achieved, then methods of increasing the understanding of professionals and users must be sought.

Helen Adams, Research Student Dr Ros Bramwell, Senior Lecturer, Department of Clinical Psychology, University of Liverpool

Corresponding author: Helen Adams, Department of Clinical Psychology, Whelan Building, Quadrangle, Brownlow Hill, Liverpool. L69 3GB UK tel. 0151 794 4160 fax. 0151 794 5537 email: helena1@liv.ac.uk

References 1. Steurer J, Fischer J, Bachmann L, Koller M, ter Riet G. Communicating accuracy of tests to general practitioners: a controlled study. BMJ 2002,324:824-6

2. Hammerton MA. Case of Radical Probability Estimation. J Exp Psychol 1973;101(2):252-4

3. Gigerenzer G, Hoffrage U. How to Improve Bayesian Reasoning Without Instruction: Frequency Formats. Psychol Rev 1995;102(4):684-704

4. Sedlmeier P, Gigerenzer G. Teaching Bayesian Reasoning in Less Than Two Hours. J Exp Psychol 2001;130(3):380-400

Competing interests: No competing interests

27 July 2002
Helen M Adams
Research Student
Ros Bramwell
Department of Clinical Psychology, Whelan Building, Quadrangle, Brownlow Hill, Liverpool. L69 3GB