Making sense of risk information on the webBMJ 2003; 327 doi: https://doi.org/10.1136/bmj.327.7417.695 (Published 25 September 2003) Cite this as: BMJ 2003;327:695
- Steven Woloshin, associate professor of medicine (, )
- Lisa M Schwartz, associate professor of medicine,
- Andrew Ellner, assistant medical editor
- VA Outcomes Group, VA Medical Center, 215 N Main Street, White River Junction, VT 05005, USA
- BMJ Knowledge, London WC1H 9JR
Don't forget the basics
Web based risk calculators are among the newest information resources available to people who want to understand the health risks they face. The advantage of these calculators is their ability to generate tailored risk information based on personal factors. But their usefulness depends on their accuracy and whether they are complete or balanced. To focus on the second issue, we present a hypothetical case history highlighting some elements of good (and not so good) risk communication.
The case: Mr Jones is a 55 year old white man worried about prostate cancer after reading about a politician who had recently been diagnosed with the disease. His first search effort–using the Google search engine to look for “prostate cancer and risk calculator” yields 8410 hits. The first hit (http://www.yourcancerrisk.harvard.edu/) seems perfect. This asks him questions about himself and, based on his age, ethnic group, family history, height, vasectomy history (he had one), and dietary habits (he eats ext-link ≤5 servings of food with animal fat a day and ≤5 servings of tomato based foods a week), tells him his risk is above average. He is now even more worried and calls his doctor.
Mr Jones's doctor explains that three things are missing in this risk assessment: clarity about the risk, context, and an acknowledgment of uncertainty.
Clarity means knowing what specific risk is under consideration (is this about getting or dying of the disease?), a number (the probability), and the time period associated with that numbe. Just being told that his risk is above average does not tell Mr Jones the chance that he will get or die of prostate cancer in some defined time frame.
A limited number of calculators are available that can generate quantitative risk estimates for various diseases such as breast cancer in the next five years,1 lung cancer in the next 10 years,2 or the combined chance of myocardial infarction or death over 10 years.3 Most, however, calculate only the chance of developing a specific disease, not the chance of dying from it. The US federal government's surveillance, epidemiology, and end results (SEER, http://seer.cancer.gov/) site provides look-up tables and an interactive calculator for estimating the risk of both getting and dying of most cancers.4 Its disadvantage is that the output can be tailored only to age, sex, and race. Its advantages are the broad array of cancers included and flexibility in specifying the time frame. Together Mr Jones and his doctor learn that over the next 10 years a 55 year old white American man's chance of getting prostate cancer is about 40 in 1000 whereas his chance of dying of it is 2 in 1000.
Nevertheless, even with the clear statement of risk, something is missing–is a 2 in 1000 chance of dying over 10 years a big or a small risk? Mr Jones needs a context–for example, how his risk of prostate cancer compares with that of the average person; or how his chance of dying of prostate cancer compares with his chance of dying of other cancers. He can get some context simply by comparing his chance of getting prostate cancer with his chance of dying of it: from this he learns that prostate cancer is not uniformly fatal since so many more men get it than die of it.
He can use the SEER site to get more context by calculating his risk of other cancers. Even more helpful would be to have data on common causes of death and death in general. This sort of benchmark information is available in the form of risk charts.5 These charts make it easy for people to compare their chance of dying of various causes and all causes. From these Mr Jones sees that the 2 in 1000 risk of dying of prostate cancer is lower than his chance of dying of colon cancer (4 in 1000) or heart disease (51 in 1000 if he is a smoker, 20 in 1000 if he has never smoked) in the same time frame and much less than his chance of dying of anything in that time (217 in 1000 if he smokes and 93 in 1000 if he has never smoked).
Mr Jones's doctor also points out the third problem: Jones has not been given any sense of the uncertainty inherent in risk predictions. To the extent that a calculator tailors predictions, it is important to know something about the strength of evidence behind the factors used in generating risk estimates. Age, for example, has been consistently shown to be an important risk factor for prostate cancer, whereas height, diet, and vasectomies have not–yet these factors were included in the risk calculator used by Mr Jones.
Unfortunately there is no simple way to judge the quality of risk information. Moreover, Mr Jones, like many people, has had little experience of thinking about risk, let alone quantifying it. A new resource–a tutorial on the BMJ Knowledge BestTreatments website6–is now available to help patients understand where numbers on risk come from (medical studies) and how to interpret them. The tutorial helps people conceptualise the probabilistic aspects of risk. It also explains the importance of how messages are presented (for example, a drug reducing someone's risk of disease from 2% to 1% can be said to reduce their risk by 1% or by half). Finally, it suggests ways for people to think about medical risks in context by comparing them with non-medical risks.
New communication technology now gives the public greater access to health information than ever before. But no matter how sophisticated the source, it takes more than good data to make useful information. We believe that risk presentations that follow the basic principles summarised in the box would help patients find meaningful answers to the questions they are asking.
Elements of risk and selected sources
Clarity about the risk
What risk is being discussed? What are the numbers?
What is the time period? How dangerous is the disease?
Getting and dying from most cancers at specified times (National Cancer Institute's surveillance,epidemiology and end results website, http://seer.cancer.gov/query/)
Getting breast cancer in the next 5 years (National Cancer Institute's breast cancer risk assessment tool, http://bcra.nci.nih.gov/brc/)
Myocardial infarction or cardiac death in next 10 years (National Cholesterol Education Program heart risk calculator,http://hin.nhlbi.nih.gov/atpiii/calculator.asp?usertype=prof)
Getting lung cancer in the next 10 years (long term smokers) (Memorial Sloan Kettering Cancer Center lung cancer risk assessment tool, www.mskcc.org/mskcc/html/12463.cfm)
How does my risk compare to risk of an average person? similar disease? leading causes of death? all-cause mortality?
Dying from various and all causes in the next 10 years (risk charts http://jncicancerspectrum.oupjournals.org/cgi/content/full/jnci;94/11/799)
Has the risk factor been shown to change risk (is it really a risk factor)? Does the risk factor really cause disease? How precise is the risk estimate?
Competing interests SW and LS none declared. AE is employed by BMJ Knowledge, which owns the BestTreatments website.