- Steven A Grover, directora,
- Ilka Lowensteyn, research fellowa,
- Katja L Esrey, research fellowa,
- Yvonne Steinert, psychologistb,
- Lawrence Joseph, biostatisticiana,
- Michal Abrahamowicza, biostatistician
- a Division of Clinical Epidemiology, Centre for the Analysis of Cost-Effective Care, Montreal General Hospital, Montreal, Quebec, Canada H3G 1A4
- b Department of Family Medicine, Sir Mortimer B Davis Jewish General Hospital, Montreal, Quebec, Canada H3T 1E2
- Correspondence and reprint requests to: Dr Grover.
- Accepted 3 February 1995
Objective: To evaluate the ability of doctors in primary care to assess risk patients' risk of coronary heart disease.
Design: Questionnaire survey.
Setting: Continuing medical education meetings, Ontario and Quebec, Canada.
Subjects: Community based doctors who agreed to enrol in the coronary health assessment study.
Main outcome measure: Ratings of coronary risk factors and estimates by doctors of relative and absolute coronary risk of two hypothetical patients and the “average” 40 year old Canadian man and 70 year old Canadian woman.
Results: 253 doctors answered the questionnaire. For 30 year olds the doctors rated cigarette smoking as the most important risk factor and raised serum triglyceride concentrations as the least important; for 70 year old patients they rated diabetes as the most important risk factor and raised serum triglyceride concentrations as the least important. They rated each individual risk factor as significantly less important for 70 year olds than for 30 year olds (all risk factors, P<0.001). They showed a strong understanding of the relative importance of specific risk factors, and most were confident in their ability to estimate coronary risk. While doctors accurately estimated the relative risk of a specific patient (compared with the average adult) they systematically overestimated the absolute baseline risk of developing coronary disease and the risk reductions associated with specific interventions.
Conclusions: Despite guidelines on targeting patients at high risk of coronary disease accurate assessment of coronary risk remains difficult for many doctors. Additional strategies must be developed to help doctors to assess better their patients' coronary risk.
This study shows that doctors in primary care accurately assess the relative risk of coronary disease in individual patients
They systematically overestimate, however, the absolute risk of coronary disease in individual patients
Doctors in primary care also overestimate the absolute benefits of modification of coronary risk factors including lowering lipid concentration, control of blood pressure, and stopping smoking
Additional strategies must be found to improve the skills in risk assessment among doctors in primary care to support their clinical decision making in individual patients
Doctors in primary care face increasing numbers of expert guidelines on the treatment of hyperlipidaemia to prevent coronary heart disease. In North America and Europe there are numerous and often divergent recommendations on the management of hyperlipidaemia.1 2 3 4 5 6 Despite all this information doctors remain uncertain about the clinical indications for treating the condition.7 8 9 10 11 One source of the confusion may be specific guidelines that suggest targeting all adults for screening and substantial numbers for intervention.12 13 Another problem may be the difficulties that doctors experience when estimating the absolute risk of the “high risk patient,” at whom their best efforts should be targeted.
Several recent studies have shown that doctors in primary care often dismiss consensus recommendations for screening and treating hyperlipidaemia.9 10 11 McBride and colleagues surveyed primary practitioners and identified doctors' concerns that serum lipid concentrations alone are too simplistic to define risk and that further education on multifactorial risk is needed.14 To this end, DeFriese and Fielding proposed individualised health risk appraisals to identify specific risk factors that place patients at increased risk of disease.15 Finally, the American Heart Association supported the use of a multifactorial coronary risk profile to identify patients who might benefit the most from modification of lipid concentrations.16
We recently carried out a randomised controlled trial, the coronary health assessment study, to evaluate the effectiveness of a computerised multifactorial risk profile to help doctors to identify patients at high risk of coronary heart disease. We also evaluated doctors' abilities to assess risk at entry into the study, and we report these results here.
We selected urban and rural communities throughout Ontario, Canada, for recruitment of doctors on the basis of the communities having sufficient doctors. Doctors in each of the 24 selected areas were invited to attend a local evening meeting in continuing medical education; the meetings included a dinner. Two of these meetings had to be held in Montreal, Quebec, after cancellations in Ontario because of low interest among doctors.
The meetings introduced the programme for the coronary health assessment study; at each meeting a one hour lecture on coronary heart disease was presented, covering such topics as primary prevention, multifactorial risk, and the limited accuracy of current guidelines for identifying high risk patients.17 18 The goals of the study were then defined, although the potential participants were not told about the randomised and controlled nature of the study. The doctors attending the meetings were eligible for up to six study credits in continuing medical education from McGill University, Montreal.
EVALUATION BY DOCTORS AND DATA ANALYSIS
During a 15-30 minute reception before the lecture the doctors were offered light snacks and a drink. During the 10-15 minutes after the lecture and before the dinner, the doctors enrolling in the study completed a brief questionnaire that evaluated their beliefs about the prevention of coronary heart disease. Although the issues in the questionnaire had been covered in the preceding lecture, the answers to the questions were not contained in the lecture. The questionnaire included a list of 14 coronary risk factors that the doctors were asked to rate on a 10 point scale according to their importance as risk factors for developing coronary heart disease.
Presented with the cases of two hypothetical patients (a 40 year old man and a 70 year old woman) without diagnosed coronary heart disease, each participating doctor was asked to estimate each patient's absolute risk of developing coronary heart disease (angina, myocardial infarction, or coronary death) over the next eight years; the absolute amount that this risk would be reduced by if one or more risk factors were modified; and the average risk for Canadian men aged 40 or women aged 70. The doctors also had to estimate the change in life expectancy that would be associated with modification of risk factors.
The male patient had a blood pressure of 150/90 mm Hg, a body mass index of 23 kg/m2, and a sedentary lifestyle, and he smoked cigarettes. He had a serum total cholesterol concentration of 6.2 mmol/l, a high density lipoprotein cholesterol concentration of 0.8 mmol/l, a serum glucose concentration of 5.5 mmol/l, and a normal resting electrocardiogram. The female patient had a blood pressure of 180/90 mm Hg, a body mass index of 23, and a sedentary lifestyle, and she did not smoke. She had a serum total cholesterol concentration of 7.2 mmol/l, a high density lipoprotein cholesterol concentration of 1.4 mmol/l, a serum glucose concentration of 5.5 mmol/l, and a normal resting electrocardiogram.
We determined the accuracy of each doctor's estimates of absolute coronary risk and changes in life expectancy by comparing them with the estimates of the coronary heart disease prevention model (the “gold standard”). The risk ratio between the two was calculated as x1/x2 where x1 is the average estimate by the doctor and x2 the model's estimate.
THE CORONARY HEART DISEASE PREVENTION MODEL
The previously published coronary heart disease prevention model was developed to calculate the annual probability of death from coronary heart disease and from other causes by using data from the Framingham heart study, the Canada health survey, and Canadian life tables.19 Risk of coronary heart disease is a function of the patient's age; sex; total and high density lipoprotein cholesterol concentrations; diastolic blood pressure; whether the patient smokes; and the presence of glucose intolerance or left ventricular hypertrophy in the electrocardiogram, or both. This model has been shown to predict accurately the results of prospective randomised clinical trials by using summary data characterising the treatment and control cohorts.19 The average risk of Canadians of a specific age and sex was calculated with 2109 profiles based on a random sample of Canadians aged 30-74 from the Canada health survey.20
In all, 445 doctors attended 24 meetings over seven months. Of these doctors, 253 (57%) agreed to participate in the study and completed the questionnaire. Table I shows the characteristics of the participating doctors. On a 10 point scale (ranging from 1 (very confident) to 10 (not at all confident)) the doctors rated their confidence in their ability to estimate a patient's risk of coronary heart disease, and 142 doctors rated their confidence as 3 or higher.
The doctors rated the relative importance of specific risk factors for developing coronary heart disease (table II). For 30 year olds the doctors rated cigarette smoking as the most important risk factor and raised serum triglyceride concentrations as the least important risk factor; for 70 year old patients they rated diabetes as the most important risk factor and raised serum triglyceride concentrations as the least important. They rated each individual risk factor as significantly less important for 70 year olds than for 30 year olds (paired Student's t tests, all P<0.001).
The doctors' estimates of the eight year coronary risk for the two hypothetical patients and average Canadians were higher than those predicted by the coronary heart disease model (table III). The doctors overestimated coronary risk for the hypothetical male patient and the average 40 year old Canadian man to a greater degree than they did for the hypothetical 70 year old female patient and the average 70 year old Canadian woman.
Although the doctors overestimated coronary risk, they were, on average, accurate in estimating the relative risk of the hypothetical patients compared with the age matched and sex matched Canadian averages. Indeed, for the hypothetical patients the mean relative risk for each was exactly the same as that predicted by the coronary heart disease model.
Doctors' estimates of the long term benefits of modifying one or more specific risk factors were consistently higher than those predicted by the coronary heart disease model (table IV), including those for the benefits of stopping smoking, lowering blood pressure, and treating hyperlipidaemia.
Our results show that experienced doctors assessing the risk of coronary disease were wrong about absolute risk but correct about relative risk. In other words, while they systematically overestimated both the baseline coronary risk of specific patients and the underlying risk of the average adult, their estimates of relative risk were, on average, accurate. This suggests that, with some recalibration and sufficient training, doctors might be able to predict relative and absolute risk quite well.
A recent meta-analysis of cholesterol lowering trials suggested that drug treatment reduces total mortality only among those at high absolute risk of coronary death.21 Despite the controversies surrounding treatment of hyperlipidaemia consensus is growing on targeting high risk patients for primary prevention.18 22 23 24 Doctors' abilities to assess risk, however, have not previously been evaluated. One important hurdle is the definition of a suitable “gold standard” by which doctors' skills may be judged. We have used the coronary heart disease prevention model, a mathematical model of coronary heart disease risk that is based on the Framingham heart study, and Canadian mortality data. The results from the Framingham study have been shown to be generalisable to other American prospective cohort studies, and the coronary heart disease prevention model has been validated and shown to predict the results of clinical trials to reduce coronary heart disease events through modification of risk factors.19 25 So while no generally accepted gold standard exists, we used a validated mathematical model based on what is arguably the best long term prospective cohort study of coronary risk to date.
Many of the doctors' responses were consistent with current medical thinking. Cigarette smoking and diabetes were ranked among the strongest risk factors for coronary heart disease, consistent with data from the Framingham study.16 Among the serum lipids, high density lipoprotein cholesterol was ranked as the most important risk factor for coronary heart disease, total cholesterol and low density lipoprotein cholesterol as slightly less important, and serum triglycerides as least important. These findings are also consistent with data from the Framingham study.26
Despite the doctors' strong knowledge base on risk factors for coronary heart disease our results suggest that doctors need to improve their skills in estimating coronary risk for specific patients. This is not surprising; while a sound understanding of the relative importance of coronary risk factors is a necessary condition for risk assessment, estimates of multifactorial risk require sophisticated calculations based on different conditional probabilities that may well be beyond the average doctor's capabilities.
The doctors in our study were not a random sample of doctors in the community. Rather, our sample represents doctors who were willing to attend an evening meeting and to participate in a continuing study aimed at improving their skills in risk assessment. None the less, our doctors show striking similarities to a random sample of Canadian primary care practitioners surveyed by Tannenbaum and colleagues11: 82% (v 79%) were male; average age was 48 (v 42.4); 40% (v 53%) were in group practice; mean number of patients seen weekly was 149 (v 130); and participation rate was 57% (v 53%).
The results of our study show that doctors tend to overestimate the absolute benefits of reducing risk factors over both the short term (eight year risk) and the long term (life expectancy). This tendency may be particularly important if such implicit estimates form the basis of their risk-benefit assessments when they decide how aggressively to treat individual patients. The doctors in the study said that they were confident about their abilities to evaluate risk of coronary heart disease. This confidence, when communicated to individual patients, could be misleading if doctors' risk estimates are systematically too high.
Recent health economic analyses have suggested that prevention of coronary heart disease may be cost effective compared with other medical interventions, particularly when high risk patients are targeted.27 28 29 30 Given the high prevalence of many of the modifiable coronary risk factors in our communities, including smoking, hyperlipidaemia, hypertension, and sedentary lifestyle, there is the potential to reduce substantially or to delay the development of coronary disease among adults.31 32
Our increasingly constrained health care budgets, however, demand that we accurately identify not only who is at increased risk but also the absolute magnitude of this risk and the potential to reduce it.33 34 The results of this study suggest that, while doctors may be knowledgeable and confident about the primary prevention of coronary heart disease, better skills in risk assessment will be necessary if we are to ensure that their evaluations of patients are right, both relatively and absolutely.
SAG, LJ, and MA are supported by the Fonds de la recherche en sante du Quebec (a health research foundation).
This study was primarily supported by a grant-in-aid from Merck Frosst Canada and was monitored by a scientific advisory committee whose members included Dr David Sackett (chairman), Mr Yale Drazin, Dr Jacques Genest Jr, Dr Brian Gore, Dr Peter McLeod, and Dr Andres Petrasovits. The authors thank Dr Andreas Laupacis for his helpful comments on earlier drafts of this manuscript. The authors acknowledge the substantial efforts of the marketing and sales divisions at Merck Frosst Canada in helping to recruit doctors to the study. We wish to thank specifically Christopher Tan, Dilip Patel, Brian Erhardt, and the coordinators for the meetings in continuing medical education, including Andrea Arnott, Sean Burke, Warren Douglas, Glenn Dow, Jude D'Souza, Eric Frendo, Doug Fulford, Christian Gagnon, Celeste Heyerichs, Kirsten Horlick, Andre Lemery, Harvey Lyman, Phil McCarthy, Fawn McLaughlin-Elder, Lynn McLeod, Ric McRae, Lori Osborne, Ute Repple, Christine Sporer, Rob Steininger, and Michael Wacasey.
We also thank Peter Wolter of Healthcare Marketing Resources for his coordination of the credits in continuing medical education and invaluable help throughout the study and Ms Nadine Bouchard for typing the manuscript.
SAG is a partner in Arcadie Health Assessment Associates, which holds the copyright to the computer risk assessment model described in this study; LJ and MA received remuneration for the development of the model.