Effectiveness and efficiency of different guidelines on statin treatment for preventing deaths from coronary heart disease: modelling studyBMJ 2006; 332 doi: https://doi.org/10.1136/bmj.38849.487546.DE (Published 15 June 2006) Cite this as: BMJ 2006;332:1419
- Douglas G Manuel, scientist ()1,
- Kelvin Kwong, graduate student1,
- Peter Tanuseputro, research coordinator1,
- Jenny Lim, research coordinator1,
- Cameron A Mustard, president and senior scientist2,
- Geoffrey M Anderson, professor3,
- Sten Ardal, director4,
- David A Alter, scientist1,
- Andreas Laupacis, chief executive officer1
- 1 Institute for Clinical Evaluative Sciences G106-2075 Bayview Avenue, Toronto, Ontario M4N 3M5 (Note to Author: We give only one address per author)
- 2 Institute for Work and Health, Toronto
- 3 Department of Health Policy, Management and Evaluation, University of Toronto
- 4 Central East Health Information Partnership, Toronto
- Correspondence to: D G Manuel
- Accepted 1 April 2006
Objective To examine the potential effectiveness and efficiency of different guidelines for statin treatment to reduce deaths from coronary heart disease in the Canadian population.
Design Modelled outcomes of screening and treatment recommendations of six national or international guidelines—from Canada, Australia, New Zealand, the United States, joint British societies, and European societies.
Data sources Details for 6760 men and women aged 20-74 years from the Canadian Heart Health Survey (weighted sample of 12 300 000 people) that included physical measurements including a lipid profile.
Main outcome measures The number of people recommended for treatment with statins, the potential number of deaths from coronary heart disease avoided, and the number needed to treat to avoid one coronary heart disease death with five years of statin treatment if the recommendations from each guideline were fully implemented.
Results When applied to the Canadian population, the Australian and British guidelines were the most effective, potentially avoiding the most deaths over five years (> 15 000 deaths). The New Zealand guideline was the most efficient, potentially avoiding almost as many deaths (14 700) while recommending treatment to the fewest number of people (12.9% of people v 17.3% with the Australian and British guidelines). If their “optional” recommendations are included, the US guidelines recommended treating about twice as many people as the New Zealand guidelines (24.5% of the population, an additional 1.4 million people) with almost no increase in the number of deaths avoided.
Conclusions By focusing recommendations on people with the highest risk of coronary heart disease, the Canadian, US, and European societies guidelines could improve either their effectiveness (in terms of hundreds of avoided deaths) or efficiency (in terms of thousands of fewer people recommended treatment) in the Canadian population.
Clinical practice guidelines help clinicians and patients make decisions about the appropriate use of treatment by supporting individual assessment of benefits and risks.1 When guidelines apply to a large population, they should also describe the benefits, harms, and costs from a societal perspective. This is especially important if guidelines recommend treatment for a group whose disease risk is small and who represent a considerable proportion of the population.
Because coronary heart disease is common and lipid lowering drugs—most notably statins—are widely dispensed, statins are among the most frequently prescribed drugs in many drug plans.2 Lipid treatment guidelines therefore have important implications both for population health and for healthcare resources.
In this study we assessed six recent national or international lipid guidelines from a population perspective.3–8 We considered three characteristics of each guideline: (a) the number of people that the guidelines recommended for statin treatment, (b) the potential community effectiveness (defined as the potential number of deaths from coronary heart disease that could be prevented if all community members were screened, treated, and compliant according to the guideline),9 10 and (c) the guideline efficiency (defined as the number of deaths from coronary heart disease prevented in relation to the number of people recommended statins and measured by the overall number needed to treat).11 We examined these three characteristics for each guideline in the Canadian population.
Participants and methods
Figure 1 shows how we compared guideline recommendations. We used data from the Canadian Heart Health Survey, a population based survey that collected physical measures including blood pressure, weight, height, and blood lipid concentrations in 6760 people aged 20-74 years,12 to estimate the risk of coronary heart disease and cardiovascular disease (coronary heart disease plus stroke) of individual respondents.13 14 The survey, conducted during 1990-2, is well suited for estimating the potential effectiveness of statins since they were not standard treatment for heart disease at this time (see appendix on bmj.com for detailed discussion of data and analysis methods).15
We identified national guidelines by searching Medline, EMBASE, TRIP, and National Guideline Clearinghouse, for articles published before 31 December 2004 using the key words “lipidemia, cholesterol blood level, cardiovascular risk, coronary heart disease, risk assessment, ischemic heart disease” either alone or in combination. We found eight guidelines that were used nationally and selected the most current guideline from those countries whose population risk of heart disease and whose healthcare resources were similar to those of Canada (from Canada, Australia, New Zealand, the United States, joint British societies, and European societies.3–8 We excluded the guidelines for Japan and South Africa.16 17
We screened each respondent to the Canadian Heart Health Survey for coronary risk and assigned treatment according to each guideline's recommendations (see table B on bmj.com for details). The US guideline had optional treatment recommendations, which we considered separately. Guidelines assigned risk and stratified people into high, medium, or low risk groups using different end points for heart or cardiovascular disease (five or 10 year risk of event or death). We used each guideline's risk charts to assign respondents into their appropriate risk category. All guideline risk charts used the Framingham heart study data or algorithms, except the European societies, which used the SCORE (systematic coronary risk evaluation) study data in their most recent guideline.18 Respondents with a history of heart disease were automatically placed in the highest risk category in all guidelines.
To compare the effectiveness and efficiency of the guidelines, we estimated five year mortality from coronary heart disease (using the Framingham algorithm19) and 10 year mortality from cardiovascular disease (using the SCORE algorithm18). For respondents with cardiovascular disease, we used the observed mortality risk of Ontario residents (a third of the national population) because Framingham or similar equations are not available for people with cardiovascular disease. For those people recommended statin treatment, we calculated the number of deaths from heart disease potentially avoided, using the product of five year baseline risk and an estimate of 27% relative reduction in mortality from cardiovascular disease.20 We adjusted all estimates using the survey sampling weights to provide population based estimates for the Canadian population aged 20-74 years in 1990-2 (12 300 000 people).
Figure 2 shows how the six guidelines differ in the proportion of Canadians aged 20-74 years recommended statin treatment and the number of deaths from heart disease potentially avoided if all the people recommended statins took them correctly for five years. (Table C on bmj.com shows details of, and deaths from, heart and cardiovascular disease in each risk group for all the guidelines.) The Australian, British, and the optional US guidelines were the most effective, potentially avoiding the most deaths from heart disease over five years (> 15 000 deaths). The New Zealand guideline was the most efficient (number needed to treat = 108), potentially avoiding almost as many deaths as the most effective guidelines (14 700 deaths) while recommending statin treatment to the smallest proportion of the Canadian population (12.9%). The optional US guideline, which was the most liberal, recommended treatment to almost twice as many people (24.5%, number needed to treat = 198).
We created a curve of “optimal community effectiveness” to show the gap between the potential and optimal effectiveness and efficiency of the different guidelines (fig 2). The optimal community effectiveness curve represents the maximum number of deaths from heart disease avoided over five years of statin treatment if treatment were distributed according to individuals' baseline risk of developing heart disease. The curve assumes that people with the highest risk are treated first and that all people receive the same relative benefit from statin treatment. The effectiveness gap for a guideline (such as line A in fig 2) represents the extra number of deaths that could be avoided while treating the same number of people as that guideline recommends. The efficiency gap (line B) represents the minimum number of people that could be treated to achieve the same number of avoided deaths.
The effectiveness gap for the six guidelines fell into two groups. The effectiveness of the New Zealand, British, and Australian guidelines approached the optimal level. These three guidelines had an average number needed to treat that was 95% of the optimal number needed to treat, whereas the European societies, Canadian, and US guidelines had an average of 81%. In absolute terms, the European, Canadian, and US guidelines could potentially avoid 2000-3000 extra deaths over five years while recommending treatment to the same number of people. Alternatively, they could potentially avoid the same number of deaths but reduce the number of people recommended treatment by a third to a half. The guidelines with the largest effectiveness and efficiency gaps failed to recommend treatment to people at high-risk of cardiovascular or heart disease, usually by introducing target lipid concentrations that excluded many high risk people (see table B on bmj.com).
The efficiency and effectiveness gaps were similar when we estimated them for different outcomes (coronary heart disease event or death from cardiovascular disease) or used different risk algorithms (SCORE for either high or low risk populations) (data not shown).
We found that different national guidelines on statin treatment varied considerably in their potential to prevent death from heart disease. More importantly, we found that the guidelines that were potentially most effective in preventing deaths often recommended treatment to fewer people than less effective guidelines. Because the target populations for lipid treatment guidelines are large and death from heart disease is a relatively common outcome, small changes in guidelines have large consequences, potentially resulting in hundreds to thousands of avoided deaths each year in the Canadian population. Currently, more than US$1bn is spent annually on statins in Canada, not including laboratory and physician costs.21 This means that changing from one guideline's recommendations to another's could cost (or save) hundreds of millions of dollars at the national level.
Possible explanations for differences
Three factors could account for the differences in the potential efficiency and effectiveness of a guideline: (a) the way the guideline developers interpreted findings from clinical trials of statins, (b) the way the guidelines incorporated the findings from clinical trials into screening and treatment recommendations, and (c) differences in the risk profile between the Canadian and other target populations (that is, that the guidelines might be well suited to the target population of origin but not suited for the Canadian population).
Interpretation of clinical trials— This is similar for all current guidelines: all reviewed the same clinical trials, and all concluded that statins are effective in reducing coronary heart disease events for people who have a high risk for these events, even when they have low serum concentrations of low density lipoprotein cholesterol.22
Incorporation of clinical findings into recommendations— This probably accounts for the largest variation in guideline effectiveness and efficiency. Unfortunately, guidelines seldom include an explicit description of how their recommendations were developed. In our study, guideline effectiveness was strongly influenced by the proportion of high risk people who were recommended statins. The New Zealand, Australian, and British guidelines recommended statins to almost all high risk people. The other guidelines recommended treatment if lipid concentrations were above specific thresholds. Guidelines that recommended screening and treatment of many low risk people were less efficient, as measured by the average number needed to treat, often with little gain in population effectiveness. The most efficient guidelines had an explicit baseline risk below which statin treatment was not recommended unless there was familial hyperlipidaemia with high concentrations of low density lipoprotein cholesterol.
Differences in population risk— Differences between Canada and other countries in population risk are unlikely to have had a large influence on the differences in guidelines' effectiveness and efficiency. The New Zealand, British, and Australian guidelines are the most efficient and effective in the Canadian population because they focus recommendations on high risk people, who bear the vast majority of cardiovascular risk in Canada (see table A on bmj.com). Inefficient guidelines—those that recommend statins for low risk people—will seem more efficient and effective under two hypothetical situations: firstly, if the burden of total population risk is distributed mostly among low risk people; or, secondly, if statins have a higher relative benefit in low risk people than high risk people. Because these two scenarios are unlikely, we expect that the New Zealand, British, and Australian guidelines will be the most effective and efficient recommendations in most, if not all, developed countries.
Our results highlight the importance of describing statin benefit in absolute terms to help doctors and patients in their treatment decisions. Most guidelines describe statin benefit only in relative terms (that is, 20% to 40% reduction in heart disease outcomes). However, many people who were recommended statins in the Canadian and US guidelines had a low risk of heart disease and therefore had only a small absolute benefit. For example, in the Canadian recommendations a quarter of the low-risk group (118 000 people) had a number needed to treat of at least 23 000 to prevent one death from heart disease with five years of statin treatment (calculations not shown).
Limitations of study
Our study is limited by the simplified examination of the different guidelines. Firstly, most guidelines suggest an individually tailored approach to lipid evaluation and treatment, which we could not achieve. However, although such an approach is important at the level of individual patients, it would not appreciably change the number of patients treated or the potential benefit of statins at the population level. Secondly, we assumed that everyone for whom statins were recommended would receive and take them. Thus, we estimated a population effectiveness that is greater than would be achieved in the real world. Of further concern, there is a well described tendency to fail to prescribe statins to people with the greatest potential benefit.23 24 Thirdly, we assumed that statins have the same relative benefit for all risk groups. Some authors suggest that statin benefit is lower for low risk people,25 but, if that were the case, guidelines that recommended treatment of low risk people would be even less effective and efficient than we estimated.
Another limitation is our use of risk algorithms to estimate Canadian risk of death from heart and cardiovascular disease. Estimating population risk using risk algorithms is attractive because this method uses readily available data and is an approach that is already used by all guidelines. However, algorithms may incorrectly estimate the risk of death. For example, there are concerns that the Framingham algorithm overestimates low levels of risk in some populations.26 That stated, our findings based on the Framingham algorithm are reasonable for at least four reasons. Firstly, the Framingham algorithm has been validated in populations similar to Canada's.27 Secondly, the results did not change when we used different algorithms and outcomes in our calculations. Thirdly, measurement error should bias results in favour of guidelines based on the Framingham algorithm compared with guidelines based on the SCORE algorithm, whereas this was not observed. Lastly, if the Framingham algorithm overestimated low risk of heart disease, then the least effective and efficient guidelines in our study would perform even more poorly in reality because only they recommended treatment for low risk people.
Guidelines that affect a large number of people and a large volume of healthcare resources should strive for maximum effectiveness and efficiency. To improve the effectiveness of guideline recommendations, we encourage guideline developers to examine their recommendations against their population's “optimal community effectiveness” curve. Treatment recommendations should have the potential to reduce deaths from heart disease in numbers as close as possible to the optimal effectiveness. The New Zealand, Australian, and British recommendations achieved this; the European, Canadian, and US guidelines did not.
Given a fixed amount of resources, recommendations that are effective will also encourage an efficient use of those resources. However, guideline developers do not typically know to what extent resources are or should be available to implement recommendations. That stated, the slope of the “optimal community effectiveness” curve shows what planners, doctors, and patients already know—that statins reduce mortality from heart disease, but that returns diminish as more, low risk people are screened and offered treatment. Guideline developers should involve payers and patients when deciding how far along the effectiveness curve recommendations should go. To help these discussions, guideline developers can plot proposed recommendations on their population's “optimal community effectiveness” curve.
What is already known on this topic
National guidelines on the use of statins to prevent coronary heart disease have important implications for population health and use of healthcare resources
What this study adds
Six national and international guidelines varied considerably in their potential to prevent death from heart disease
The three most efficient and effective guidelines recommended treatment to fewer people while at the same time potentially reducing more deaths compared with the other guidelines
The less effective and efficient guidelines failed to recommend treatment to people at high risk of cardiovascular or heart disease, usually by introducing target lipid concentrations that excluded many such people, but recommended treatment of lower risk people
Further details of data used and analysis methods are on bmj.com
Contributors DGM, GMA, CAM, and DAA conceived of and designed the study. DGM obtained the data. DGM, PT, JL, and KK analysed and interpreted the data. DGM, KK, DAA, SA, JL, CAM, and AL wrote the manuscript, and all authors critically revised it. DGM, PT, JL, and KK performed the statistical analysis and provided administrative, technical, or material support. DGM is guarantor for the study.
Funding Support for this project was received from the Canadian Population Health Initiative. The Institute for Clinical Evaluative Sciences is supported in part by a grant from the Ontario Ministry of Health. However, the results, conclusions, and opinions expressed are those of the authors alone.
Competing interests None declared
Ethical approval From Sunnybrook and Women's College Health Sciences Centre and privacy impact assessment from the Institute for Clinical Evaluative Sciences.