What can be concluded from the Oxcheck and British family heart studies: commentary on cost effectiveness analysesBMJ 1996; 312 doi: https://doi.org/10.1136/bmj.312.7041.1274 (Published 18 May 1996) Cite this as: BMJ 1996;312:1274
- David Wonderling, research fellow in health economicsa,
- Susan Langham, lecturer in health economicsb,
- Martin Buxton, professor of health economicsa,
- Charles Normand, professor of health policyb,
- Christine McDermott, research fellow in health economicsa
- a Health Economics Research Group, Brunel University, Uxbridge, Middlesex UB8 3PH
- b Department of Public Health and Policy, London School of Hygiene and Tropical Medicine, London WC1E 7HT
- Correspondence to: Professor Buxton.
- Accepted 22 March 1996
Objectives: To provide a commentary on the economic evaluations of the Oxcheck and British family heart studies: direct comparison of their relative effectiveness and cost effectiveness; comparisons with other interventions; and consideration of problems encountered.
Design: Modelling from cost and effectiveness data to estimates of cost per life year gained.
Subjects: Middle aged men and women.
Interventions: Screening for cardiovascular risk factors followed by appropriate lifestyle advice and drug intervention in general practice, and other primary coronary risk management strategies.
Main outcome measures: Life years gained; cost per life year gained.
Results: Depending on the assumed duration of risk reduction, the programme cost per discounted life year gained ranged from £34 800 for a 1 year duration to £1500 for 20 years for the British family heart study and from £29 300 to £900 for Oxcheck. These figures exclude broader net clinical and cost effects and longer term clinical and cost effects other than coronary mortality.
Conclusions: Despite differences in underlying methods, the estimates in the two economic analyses of the studies can be directly compared. Neither study was large enough to provide precise estimates of the overall net cost. Modelling to cost per life year gained provides more readily interpretable measures. These estimates emphasise the importance of the relatively weak evidence on duration of effect. Only if the effect lasts at least five years is the Oxcheck programme likely to be cost effective. The effect must last for about 10 years to justify the extra cost associated with the British family heart study.
A more meaningful measure, cost per life year gained, requires modelling of the longer term effect of that risk reduction
In terms of life years gained, the more intensive British family heart study intervention was more effective but less cost effective than the Oxcheck intervention
The cost effectiveness of these relative to other interventions crucially depends on the assumed duration of the risk reduction, which must persist for at least five years for either programme to be viewed as cost effective
Larger trials with longer follow up would be required to fully assess the long term effectiveness and overall cost effectiveness of population cardiovascular screening
The reports of the main findings of the Oxcheck study1 2 and the British family heart study3 show that it is possible to achieve some change in coronary risk factors in primary care based, nurse led personal health checks. The reaction to both sets of results was generally negative,3 4 suggesting that the improvements were too small to justify the intervention. This reaction ignores the fact that an intervention that is relatively cheap and achieves a little can be more cost effective than one that achieves a lot at a high cost. Before empirical data from the Oxcheck and British family heart studies became available, a recent study attempted to model the cost effectiveness of screening for cardiovascular risk factors.5 The empirically estimated costs and cost effectiveness of the two interventions have now been reported.6 7
Our aim here is to provide a commentary on these two papers: firstly, to compare directly the relative cost effectiveness of the two interventions; secondly, to make some cost effectiveness comparisons with other interventions; and, thirdly, to consider some of the important practical and methodological problems encountered in carrying out these kinds of analyses.
Comparison of methods
The Oxcheck and British family heart studies were separate but concurrent attempts to explore the usefulness of health checks in primary care to reduce heart disease risk. Both studies were population based and nurse led, and both screened several risk factors including blood pressure, cholesterol concentration, smoking habit, weight, and alcohol consumption. The philosophies of the studies were, however, quite different, with a much more intensive follow up in the British family heart study. In the Oxcheck intervention, follow up was negotiated between the nurse and participant on an individual basis governed by a protocol for each risk factor, whereas participants in the British family heart study intervention were invited for follow up according to a strict protocol for both high overall risk and separately for individual high risk factors. Both studies provided intervention for one year only; some of the subjects in the Oxcheck trial were given additional annual screening, but these screens did not influence the observed clinical effect.2 Another difference was that the Oxcheck intervention targeted middle aged individuals but the British family heart study, while targeting middle aged men, invited whole families to attend.
The two independent study teams came together to coordinate the economic evaluations so as to avoid artefactual differences resulting from arbitrary variations of methodological detail or cost information. The costing methods of the two studies were very different, as they were constrained by the data collection strategies developed earlier in the trials' histories. The British family heart study used a bottom up approach, ascribing a cost to the observed trial resources deemed to be necessary in a service setting, whereas the Oxcheck trial adopted a top down approach, necessitating the removal of research costs from the actual trial expenditure. Together the two teams determined which of the resources used were service costs and which were research costs. Where service resource use was determined to be the same for both interventions, such as for the cost per nurse of overheads, an identical cost was used.
In assessing the cost effectiveness of health checks in reducing the burden of coronary heart disease it is important also to consider the costs of differences in treatment consequential on the findings of the health check. This may represent an increase (if, for example, the health check leads to the prescription of lipid lowering drugs) or a decrease if some costs of acute episodes of illness are avoided. In principle the only way to evaluate these costs accurately is to measure prospectively the use of health services over a long period for the subjects in each arm of the trial. The Oxcheck study collected data on general practitioner consultations attended by a sample of subjects and also recorded the prescribing of antihypertensive and antihypercholesterolaemic drugs. The British family heart study recorded the names and numbers of all drugs prescribed as well as the frequency of a variety of health service visits at five of the centres. The estimated mean costs of this broader resource use had large confidence intervals which included zero. Neither study followed up the pattern of such resource use after the interventions had finished. In comparing the two interventions we must therefore ignore the broader and longer term, assuming that the similarity of the studies may imply a similarity in their effects on other sectors of the health service.
In addition to coordinating the costing methodology, the study teams agreed to use a single indicator of effectiveness. Given the relatively short follow up in the studies, it was agreed that this would be the reduction in population coronary risk as derived from the measured changes in Dundee risk score8 as originally reported for the British family heart study in the clinical results3 and calculated for the cost effectiveness paper by the Oxcheck study team.6 Percentage changes in the Dundee risk score are not easily interpreted in themselves but can be used to feed into calculations of population coronary risk to show the likely effects of the interventions on extending life.
Comparison of results
In terms of the clinical effects, a comparison of the British family heart study at one year with those for Oxcheck at three years shows that although the British family heart study had a greater effect on blood pressure, the Oxcheck study was more effective in terms of reductions in both smoking prevalence and cholesterol concentration (table 1). The effectiveness of the Oxcheck study was particularly evident for the female participants. It was estimated that about half of the reduction in blood pressure observed in the British family heart study was due to accommodation bias and that all of the observed reduction in smoking prevalence could be explained by non-returner bias. The Oxcheck study results were also subject to potential non-returner bias. The estimates of coronary risk reduction presented in table 1 have been adjusted to account for these biases and have been converted from an individual risk to a population risk.
The British family heart study achieved a 13% reduction in coronary risk for men and a 10% reduction for women, whereas the Oxcheck study achieved a 7% reduction for men and a 13% reduction for women. There was little difference in their overall effectiveness: the Oxcheck study had a 13% reduction in risk compared with 12% for the British family heart study. Although the Oxcheck intervention seems to be slightly more effective overall, in terms of relative change in risk, if effect is defined as change in life expectancy then it is likely to be slightly less effective than the British family heart study intervention as the majority of the change was for women who are at a lower absolute level of risk.9
The Oxcheck intervention was more effective for women than the British family heart study, which was more effective for men. The Oxcheck study concentrated equally on men and women; in the British family heart study the primary target population was men, although a family centred approach was adopted. A health promotion programme where people are strictly required to attend follow ups regularly and where the whole family is involved may be a more effective way of treating men, whereas negotiating follow up may be more effective for women.
The less intensive Oxcheck programme cost substantially less than the British family heart study: £29.27 per participant (male or female) compared with £66.01 for men and £57.82 for women. For the Oxcheck intervention the cost of a 1% reduction in coronary risk was £4.18 per man and £1.72 per woman, compared with £5.08 per man and £5.78 per woman for the British family heart study. Hence the ratio of the programme cost to the percentage reduction in risk was better for Oxcheck than the British family heart study both for men and for women.
CONVERSION TO LIFE YEAR GAINS
To make broad comparisons between these two interventions and other interventions that aim to reduce the incidence of heart disease, it is necessary to model from the changes in coronary risk factors to estimates of gains in life expectancy. For this commentary we estimated the cost per life year gained of both interventions. To have estimated cost per quality adjusted life year gained would have required knowing the utility values of the long term morbidity.
The implications of a 1% reduction in risk, in terms of coronary events and mortality, vary greatly according to the level of a person's initial risk and therefore according to their age and sex. To illustrate this we estimated the expected life year gains for a man and for a woman aged 50 for both studies. To calculate the life years gained, a life table10 giving the disease specific mortality9 by age for a hypothetical cohort of 50 year old men was constructed, and the coronary death rate was reduced by the appropriate percentage reduction in coronary risk for a given number of years starting at the age of 50. The life expectancy was calculated before and after the reduction, and the gain in life expectancy was estimated. The same was done for a cohort of 50 year old women. We do not know how long the observed reduction in risk will persist after the initial intervention. The British family heart study followed patients only for one year but the results of the Oxcheck study imply that risk is reduced for at least three years.2 The gains in life expectancy have been estimated assuming that the risk reduction persists for a range of durations, between one year and 20 years.
Tables 2-4 present the discounted and undiscounted estimates of life years gained. Although it could be argued that such benefits should not be discounted,11 or should be discounted at a lower rate than costs,12 most existing studies have discounted benefits at the same rate as costs; therefore for the purposes of comparison with other evaluations in the published literature the discounted estimates are the most relevant.
The mean number of life years gained per person screened from the British family heart study ranged between 0.0062 (assuming a one year effect) and 0.2035 (assuming a 20 year effect) for men and between 0.0011 and 0.0626 for women. In comparison, the mean number of life years gained from the Oxcheck study ranged between 0.0034 (assuming a one year effect) and 0.1093 (assuming a 20 year effect) for men and between 0.0018 and 0.1065 for women. Although these gains in life expectancy seem small, it is important to note that these are the expected gains for the population—at an individual level, some may gain many years and others none at all. The cost per discounted life year gained associated with British family heart study ranged between £1100 (assuming a 20 year effect) and £24 400 (assuming a one year effect) for men and between £3300 and £144 500 for women. In comparison, the cost per discounted life year gained associated with the Oxcheck study ranged between £900 (assuming a 20 year effect) and £20 900 (assuming a one year effect) for men and between £1000 and £41 800 for women. The cost effectiveness of the interventions is, not surprisingly, highly dependent on the assumption of how long the risk reduction persists.
It seems that, for men and women together, the British family heart study was more effective, in terms of increasing the life expectancy of the population, than the Oxcheck study but was less cost effective (table 4). The incremental cost of those life years gained from the British family heart study intervention over and above the gains from the Oxcheck intervention ranged from £1300 to £45 900 according to the assumed duration of effect.
In addition to reductions in coronary mortality, there may have been changes in other types of mortality, arising either directly from the intervention or from the changes in prescribing and consultation rates. For example, we would expect the reduction in blood pressure to bring about a reduction in mortality from stroke, and similarly the reduction in smoking prevalence would reduce deaths from lung cancer. The estimation of these effects along with the estimation of the long term impact of the programmes on resource use and cost is not feasible in this commentary, given the limitations of the evidence.
COMPARISONS WITH OTHER HEALTH INTERVENTIONS
In terms of aiding health care decision making, the obvious question is how the programmes compare with other health check strategies. A league table compiled from estimates of life years gained for such strategies might be useful in informing decisions on the allocation of resources among competing health care needs. It provides an indication of the relative cost effectiveness of strategies or similar interventions to aid policy makers, but caution must be exercised in interpreting these rankings. It would be wise to be aware of the methodological deficiencies and the dangers of drawing conclusions too hastily or relying on their validity, especially across broad ranges of variables. Some of the problems in comparing interventions in this way have been identified by Mason et al13 in a critical analysis of a league table, produced by Maynard,14 which comprised varied interventions across various diseases. Mason et al noted that the source studies spanned different years of origin, differing discount rates had been used, different settings were being compared, the breadths of economic evaluation varied (different costs had and had not been included), and different health state valuations were being compared. The interpretation of league tables is less problematic when alternative interventions within a narrow clinical area are being compared; in this case, coronary prevention. Nevertheless, because of the imprecision of estimates and problems with comparability, we emphasise the broad rankings rather than the specific figures; these rankings may be helpful in beginning to assess the relative cost effectiveness of multiphasic screening interventions in comparison with other more targeted interventions.
There is a paucity of research into the cost effectiveness of population-wide education and lifestyle programmes in general practice, and few studies present their results in terms of cost per life year gained or saved.15 We searched a number of sources, including the register of cost effectiveness studies16 and the health economics evaluations database.17 The published cost per life year gained was converted to the pound sterling equivalent, using gross domestic product purchasing power parities,18 and then inflated to current (1994) prices, using the gross domestic product nominal prices index.18
The published evidence in table 5 shows a wide range of cost per life year gained estimates for health check interventions.19 20 21 22 23 24 25 It serves to emphasise that the relative cost effectiveness of the British family heart and Oxcheck studies is critically dependent on the presumed length of effect of the risk reductions from the one year programme. If the conservative assumption of a one year effect is adopted then these two studies would appear close to the bottom of this league table: neither programme for men or women would be relatively cost effective. If a 10 or 20 year effect is assumed, the programmes are very cost effective for men and reasonably so for women. The Oxcheck evidence implies at least a three year effect,2 in which case the programmes are certainly not cost effective for women but begin to look worth considering for men.
Although it helps our understanding of these interventions to look at outcomes for men and women separately, in reality a practice is likely to want to choose a single strategy for both sexes together. In this case what would we conclude? Thresholds for acceptable cost effectiveness ratios in the United Kingdom can only be surmised from policy decisions and should depend on local opportunity costs. However, if the duration of risk reduction is between one and three years then neither programme is likely to be sufficiently cost effective. With a five year risk reduction, Oxcheck is reasonably cost effective but the additional effectiveness of the British family heart study does not justify the additional gains. If the effect persisted for 10 or more years then the incremental cost per life year gained would probably be justified. But all these comparisons consider only programme costs and estimated mortality gains and do not include the wider net clinical and cost effects (which the trials were not powered to detect) and the longer term clinical and cost effects (which would occur well beyond the observational period of the trial). Without much fuller information we cannot unreservedly judge whether investment in these interventions is cost effective.
Difficulties in carrying out cost effectiveness studies in these contexts
The two studies have provided important lessons for the economic evaluation of clinical programmes. It was found that even when trials kept detailed financial records, a number of assumptions were required to translate these into routine service costs, firstly, because of the difficulty of separating service costs from research costs, and, secondly, because financial records often do not include important economic costs such as overheads.
The British family heart study found that the broader costs to the health service, as opposed to the narrower programme costs, may be substantial and are therefore important when it comes to estimating cost effectiveness. Such costs are highly variable, more variable than narrowly defined clinical variables, and therefore much larger sample sizes will be required if they are to be measured with any amount of precision in the future. The broader clinical effects, like the broader costs, may have a considerable influence on the cost effectiveness of the programmes. The change in coronary mortality brought about by the interventions will be augmented by changes in other types of mortality and by changes in morbidity, both of which may need to be measured if the effectiveness and cost effectiveness of health promotion programmes are to be assessed methodically.
Health check interventions are likely to have effects in the longer term as well as broader short term effects. The duration of the observed reduction in coronary risk is unknown, although the Oxcheck study suggests that it can last at least three years. The relation between risk factors and the number of events, in this study defined by the Dundee risk equation, is still not certain. Future coronary events will inevitably impose further costs on the health service as well as contributing to mortality and morbidity. Fully understanding the cost effectiveness of health promotion programmes requires that we measure reliably broader and longer term costs and benefits. This poses a challenge for future clinical and economic evaluations.
We thank our clinical colleagues from these studies for their advice, support, and comments but emphasise that the views expressed in this paper are those of its authors alone.
Funding The British family heart study was funded by the Family Heart Association with an educational grant from Merck Sharp and Dohme, the family health service authorities and Fife Health Board, Boehringer Mannheim UK, Wessex Regional Health Authority, the Health Education Authority, the Scottish Home and Health Department, and the Department of Health, which provided specific funding for the economic analysis. The Oxcheck study was funded by the Imperial Cancer Research Fund. SL was supported by the Health Gain Project, which is jointly funded by North Thames Regional Health Authority and the Health Education Authority.
Conflict of interest None.