Education and debate

Measuring outcomes in economic evaluations

David Torgerson, research fellow ^a,  James Raftery, professor ^b. 

^a National Primary Care Research and Development Centre, Centre for Health Economics, University of York, York YO1 5DD, ^b Health Economics Facility, Health Services Management Centre, University of Birmingham, Birmingham B15 2RT

Correspondence to D Torgerson

To make judgments about efficiency economic evaluation of health care has to compare health outcomes, however measured, with costs. Three main approaches exist to measuring outcomes: clinical end points, quality of life measures, and willingness to pay.

The simplest outcome measure to use in a trial is a clinical one, such as a reduction in the number of strokes or changes in blood pressure. Health economists use such measures to construct cost effectiveness ratios.¹ For example, in a trial aimed at preventing hip fractures a cost effectiveness ratio might be cost per averted hip fracture.

Measuring outcome in terms of clinical endpoints has the disadvantage that comparisons between different healthcare treatments are difficult. This is only partly solved when trial endpoints include mortality. Although estimates of cost per life gained or life year gained allow comparisons between very different therapies, using survival as an outcome measure for an economic evaluation is problematic. Firstly, few clinical trials are powered to detect mortality differences. Secondly, many treatments affect morbidity rather than mortality. Thirdly, even when survival is an appropriate end point, reductions in mortality may be at the expense of reductions in quality of life.

Measures of quality of life which go beyond both clinical and mortality end points are becoming more common. Quality of life measures may be condition specific, generic, or utility based. Condition specific measures comprise questions about particular symptoms which treatment aims to resolve. For example, the Roland and Morris back pain scale asks about a patient's back pain and how this limits functional activity.² In contrast, a generic measure such as the SF36³ asks questions about an individual's general health. Finally, utility measures, such as the EuroQol,⁴ go beyond generic measures: they have interval/ratio properties and are preference based.

By being based on scales with interval properties, utility measures enable different interventions to be compared. If health intervention A improved patients' health, on average, by 10 points on a utility scale and intervention B by 5 points, then intervention A is twice as effective. Most generic quality of life measures lack interval properties.

Furthermore, the valuation of utility measures is based on societal preferences. Although many condition specific measures are based on patients' valuations, those used in generic and utility based measures tend to use population valuations.

Utility measures tend to be relatively insensitive to important changes in health status. Unless sample sizes are extremely large, reliance on utility measures alone runs the risk of type II errorsconcluding that there is no important quality of life gain when there is. Health economists often recommend using utility measures alongside other, more sensitive, measures of outcomes.

Sometimes the benefits of healthcare interventions go beyond clinical or quality of life changes. Most couples undergoing in vitro fertilisation will not have a baby: for those, coming to terms with their infertility may be a benefit.⁵ Similar considerations apply to non-health benefits, such as respect of patients' autonomy and dignity. To measure these benefits, the techniques of willingness to pay and conjoint analysis have been suggested.⁶

In willingness to pay, the outcome of a healthcare procedure and its alternative(s) are described and patients asked how much they would be willing to pay. Procedures with the highest values are preferred. Besides capturing non-health dimensions, this technique enables benefits to be expressed in monetary terms, allowing cost benefit analysis to take place.¹ Use of willingness to pay in evaluating cystic fibrosis screening showed that benefits other than knowing carrier status were important.7

Conjoint analysis presents patients with a list of pairwise choices of a health intervention.⁶ For example, whether patients preferred their general practitioner's surgery to have longer opening times combined with a night time deputising service or shorter day time opening combined with the general practitioners doing their own on call. The various attributes of alternatives can be weighted to generate utilities. That option with the best cost utility ratio is the most efficient.

Footnotes

These notes are edited by James Raftery (J.P.RAFTERY{at}bham.ac.uk)

References

1.	Palmer S, Byford S, Raftery J. Types of economic evaluation. BMJ 1999; 318: 1349[Free Full Text].
2.	Roland MO, Morris RW. A study of the natural history of back pain 1: development of a reliable and sensitive measure of disability in low back pain. Spine 1983; 8: 141-144[Medline].
3.	Garratt AM, Ruta DM, Abdalla MI, Buckingham JK, Russell IT. The SF-36 health survey questionnaire: an outcome measure suitable for routine use within the NHS? BMJ 1993; 306: 1440-1444.
4.	EuroQol Group. EuroQol: a new facility for the measurement of health related quality of life. Health Policy 1991; 16: 199-208.
5.	Ryan M. Using willingness to pay to assess the benefits of assisted reproductive techniques. Health Economics 1996; 5: 543-558[Medline].
6.	Ryan M, Shackley P. Assessing the benefits of health care: how far should we go? Quality in Health Care 1995; 4: 207-213[Medline].