Letters

Measuring outcomes in economic evaluations

This economics note is misleading

Author's reply

This economics note is misleading

EDITORThe series of occasional notes on economics has undoubtedly helped clinicians to understand the key concepts and jargon used by economists. The note discussing the use of outcome measures in economic evaluation is, however, misleading.¹

Firstly, condition specific outcome measures and generic quality of life scales should not, in general, be used in cost effectiveness analysis.² The primary reason for this is that such scales do not have the requisite interval properties. The scores produced by the short form-36 questionnaire (SF-36), for example, are little more than transformed ordinal rankings.

Even if interval properties can be shown, the use of generic quality of life scales in cost effectiveness analysis is severely restricted by their production of a set of scores reflecting different domains of health. For example, if the SF-36 is used in an evaluation, it can produce conflicting cost effectiveness ratios with respect to its various dimension scores.² It is best to restrict the set of outcome measures appropriate for cost effectiveness analysis to those measured in natural units.³ Following such an approach would limit the direct use of health scales to cost consequences analysesthat is, the presentation of cost and outcome data in a disaggregated format.⁴

Secondly, the paper restricts its commentary on cost utility analysis to the use of utility based quality of life scales as the measure of outcome. Use of such scales is increasing and will become the dominant form of utility measurement in clinical research. However, health state valuation techniques, such as the standard gamble and the time trade off,⁵ may be used to produce study specific utility values. Such techniques are also the basis for the values awarded by the utility scales.

Thirdly, the economics note should point out that the use of willingness to pay and conjoint analysis is developmental, and their use in cost benefit analysis (as opposed to simple investigation of the patient's preferences) is extremely rare. It should also be noted that both of these approaches are valuation techniques, as opposed to outcome measures. As such, they can be used only if the effects or characteristics of the interventions have already been measured.

We recognise that this series of short articles must simplify important issues to remain concise. As it stands, however, this economics note is likely to mislead readers regarding good practice and commonly accepted practice.

Simon Dixon, lecturer. 
Colin Green, research fellow. 
Sheffield Health Economics Group, School of Health and Related Research, University of Sheffield, Sheffield S1 4DA

Author's reply

EDITORWe disagree with Dixon and Green that our note is misleading about what is good practice and commonly accepted practice. We did not suggest that cost effectiveness ratios should be constructed using non-utility measures such as the short form-36 questionnaire (SF-36). Rather, we argued for the inclusion of utility based measures as well as other quality of life measures. But utility measures are relatively insensitive to important changes in both clinical quality of life and quality of life relevant to the patient, which is one reason why we are reluctant to recommend that they be used alone.

In a recent evaluation of an intervention for low back pain the experimental group of patients had on average a 2% increase in utility (as measured by the EuroQol (European quality of life instrument)), but this was not significant (P=0.47).¹ Both of the two back-pain condition specific measures, however, showed a significant improvement in back pain (P=0.01 and P=0.02), which suggests that the non-significant utility benefit as measured by the EuroQol was true. Had these condition specific measures shown no improvement there would be doubt as to whether there was any utility benefit of this intervention. Thus, as we suggested in our note, the use of non-utility outcome measures can aid interpretation of a trial's results and compensate for poor sensitivity of utility measures. We think that it is good practice to include both a utility based measure of outcome and non-utility measures of quality of life.

Dixon and Green's second point regarding health state valuation techniques is valid, but we restricted ourselves to talking about a utility measure; given the mainly clinical readership of the BMJ, this is more likely to be familiar to them. We also agree with the authors about the relative rarity of willingness to pay and conjoint analysis; as the use of these techniques in health economics has increased in recent years, however, we thought that they ought to be mentioned.

David J Torgerson, senior research fellow. 
University of York, York YO23 5DD