Letters

Cost utility of drugs for multiple sclerosis

Systematic review places study in context

Methods used don't calculate true benefit

Analysis goes too far

Authors' reply

Systematic review places study in context

EDITORWe were disappointed that Forbes et al's population based cost utility study of interferon beta-1b in secondary progressive multiple sclerosis was not placed in the context of a systematic review of relevant published work.¹ We recently completed such a review of the effectiveness, costs, and utility of a range of disease modifying drugs in multiple sclerosis for the health technology assessment programme. As part of this, we searched various electronic databases for published studies of the cost utility of interferon beta in multiple sclerosis: Cochrane Database of Systematic Reviews, Cochrane Controlled Trials Register, Database of Abstracts of Reviews of Effectiveness and NHS Economic Evaluations Database, Medline (Silverplatter), PubMed, Embase, and National Research Register (for the period 1980 to July 1999 and limited to English language studies). In addition, we contacted experts in the field.

In addition to Forbes et al's study, we identified three further studies: one of beta interferons (1a and 1b) in both relapsing-remitting and secondary progressive sclerosis² and two further studies of interferon in relapsing-remitting multiple sclerosis. ^{3 4}

The table shows a summary of the results of the review. The studies in secondary progressive multiple sclerosis are based on the same trial (n=718) but use different cost utility methodologies. Forbes et al's conclusion that the cost per quality adjusted life year (QALY) was very high (£1 024 667) owing to the high drug cost and modest clinical effect concurs with the previous study's estimate of £874 600 per QALY. The studies in relapsing-remitting multiple sclerosis also use the results of the existing randomised controlled trials in conjunction with various sources of cost data, quality of life measures, and different degrees of decision analytic modelling. Results show that the cost per QALY per relapse avoided is high in all studies.

These cost utility studies were critically appraised by using standard criteria for decision analysis and economic evaluations. Although all used explicit and sensible processes to identify, select, and combine the evidence into probabilities, and to obtain utilities and costs, with uncertainties being addressed by sensitivity analyses, none of them provide a full economic comparison of all possible healthcare strategies. The conclusion of Forbes et al that "far more benefit would be obtained from directing funds into improved supportive care," though plausible, is not supported by the evidence presented. Further economic evaluations of a wider range of possible interventions, such as improved supportive care, are needed so that limited resources can be used effectively.

Jackie Bryant, research fellow. 
Andrew Clegg, senior research fellow. 
Ruairidh Milne, senior lecturer in public health medicine. 
Wessex Institute for Health Research and Development, University of Southampton, Southampton SO16 7PX

Competing interests: None declared.

Methods used don't calculate true benefit

EDITORWe would like to draw attention to methodological issues in calculating cost effectiveness in the study by Forbes et al.¹ They have used the only trial data available, curtailed at 36 months.² For the outcome they used, time to wheelchair dependence, the median point was not reached. Instead, the trial reports, rather vaguely, that the benefit was "a delay of up to 9 months" in time to wheelchair dependence.² This seems to be based on the 83rd centile: at 30 months for the treatment arm and 20.5 months for the control arm (see figure); as such it is an arbitrary measure of the difference between the distribution curves, determined by the length of the trial. As a second dimension of the effectiveness, Forbes et al use the number needed to treat at 30 months (18), based on the relative risk reduction from the trial and the (lower) rate of reaching wheelchair dependency from their Tayside cohort. This gives an average delay for a patient of just 0.5 months for the Tayside patients (0.9 months for the trial cohort,² with a number needed to treat of 10 calculated, using the figure below, from the difference between the numbers of treated and control patients at 30 months (100/8373)).

View larger version (19K): [in this window] [in a new window]   Survival curve to wheelchair dependence in patients with multiple sclerosis treated with interferon beta-1b and controls. Area between curves/ 100=average delay; dotted lines indicate 83rd centile

The figure plots the estimated probabilities of not being wheelchair dependent from the trial data,² as applied to a cohort of 100 patients. The area under each curve was estimated by summing the areas of the polygons for each month, using a spreadsheet. The difference between them is the difference in the time spent in a wheelchair, and generates a true mean at 30 months of 1.7 months' delay (compared with the 0.9 months above). The method used by Forbes et al estimates only the (marginal) benefit, as represented by the area between the curves from 21 to 30 months.

We believe that this method of the area under the curve offers the best means of calculating true benefit in most cases, especially when data are limited, though it has its own problems. Further debate is needed as methodologies for health economic assessments develop.

As for individual patients, they may be led to believe that treatment with interferon beta-1b offers a nine month delay in becoming wheelchair dependent, whereas in reality it is just 1.7 months on average. Would this knowledge change their perspective?

Richard Richards, consultant public health physician. 
North Nottinghamshire Health Authority, Rainworth, Mansfield NG21 0ER r.g.richards{at}shef.ac.uk

Amanda Burls, senior clinical lecturer in public health and epidemiology. 
Department of Public Health and Epidemiology, University of Birmingham, Edgbaston, Birmingham B15 2TT

Nick Payne, clinical senior lecturer. 
School of Health and Related Research, University of Sheffield, Sheffield S1 4DA

Competing interests: None declared.

Analysis goes too far

EDITORThe paper by Forbes et al gives the appearance of applying scientific methodology to the challenge of resource allocation.¹ However, if the methodology is flawed such papers have the potential of influencing the political debate, resulting in patients being deprived of useful treatment.

The paper takes a local audit of people with multiple sclerosis, applies the results from one double blind placebo controlled trial, using a single aspect of disability (wheelchair dependence) and then extrapolates quality adjusted life years (QALYs) to which a price tag is attached.² This results in the apparently exorbitant cost per QALY of £832 399.

The study of the use of interferon beta-1b cited by Forbes et al was designed as an efficacy study. Any further analysis of the data has to be viewed as hypothesis generating rather than as strong evidence in its own right. Post hoc manipulations of data are notoriously unreliable, as was shown by the unfortunate safety scare over the use of selegiline in Parkinson's disease,³ which has not been confirmed in other studies.

Although wheelchair dependence is an important aspect of disability in multiple sclerosis, it is not the only one. Any study into the overall benefit of a treatment in multiple sclerosis that used only this one aspect of disability would seriously underestimate the problems associated with this disease.

Assessment of the effect of medical interventions is problematic,⁴ and the underlying assumption behind QALYs that seriousness of a disability is a fixed quantity is unlikely to be true. Many healthy people might say they would prefer to be dead rather than live with a severe disability, but if you ask disabled people, they generally prefer to be alive. The severity of a disability depends on one's perspective and is not fixed.

Forbes et al's paper will be used as strong evidence to limit availability of treatment to patients, and it may constitute another example of the misuse of so called evidence based medicine for rationing purposes.⁵ This sort of exploration should be used as hypothesis generating to inform the design of a proper cost benefit trial.

Simon J Ellis, consultant neurologist. 
North Staffordshire Royal Infirmary, Keele University, Stoke on Trent ST4 7LN

1.	Forbes RB, Lees A, Waugh N, Swingler RJ. Population based cost utility study of interferon beta-1b in secondary progressive multiple sclerosis. BMJ 1999; 319: 1529-1533. (11 December.)
2.	European Study Group on Interferon beta-1b in Secondary Progressive Multiple Sclerosis. Placebo-controlled multicentre randomised trial of interferon beta-1b in treatment of secondary progressive multiple sclerosis. Lancet 1998; 352: 1491-1497.
3.	Ben-Shlomo Y, Churchyard A, Head J, Hurwitz B, Overstall P, Ockelford J, et al. Investigation by Parkinson's Disease Research Group of United Kingdom into excess mortality seen with combined levodopa and selegiline treatment in patients with early, mild Parkinson's disease: further results of randomised trial and confidential inquiry. BMJ 1998; 316: 1191-1196[Free Full Text].
4.	Wright JC, Weinstein MS. Gains in life expectancy from medical interventionsstandardizing data outcomes. N Engl J Med 1998; 339: 380-386[Abstract/Free Full Text].
5.	Kryger M. Sleep apnoea and the misuse of evidence based medicine. Lancet 1997; 349: 803-804[Medline].

Competing interests: Dr Ellis is a blinded assessor in a study of interferon beta run by Serono but with no pecuniary advantage. Serono has paid travel and accommodation expenses for attendance at investigators' meetings.

Authors' reply

EDITORDespite the published responses we believe that the conclusion of our study is still a robust one: that the opportunity cost of interferon beta for secondary progressive multiple sclerosis is considerable.

Bryant et al emphasise the need to compare our study with other economic evaluations of interferon beta. We had not quoted studies of relapsing-remitting multiple sclerosis as this is considered a separate entity to secondary progressive disease. Comparisons between studies of secondary progressive and relapsing-remitting multiple sclerosis do not add to our analysis, except to emphasise that interferon beta remains an extremely expensive intervention, whatever the disease type. If quality adjusted life years are a generic measure of health benefits, then our conclusion that supportive care might represent better use of funds is likely to be valid, as few healthcare interventions produce health benefits at such a high cost.

According to Richards, we underestimated the benefits of interferon beta. Even taking such an error into account, our final conclusion is not altered, as the cost utility ratio is driven by the high cost of the drug.

In response to Ellis, an economic model which applies observational data to efficacy data from a randomised controlled trial is preferable to conducting trials with economic end points, as economic data gathered in the context of a trial are unlikely to be generalisable outside of trial settings.¹ There is also the issue of who should conduct (and pay for) such studies. We accept that in the United Kingdom economic analysis is being used to limit access to interferon beta. However, services for people with neurological disorders remain underdeveloped, and, it is well known that the United Kingdom has fewer neurology specialists than comparable developed nations. In multiple sclerosis, it would be a great pity if huge sums of money were devoted to a drug of marginal benefit to a small number of people, when alternative uses of those funds could deliver a high standard of service and care to a much wider group of people. Technological advances inevitably lead to demands for increased healthcare funding. Those who are developing neurology services should either allow unrestricted prescription of new expensive drugs, such as interferon beta, or consider the relative costs and benefits of new technologies, as alternative uses of development funds (such as simple interventions to reduce symptoms) might better serve the patients under our care.

Raeburn B Forbes, specialist registrar. 
Department of Neurology, Royal Victoria Hospital, Belfast BT12 6BA raeburn.forbes{at}royalhospitals.n-i.nhs.uk

Robert J Swingler, consultant neurologist. 
Ninewells Hospital and Medical School, Dundee DD1 9SY

Competing interests: Drs Forbes and Swingler have investigated and evaluated neurologists for a clinical trial of Antegren in multiple sclerosis, sponsored by Athena Neurosciences. Dr Forbes has received funding to attend a conference from Serono. Dr Swingler has received grants from Serono, Schering Healthcare, and Biogen to fund attendance at conferences.