Education and debate

Problems with UK government's risk sharing scheme for assessing drugs for multiple sclerosis

Cathie L M Sudlow, Wellcome clinician scientist ^a, Carl E Counsell, senior lecturer ^b. 

^a Department of Clinical Neurosciences, University of Edinburgh, Western General Hospital, Edinburgh EH4 2XU, ^b Department of Neurology, Aberdeen Royal Infirmary, Aberdeen AB25 2ZN

Correspondence to: C L M Sudlow csudlow{at}skull.dcn.ed.ac.uk

The government plans to make interferon beta and glatiramer available to patients with multiple sclerosis through a risk sharing scheme, despite lack of evidence of cost effectiveness. Sudlow and colleagues argue that the money would be better spent on independent research

The National Institute for Clinical Excellence (NICE) recently announced that interferon beta and glatiramer acetate were not cost effective treatments for multiple sclerosis and could not be recommended for NHS funding.¹ As a result, the Department of Health and the manufacturers developed a "risk sharing scheme" aimed at providing these drugs more cost effectively. ^{2 3} Treatment will be provided to ambulating patients with two or more disabling relapses in the past two years (about 15% of all patients with multiple sclerosis)⁴ and their progress monitored over 10 years. However, the scheme has several scientific and practical problems that we believe limit its ability to improve the care of patients in the long term. In this paper, we review the quality of the evidence on which NICE and the Department of Health reached their decisions, consider some of the problems of the risk sharing scheme, and suggest an alternative approach.

Summary points NICE has announced that neither interferon beta nor glatiramer can be recommended for multiple sclerosis in the NHS The Department of Health plans to make these drugs available through a risk sharing scheme that is scientifically unsound and impractical Randomised trials suggest that azathioprine (which is 20 times cheaper) may be just as effective The long term effectiveness of these drugs is unknown Government money would be better spent on a long term randomised trial comparing interferon beta or glatiramer with azathioprine and no treatment

	Methods

We identified randomised trials of disease modifying drugs in patients with multiple sclerosis from systematic reviews,^5-7 the Cochrane controlled trials register, and the treatment guidelines of the Association of British Neurologists.⁴ We also got information from discussion with colleagues, including several international experts in multiple sclerosis. We used Cochrane RevMan software to produce summary relative risks.

	What is the evidence that the drugs are effective?

NICE considered data from placebo controlled trials of interferon beta and glatiramer acetate but did not assess azathioprine, which has also been widely tested in multiple sclerosis.^5-31 The three drugs produce a similar reduction (15-30%) in the relative risk of a relapse at two years (fig 1). Interferon beta and glatiramer may also reduce disability (fig 2), but appropriate data were not available for azathioprine. Although interferon beta, glatiramer, and azathioprine were all associated with more patient withdrawals than placebo, the side effects were generally mild. Azathioprine may be associated with a small increased risk of neoplasia after 10 years of treatment, ^{32 33} but there is not enough long term experience with either interferon beta or glatiramer to exclude an increased risk of cancer.

These results, although promising, are based on limited, short term data (a few hundred patients for each drug, usually followed up for no more than two years). We therefore do not know whether the effects are sustained over the long term. Several other previously noted methodological problems⁷ also limit the interpretation of the results and may have biased them in favour of active treatment. These include uncertainty about the adequacy of randomisation,³⁴ which is not always clearly described; unavoidable patient unblinding³⁵; difficulty interpreting the outcome of confirmed progression of disability, which was generally based on the widely criticised expanded disability status score ^{36 37} ; substantial losses to follow up in a few trials; publication bias (the largest randomised trial assessing interferon beta in secondary progressive multiple sclerosis showed no overall effect on progression of disability but remains unpublished¹⁷); and funding of the trials by pharmaceutical companies, which own the data and were involved in the trials' design, conduct, analysis, and reporting.^9-24Although many trials had independent data monitoring committees, it is concerning that some data from some trials have not been placed in the public domain.

View larger version (24K): [in this window] [in a new window]   Fig 1.   Risk of relapse of multiple sclerosis at about two years. Numbers were obtained from a systematic review of interferon beta in relapsing-remitting multiple sclerosis5 as well as individual trials. There was no statistical heterogeneity between individual trials contributing to summary statistics

View larger version (20K): [in this window] [in a new window]   Fig 2.   Risk of progression of disability at about two years. Numbers were obtained from a systematic review of interferon beta in relapsing-remitting multiple sclerosis5 as well as individual trials. There was no statistical heterogeneity between individual trials contributing to summary statistics

	Limitations of NICE's assessment of cost effectiveness

NICE's conclusions on cost effectiveness were based mainly on an analysis commissioned from the Sheffield University School of Health and Related Research. ^{1 38} The researchers calculated the cost per quality adjusted life year (QALY) gained at 5, 10, and 20 years after starting treatment (table). The model suggests that the threshold of £36 000 per QALY (set by the Department of Health for the risk sharing scheme) is approached only after 20 years.

Although this economic model is probably the best available for multiple sclerosis, it has several unavoidable flaws. Firstly, it depends on the quality of the evidence for effectiveness of treatment, which, as highlighted above, has major deficiencies. Treatment effects were estimated mainly from published reports. Two companies provided some additional confidential data, one refused, and one withdrew its additional data after seeing its effects on the results. ^{1 38}

Secondly, because of the lack of long term placebo controlled randomised trials, the model compares the effects of treatment with the experience of a cohort of 1000 Canadian patients with multiple sclerosis recruited in the 1970s and 1980s and followed for an average of 25 years. ^{39 40} It assumes that treatment remains effective for as long as the patient takes it and that the benefit accrued is maintained after treatment stops. NICE acknowledges that this extrapolation of treatment effects becomes increasingly unreliable as the time horizon is increased. ^{1 38}

Thirdly, the model is heavily influenced by assumptions about future discounting of costs and benefits, the proportion of patients who stop treatment prematurely and what happens to them, and the way in which the costs of disability related to multiple sclerosis are estimated (table).³⁸ Finally, it does not consider azathioprine, which has an annual treatment cost of only about £300 a patient.

	Scientific flaws of risk sharing scheme

The risk sharing scheme plans to use the Sheffield model as a basis for assessing and adjusting the real life cost effectiveness of interferon beta and glatiramer. Azathioprine has been ignored. Patients meeting the Association of British Neurologists treatment criteria⁴ will be assessed annually for 10 years to determine the rate of progression from no disability (expanded disability status score <4), through mild (4-5.5) and moderate (6-6.5), to severe disability (7). The effects of each treatment will be determined every two years by comparison with the expected progression without treatment derived from the Canadian cohort. ^{3 39 40} Target treatment effects have been agreed with the drug companies, and if these are not achieved, the drug costs will be reduced to maintain cost effectiveness at a threshold of £36 000 per QALY over 20 years.³ Unfortunately, the scheme has several major problems.

Non-randomised comparisons
The Department of Health circular states that the scheme is not a further trial of clinical effectiveness but a study to establish long term cost effectiveness.³ However, a reliable estimate of long term cost effectiveness first requires a reliable estimate of long term clinical effectiveness. This will not be achieved by comparing a modern cohort of patients treated in the United Kingdom with a historical cohort of Canadian patients since non-randomised comparisons give unreliable, biased results. ^{41 42}

Lack of power calculations
The scheme will include about 7000 patients in England and Wales (plus more patients from Scotland), but the circular gives no power calculations to justify this number.³ It recognises that the non-randomised comparison will be biased and that chance may lead to imprecise measures of treatment effect. However, rather than randomising large numbers of patients, the Department of Health proposes to incorporate a tolerance margin of 10-20% in the comparison between the treated and untreated cohorts.³ It does not explain how this margin was chosen; nor is it clear whether the margin represents a relative or absolute difference in outcome.

Other biases
The risk sharing scheme is subject to several other biases. Firstly, patients already receiving treatment at the start of the scheme will be included if they fulfilled the inclusion criteria at the start of treatment and their pre-treatment disability score and other prognostic data are available.³ This will bias the comparison in favour of treatment because patients who started and then stopped treatment before the scheme because of adverse effects or perceived lack of effectiveness will not be included.

Secondly, the circular states that patients who stop taking treatment during the scheme are "to be monitored as far as possible."³ This is not good enough. It is essential to follow up such patients because we do not know how patients respond once they stop treatment. This information is critical to the cost effectiveness calculations.

Thirdly, the scheme does not intend to have blinded assessment of outcome. Unblinded assessment of outcome in multiple sclerosis trials can result in overestimates of the effect of treatment on progression of disease.³⁵ Hence, an apparent treatment benefit may simply be due to the expectation bias of patients and their neurologists or specialist nurses. An additional competing interest bias may be introduced by unblinded assessment of outcome being done by specialist nurses whose salaries are paid for by pharmaceutical companies.

Calculation of cost effectiveness
The estimates of cost effectiveness depend critically on the various assumptions used in the modelling process, but the actual assumptions to be used are not mentioned in the circular. These will have to be made explicit and justified before the scheme starts.³

Other issues
The circular does not state what will happen if a treatment seems ineffective. Neither does it tell us which patients will be included in the analyses or whether these will be on a truly intention to treat basis. Ideally, a proper intention to treat analysis would be ensured by information on patients giving their consent being telephoned or faxed immediately to a central site. This would avoid the loss or non-registration of patients who do not do well on their chosen treatment. However, no details of this sort have been provided.

	Practical problems with risk sharing scheme

The Department of Health proposed that patient recruitment would start on 6 May 2002. However, the national coordinating team was not appointed until July 2002, ethical approval has had to be sought, and many neurologists have yet to see a detailed protocol.

The cost of the drugs (more than £50m a year) for the scheme will have to be met from existing NHS budgets. In addition, collecting data is likely to put further strain on NHS resources. The circular states that "the scheme should as far as possible build on normal clinical practice without requiring elaborate additional infrastructure" and that "data entry should be as simple as possible and arise out of normal patient contacts."³ However, neurological services are already extremely stretched (median outpatient waiting times are about 26 weeks), and many potentially eligible patients do not have regular contact with a neurologist.⁴³ Many additional consultant neurology and specialist nurse sessions (with appropriate administrative support) will be needed to evaluate patients who may be eligible and to follow up those who join the scheme. Normal patient contacts do not include assessment of the expanded disability status score, and so appointments will have to be longer to allow for this. It is not clear how all these additional sessions can be provided without seriously compromising the existing service, although the pharmaceutical companies will fund some. Local staff (probably specialist nurses) will need to be trained to collect, store, and transfer the additional data.

	Alternative proposal

We believe that the government could spend the extra resources for patients with multiple sclerosis more effectively. Firstly, it should commission an independent, individual patient data overview of all relevant published and unpublished randomised trials of disease modifying drugs for multiple sclerosis. The overview would address unanswered questions about the trials and may go some way towards resolving the uncertainties about the effects of interferon beta, glatiramer, and azathioprine.

Secondly, the risk sharing scheme should be modified to include a concurrent, randomised control group rather than a historical cohort. Given that the Department of Health is committed to providing resources for the assessment, long term follow up, and drug costs for several thousand patients with multiple sclerosis, a long term randomised trial, run independently of the pharmaceutical industry, would be a far more scientifically (and so ethically) justifiable use of this money. Patients could be randomised three ways (interferon beta or glatiramer versus azathioprine versus no treatment) and followed up in the same manner and with the same outcomes as the existing scheme. Additional resources would be required for blinded outcome assessment and perhaps inclusion of a quality of life outcome, but a trial would probably be less expensive than the present scheme because only one third of patients would be taking an expensive drug. Careful explanation would have to be given to patient groups about why a randomised trial is the best way forward as fewer patients would be receiving active treatment. However, it is the patients who have most to gain by reliably establishing the long term clinical effectiveness as well as cost effectiveness of these treatments.

	Conclusions

Any additional resources for patients with multiple sclerosis are welcome. However, these should be used to provide services that we know benefit patients and to support further, properly designed research into interventions about which there is still uncertainty. Uncertainty remains about the effectiveness and cost effectiveness of interferon beta and glatiramer, and the risk sharing scheme will neither resolve these nor determine the possible role of promising but far less expensive drugs such as azathioprine. All patients with multiple sclerosis, whether eligible for treatment under the terms of the scheme or not, deserve much better than this. The government should consider a more appropriate use of this large amount of public money.

	Acknowledgments

Contributors: CS identified relevant articles and analysed the data. Both authors drafted and modified the paper and both will act as guarantors.

	Footnotes

Funding: CS is supported by a Wellcome Trust clinician scientist award.

Competing interests: None declared.

	References

1.	National Institute for Clinical Excellence. Beta interferon and glatiramer acetate for the treatment of multiple sclerosis. Technology Appraisal Guidance No 32. www.nice.org.uk/Docref.asp?d=39265 (accessed 20 November 2002).
2.	Department of Health. Payment by results breakthrough ends years of uncertainty for MS patients. Press release 2002/0056, 4 February 2002. www.info.doh.gov.uk/doh/intpress.nsf/page/2002-0056?OpenDocument (accessed 20 November 2002).
3.	Department of Health. Cost-effective provision of disease modifying therapies for people with multiple sclerosis. Health Service Circular 2002/004, February 2002. www.doh.gov.uk/doh/coin4.nsf/Circulars?ReadForm (accessed 25 November 2002).
4.	Association of British Neurologists. Guidelines for the use of beta interferons and glatiramer acetate in multiple sclerosis. London: ABN, 2001. www.theabn.org/downloads/msdoc.pdf (accessed 20 November 2002).
5.	Rice GPA, Incorvaia B, Munari L, Ebers G, Polman C, D'Amico R, et al. Interferon in relapsing-remitting multiple sclerosis. Cochrane Database Syst Rev 2001;(4):CD002002.
6.	Yudkin PL, Ellison GW, Ghezzi A, Goodkin DE, Hughes RAC, McPherson K, et al. Overview of azathioprine treatment in multiple sclerosis. Lancet 1991; 338: 1051-1055[CrossRef][ISI][Medline].
7.	Clegg A, Bryant J, Milne R. Disease-modifying drugs for multiple sclerosis: a rapid and systematic review. Health Technology Assessment 2000;4:No 9.
8.	Northern and Yorkshire Regional Drug and Therapeutics Centre. Assessment of interferon-beta and glatiramer for the treatment of multiple sclerosis. Report commissioned by the NHS Health Technology Assessment Programme on behalf of the National Institute for Clinical Excellence, April 2000. www.nice.org.uk/pdf/OriginalHTAReportApril2000.pdf (accessed 25 November 2002).
9.	The IFNB Multiple Sclerosis Study Group. Interferon beta-1b is effective in relapsing-remitting multiple sclerosis. I. Clinical results of a multicenter, randomized, double-blind, placebo-controlled trial. Neurology 1993; 43: 655-661[Abstract/Free Full Text].
10.	The IFNB Multiple Sclerosis Study Group, University of British Columbia MS/MRI Analysis Group. Interferon beta-1b in the treatment of multiple sclerosis: final outcome of the randomized controlled trial. Neurology 1995; 45: 1277-1285[Abstract].
11.	Jacobs LD, Cookfair DL, Rudick RA, Herndon RM, Richert JR, Salazar AM, et al. Intramuscular interferon beta-1a for disease progression in relapsing multiple sclerosis. The Multiple Sclerosis Collaborative Research Group. Ann Neurol 1996; 39: 285-294[CrossRef][ISI][Medline].
12.	Rudick RA, Goodkin DE, Jacobs LD, Cookfair DL, Herndon RM, Richert JR, et al. Impact of interferon beta-1a on neurologic disability in relapsing multiple sclerosis. The Multiple Sclerosis Collaborative Research Group. Neurology 1997; 49: 358-363[Abstract/Free Full Text].
13.	Prevention of Relapses and Disability by Interferon beta-1a Subcutaneously in Multiple Sclerosis Study Group. Randomised double-blind placebo-controlled study of interferon beta-1a in relapsing/remitting multiple sclerosis. Lancet 1998; 352: 1498-1504[CrossRef][ISI][Medline].
14.	Once Weekly Interferon for MS Study Group. Evidence of interferon beta-1a dose response in relapsing-remitting MS. Neurology 1999; 53: 679-686[Abstract/Free Full Text].
15.	European Study Group on Interferon -1b in Secondary Progressive MS. Placebo-controlled multicentre randomised trial of interferon -1b in treatment of secondary progressive multiple sclerosis. Lancet 1998; 352: 1491-1497[CrossRef][ISI][Medline].
16.	Secondary Progressive Efficacy Clinical Trial of Recombinant Interferon-beta-1a in MS Study Group. Randomized controlled trial of interferon-beta-1a in secondary progressive MS. Clinical results. Neurology 2001; 56: 1496-1504[Abstract/Free Full Text].
17.	Goodkin DE, North American Study Group in Interferon beta-1b in Secondary Progressive MS. Interferon beta-1b in secondary progressive MS: clinical and MRI results of a 3-year randomized controlled trial. Neurology 2000; 54: 2352.
18.	Jacobs LD, Beck RW, Simon JH, Kinkel RP, Brownscheilde CM, Murray TJ, et al. Intramuscular interferon beta-1a therapy initiated during a first demyelinating event in multiple sclerosis. N Engl J Med 2000; 343: 898-904[Abstract/Free Full Text].
19.	Comi G, Filippi M, Barkhof F, Durelli L, Edan G, Fernandez O, et al. Effect of early interferon treatment on conversion to definite multiple sclerosis: a randomised study. Lancet 2001; 357: 1576-1582[CrossRef][ISI][Medline].
20.	Bornstein MB, Miller A, Slagle S, Weitzman M, Crystal H, Drexler E, et al. A pilot trial of Cop 1 in exacerbating-remitting multiple sclerosis. N Engl J Med 1987; 317: 408-414[Abstract].
21.	Johnson KP, Brooks BR, Cohen JA, Ford CC, Goldstein J, Lisak RP, et al. Copolymer 1 reduces relapse rate and improves disability in relapsing-remitting multiple sclerosis: results of a phase III multicenter, double-blind, placebo-controlled trial. Neurology 1995; 45: 1268-1276[Abstract].
22.	Johnson KP, Brooks BR, Cohen JA, Ford CC, Goldstein J, Lisak RP, et al. Extended use of glatiramer acetate (Copaxone) is well tolerated and maintains its clinical effect on multiple sclerosis relapse rate and degree of disability. Neurology 1998; 50: 701-708[Abstract/Free Full Text].
23.	Comi G, Filippi M, Wolinsky JS, European/Canadian Glatiramer Acetate Study Group. European/Canadian multicenter, double-blind, randomized, placebo-controlled study of the effects of glatiramer acetate on magnetic resonance imaging-measured disease activity and burden in patients with relapsing multiple sclerosis. Ann Neurol 2001; 49: 290-297[CrossRef][ISI][Medline].
24.	Bornstein MB, Miller A, Slagle S, Weitzman M, Drexler E, Keilson M, et al. A placebo-controlled, double-blind, two-center, pilot trial of Cop 1 in chronic progressive multiple sclerosis. Neurology 1991; 41: 533-539[Abstract/Free Full Text].
25.	Swinburn WR, Liversedge LA. Long-term treatment of multiple sclerosis with azathioprine. J Neurol Neurosurg Psychiatry 1973; 36: 124-126[ISI][Medline].
26.	Mertin J, Rudge P, Kremer M, Healey MJR, Knight SC, Compston A, et al. Double-blind controlled trial of immunosuppression in the treatment of multiple sclerosis: final report. Lancet 1982; ii: 351-354.
27.	Goodkin DE, Bailly RC, Teetzen ML, Hertsgaard D, Beatty WW. The efficacy of azathioprine in relapsing-remitting multiple sclerosis. Neurology 1991; 41: 20-25[Abstract/Free Full Text].
28.	Ghezzi A, Di Falco M, Locatelli C, Zaffaroni M, Caputo D, Marfono S, et al. Clinical controlled randomized trial of azathioprine in multiple sclerosis. In: Gonsette RE, Delmotte P, eds. Recent advances in multiple sclerosis therapy. Amsterdam: Elsevier, 1989:345-346.
29.	British and Dutch Multiple Sclerosis Azathioprine Trial Group. Double-masked trial of azathioprine in multiple sclerosis. Lancet 1988; ii: 179-183.
30.	Milanese C, La Mantia L, Salmaggi A, Eoli M. A double-blind study on azathioprine efficacy in multiple sclerosis: final report. J Neurol 1993; 240: 295-298[CrossRef][ISI][Medline].
31.	Ellison GW, Myers LW, Mickey MR, Graves MC, Tourtellotte WW, Syndulko K, et al. A placebo-controlled, randomized, double-masked, variable dosage, clinical trial of azathioprine with and without methylprednisolone in multiple sclerosis. Neurology 1989; 39: 1018-1026[Abstract/Free Full Text].
32.	Kinlen LJ. Incidence of cancer in rheumatoid arthritis and other disorders after immunosuppressive treatment. Am J Med 1985; 78(suppl 1A): 44-49[ISI][Medline].
33.	Confavreux C, Saddier P, Grimaud J, Moreau T, Adeleine P, Aimard G. Risk of cancer from azathioprine therapy in multiple sclerosis: a case-control study. Neurology 1996; 46: 1607-1612[Abstract/Free Full Text].
34.	Kunz R, Oxman AD. The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ 1998; 317: 1185-1190[Abstract/Free Full Text].
35.	Noseworthy JH, Ebers GC, Vandervort MK, Farquhar RE, Yetisir E, Roberts R. The impact of blinding on the results of a randomised, placebo-controlled multiple sclerosis trial. Neurology 1994; 44: 16-29[Abstract/Free Full Text].
36.	Kurtzke JF. Rating neurologic impairment in multiple sclerosis: an expanded disability status scale (EDSS). Neurology 1983; 33: 1444-1452[Abstract/Free Full Text].
37.	Rice G, Ebers G. Interferons in the treatment of multiple sclerosis. Do they prevent the progression of the disease? Arch Neurol 1998; 55: 1578-1580[Free Full Text].
38.	Tappenden P, Chilcott J, O'Hagan T, McCabe C, Cooper N, Abrams K, et al. Cost-effectiveness of beta interferons and glatiramer acetate in the management of multiple sclerosis. Final report to the National Institute of Clinical Excellence, July 2001. www.nice.org.uk/pdf/msscharrreport.pdf (accessed 25 November 2002).
39.	Weinschenker BG, Rice GP, Noseworthy JH, Carriere W, Baskerville J, Ebers GC. The natural history of multiple sclerosis: a geographically based study. 3. Multivariate analysis of predictive factors and models of outcome. Brain 1991; 114: 1045-1056[Abstract/Free Full Text].
40.	Kremenchutzky M, Cottrell D, Rice G, Hader W, Baskerville J, Koopman W, et al. The natural history of multiple sclerosis: a geographically based study. 7. Progressive-relapsing and relapsing-progressive multiple sclerosis: a re-evaluation. Brain 1999; 122: 1941-1949[Abstract/Free Full Text].
41.	Collins R, Peto R, Gray R, Parish S. Large-scale randomized evidence: trials and overviews. In: Weatherall DJ, Ledingham JGG, Warrell DA, eds. Oxford textbook of medicine. Oxford: Oxford University Press, 1996:21-32.
42.	Sacks H, Chalmers TC, Smith Jr H. Randomized versus historical controls for clinical trials. Am J Med 1982; 72: 233-240[CrossRef][ISI][Medline].
43.	Association of British Neurologists Working Party on Care of Neurological Emergencies in Adults. Acute neurological emergencies in adults. London: ABN, 2002. www.theabn.org/downloads/AcuteNeurology.pdf (accessed 20 November 2002).

(Accepted 21 October 2002)