Analysis

Multiple sclerosis risk sharing scheme: a costly failure

BMJ 2010; 340 doi: http://dx.doi.org/10.1136/bmj.c1672 (Published 03 June 2010) Cite this as: BMJ 2010;340:c1672
  1. James Raftery, professor of health technology assessment
  1. 1Wessex Institute, School of Medicine, University of Southampton, Southampton SO167NS
  1. raftery{at}soton.ac.uk
  • Accepted 16 March 2010

The NHS is paying for thousands of patients with multiple sclerosis to receive drugs that monitoring data suggest are not effective. James Raftery examines what went wrong with the access scheme that facilitated their use

The largest and most ambitious patient access scheme, the UK multiple sclerosis risk sharing scheme, was set up in 2002 after the National Institute for Health and Clinical Excellence (NICE) recommended against use of interferon beta and glatiramer acetate.1 Under the scheme patients were closely monitored to confirm the cost effectiveness of the drugs, with an agreement that prices would be reduced if patient outcomes were worse than predicted.2 The first report on the scheme in 2009 showed patient outcomes were much worse than predicted but judged that it was premature to reduce prices.3 Why did this happen and what can we do to prevent it recurring?

Development of patient access schemes

The high prices charged for many new drugs have led to many health systems requiring proof of cost effectiveness before they are funded. However, decisions often have to be made with limited information: licensing trials are typically short term and of little value for assessing cost effectiveness. Once licensed, drug prices are listed at prices that seldom change. Patient access schemes have developed whereby agreements are made with the manufacturer to vary drug prices according to factors such as the number of patients treated or doses used, the response of the patient, or longer term patient outcomes.

The 2009 Pharmaceutical Price Regulation Scheme, which governs drug pricing, outlined two types of access scheme: those based on finance and those based on outcomes (table).4 Schemes based on finance involve either changes in list price or discounts linked to the number of patients or doses or to patient responses. Schemes based on responses can help identify patients who will benefit. In outcome based schemes, such as the multiple sclerosis scheme, prices are linked to longer term patient outcomes and hence to cost per quality adjusted life year (QALY). If prices can rise or fall, the risk is shared.

Types of patient access schemes with UK examples

View this table:

How the multiple sclerosis scheme works

The multiple sclerosis risk sharing scheme established by the Department of Health in 2002 “involves detailed monitoring of a cohort of patients to confirm the cost effectiveness of treatments” (box).2 All UK patients with relapsing remitting disease were eligible, and 5000 to 7000 were expected to join, making it the biggest and longest cohort study in multiple sclerosis.5 Prices for the drugs were set close to commercial levels (around £8000 (€9000; $12 000) a patient/year) but were to be reduced if patient outcomes were less than those required to meet a cost per QALY of £36 000.

Key elements of UK multiple sclerosis risk sharing scheme35

  • Aim of scheme—To monitor outcomes of drug treatments in cohort of patients in order to adjust drug prices if outcomes are worse than predicted

  • Eligibility—All UK patients who meet Association of British Neurologists criteria for relapsing remitting multiple sclerosis or secondary progressive disease in which relapses are the dominant clinical feature

  • Type of study—Prospective observational cohort

  • Cohort study—Subset of eligible patients

  • Outcomes—Score on extended disability status scale (EDSS), measured annually

  • Outcome statistic—Change relative to baseline of weighted average of proportion of patients who have progressed to disability scores of 4, 6, and 7

  • Predicted outcome—EDSS score required to achieve £36 000/QALY based on the ScHARR model used by NICE

  • Controls—Historical (Ontario dataset of patients followed up over 25 years)

  • No of included patients—7500-9000 eligible; 5500 to 7000 likely to be included in formal comparison

  • Target dates—Start May 2002. Recruitment to end Nov 2003 (not achieved until 2005)

  • Duration of scheme—10 years with price setting reviews every two years

  • Stopping rule—Discontinue when no longer effective, intolerable adverse events, or preparing for pregnancy

  • Funding—NHS bodies to fund drug treatment. Monitoring study funded jointly by Department of Health and four drug companies

  • Governance—Steering group with representation from four participating companies, the Multiple Sclerosis Society and Trust, Association of British Neurologists, Royal College of Nursing/Association of Multiple Sclerosis Nurses, and four UK health departments. Adjudication panel with independent chair, biostatistician, non-aligned industry figure, plus a representative from each of the health departments, Multiple Sclerosis Society, Multiple Sclerosis Trust, and Association of British Neurologists.

The NHS was statutorily obliged to fund patients in the scheme. Besides the Department of Health, the scheme’s steering group included the four relevant drug companies, the Association of British Neurologists, the Multiple Sclerosis Society and Trust, the Royal College of Nursing, and the Association of Multiple Sclerosis Nurses.

Failings

The first report on the scheme was published in late 2009, with details of patients’ outcomes for 2005-7.3 Disease progression was not only worse than predicted by the model used by NICE,1 it was worse than that in the untreated control group. The primary outcome—the difference between actual and expected benefit as a percentage of expected benefit—was 113%, well above the 20% tolerance for price changes (any value above 0 indicates that benefit is less than expected). The report stated “the outcomes so far obtained in the pre-specified primary analysis suggest a lack of delay in disease progression.”3

This dramatic finding did not, however, trigger any price reduction. Instead, the paper reports: “The scientific advisory group considered that it was premature at this stage to reach any decision about re-pricing the drugs without further follow-up and analyses.” Various reasons were given, including possible underestimation in the model, that use of historical controls may miss changes in the disease, and the effects of a “no improvement” assumption. Each of these arguments has been strongly contested (convincingly to my mind) by McCabe and colleagues,6 most of whom took part in the original modelling. However, given the terms of the scheme, such retrospective arguments should be irrelevant.

Besides failing to link prices to outcomes, the scheme proved slow to establish and was costly. It took three years rather than the planned 18 months to recruit 5000 patients. Around 120 extra multiple sclerosis nurses had to be hired to enable the scheme to operate at 73 centres. At a gross cost of £50 000 for each nurse this is £6m per year. The biggest cost was drugs, at around £40m for 5000 patients. Monitoring the scheme costs around £1m a year, making the total annual cost close to £50m. Since not all patients who are receiving drugs are included in the study the number of patients treated under the scheme is probably higher. If, as suggested by the Multiple Sclerosis Society7 and the undersecretary for health,8 10 000 patients are receiving drugs through the scheme, the cost will be almost double. This may well be the most expensive publicly funded ongoing health related study in the UK, and probably anywhere, ever.

Independent decisions?

The decision not to recommend price changes was attributed to the scheme’s “independent scientific advisory group.”3 No such group was mentioned in the 2002 documentation for the scheme,2 but it seems to be the original “adjudication panel.” The independence of this group is questionable. When established it had three independent members (the chair, a biostatistician, and a non-aligned industry figure) plus a representative from each of the health departments, the Multiple Sclerosis Society, the Multiple Sclerosis Trust, and the Association of British Neurologists. The last three of these had appealed against the NICE recommendation not to use these drugs. In addition, the four drug companies had the right to make written or oral representations to the committee and see the proposed price adjustments in draft.

A team from the Sheffield School of Health and Related Research (ScHARR) was originally charged with monitoring patient outcomes. However, it withdrew over concerns about the governance of the scheme. A paper on the scheme written by many of the ScHARR team states: “The commercial risks involved with the scheme led to the involvement of the companies as well as the Department of Health in the scheme’s governance. The scheme was re-tendered, but the ScHARR consortium decided not to apply as they [sic] were not happy with the proposed arrangements for data access and publication rights, and the scheme is now being undertaken by a contract research organisation.”9

The paper’s acknowledgements state that the companies “were also represented on the scheme’s scientific advisory group, which comments on the draft paper and gave permission for publication.” This is hardly a description of an independent scientific advisory group.

No annual reports have been published on the scheme. A report by the Sheffield group to the Department of Health remains unpublished despite parliamentary questions10 and requests from the Commons health committee.11 My understanding is that members of the Sheffield team have had to sign confidentiality agreements. The appendix gives what has been published on the terms of reference of the scheme and the known committee members.

Governance independent of vested interests is a key principle in the design of such schemes.12 An attempt to assess the scheme before the publication of its first results13 found that its governance arrangements “do not appear to be designed to promote and protect the independence of the scheme.” It suggested “given that the majority of the scientific advisory group appear to have a vested interest in scheme’s findings, the enforceability of the scheme is open to question.”

Other aspects of the scheme must also be questioned. Firstly, designed to confirm cost effectiveness, it seems to have been unprepared for the possibility that the drugs would be ineffective, something noted in 2003.14 The use of historical controls was an inherent part of the scheme, which was accepted and refined by the scientific advisory committee in 2002. The decision to pursue “alternative sources of data of progression in untreated patients”3 at this stage is extraordinary. Secondly, the design of the scheme around the Sheffield cost effectiveness model used by NICE has proved difficult. Several assumptions in the model did not hold in the scheme. One was that those patients whose disease progressed would discontinue taking the drugs—many did not. Another assumption, necessitated by the way the historical control data were reported, was that treatment could at best only slow the decline in patients’ disability scores. In the scheme, 38% of patients had improved scores at one year. However, the limitations of the extended disability status scales as a measure of disability were noted around the start of the scheme.14

None of these problems constitute compelling reasons for disregarding the terms of the scheme, which specified using the two year results to set the price. The fact that the prices would have had to be cut to zero (given outcomes were less than those of the controls) is hardly a reason for not proceeding. If patients are to continue to receive drugs through the scheme, big price cuts seem necessary.

Winners and losers

Interested parties will differ on the success or failure of the scheme. The health ministers who set up the scheme may see it as having postponed a difficult confrontation with the multiple sclerosis lobby. All the health ministers at the time the scheme was set up have moved on, as predicted at the time.5 NICE may see its decision not to recommend use of these drugs as justified. However, it has had to recommend use of a newer and even more expensive (£15 000 per patient) drug for multiple sclerosis, natalizumab, because the scheme means that interferon beta is the “current standard of practice” in the NHS.15

The Multiple Sclerosis Society claimed success for the scheme up to 2007 because it “improved care and support for people with MS.”7 However on publication of the results of the scheme in 2009 it withdrew its support. It states that the Department of Health has confirmed that “access to the disease modifying drugs will continue regardless of the success or failure of the research. . . . With up to 10 drug options expected to be available in 2011, action is needed now to ensure fast and equal action to latest treatments.”16 If NICE assesses the cost effectiveness of these new drugs against interferon beta, as happened with natalizumab, the scheme will have advantaged multiple sclerosis over all other diseases.

The views of patients with multiple sclerosis are not known, but, notably, although all eligible patients were assumed to want to enter, a considerable number refused. The two year review acknowledged that “it would have been desirable to have identified patients who were eligible for the scheme but chose not to participate.”3

The scheme was a success for the drug companies, who sold at close to full price to the NHS. It has also been a success for the companies making natalizumab (Biogen Idec and Elan International), which would not have been recommended by NICE in the absence of the scheme.

For the NHS, however, the scheme can be judged only “a costly failure”11 as suggested by the House of Commons Health Committee which has been raising concerns about the Scheme for several years. The biggest losers are the other NHS patients who would otherwise have benefited from the money spent on the scheme.

The only other patient access scheme that uses outcomes—the Australian scheme for bosentan in pulmonary artery hypertension—may have been more successful. This is partly helped by a smaller patient group (528 patients), a well defined outcome measure (death), and a health system in which negotiation of drug prices is common.17 However, it too has struggled with issues of governance, ethical permission, and data collection.18

What are the lessons more generally for patient access schemes? One is that financially based schemes are preferable to those based on outcomes. Outcome based schemes should probably be avoided if at all possible. For those that do go ahead, it is vital to ensure appropriate governance given the inevitable conflicts of interest. Since a robust control group is essential, policy oriented randomised clinical trials may be required. Monitoring and evaluation of outcomes must be independent of the companies involved. Transparency is essential, involving annual reports, access to data, and rights to publish. Any of these might have helped avoid the current fiasco.

Notes

Cite this as: BMJ 2010;340:c1672

Footnotes

  • Analysis, doi:10.1136/bmj.c1786
  • Contributors and sources: JR has provided health economics input to six technology assessment reports for NICE and published several reviews of NICE. He has observed the multiple sclerosis risk sharing scheme since its inception.

  • Competing interests: The author has completed the unified competing interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declares (1) no financial support for the submitted work from anyone other than their  employer; (2) no financial relationships with commercial entities that might have an interest in the submitted work; (3) no spouses, partners, or children with relationships with commercial entities that might have an interest in the submitted work; and (4) no non-financial interests that may be relevant to the submitted work.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

References