Intended for healthcare professionals

Education And Debate

Perspectives of commissioners and cancer specialists in prioritising new cancer drugs: impact of the evidence threshold

BMJ 1999; 318 doi: (Published 13 February 1999) Cite this as: BMJ 1999;318:456
  1. Robbie Foy, senior registrar (R.Foy{at},
  2. June So, chief pharmacistb,
  3. Elizabeth Rous, consultanta,
  4. J Howard Scarffe, professor of medical oncologyb
  1. aStockport Health Authority, Stockport SK7 5BY
  2. bChristie Hospital NHS Trust, Manchester M20 4BX
  1. Correspondence to: Dr R Foy, Scottish Programme for Clinical Effectiveness in Reproductive Health, Department of Obstetrics and Gynaecology, University of Edinburgh, Edinburgh EH3 9AW
  • Accepted 29 September 1998

The provision of high quality cancer services is a major priority for the NHS.1 However, greater budgetary pressures are being placed on specialist hospitals and health authorities by several factors:

  • Recent development of several new cancer drugs

  • Expanding use of existing drugs

  • An ageing population

  • Introduction of cancer screening programmes

  • Increasing expectations of patients

Christie Hospital NHS Trust in Manchester provides specialist cancer care to a population of 4.5million people from North Wales to Cumbria. The trust's drug budget of £4 million accounted for 18% of the total income from patient services in 1996-7,creating an obvious focus for annual negotiations with commissioning health authorities. Historical underfunding of the trust's drug budget, relative to that of other oncology centres, further supported the trust's case for increased resources. Clinicians highlighted the paradox whereby only 1% of the annual NHS drug budget (£58 million) was allocated to cytotoxic drugs for treating a disease that would affect at least one in three of the population, while £200 million was spent on treating constipation.24

The trust was concerned that individual health authorities within its catchment area would negotiate different contracts, resulting in inequitable access to new developments, as has previously occurred with cancer treatments.5 A Greater Manchester consortium of six health authorities was established, partly as a response to this and partly to develop more efficient commissioning. The consortium, responsible for purchasing around two thirds of services from the hospital, also advised neighbouring health authorities in the catchment area. It soon became clear that the trust's proposed developments could not be met in full from available resources and the consortium agreed to set priorities in funding drug developments. This paper describes our joint experiences in prioritising drug proposals for 1997-8.

Summary points

  • Providers and commissioners of health care are under pressure to introduce new and expensive cancer drugs within limited budgets

  • Collaboration between a specialist cancer hospital and a consortium of six health authorities helped in appraising drug developments and drawing up priorities for 1997-8

  • Obstacles to agreement included the different values placed on the methods and outcomes of research, lack of national guidance, and a low underlying investment in oncology services

  • Decisions on funding treatments were based on evidence thresholds—cut off points determined from information on effectiveness

  • Evidence thresholds can be influenced by the value placed on some clinical outcomes, political pressures, and financial constraints


Each year the trust's drugs and therapeutics committee consults all senior medical staff to register likely new drugs, new indications, and growth areas. Doctors provide supporting evidence for new developments. The costs of these are based on the expected number of patients requiring treatment over the year (or from expected launch dates for new drugs). The trust's planning and advisory committee assigned a high priority to the drug proposals submitted to the commissioners. Supporting documentation summarised evidence from a range of sources, including randomised trials reported in peer reviewed journals, independent reports,6 and unpublished data, alongside statements of funding needs and implications.

Collecting evidence

The commissioners then obtained further evidence by contacting doctors, searching Medline for relevant drug trials that had taken place between 1970and 1997,retrieving secondary references, and contacting pharmaceutical companies. Published reports were appraised for each drug proposal, to assess its validity, relevance, and cost effectiveness.7 The impact of new interventions on survival and quality of life were compared, where evidence was available, with standard treatment. Other outcomes of treatment assessed included tumour response rates, symptom responses, side effects, and overall costs.


As newer, better tolerated treatments are offered, the number of patients will expand

Categorising evidence

We drew on previous work on rating of evidence8 to classify proposals as shown in the box. Cost effectiveness for category I drugs was estimated as: costs per life year gained, costs per year in remission gained, and costs per adverse event prevented. Although several agents were primarily indicated for palliation, validated quality of life measurements were not often available from trial data. An attempt to derive quality adjusted life years was abandoned because outcome data were insufficient.

Categories of evidence

  • CategoryI—proved clinical effectiveness over and above existing treatments

  • Category II—promising, according to surrogate measures of clinical outcome, but further research and development required

  • Category III—research to date does not show appreciable benefit over existing treatment

Drug costs

Drug costswere derived from the trust's proposals. Direct costs were used as marginal costs for three reasons. Firstly, most drug proposals were indicated for advanced cancer that had previously been managed with best supportive care. Secondly, the number of patients treated would expand as newer, better tolerated treatments were offered. Thirdly, new regimens tended to be considerably more expensive than existing treatments.


The ranked proposals were relayed back to clinicians and discussed with them. We then reviewed any subsequent, updated evidence on drug proposals for which the earlier evidence had been categorised as II or III, and amended the overall rankings slightly. Patients did not participate in this approach, although at least one health authority has since started involving patients in setting priorities.

Outcomes of the assessment

The four case studies given in the box illustrate some of the issues we had to deal with. Commissioners judged that phase III randomised trials provided the best quality evidence (case 1). However, proof of effectiveness was also accepted where phase II studies showed exceptional benefits related to quality of life or survival, or where randomised trials seemed difficult to organise (case 2). Unpublished data were accepted (case 3), although the case was strengthened by peer reviewed published reports.

Case studies

Case1—paclitaxel in advanced ovarian cancer (accepted)

A major randomised trial showed that first line use of a combined drug regimen including paclitaxel increased median survival by 14months.9 The estimated cost per added life year was £7710. During feedback, clinicians favoured giving paclitaxel a higher ranking as funding would be subsidised for what was viewed as the definitive Medical Research Council multicentre trial.

Case2—lipid associated amphotericin in invasive pulmonary aspergillosis (accepted)

Aspergillus infection is treated conventionally with amphotericin, although the use of this drug is limited by its side effects. Most trials suggesting benefit from lipid associated amphotericin in patients who did not tolerate or respond to conventional amphotericin treatment did not include controls.10 The case for funding was accepted because a small, unpublished, local study showed a large fall in mortality, compared with historical rates of 50-100%, in acutely ill patients with leukaemia. The estimated cost per added life year was £1380, assuming an additional 2.9life years for every patient treated.6

Case3—anastrozole in advanced breast cancer (accepted)

Early evidence supporting anastrozole indicated that tumour response rates were improved, but there were no corresponding benefits in survival or quality of life. Although patients taking anastrozole gained less weight than those given standard treatment, this agent was rejected because 17patients would need to be treated to prevent one gaining 2kg in weight.11 However, during our deliberations evidence emerged of improved survival after longer term follow up.12 The estimated cost per added life year was £7300.

Case4—docetaxel in advanced breast cancer (rejected)

An uncontrolled study showed that patients with tumours which responded to docetaxel survived for 11months longer than patients whose tumours continued to progress.13 However, median survival for all patients treated was 8months (95% confidence interval 6.7to 10.5months) compared with 2-3months for historical control subjects given the best supportive care and up to 6months for patients treated with current second line monotherapy. Potential selection bias and uncertainty over the size of the clinical effect showed that randomised trials were needed. Early results from a recent randomised trial suggest clinical benefit, but this is being assessed further.14

Methodological problems

Several drug proposals were rejected because the supporting evidence was flawed by methodological limitations (case 4). Well conducted randomised trials showing only an improved tumour response rate—a surrogate outcome measure—were not accepted as proof of benefit. Tumour response rates suggest clinical progress, but they do not necessarily correlate with better survival or quality of life.15

Outpatient versus inpatient treatments

Submissions made for two drugs, raltitrexed and gemcitabine, were based on the finding that although clinical outcomes were similar to those for existing agents, these drugs could be used in outpatient settings. The reduced need for inpatient care might result in a lower total cost to the trust. 16 17 While challenging the validity of these economic data, the commissioners suggested that the trust could opt to fund these drugs from existing resources. However, this was not financially viable for the trust because inpatient activity is reimbursed at a higher rate than outpatient activity under the current contracting system. The white paper, The new NHS,seeks to address this issue.18

Priority list for 1997-8

Of 16proposed drug developments, seven were classified under category I, eight under category II, and one under category III (table). The Greater Manchester consortium agreed that all category I drugs should be funded.

Categorisation of drug proposals and projected additional costs to commissioners for 1997-8

View this table:

Lessons and implications

The Audit Commission recommended that health authorities work in consortia to commission specialist services, sharing risk and explicit criteria for prioritising treatments with trusts. 19 20 Although feasible, such collaboration requires considerable investment in time, coordination, and the tolerance (or use) of tensions between specialist clinicians and generalist commissioners.

Disagreements often centred on the different values placed on some trial outcomes (for example, tumour response rates) or the clinical importance of improvements in mortality or morbidity. Ideally, criteria on what constitutes satisfactory evidence of effectiveness would be agreed before negotiations. If more new drug trials incorporated validated, relevant health and economic outcomes it would be easier to resolve these differences and, perhaps, the question of funding for routine use.

Earlier guidance commends using expert opinion when only limited evidence is available.21 Clinicians perceived this systematic attempt to set priorities for new cancer drugs as unresponsive to individual patient needs. The director of the NHS Research and Development Programme has commented that final decisions over treating individual patients “should be made by those responsible for delivering the service.”22 However, commissioners also share responsibility for delivering the service and need to balance a wide range of priorities. The cost effectiveness calculations were crude and often limited by the lack of comparative data, but they were necessary to justify health authority funding.

The reality of dealing with a continuum of evidence was most noticeable in case 3(anastrozole). Here, survival data became available only late in negotiations and tipped the balance in favour of funding anastrozole. Otherwise, anastrozole would probably not have received funding for 1997-8.Longer term commissioning cycles of three to five years might ease the pressures related to short term contracting.18 Local mechanisms for rapid review of developments with commissioners, based on a joint plan for managing additional patient activity within the financial year, have yet to be agreed.

Commissioners face a dichotomy of political rhetoric on setting priorities. They are expected to divorce competing budgetary pressures from the objective assessment of new interventions and set priorities by assessing needs.23 In practice, the cut off point for funding based on the available information about effectiveness—the evidence threshold—might easily be influenced by several factors. Science is not a purist pursuit and scientists are human in their response to the social and political climate.24 We experienced pressure from some health authorities to move the threshold up because of their financial constraints and pressure from doctors to move the threshold down to reflect their special interests and the expectations of their patients.

Raising the evidence threshold, say by accepting evidence of benefit from randomised controlled trials only, would have reduced the number of drugs approved by two and increased the assessors' discomfort with the final decision on priorities. Lowering the evidence threshold might result in inappropriate funding and the adoption for routine use of treatments that have been evaluated insufficiently. Pressures to fund new treatments often begin before the results of relevant trials have been reported because the time lag before results are published may be considerable.

Some health authorities were uncomfortable about the fact that the evidence threshold was not sensitive to the financial position of individual health authorities. They argued that since the evidence itself was not black and white, the evidence threshold should be influenced by financial constraints. However, to preserve equity of treatment between patients from different districts, health authorities would either have to subsidise one another or accept the evidence threshold appropriate to the most financially constrained authority. Neither option was acceptable, and the health authorities were therefore advised that the cost of their contract would have to be increased by an appropriate percentage to meet the costs of adopting treatments backed by category I evidence. However, the trust still needed to negotiate a separate settlement with each health authority, and risk inequity.

The Calman-Hine report on oncology services recommends that new treatments “should initially be delivered only by the Cancer Centre.”1 However, cancer specialists felt frustrated and undermined throughout negotiations because several new drugs not funded at their hospital were being prescribed elsewhere within Greater Manchester. As current mechanisms could not control this trend, the trust supported a model whereby it received a devolved budget from commissioners to fund new oncology drugs within its catchment area.25 The advent of clinical governance, greater use of clinical networks envisaged under the Calman-Hine model, and national guidance may also promote more equitable and consistent prescribing between districts.


Ideally, commissioning decisions would be made with sound knowledge of the effectiveness of interventions. However, in practice, evidence based commissioning is hindered by a limited evidence base and influenced by political and financial pressures. Given the same evidence assessed here, it is possible that other commissioners would have set different priorities, exacerbating geographical inequities in access to new cancer treatments. National guidance to support commissioners in setting explicit priorities could reduce duplicated effort, promote equity, and ensure that patients receive the most cost effective interventions. It remains to be seen whether the National Institute for Clinical Excellence will support this function. However, responsibility also lies with those developing or advocating new interventions to provide valid and relevant evidence of their clinical and cost effectiveness.


When the approach to priority setting described above was repeated during the contract negotiations for 1998-9, paclitaxel as first line treatment in ovarian cancer (at a marginal cost of £530000) retained category I status, whereas docetaxel in advanced breast cancer (at a marginal annual cost of £240000) acquired category I status. Although the evidence threshold delineated these drugs from other developments, financial constraints prevented commissioners from funding their application fully. Concerns about inequity of access to new treatments across the North West Region persist. However, how to resolve these and several other difficult issues is still being debated vigorously between the trust and health authorities.


We acknowledge the assistance of senior medical staff at Christie Hospital and members of the Greater Manchester health authority consortium.

Funding: None.

Competing interest: None declared.


View Abstract