NICE is dead; long live NICEBMJ 2013; 346 doi: http://dx.doi.org/10.1136/bmj.f2546 (Published 24 April 2013) Cite this as: BMJ 2013;346:f2546
Fourteen years ago, Parveen Kumar was unexpectedly summoned to a meeting in Whitehall. It was on a cold Friday morning, soon after her interview to be one of the directors of a new government body to evaluate medical treatment. After being ushered in she was told that she was to see the health secretary, Frank Dobson. He recalls, “The toughest challenge was to get good people on board. I insisted on seeing all of the nominated directors, with the right of veto if I didn’t take to any of them.”
At the end of this second interview, Kumar asked, “One last question. Will it work?”
Dobson replied, “Probably not, but we’ll have a bloody good try at it.”
He was right to be cautious about the prospects of the National Institute for Clinical Excellence (NICE) back in 1999. It was launched on April Fools’day with “no premises, no money, no staff, and no chief executive,” and an acronym that was “going to get lampooned in the press all the time,” according to Sir Michael Rawlins, the founding chair.
It was “a world where some quangos have disappeared within 18 days of being created,” as Richard Smith, former editor of the BMJ, put it.
Another journal editor, the Lancet’s Richard Horton, pondered its role in rationing, the “most electorally incendiary of subjects.” To provide some of these fireworks, Pfizer was preparing a legal challenge against Dobson’s decision to limit the use of sildenafil (Viagra) across the NHS, patient groups were preparing to campaign about limited access to new drugs, and Glaxo Wellcome was preparing for a “gloves off” fight with NICE.1 Other drug companies saw NICE as a potential barrier or at least a bottleneck for new drugs.2 An opposition party MP—Philip Hammond, now secretary of state for defence—was already warning. “This NICE will be nasty.”3
Now, as Mike Rawlins steps down as chair, NICE is “stronger than ever” due to legislation by the current government.4 Pfizer says NICE “has much to be proud of,” and GlaxoSmithKline says it “performs a valuable role.”5 6
Understanding the changing attitudes means understanding the evolution of NICE. The National Institute for Health and Clinical Excellence ceased to exist at the end of March. All its staff signed new contracts to work at the National Institute for Health and Care Excellence from the start of April, with a new chair and new responsibilities. But how far has NICE come in meeting its original aims?
Does NICE assess all treatment evidence?
NICE’s first published guidance started with the words “The practice of prophylactic removal of pathology-free impacted third molars should be discontinued in the NHS.” Since then, it has produced 274 more technology appraisals of surgical procedures, medical devices, tests, and, most famously, new drugs. NICE was set up to end postcode prescribing—the “lottery in care” depending on where you live.7 The idea was to replace covert local subjective judgments about funding made by primary care trusts with overt national evidence based decisions. But there have been some limitations.
One is the number of treatments that can be appraised. NICE does not look at all NHS treatments, and it has been rebuked for not doing more on older therapies.8 It does not even analyse all new drugs—it could look at less than half of those introduced in the past decade.9 All appraisals must first be referred by the health secretary and are prioritised if they have potential for significant benefit as well as having a significantly different price.10 As a consequence, only a small percentage of decisions in primary care or public health are affected by NICE approvals, and it has been argued that NHS spending becomes skewed towards expensive treatments in secondary care.8
NICE also produces broader national clinical guidelines to standardise good practice using evidence. But even for these, says David Haslam, the new chair of NICE and a former general practitioner, primary care clinicians are the least likely to use them. On the other hand, drug manufacturers and patient groups complain about “NICE blight,” which occurs when the NHS is reluctant to prescribe an expensive new drug unless it has been recommended by NICE.11
Another limitation is the type of evidence. Like many others, NICE places greatest emphasis on evidence from randomised controlled trials and systematic reviews. The Association for Family Therapy and Systemic Practice complains that a “constricted focus” on randomised trials creates “a very poor fit to the needs of the population” when NICE looks at mental health and social care.12 The British Psychological Society agrees. Rawlins has stated the need to consider the methodological flaws of the null hypothesis, problems with generalisability of findings, and the substantial resources needed—the median cost of randomised controlled trials by five funders in 2005-06 was £3.2m (€3.7m; $4.9m).13 Outside industry, few can afford to carry out such trials.
Inside industry, trials may be carried out but not published. In 2003, NICE considered the use of selective serotonin reuptake inhibitors, such as paroxetine, in children. Using published data only, the guideline group says it would have recommended such drugs. Fortuitously, the Medicines and Healthcare Products Regulatory Agency was investigating concerns about increased suicidal behaviour in children taking these drugs and it released all available unpublished trial data at the same time. NICE reanalysed the combined data and recommended alternative treatments.
Of course, sometimes there really is little or no evidence. Carl Heneghan, director of the Centre for Evidence Based Medicine at the University of Oxford, says that in those instances it is better not to make a recommendation: “Many users implement guidance in its entirety, but we should not be using low quality evidence that still needs research. Expert opinion leads to differing guidance around the world, which can be confusing to clinicians.”
Gillian Leng, deputy chair of NICE, counters “Where there is no published evidence, we use consensus methods, formal and informal, to fill those gaps because those questions need to be answered.”
How independent is NICE from political meddling?
The previous health secretary, Andrew Lansley, believed that an organisation as important and internationally recognised as NICE needed to be defined in an act of parliament—previously its existence could be ended by the stroke of a ministerial pen. Therefore, the Health and Social Care Act 2012 made NICE a non-departmental public body from 1 April 2013.
Although health secretaries have never exercised their right to overrule a NICE decision, there are at least two examples of perceived political interference.8 The first was NICE’s recommendation in 2002 rejecting the use of beta interferon and glatiramer acetate, disease modifying drugs for multiple sclerosis, because of cost. In response to the public furore, health ministers set up an innovative risk sharing scheme, in which the drug companies would repay the NHS for the drugs if they did not prove to be cost effective. Unfortunately, the ongoing study is mired in controversy, such that only results from a two year follow-up are available and it is unlikely that patients will know the exact benefits of the drugs or that drug companies could be forced to repay the NHS.14 15 16 17
In the case of trastuzumab (Herceptin), a drug already used in advanced breast cancer, a new health secretary decided to do NICE’s job for it. In October 2005, Patricia Hewitt ordered PCTs to test women with early breast cancer for trastuzumab eligibility and to give it to all women who needed it. When a PCT refused the drug for one patient she demanded to see its evidence herself; the PCT reversed its decision the next day.18 At that time Roche, the manufacturer, had not even applied for a licence to use it in early cancer and only interim trial data were available.19 Unsurprisingly to some, NICE approved the new indication for trastuzumab the following year.
Has NICE become too lenient?
Karol Sikora, an outspoken oncologist who has called NICE a Stalinist quango for refusing cancer treatments, thinks it buckles too easily under political pressure and cites NICE’s U turn and approval of sunitinib, a drug for renal cell carcinoma.20 He also says the more recent cancer drugs fund, set up by Lansley in 2010 with £200m per year, is a political way of circumventing NICE.21 Here treatments are chosen by regional panels that include cancer specialists and patients on the basis of clinical and cost effectiveness. But in apparent Orwellian doublespeak, the fund is for drugs that do not meet NICE’s cost effectiveness threshold.22
As expected, the Association of the British Pharmaceutical Industry (ABPI) successfully lobbied against NICE having a role in the cancer drugs fund.23 Rawlins says, “The most tricky part in the job was overcoming the hostility of the pharmaceutical industry. Relationships have become very much better than they were 14 years ago but all is not sweetness and light of course. If they were, we would probably not be doing a good job.”
NICE now has a consultancy service to work with industry on new drugs, and Rawlins regularly met drug company bosses at their headquarters. Jean Pierre Garnier, the former chief executive of GlaxoSmithKline, said to Rawlins that he hoped other organisations would be as fair as NICE. Does this mean NICE is becoming too lenient? After all, it recommends 80-90% of drugs for clinical use, with or without restrictions.24 Studies of similar international bodies find the recommendation rate is similar in Scotland (80%), but less in Canada (50%) and Australia (54%).25 26 27 However, the authors note that NICE was unique in studying only some drugs and having a formal appeals process in which up to a third of decisions are reversed.
But the ABPI argues that NICE is too stringent and that overall spending on drugs is less in the UK than in France, Germany, Ireland, Japan, the United States, and Canada, both as a percentage of gross domestic product and as spending per capita—backed up by figures from the World Health Organization.28 29 In particular, it points out that NICE does not fully value innovation or the wider benefits to society and that this damages the UK pharmaceutical industry, the fourth largest in the world with £21bn in annual exports and 70 000 employees. Others say that the industry already has significant government protection and that many new drugs have little benefit.30
QALYS: a flawed system but the best we’ve got?
Can finite healthcare resources be justly distributed among patients? “The moment you accept that it’s inappropriate to spend 100% of the United Kingdom’s gross domestic product on looking after one patient, then the discussion is no longer about whether there is a line, it’s simply about where you draw the line,” says Haslam.
Drawing the line can be difficult. After trastuzumab was approved, one oncology department said it would need £2.3m to fund the drug for 75 patients and said this could mean forgoing chemotherapy for 355 patients, of whom 16 could be expected to go into remission.31 Rawlins says “We felt it was cost-effective but Herceptin imposed a substantial cost on the NHS. In order to find that money, one trust closed down its palliative services at home and another trust closed down its diabetic eye clinics. When you remember that there is an opportunity cost, it’s a very real problem.”
Famously, NICE uses quality adjusted life years (QALYs) to quantify the “amount of health” and seeks to maximise the number of QALYs for a given cost. One QALY is a year in perfect health and zero QALYs is death. Questionnaires, such as the EQ-5D, can be used to calculate quality of life inbetween based on mobility, everyday activities, self care, pain, and mood.
If a new treatment is cheaper than current practice with similar or better outcomes then it is easy to recommend the new treatment. In most cases though, new treatments are more expensive with potentially better outcomes. Here NICE assesses the extra increase in health divided by the extra increase in spending to give an incremental cost effectiveness ratio (ICER). For example, when beta interferon was declined, NICE calculated the ICER to be £70 000 for one extra QALY gained by a patient.32
Initially NICE denied the existence of an upper threshold for acceptable ICERs but later analysis suggested it was between £20 000 and £30 000 per QALY for most decisions.
But QALYs have been criticised by patients and industry. They are subjective, and patients may rate themselves on a visual analogue scale or even simply be asked to estimate their health as a fraction of full health.33 As a result, NICE’s point estimates of ICERs may mask wide confidence intervals spanning tens of thousands of pounds.34
QALYs may also be biased.8 Quality of life in chronic conditions is said to be undervalued. Treatments for life threatening conditions will do better than for long term conditions. Mild conditions may be weighted too much because an increase in quality of life from 0.2 to 0.4 has the same value as an increase from 0.8 to 1.0, although the former is a doubling in quality. A cure for younger people is at an advantage because it improves quality for more years—but at the same time, a treatment that adds five years to an 80 year old’s life is not valued differently from one that adds five years to a 20 year old’s. Most of these criticisms go back to the fundamental notion of placing a value on quality of life. Despite them, Andrew Dillon, the chief executive of NICE, explains that QALYs are the best tool we have to compare the costs of different treatments in a consistent way.8
QALYs only measure direct health related benefits to the patient, excluding many other potential gains catalogued by health economists, patient groups, and the ABPI.35 For example, patients may experience increased happiness, comfort, dignity, and earnings; carers may have better mental and physical health; and society may benefit from reduced unemployment and social care spending. The government deliberately excluded all of these when it set up NICE, which is allowed to consider only patient costs to the NHS.
NICE is aware of the limitations and says QALYs “only inform, and not determine, NICE guidance.”8 Committees have the discretion to take account of the uncertainty of evidence and end of life situations. The NICE Citizens Council has said that a greater value should be placed on children and severe diseases. Therefore riluzole was approved despite an ICER of about £40 000 because of the severity of motor neuron disease, poor prognosis, and the lack of any other treatments. Rawlins also highlights the approval of permetrexed for mesothelioma by one committee—“They said yes to that, because the people who got this were those subjected to asbestos and we as a society had an obligation to try to help them.”
John Appleby, chief economist at the King’s Fund said the ICER threshold had no basis in theory or evidence 36 and is higher than NICE admits—being closer to £45 000.
Furthermore, NICE did not know the cost effectiveness of existing treatments in the NHS. Recent work looking at NHS expenditure and mortality says that the NHS cost effectiveness threshold is £18 317 per QALY, with a nearly two thirds chance that the NHS threshold is under £20 000.37 This means that the NHS may be twice as cost effective as some new treatments recommended by NICE.
The government acknowledges that reform is needed. It has proposed value based pricing from next year to replace the current pharmaceutical price regulation scheme, which allows companies to set drug prices on the basis of a target profitability (of about 21%).38 Details are still sketchy, but one possibility is that a fixed threshold could be chosen—say, £20 000 per QALY—and prices are negotiated with manufacturers to obtain this value.39 The government says it will define criteria for a higher threshold and that these could be diseases that are severe or without treatment and drugs that show greater innovation or wider benefits to society.8 In March, it announced that NICE would carry out all of the new value assessments.40
Exporting the NICE brand
NICE has been interviewed, investigated, lambasted, or lauded in thousands of publications, hundreds of newspaper articles, three parliamentary inquiries, and two BBC TV documentaries. On top of this, the organisation asked for two World Health Organization reviews of its work in 2004 and 2005.
Suzanne Hill took part in the reviews and now chairs the Australian equivalent of NICE, the Pharmaceutical Benefits Advisory Committee. This used cost effectiveness measurements six years before NICE and heavily influenced its establishment, with the founders of NICE visiting Australia to see how appraisals and committees worked. She has seen NICE grow and says, “It’s obviously incredibly well recognised as a brand for health technology assessment. I think it’s highly regarded for having a very comprehensive and sound approach to guidelines, rigorous health technology assessments, and involving consumers effectively.”
Rawlins adds, “I never imagined anyone outside Britain would be interested in NICE, but it’s got to the stage now where lots of countries are interested in what we’re doing. We also know that a no from NICE on a new drug is globally damaging. As a consequence, companies will give discounts to get a yes out of NICE, and that’s brilliant for the British public.”
He is referring to patient access schemes in which companies may offer free portions of treatment to reduce the overall cost per QALY, but the official price stays the same. The ABPI agrees that “NICE is the single most talked about health technology assessment agency in Europe. It is by no means now the only one, but NICE does set the tone in many places.”8
Hill also admires the openness of NICE from public board meetings to full availability of its documents. “Historically, the PBAC inherited UK secrecy laws that meant you could not see what was on our committee’s agenda up until 2005. Now, ironically, it is NICE that has been strongly championing transparency, and I see it as leading the way on that.”
The biggest difference between the two bodies is that the Australian committee only does new drug appraisals and has none of NICE’s other functions.41 One strength that Hill notes is that NICE has an effective monopoly on guidelines in England and Wales and she says, “We have no comprehensive, central, and sensible system for guideline development here in Australia.”
In fact, Australia joined Italy, Spain, and California in adopting NICE’s very first clinical guideline on schizophrenia.42 In an international comparison of the quality of schizophrenia guidelines across 21 countries, NICE’s guideline outperformed all others with a score of 90%.43 The next best were from professional associations in the US and Australia, with 71% and 62% respectively. The authors concluded “The NICE guideline’s strength was in its rigour of development and applicability, and its recommendations were evidence based with a clear description of how evidence was synthesised.”
The Pharmaceutical Benefits Advisory Committee does have one role that NICE does not. It is responsible for implementation of its decisions, through pricing recommendations and Hill wonders if NICE has focused too much on what to recommend and not how.
Has NICE put an end to postcode prescribing?
Since 2002, all technologies recommended by NICE have had to be made available within three months by NHS services. To help with this, NICE set up an implementation programme and now has a field team of six consultants who liaise with commissioning groups. By 2007, the Healthcare Commission said 85% of NHS organisations reported full compliance with NICE appraisals, although the Audit Commission found only 25% could verify compliance.8
Pharmaceutical companies and device manufacturers are frustrated by this “slow and low” uptake of NICE approved treatments.6 Most PCTs said that cost was the main barrier to full implementation. Approvals by NICE account for about 1% or £1.2bn of annual NHS expenditure, and the Department of Health has said this can be managed within the NHS budget.8 The industry is calling for financial incentives or penalties to be linked to adoption of NICE recommendations.
Progress is already being made. In 2012, the Department of Health implemented its own plans to set up a NICE compliance regime.44 45 In January this year, an “innovation scorecard” was launched to show the variation in use of each technology appraised by NICE.46 The ABPI welcomed this move and pointed out wide variations in uptake around the country.47
Finally, this April another change has occurred, set in motion by NICE’s assimilation of the National Prescribing Centre last April. Local hospital formularies will now automatically include NICE approved drugs and will become available for online scrutiny. The NHS chief executive ordered the move and warned all NHS organisations that this will become a standard term and condition in NHS contracts.48 Rawlins has no doubt that drug companies will soon be poring over local formularies to ensure full compliance is attained.
What next for NICE?
The headline change at NICE is the expansion into social care, reflected in the name change. Another change was that of the chair, with David Haslam taking up the post. At his parliamentary pre-appointment hearing, he argued the benefits of integrating health and social care guidance.
“If you were designing a system from scratch, you would not split it into health and social care. The public doesn’t recognise that [but] it is not something that you can legislate for. You have to build up trust and understanding and bring people together. I see my role as chair very much, hopefully, as working closely with senior people from the social care world to understand, indeed, their fears about NICE,” he said.8
Andrea Sutcliffe, former deputy chief executive of NICE and current chief executive at the Social Care Institute for Excellence, is well placed to know the difficulties with integrated guidelines. At a recent seminar, she described the scene in which NICE will work.
“There is a very, very challenging context—a tremendous squeeze on public and private finance, which we can see with the impact on local authorities . . . We’re going through our own legislative change in social care now as well,” she announced.
In her view, the challenges for NICE include the increasing number of older people, the diversity of the social care workforce, the much smaller evidence base, and the different culture in social care, with sometimes very different languages.
Fourteen years later, Parveen Kumar, who served as non-executive director of NICE for three years, and Frank Dobson again share concerns—this time over the expansion of NICE’s role into social care. Kumar says, “NICE is brilliant at what it does and has risen astonishingly. Its remit has grown as well and moved away from just drugs and technology. But is it the right place to do social care? We are trading on the name of NICE and I personally think it’s being asked to do things that do not belong there.”
Dobson says, “I don’t treasure that NICE should stay the way I set it up, but I am dubious about integrating health and social care. For example, the Care Quality Commission has not been as successful with a similar integration, and bringing together education and children’s services has not been a success in local authorities. I fear that NICE may be diluted.”
NICE confounded Frank Dobson’s fears once before. Let’s hope it does so again.
Cite this as: BMJ 2013;346:f2546
Competing interests: I have read and understood the BMJ Group policy on declaration of interests and declare I am a member of the guideline development group at the National Clinical Guidelines Centre.
Provenance and peer review: Commissioned; not externally peer reviewed.