Margaret McCartney: Innovation isn’t the same as evidence

The NHS is keen on innovation. “Innovation is not an option but a necessity if we are to build a sustainable NHS,” says Bruce Keogh, national medical director of NHS England. “Innovation is key to a viable NHS,” says Mahiben Maruthappu, cofounder of the NHS Innovation Accelerator.

NHS England has approved eight innovations that it says are “evidence-based and cost-saving and focus on providing solutions to key challenges facing the NHS, including better prevention of ill health, improved management of long term conditions, and early intervention into diseases.”1 Technology, early diagnosis, and innovation are the triumvirate of fashionable policy making.

The airwaves were saturated with pronouncements on an innovation that would, we were promised, be a step change in how sore throats are managed. Patients would go for rapid antigen tests at pharmacies, where they could also have antibiotics prescribed.

“It is hoped the scheme could result in fewer visits to GPs—potentially saving the NHS millions of pounds a year,” the BBC said. A microbiologist remarked, “Anything that reduces our reliance and our inappropriate use of antibiotics is a good thing.”2

This is innovation masquerading as evidence based policy making. The paper that seems to have spurred this policy decision in England was published in the Journal of Antimicrobial Chemotherapy in July 2016.3 It was a pilot study of outcomes of 367 people in which pharmacy staff identified them as attending with a sore throat, calculated their Centor criteria (which give a predictability score for sore throat being due to a bacterial infection4), and prescribed antibiotics if a point of care test for streptococcus A was positive.

Spot the problems. It wasn’t a randomised controlled trial, so we can’t know whether antibiotic prescriptions were reduced by the scheme. Neither do we know whether GP consultations decreased (15% of patients were referred to their GP by the pharmacist) or even increased (by encouraging more professional advice seeking in managing minor illness). It had no follow-up to detect benefits or harms; and other infections, such as chlamydia, were not tested for.

Neither NICE nor SIGN recommends rapid antigen testing for strep A,5 as the test’s poor sensitivity provides little additional benefit to clinical decision making. Group A streptococcus is self limiting in most cases.6 In culture confirmed infections the number needed to treat with antibiotics is eight to produce an improvement in recovery speed of less than a day, with no change in time off work or school, and 145 to prevent otitis media in adults. A 10th of people who take antibiotics experience side effects.7 Those aren’t great odds.

Innovation is being used as a smokescreen to smuggle poor quality evidence past the need for systematic review

Another innovation is “low intensity telehealth,” which aims to reduce “avoidable admissions.” But figures showing “reduced hospital admissions by up to 75%” were not obtained in a randomised controlled trial. And, with only 43 and 92 patients in the pilots, is this really a scheme worth funding, when other research points towards minimal and expensive benefits?8

New ideas are often useful. But innovation is being used here as a smokescreen to smuggle poor quality evidence past the need for systematic review and the cost effectiveness controls designed to stop the NHS wasting money, time, and effort on interventions that don’t come with a good enough rationale.

We need evidence based policy making. Labelling this trial “innovation” to get it into the NHS is not it.