Intended for healthcare professionals

Feature Assessing Patient Care

NHS goes to the PROMS

BMJ 2008; 336 doi: (Published 26 June 2008) Cite this as: BMJ 2008;336:1464
  1. Nicholas Timmins, public policy editor
  1. 1Financial Times, London
  1. Nick.timmins{at}

Sir Bruce Keogh has been credited for encouraging cardiac surgeons to publish their results. Now, as the medical director of the NHS, he’s turning his attention to other specialties. Nick Timmins investigates

More than 12 years after the Conservative government first announced that the NHS would publish measures of outcome and mortality, the data are coming.

Previews of the first risk adjusted data on mortality from hip and knee replacements and aortic aneurysms are already on the NHS Choices website. Lord Darzi’s next stage review will promise much more, along with the development and publication of patient reported outcomes (PROMS), which use questionnaires on pain, mobility, depression and anxiety, and the ability to undertake the normal activities of daily life. Advocates of the questionnaires, which patients complete before and after treatment, say patient reported outcomes provide a remarkably sophisticated measure of whether a treatment has worked in the rather important sense of whether the patient feels better, and how much better.

Writing on the wall

How much hospitals get paid for a procedure may soon (possibly too soon) depend in part on such measurements of outcome. The man in charge is Sir Bruce Keogh, former president of the Society for Cardiothoracic Surgery and now the NHS medical director. “The writing has been on the wall for some time,” he says. “There has been a lot of talk for a long time about quality, and now we need to move forward quite quickly in trying to measure clinical outcomes.”

For Professor Keogh, the moment of truth came when he was summoned, as the man in charge of the cardiac surgical register, to Sir Ian Kennedy’s inquiry into the paediatric cardiac surgical deaths at Bristol Royal Infirmary in the mid-1990s. “Two things became apparent,” he said. “One was that our data weren’t as robust as they could be. And the second was that when Ian Kennedy reported, there was going to be a significant focus on clinical outcomes.”

After a long battle cardiac surgeons now routinely publish their results, with 80% of them publishing their individual results. And this seems to have happened without patients becoming terrified or surgeons being pilloried, or, as far as one can tell, surgeons avoiding difficult cases for fear that they will lower their unadjusted outcome figures.

Sophisticated outcome measures

For cardiac surgery, however, surgeons developed their own sophisticated outcome measures. “We developed a whole stand alone parallel system for collecting the data because we didn’t think the information systems in the NHS”—chiefly the hospital episode statistics (HES) data—“were good enough,” Professor Keogh says.

“But we can’t do that for every specialty. It is not right to try to. I’ve argued over the years as loudly as anybody that HES is not appropriate for this job. But we’ve got to start somewhere.”

So Professor Keogh and the chief nursing officer have written to hospitals to ask what HES based metrics they use, hoping that they will send back examples from various forms of clinical audit. Those examples will be consulted on and published on the NHS comparators website. Once people agree they are of value they will go on to the public NHS Choices website.

Professor Keogh hopes about 50 measures will emerge in addition to the simple death rates for major surgical interventions, such as hip and knee replacements, which are being rolled out now “to send a signal to the service that we are now in the business of measuring outcomes.”

Doctors will then be asked to develop the sorts of data for clinical audit “that people would really like to see, given a blank sheet of paper.” In developing these more sophisticated measures, Professor Keogh is keen to make it clear that he does not want his department’s dire track record “of making rapid decisions that are not evidenced based” to continue. So aside from drawing “on the intellectual firepower of doctors and other clinicians in the health service,” he will also look at the measures the northwest strategic health authority is piloting with US owned Premier Healthcare, along with measures that are used and being developed in the United States, Germany, and elsewhere.

“Once we have identified good ones, I would be keen to ask the strategic health authorities to pilot two of these a year—to get them to iron out the glitches—then take them national and then pilot some more”—in effect building up a library of measures. That “hasn’t been formalised yet,” he said. But some of the strategic health authorities “feel that clinical outcome measures are, in a sense, their core business, so they would be prepared to help with this.”

Mortality gets you only so far

And alongside all that will be a drive to include patient reported outcomes to overcome the cliché that “the operation was a great success, but the patient died.” Because, Professor Keogh notes, death rates get you only so far. Even for elective cardiac surgery, the death rate is below 1%.

Andrew Vallance-Owen, group medical director for the private health insurance company Bupa, which has been using patient reported outcomes for some years, says that mortality on its own, “tells you nothing at all about the quality of care for well over 90% of procedures.” And clinical outcomes alone do not tell you about the patient’s experience. A surgeon might report that the blood flow from a coronary artery bypass looks great, but that is not much use if the patient reports they still get out of breath and pained on exercise.

Patient reported outcomes provide the answer to that. The first ones, for four conditions (hip replacement, knee replacement, inguinal hernia, and varicose veins)are to be piloted nationally from next April. More are likely to follow, and Professor Keogh has started discussions within the health department on a research programme that could see new patient reported measures developed, not least so that they cover the shift of care out of hospital and into other settings. “We then need some sophisticated way of linking up pure clinical outcomes measures and the patient measures,” Professor Keogh says, because ultimately what needs to be ensured is that “we get the right treatment to the right patient, at the right time, and with the right result.”

When it owned its own hospitals Bupa used similar patient reported outcomes to inform its twice yearly review of whether consultants should retain admitting rights. The company used the data when a consultants’ practice was called into question as well as to drive up quality.

“Doctors are quite competitive,” Dr Vallance-Owen says. “Once they see each other’s data they want to do as well or better. So you get continuous quality improvement out of this.”

Professor Keogh notes that many forces are driving the need for individual and unit data on outcomes—revalidation, commissioning (when commissioners need to know where to go for the best care), distinction awards, service line reporting, regulation of foundation trusts by Monitor, and assessment of patient care by the Healthcare Commission. More people need to know about quality.

Coding the data

One likely outcome, he says, is that—as in the US, for example—it is doctors who will have to take responsibility for the coding of data.

“You hear people slagging off coding clerks,” he says, with doctors complaining that the HES data are dirty and unreliable. “But I think consultants should take responsibility for the coding and reporting of their activity and outcome. You have a moral and professional obligation to know what you are doing and how well you are doing it. And part of that obligation is building up the evidence based proof that you are doing well.”

Professor Keogh says he does not believe individual consultant data should be published yet and probably not “for some time down the line:” not until people are convinced that the data are accurate, and even then only if the individual data are meaningful. Doctors increasingly work in teams, he notes, so individual data may not always tell the story. He notes, however, that “teams have—or should have—leaders … and I believe leaders of teams need to be accountable.”

Financial incentives

One thing that worries him—and worries the external advocates of patient reported outcomes—is Gordon Brown’s suggestion in the draft Queen’s speech that they may be used to adjust how much hospitals get paid per procedure. “I’m quite keen in time, as we get more sophisticated measures, to see financial incentives put in place to help improve quality,” Professor Keogh says, “but not too quickly. Absolutely not. It really destroys everything if we do it too quickly.”

More than 150 years ago, Florence Nightingale used to record that her patients left hospital “dead, relieved, or unrelieved.” But for 60 years the NHS has routinely recorded only whether they left dead or alive. And that is about to change.

Throughout Professor Keogh stresses that “I want all the help I can get on this” from doctors and the service, “and with all the firepower out there I am sure we can find a solution to the inevitable problems.

“But I am keen to send a very clear message to the service that there is no going back on this. It is crazy that we are 60 years into the NHS and measuring outcomes on only a handful of things.”


  • Competing interests: None declared.

View Abstract