Patients’ rating of treatment tells you more about patients than hospitals, research concludes

BMJ 2013; 347 doi: (Published 18 November 2013) Cite this as: BMJ 2013;347:f6916
  1. Nigel Hawkes
  1. 1London

The way patients rate the outcome of their treatments in different hospitals says more about the patients than it does about the hospitals, a study at the Centre for Health Economics at the University of York has found.

Using patient reported outcome measures (PROMs) collected by the NHS in England since 2009, a team led by Andrew Street has found that most of the variation was accounted for by the types of patient treated rather than by the competence of the hospital treating them.

The findings, presented at a meeting in London on 14 November organised by Healthcare Conferences UK, undermine the idea that led to the introduction of PROMs in England, which was to improve the quality of care by identifying poor performers and enabling informed patients to choose to go to better ones.

Speaking at the meeting, Nick Black of the London School of Hygiene and Tropical Medicine contrasted this objective with the vision in the United States and Sweden, where, he said, PROMs were seen as a means to improve care through shared decision making.

PROMs assess the outcomes of treatments such as hip or knee replacement by asking patients a series of questions about their state of health before the operation and repeating the exercise three or six months afterwards, when they have had time to recover.

The results have shown that for hip and knee implants and varicose vein and hernia surgery—the four procedures with which the programme started—most patients do report benefits. In the case of hip implants, Black said, 77% of patients reported a useful benefit, while 30% also reported postoperative problems, though these included what may simply be a transient problem with the wound.

PROMs vary between hospitals. Street and his colleague Manual Gomes examined four possible reasons for the differences: patients were different; the cost of their care was different; they were treated differently in different hospitals; or data were incomplete. They found that these were not independent variables, as patients who had the most comorbidity or who came from the most deprived areas were more likely not to complete their forms than patients who were healthier, were white, or were treated in private hospitals.

Straightforward comparison of the PROMs results showed that 22 hospitals were outliers, with results significantly worse than average, but after adjustment for risk this number fell to nine, and after further statistical analysis with multiple imputation this fell to just two. Street and Gomez concluded that there was little point in varying the tariff system to reward hospitals that had better outcomes, because there was little evidence of systematic differences in the quality of care. If money were to be spent, it would be better to reward providers for collecting data more efficiently, they said.

Despite these findings, Jo Partington of NHS England said that the motivations for collecting PROMs included benchmarking, holding providers to account, quality monitoring, and resource allocation. She said that future plans included using PROMs in treatment for cancer, major trauma, HIV, musculoskeletal disorders, depression, and dementia and in psychological therapies.

Black said that after a hiatus caused by the government’s reorganisation of the NHS in England, the PROMs programme was now moving ahead, and the first meeting of a PROMs Advisory Committee was planned by NHS England to be held soon.

Among other things he hoped that the committee, of which he will be a member, would discuss the presentation of PROMs data by the NHS Information Centre. “All attention has been to input, none to output,” he said. “The spreadsheets published by the Information Centre are a complete turn-off. At least 20% of the budget should be spent on outputs.”

He also said that higher response rates were needed, which were possible because some hospitals had already achieved them. And he urged commissioners not to misuse the information that PROMs provided, such as by trying to use the data to determine what kind of patient was suitable for surgery.

More work needed to be done on using PROMs for treatments other than surgery, he said, such as emergency care and treatment for long term conditions. “We’ve had a bit of a lull, but I’m now optimistic that we can kick start the use of PROMs again.”


Cite this as: BMJ 2013;347:f6916

View Abstract

Log in

Log in through your institution


* For online subscription