Patient reported outcome measures: how are we feeling today?BMJ 2012; 344 doi: http://dx.doi.org/10.1136/bmj.d8191 (Published 11 January 2012) Cite this as: BMJ 2012;344:d8191
Don Berwick, distinguished health quality guru and, until recently, in charge of the Center for Medicare and Medicaid Services, long ago pointed out that “The ultimate measure by which to judge the quality of a medical effort is whether it helps patients (and their families) as they see it.”1 This may seem a statement of the obvious, but it has taken a surprisingly long time for any health service to systematically collect information from patients about their health status.
Many countries have been collecting patient reported outcome measures (PROMs) as part of clinical trials or national or regional clinical registries, for example. However, it is the PROMs initiative in the English NHS2—and in particular its ambition to cover not just elective surgery but people with long term conditions such as diabetes, asthma, and chronic obstructive pulmonary disease—that perhaps sets it apart from other national efforts in this area.
Up to June 2011 the NHS in England had amassed a database of nearly a quarter of a million records detailing patients’ assessments of their health status both before and after surgery for one of four procedures (hip replacement, hernia repair, varicose vein surgery, and knee replacement).3 For each procedure patients complete a generic (EQ-5D) and a disease specific questionnaire (such as, the Oxford hip and knee score).
As more data accumulate—the dataset has been growing at an average of around 8000 new records every month since April 2009—patterns are beginning to emerge that are starting to raise questions about, for example, the significance of differences in health gain between procedures, why hospitals vary in the health gains they achieve, and, importantly, whether differences before and after surgery are meaningful in a clinical sense or to patients themselves.
Results for the first full year of data (fig1⇓) show that while half of groin hernia patients (9175 out of the 18 280 with complete EQ-5D records) reported an improvement in their health related quality of life, the other half reported either no change (32%) or poorer health after surgery (18%).4 The outcome was similar for varicose veins. On the other hand, nearly 9/10 patients having hip replacement and 8/10 having knee replacement reported an improvement after surgery.
Given the apparently different effects of the four procedures on patients’ health related quality of life, it’s tempting to suggest that, at a time when the NHS is looking to improve value for money, it should switch from hernia repairs to hip replacements. But given general agreement that the NHS aims (if not always successfully) to provide a comprehensive service, this temptation should be resisted. There is no doubt, however, that as PROMs data expand and links are made to other data on costs, the value we get from different healthcare intervention—something that has remained largely hidden—will start to expose potential trade-offs and increasingly difficult decisions.
Interesting results also emerge when comparing hospitals. What do some independent sector providers do to get better results from a hip operation than NHS hospitals (fig 2⇓)?4 (The funnel plot (fig 2⇓) is the preferred presentational device for identifying possibly important outliers.) But also, are the differences meaningful or chance findings? Other reasons for apparent better results may include the fact that independent treatment centres generally do not accept more difficult or complicated cases, though better preoperative health tends to be associated with smaller, not larger, health gains (fig 3⇓). Comparisons between types of hospitals or procedures can be full of statistical pitfalls; more research (inevitably) is needed.
Cite this as: BMJ 2011;343:d8191
Competing interests: The author has completed the ICJME unified disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declares no support from any organisation for the submitted work; no financial relationships with any organisation that might have an interest in the submitted work in the previous three years; and no other relationships or activities that could appear to have influenced the submitted work.
Provenance and peer review: Commissioned; externally peer reviewed.