Intended for healthcare professionals

Analysis

Patient reported outcome measures could help transform healthcare

BMJ 2013; 346 doi: https://doi.org/10.1136/bmj.f167 (Published 28 January 2013) Cite this as: BMJ 2013;346:f167
  1. Nick Black, professor of health services research
  1. 1London School of Hygiene and Tropical Medicine, London WC1H 9SH, UK
  1. Correspondence to: nick.black{at}lshtm.ac.uk

Abstract

Routine use of patient reported outcome measures (PROMs) has the potential to help transform healthcare, says Nick Black. Not only can PROMs help patients and clinicians make better decisions, but they can also enable comparisons of providers’ performances to stimulate improvements in services

Patient reported outcome measures (PROMs) can drive the changes in how healthcare is organised and delivered. Key to this will be to link doctors' use of PROMs in the treatment of their patients with collection and aggregation of the data for assessing and comparing the performance of providers—all to improve healthcare quality.

What are PROMs?

Involvement of patients has moved on from simply seeking people’s satisfaction with their care. PROMs seek to ascertain patients’ views of their symptoms, their functional status, and their health related quality of life. PROMs are often wrongly referred to as so called “outcome measures,” though they actually measure health—by comparing a patient’s health at different times, the outcome of the care received can be determined. It’s important to distinguish PROMs from patient reported experience measures (PREMs), which focus on aspects of the humanity of care, such as being treated with dignity or being kept waiting.

PROMs were initially developed for use in research, which has culminated in some regulatory bodies mandating their use. From there, PROMs were adopted by some doctors to enhance the clinical management of individual patients. In recent years they have been used to assess and compare the outcomes achieved by healthcare providers, with support of leading clinicians and encouragement of politicians. Some doctors still question their use, but most recognise the benefits of incorporating the views of patients (see box 1) alongside their own.

Box 1: Why consider patients’ views?

  • Most healthcare aims to reduce symptoms, minimise disability, and improve quality of life—these are aspects that only patients can assess

  • Patients welcome being involved, and this may have health benefits in itself

  • Patients’ response rates are invariably better than clinicians’ (a patient only has to complete one questionnaire whereas a clinician has to do it for every patient)

  • The measure avoids observer bias (inevitable if asking clinicians to assess their own practice)

  • Considering patients’ views increases public accountability of health services and healthcare professionals

Current approaches to measuring patient reported outcomes

Broadly there are two types of PROM: disease specific and generic. The former, of which there are thousands, are tailored to the symptoms and impact on function of a specific condition. Generic PROMs consider general aspects such as self care and mobility (see box 2). Often both types are used, the former having greater face validity and credibility, the latter allowing comparisons across conditions. The reliability of PROMs is similar to that of clinical measures such as diastolic blood pressure or blood glucose.[1]

In addition to such multi-item PROMs, patients might also be asked single questions about the extent of any change in their health resulting from treatment (so called single transitional items) and also questions about any adverse consequences (complications).

Little is yet known about the impact of PROMs, although randomised trials of their use in clinical practice have demonstrated improvements in processes (such as diagnosis) and, less convincingly, in health outcomes.[2] The clearest benefits have been found in the diagnosis of depression. The more recent adoption of PROMs in comparing providers’ performance means that their impact has not yet been evaluated.

Box 2: Example of a disease specific and a generic PROM

Disease specific PROM: Oxford Hip Score

Twelve questions about how the patient has been over the previous 4 weeks covering pain (4 items), mobility (3 items), and activities (5 items). Five possible answers scored from 0 to 4, creating overall scale of 0 (severe disease) to 48 (no problems).

Example questions:

  • During the past 4 weeks have you been able to climb a flight of stairs?

  • Yes, easily/With little difficulty/With moderate difficulty/With extreme difficulty/No, impossible

  • During the past 4 weeks how would you describe the pain you usually had from your hip? None/Very mild/Mild/Moderate/Severe

  • During the past 4 weeks could you do the household shopping on your own?

  • Yes, easily/With little difficulty/With moderate difficulty/With extreme difficulty/No, impossible

Generic PROM: EuroQol EQ-5D

Five questions seeking information that best describes the patient’s health that day, covering mobility, self care, usual activities, pain/discomfort, anxiety/depression. Three possible answers: no problem; some problem; severe problem.

Example questions:

Self care: I have no problems with self care/I have some problems washing or dressing myself/I am unable to wash or dress myself.

Anxiety/depression: I am not/moderately/extremely anxious or depressed.

How widely have PROMs been implemented in routine practice?

Individual clinicians and hospitals are increasingly using PROMs, but widespread use by health systems is still uncommon and largely restricted to England, Sweden, and parts of the United States. In contrast to England, where their adoption has been driven by government wishes for public comparisons of providers’ performance, in Sweden and the US it has been the medical profession that has led the way, focused on improving the clinical care of individual patients.

In England, the principal use has been in elective surgery. The first nationwide application was in 2008 in a voluntary audit of mastectomy and breast reconstruction,[3] followed in April 2009 by a mandatory audit of all providers of hip and knee replacement, groin hernia repair, and varicose vein surgery (see box 3).[4] There are plans for more procedures to be added to this list, starting with coronary revascularisation in 2013. In addition, the feasibility of extension to long term conditions, cancer survivors, and people with dementia, is being explored. Given that there are over 50 established national clinical audits (all but one limited to clinicians’ reports of processes and outcomes), opportunities for wider use of PROMs are readily available.

Box 3: National PROMs programme in England for elective surgery

From April 2009 it has been mandatory for all providers (NHS hospitals, independent sector treatment centres, private hospitals) treating NHS patients for any of four elective procedures to participate in the national PROMs programme. All patients undergoing a hip or knee replacement, groin hernia repair, or varicose vein surgery should be invited to complete a questionnaire before surgery, either at the pre-assessment clinic or on the day of admission.

The preoperative questionnaire collects data on the patient’s sociodemographic characteristics, the duration of their condition, their general health, any comorbidities, and whether they are undergoing a repeat/revision procedure. In addition, they are asked to complete a disease specific PROM (Oxford Hip Score, Oxford Knee Score, or Aberdeen Varicose Vein Score; there is no available instrument for hernia repair) and a generic PROM (EQ-5D index and EQ-Visual Analogue Scale).

Patients who complete a preoperative questionnaire are mailed a postoperative questionnaire after three months (hernia repair, varicose vein surgery) or six months (hip or knee replacement). Non-responders receive one reminder letter. The questionnaire includes the same PROMs as the preoperative one plus single transitional items on their overall view of the result of surgery and the extent of any improvement. They are also asked to report on adverse outcomes (complications, readmission, and further surgery).

Over the first two years, of the 485 000 eligible patients, 329 000 (68%) were recruited, though this varied from about 80% recruitment for hip and knee replacement to 60% for hernia repair and 50% for varicose vein surgery. Postoperative response rates also differed by procedure from 85% (hip and knee replacement) to 75% (hernia repair) and 65% (varicose vein surgery).

PROMs data are linked to Hospital Episode Statistics by the Health and Social Care Information Centre who provide regular analysis of each provider’s preoperative patient characteristics (age, sex, severity) and the mean change in the PROM scores adjusted for case mix.[4] Providers are identified and compared by means of funnel plots that show whether or not any provider’s outcome is significantly different from what would be expected.

Nationwide use of PROMs commenced earlier in Sweden using the disease specific clinical databases (quality registers) established there by the medical profession since 1975.[5] PROMs began to be introduced in some in 2000.[6] In the US, widespread implementation of PROMs has been more restricted: for spinal conditions in northern New England,[7] for primary care in Pittsburgh,[8] and for depression in Minnesota.[9] The only nationwide use has been to compare health plans that purchase care for those over 65 years of age (Medicare).[10] In 2015, the federal government plans to extend the use of PROMs to reimbursement mechanisms for accountable care organisations (health maintenance organisations with a focus on outcome measurement). It is hoped that this will enable the level of reimbursement to reflect the value that patients’ ascribe to the outcome of their treatment.[8]

How are PROMs being used in England?

All three ways that PROMs can improve care are being pursued in England: assisting clinicians to provide better and more patient centred care; assessing and comparing the quality of providers; and providing data for evaluating practices and policies.

As regards the first, PROMs are being used to monitor patients’ conditions to help them and their doctors make well informed decisions about their treatment.[11] For example, three monthly measurements by people with hip osteoarthritis to help clinicians decide if and when to operate.[12] Similarly, regular reporting of PROMs is being used to help patients and doctors share the management of long term medical conditions.[13] PROMs help clinical decision making in the same way clinical investigations do. They are not used as absolute determinants (“patients with an Oxford Hip Score under 30 should have surgery,” for example) because their predictive validity for individuals is not strong enough.

The second use, for provider comparisons, aims to stimulate improvements in quality in several ways.

Firstly, patients can choose where to be treated on the basis of the outcome reports of other patients, though in practice many other factors (such as distance from home) also influence a patient’s preference.[14] Secondly, PROMs are included in the NHS outcomes framework, which will be used to performance manage the NHS Commissioning Board and clinical commissioning groups. Thirdly, by having to report PROMs in their annual quality account, NHS providers account publicly for their performance to their local community. Fourthly, PROMs data can contribute to the revalidation of doctors. Finally, PROMs provide a means of enhancing the calculation of healthcare productivity by including the outcome as well as the quantity of care.

As for research, routine PROMs provide data on large numbers of patients representative of typical, everyday practice, thus facilitating research on the effectiveness (rather than efficacy) of treatments.[15] The inclusion of generic PROMs (such as the EuroQol EQ-5D—see box 2) allow patient utilities to be derived for cost effectiveness analysis. Such data can also be used to evaluate policies quickly and cheaply, such as the introduction of new ways of providing care[16] and the equity of services.[17]

What are the challenges and how can they be met?

Despite good progress in introducing PROMs into routine practice, more widespread implementation faces several challenges.

Minimising the time and cost of collection, analysis, and presentation of data

Information technology is important. Patient reported measurement systems are already being developed and web based entry has been introduced not only in clinical settings but also in patients’ homes.[5] [7] [12] [13] However, implementation is not necessarily straightforward. For example, although rheumatology departments in Sweden started converting to web based entry in 2003, by 2012 only 39% had done so. Hip replacement patients are more likely to respond to mailed (92%) than internet based questionnaires (49%).[18] Another option for minimising cost is to reduce the number of data collected by replacing multi-item PROMs with single transitional items.[19] This approach is the basis of the current quest by the Department of Health in England for a short questionnaire that could be used for all conditions and interventions.

Achieving high rates of patient participation

The challenge is how to achieve high rates particularly among older, sicker, more deprived, and non-white patients who tend to be under represented.[20] It is harder to recruit patients with minor conditions or those undergoing minor (or no) procedures, and those who are outpatients rather than inpatients.[21] As for primary care, little is yet known but it is likely to require different, innovative approaches, particularly for repeated assessments of patients with long term conditions.

Recognising all three dimensions of quality: safety, effectiveness, experience

PROMs focus mostly on the effectiveness of care, but safety and experience, the other two key dimensions of quality, must not be ignored. It is known that poor safety (such as complications) has an adverse impact on patient’s perception of the effectiveness of care.[22] The impact of patients’ experience of the humanity of care (such as dignity and respect) has started to be considered but requires much more investigation. It may be that judgments of a provider’s effectiveness will need to be adjusted to take into account patients’ experiences and vice versa.

Attributing outcomes to the quality of care

This presents several challenges.

Firstly, meaningful comparisons of providers require sufficiently robust adjustment for differences in case mix to achieve credibility. In addition to collecting data on known confounders from patients, more use could be made of obtaining data through linkage with other databases.

Secondly, judging the best time to assess outcome after an intervention so as to be able to attribute it to that intervention is often contentious: delaying follow-up ensures patients have gained all possible benefit but may undermine attribution to the intervention in question.

The third issue is determining the appropriate level of analysis for attributing responsibility for a patient’s outcome. Currently most PROMs are reported at institutional level (such as that of hospital, trust, commissioner). While this may be appropriate for some interventions, for others the individual practitioner is perceived as the attributable level. This is true of surgery—patients and surgeons (and many politicians) are enthusiastic for data at this level.[14] In contrast, the treatment of long term conditions depends on both primary and secondary care, so whole health economies may be the appropriate entity to consider.

Fourthly, emergency admissions present a challenge in the attribution of impact of care when PROMs are only available after the event. Solutions that need exploring are a patient’s recall of their pre-event health and the use of population norms.

Providing appropriate output to different audiences

Most questionnaires include more than one PROM (for example, a disease specific and a generic measure), each of which may draw different conclusions about a provider’s performance. In addition, different metrics can be derived from a measure (such as the mean PROM score or the proportion of patients achieving a certain level of improvement), and these may also assess providers’ performances differently. Further, having decided on an indicator, defining what constitutes unacceptable performance requires careful consideration (fig 1). The comparative risks of missing a poor performer must be weighed against unfairly criticising a provider. It is unclear if the rules used for clinical outcomes (such as mortality) are appropriate for PROMs. Is a PROM score more than three standard deviations from the mean as serious as a death rate that far from the mean? Also, deciding how to present the data needs to be tailored to the intended audience.[23] [24]

Figure1

Fig 1 Funnel plot of mean change in Oxford Hip Score following primary hip replacement for 88 NHS trusts (real data; fictitious names). Note that trusts more than three standard deviations below average (“much worse than average”) have mean scores only 2-3 points below average

Avoiding misuse of PROMs

There is a danger of PROMs being used crudely to ration care. Data from the national PROMs programme have already been misinterpreted as showing that 20 000 hernia and varicose vein operations and 16 000 hip and knee replacements each year should not take place.[25] While some patients will not benefit from surgery, unfortunately they cannot necessarily be identified preoperatively using PROMs. Another potential misuse is using PROMs to decide on competing demands between treatments for funding. If only short term outcomes are considered and longer term aspects, such as the natural history of the disease or long term outcomes, are ignored, poor conclusions may be drawn.[26]

Where next for PROMs?

The routine use of PROMs provides an opportunity to help drive changes in how healthcare is organised and delivered. There are five priorities for maximising their contribution.

Firstly, despite their separate development to date, we need to combine initiatives to use PROMs for clinical management and for provider comparisons, to contribute to both goals. Secondly, we need to encourage the adoption of new data collection technologies such that PROMs become part of everyday care. Thirdly, given that it is not feasible to extend provider comparisons to all healthcare, priority diseases and treatments need to be identified. Fourthly, we need to tackle the methodological challenges that remain unresolved to ensure PROMs are used appropriately. And finally, we must make use of the opportunity that PROMs presents to develop value based care in which health services can be driven by health outcomes per pound spent.[27]

Key messages

  • PROMs can be used to support the clinical management of patients, assess provider performance, and provide a basis for evaluative research

  • Nationwide use is most advanced in England (particularly for performance comparisons) and Sweden (for supporting clinical practice)

  • Several challenges need to be addressed including minimising costs, achieving high participation, attributing causality, providing appropriate outputs and discouraging misuse of PROMs data

  • The separate development of PROMs in clinical practice and in provider comparisons needs to be brought together for the benefit of both tasks

  • The impact of PROMs on clinical practice and on stimulating improvements in the quality of health services still needs to be established

Notes

Cite this as: BMJ 2013;346:f167

Footnotes

  • Contributor: NB has played a leading role in the development and establishment of the NHS national PROMs programme in England. As a grant holder for methodological studies of PROMs from the Department of Health Policy Research Programme, he was a member of the DH PROMs Operations Board. He is also a member of the DH PROMs Stakeholder Reference Group and chairs the DH National Advisory Group for Clinical Audit and Enquiries. NB is sole contributor and guarantor.

  • Competing interests: The author has completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declares: no support from any organisation for the submitted work; a financial relationship with the Department of Health for research on PROMs and membership of the DH Operations Board for the National PROMs Programme.

  • Provenance and peer review: Commissioned; externally peer reviewed.

References

View Abstract