Intended for healthcare professionals


Maximising the impact of patient reported outcome assessment for patients and society

BMJ 2019; 364 doi: (Published 24 January 2019) Cite this as: BMJ 2019;364:k5267
  1. Melanie Calvert, professor of outcomes methodology1,
  2. Derek Kyte, lecturer in health research methods and NIHR fellow1,
  3. Gary Price, patient partner, member of CPROR executive1,
  4. Jose M Valderas, professor of health services and policy research2,
  5. Niels Henrik Hjollund, clinical professor3
  1. 1Centre for Patient Reported Outcomes Research (CPROR), Institute of Applied Health Research, and NIHR, Birmingham Biomedical Research Centre, University of Birmingham B15 2TT UK
  2. 2NIHR PenCLAHRC and Institute for Health Services Research, University of Exeter Medical School, St Luke’s Campus, St Leonards, Exeter EX1 2LU, UK
  3. 3AmbuFlex/WestChronic, Regional Hospital West Jutland, Herning, Denmark, and Department of Clinical Epidemiology, Aarhus University Hospital, Aarhus, Denmark
  1. Correspondence to: M Calvert m.calvert{at}

Patient reported outcome measures can help drive global patient centred healthcare reform, but we need a more efficient coordinated approach to assessment if we are to fully realise benefits for patients and society, say Melanie Calvert and colleagues

Key messages

  • Patient reported outcome data are increasingly being used by a range of stakeholders in healthcare

  • These data may offer major benefits to patients and society, but current use is fragmented and suboptimal

  • We propose an integrated evidence based approach to data collection to meet multiple stakeholder needs

Over the past decade we have seen a global rise in the involvement of patients in coproducing research and decisions about their health and care. “Measuring what matters to patients” is recognised as central to improving patient care and service delivery, but patients need to be involved in deciding what to measure and how.1

One way to measure what matters is using patient reported outcome measures (PROMs), which are questionnaires completed by patients to assess the effects of disease or treatment (or both) on symptoms, functioning, and health related quality of life from their perspective. PROM data can be used to inform health technology assessment, pharmaceutical labelling claims, health policy and service improvement, and can support communication between patients and healthcare professionals.23

Here we discuss the current applications and potential benefits of PROMs in healthcare and challenges that reduce their potential to drive improvements in patient care. We focus on recent developments in the use of PROMs and consider strategies for efficient PROM data collection to maximise benefits for patients and society.

Current use and benefits

PROM assessment in research and routine clinical practice offers a range of potential benefits for individual patient care and for clinicians, regulators, healthcare management teams, commissioners, and policy makers (table 1).

Table 1

Use of PROM data by stakeholders, and current data collection

View this table:

The use of PROMs in research, particularly in clinical trials and observational studies, is well established and can provide valuable evidence on the burden of disease and the efficacy, effectiveness, and cost effectiveness of interventions from a patient perspective.261617 PROM data are increasingly being used to provide evidence for drug and device approval. Emphasis is being placed on involving patients throughout the innovation pathway, including the appropriate collection of PROMs informed by FDA and EMA guidance.2728Aggregate PROM data have been used in routine practice for several years; for example, in the UK to assess provider performance in the primary care Quality and Outcomes Framework (QOF)25 and in the NHS PROMs initiative.229 Three PROMs (the PHQ9, HADS, and Beck Depression Inventory-II) were used as part of QOF to assess the severity of depression, to support clinical decision making, and to assess provider performance. However, the QOF indicator was dropped in 2013 owing to criticism regarding over-diagnosis using the tools and the potential for gaming.2530 PROMs have been used to measure health gain in patients undergoing hip or knee replacement, among other procedures, based on responses to questionnaires before and after surgery. Patients and referrers can use PROM data to help decide where to receive treatment: NHS Choices publishes provider level outlier data for PROM eligible procedures as part of a “score card.”31 The evidence to support using PROMs in this way is, however, limited,32 and challenges with paper based “top down” PROM capture include high rates of missing data and lack of accessible feedback for use by clinicians and patients.29

By contrast, using PROMs at the individual patient level is relatively new. PROMs can be used to inform clinical decision aids, for shared decision making, and to tailor care to individual patient needs. Electronic capture of PROMs in clinic and between appointments allows real time monitoring of symptoms, flexible scheduling of hospital appointments in response to PROM data, early detection of problems, and prompt clinical intervention.33 A US randomised trial of web based symptom monitoring in patients receiving chemotherapy showed that use of the tool was associated with better quality of life, fewer emergency hospital admissions, and increased survival.89 In Denmark, the AmbuFlex telehealth system is being used to schedule outpatient appointments for chronic conditions, including asthma, chronic obstructive pulmonary disease, epilepsy, sleep apnoea, and cancer.1213 PROMs are completed by patients at home and used for decision support to evaluate the need for a consultation, reducing the need for unnecessary outpatient appointments. To date, 31 000 outpatients have been referred to AmbuFlex follow-up, and 115 000 telePRO based contacts have been completed.

Challenges to consider

Several challenges have hindered both uptake and benefit to patients of PROMs (box 1), with major problems found in PROM study design, implementation, reporting, and interpretation.26 PROM data collection is fragmented, with limited coordination—if any—between teams responsible for research and routine care. Clinical disciplines often lack a standardised approach to assessment. Patients may be asked to complete multiple questionnaires, often with overlapping items, which can be burdensome and confusing. Furthermore, PROMs are often poorly or not reported, which limits their effects on patient care and is unethical.2938 Evidence shows that clinicians find that collecting PROMs improves clinical care and workflow and is “beneficial rather than burdensome.”4 But some clinicians think that these data are “subjective” and therefore biased or unimportant compared with laboratory findings.

Box 1

Current challenges in PROM assessment

PROM selection

  • PROMs are not always designed and selected with patient input to ensure that they measure what matters

  • Measurement properties, patient acceptability and burden, cultural validity, and interpretation guidelines are not always considered

  • Inconsistency in PROMs used within and across disease specialties make comparisons difficult

Ethical concerns

  • Patients may be unsure why they are being asked to complete a PROM, who will access their responses, and how the data will be used

  • Patient burden of completing multiple questionnaires

  • Inconsistent management of situations where PROM data show “concerning levels of psychological distress or physical symptoms that may require an immediate response”34

  • Poor quality or no reporting of PROM data means that patients may complete multiple questionnaires for no discernible purpose

  • Lack of PROM specific ethical guidance

Data collection, analysis, reporting, and interpretation

  • Engagement and acceptance from stakeholders for PROM collection may be lacking

  • Many clinical trials do not provide a clear rationale for PROM assessment26

  • How the data will be used to maximise patient care has not always been fully considered, even in routine clinical practice29

  • PROM data in research is commonly collected from a relatively small subset of the population, hindering wider applicability of findings. This may be more pronounced in trials with a “substudy” approach, or where appropriate, culturally validated, alternative language PROMs are not available

  • Missing data hinder reporting and use, and approaches to minimising missing data are highly variable26

  • Lack of consensus regarding analytical approach35

  • Many clinical applications of PROMs have been developed in silos and remain unpublished, limiting sharing of implementation strategies, good practice, and results

  • PROM results are often poorly reported and are difficult to access and interpret by patients and clinicians36

Data logistic problems

  • Incompatible IT systems without integration with electronic health records and use across service providers

  • Data stored in different formats

  • Lack of relevant IT/health informatics expertise

Inefficient uncoordinated approach

  • Development in silos leads to duplication of effort and inconsistency in collection methods, measures used, and data collected

  • Lack of integration between routine data collected for population level initiatives and individual symptom monitoring, and between routinely collected PROMs and research data

  • Missed opportunity to upscale datasets and enhance efficiency; no opportunity to “collect once, use many times”37


Integrated approach to PROMs

We need a strategic, coordinated, integrated approach to PROM assessment, a view supported by international qualitative research.39 This approach should be aimed at creating a non-burdensome pathway for patients to provide meaningful PROM data that may be used to support shared decision making, as well as provide a patient centred data pipeline for audit, benchmarking, research, and real world evidence (fig 1). Routine remote PROM monitoring could be used to support not only patients at high risk of emergency admission but also the millions of people who have multiple long term health conditions to reduce unnecessary outpatient appointments, promote medicine adherence, and tailor care to individual needs. Beyond optimising healthcare resources, this approach offers broader benefits to patients and society, with potential reductions in time off work, carer burden, and carbon footprint.

Fig 1
Fig 1

Integrated assessment of PROMs to meet multiple stakeholder needs

Crucially, the same PROM data could be aggregated to inform commissioning and service delivery decisions. Concurrently, the data could be incorporated into pragmatic trials to provide real world evidence of effectiveness and safety. Large scale “PROmic” data could be integrated with genomic, proteomic, metabolomic, clinical, and biomarker data and be used in prognostic models to inform patients of likely courses of symptom burden and functioning.40

Integrative approaches to PROM assessment should consider ways to reduce inefficiencies in data acquisition: a harmonised approach to the selection, collection, analysis, and reporting of PROMs, integration into the electronic health record, and guidance on the optimal presentation and use of data (fig 2).

Fig 2
Fig 2

Steps to realising a fully integrated PROM system

Stakeholder engagement and cooperation

A national PROM strategy should be developed with input from patients, clinicians, academics, industry, regulators, ethicists, and policy makers to ensure that the system and data meet stakeholder needs. We have found that engagement from patients, senior management, nurses, consultants, and allied healthcare professionals is essential to successful delivery of PROM specific strategic goals.101329

Establish which outcomes to measure

PROMs should measure outcomes that correspond to stakeholder needs. Identifying these outcomes and what matters to patients should be a priority. Regulatory agencies may focus on physical symptoms and functioning to inform licensing and labelling claims, whereas patients and health policy makers may be more interested in other domains of health related quality of life, such as participation in social activities and emotional wellbeing.41 Stakeholder relevant PROMs can be identified through patient involvement, qualitative research, or core outcome sets. These provide a set of standardised outcomes to be assessed in routine practice or effectiveness trials. They often include traditional clinical outcomes, such as all cause mortality, alongside measures of symptom burden, functioning, and disease control, which can be measured using PROMs. Several core outcome sets are available from the International Consortium for Health Outcomes Measurement42 and the Core Outcome Measures in Effectiveness Trials initiative43; further efficiencies may be gained, however, if a single core outcome set can be generated for research and routine practice for a clinical area or a broader set of conditions, such as inflammatory diseases.

Selection of PROMs

Identifying and selecting valid, reliable tools that are acceptable to patients from the target population may be challenging. The Consensus Based Standards for the Selection of Health Measurement Instruments initiative and the Evaluating the Measurement of Patient Reported Outcomes programme provide useful guidance to support the review of measurement properties.4445 Selected PROMs should have been developed with patient input,27 but this is not the case for many commonly used measures. A further challenge is the use of different measurement scales, which make it difficult to compare across measures. The Patient Reported Outcomes Measurement Information System (PROMIS) aims to provide measures scored on a common scale across global, physical, mental, and social health domains. The PROMIS items can be used for computerised adaptive testing to reduce patient burden.46 Further benefits of PROMs may be realised through individualised measures, but research in this area is scarce.32 Ultimately, the utility of the measure may differ depending on context and purpose, but wherever possible systems should be designed with multiple use in mind.

Developing a governance framework

Patients need to know who will access their data and how their data will be used. Clearly this has important consent implications for integrative PROM collection, which must meet data protection regulations. PROM data may reveal worrying levels of psychological distress or physical symptoms that may require an immediate response, known as a “PRO alert.”34 Clear response pathways for the management of PRO alerts should be in place, and issues around legal liability for failure to act must be considered. Overarching guidance on governance would help implementation of PROMs and promote efficiency in delivery.

Integrated approach to electronic capture of PROMs

Health informatics systems should be developed to capture PROMs in a standardised way that will allow patients and clinicians to access data at the point of care, incorporating flexible permissions that allow the patient to choose how their data will be used. National institutes for data science in healthcare should play a pivotal role in infrastructure developments. Operational and logistical matters—such as patient identification, usability of the system by diverse patient groups, automated reminders, algorithms for PROM alert management, and reporting mechanisms—require careful consideration to encourage compliance, ensure smooth workflow, and promote data quality. To fully realise the benefits and to meet multiple stakeholder needs, PROM data should be aggregated with clinical and “omic” data; for example, to facilitate case mix adjustment for comparison of service providers and for use in prognostic models.40

Analysis, reporting, interpretation, and dissemination of PROM data

Data will need to be analysed and reported using different templates tailored to stakeholder needs. PROM data are currently presented in a wide range of formats and further research is needed to optimise their presentation for accurate interpretation of the data and to make it useful.47 We have found that training and support is needed during early adoption.

System evaluation

Integrating PROM data will require iterative development and improvement. The cost effectiveness of PROM systems, impact on workflow and user satisfaction, and potential biases associated with multiple uses of data should be rigorously assessed.48


Routine collection, processing, and sharing of PROMs may offer huge benefits to society through better health outcomes and use of resources. There is clearly much to do to maximise the benefits of PROMs for patients and society. A crucial first step is to establish a national multi-stakeholder steering group, involving patients, clinicians, PROM methodologists, regulators, policy makers, and NHS digital to standardise PROM data and to establish and share knowledge and good practice. Integrated approaches to data collection will help promote optimal efficient collection, reduce patient burden, and enable us to harness patient centred data alongside health and biomedical outcomes to tackle healthcare challenges at scale. Greater collaborative multi-stakeholder efforts are required both nationally and internationally for the benefits of PROMs to be realised.


  • Contributors and Sources: MC wrote the first draft of the article, which arose from a series of discussions about the topic with coauthors. The piece draws upon broader discussions from the Centre for Patient Reported Outcomes Research (CPROR) Institute of Advanced Studies workshop led by MC, DK, and GP in February 2018 and attended by JMV and NH and from discussions with Ethan Basch, distinguished visiting professor, in November 2017. MC is director of CPROR at the University of Birmingham and leads cross cutting PROM research at the NIHR Birmingham Biomedical Research Centre and the NIHR Surgical Reconstruction and Microbiology Research Centre at the University Hospitals Birmingham NHS Foundation Trust and the University of Birmingham and Health Data Research UK Midlands. She led development of the SPIRIT-PRO and CONSORT-PRO extensions. DK is an NIHR fellow, deputy director of CPROR and is leading development of PRO systems in routine NHS practice. GP is a patient partner and member of the CPROR executive. JMV is president of the International Society for Quality of Life Research, has recently provided advice to the Organization for the Economic Cooperation and Development on developing patient reported indicators of health system performance, and recently completed an NIHR funded programme of research on PROMS in primary care (NIHR/CS/010/024). NHH led the development and implementation of the AmbuFlex PRO system in Denmark and is leader of the WestChronic research unit at the Aarhus University Research Clinic, Herning, Denmark. MC is guarantor.

  • Funding: This is an independent analysis funded by the NIHR Birmingham Biomedical Research Centre and the NIHR Surgical Reconstruction and Microbiology Research Centre at the University Hospitals Birmingham NHS Foundation Trust and the University of Birmingham. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, or the Department of Health.

  • Competing interests: We have read and understood BMJ policy on conflicts of interest and declare the following: MC has received personal fees from Astellas, Takeda, and Glaukos outside the submitted work and grants from the NIHR and is a coinvestigator at Health Data Research UK Midlands. DK is supported by the NIHR postdoctoral fellowship scheme (PDF-2016-09-009). JMV has received grants and fellowships from NIHR and Instituto de Salud Carlos III (Spain), grants from MRC, CRUK, Royal Devon and Exeter Trust, Fondo de Investigaciones Sanitarias (Spain), consultancy fees from the World Health Organization, and Technische Krankenkasse (Germany) and currently is chairman (unpaid) of the International Consortium for Health Outcomes Measurement panel for the development of core sets of outcomes for Health Overall Adult Health. GP and NHH declare no competing interests.