Education and debate

Methods for incorporating patients' views in health care

Michel Wensing, senior researcher ^a, Glyn Elwyn, professor ^b. 

^a Centre for Quality of Care Research, University Medical Centre St Radboud, PO Box 9101, 6500 HB Nijmegen, Netherlands, ^b Primary Care Research Group, University of Wales Swansea Clinical School, Swansea SA2 8PP

Correspondence to: M Wensing m.wensing{at}wok.umcn.nl

Efforts to improve health care will be wasted unless they reflect what patients want from the service. But to be sure that surveys of patients' views are valid and have an effect on care, the methods used must be evaluated rigorously

Society now acknowledges the importance of the views of users in developing services, and the healthcare sector has used a range of methods to identify the views of patients and the public. Examples are questionnaires to assess patients' needs before a consultation with the clinician, shared decision making, focus groups with patients to include their views in clinical guidelines, and surveys among patients to provide feedback to care providers or the public. Such methods need to be examined in terms of validity, effectiveness, and implementation.¹ We describe some of the important issues related to measuring patients' views and evaluating their use in improving health care.

Summary points Patients can contribute to debates on health care by giving their preferences for care, evaluations of what occurred, or factual reports of care Measures of patients' views should be assessed for validity, preferably by rigorous qualitative studies Methods to include patients' views must be shown to affect the processes and outcomes of health care; possible negative consequences should also be considered

	Types of measures

The methods used to determine patients' views can be divided into three types: measures of preferences, evaluations by users, and reports of health care (box). The types of measure used will depend on what aspect of health care is being assessed, but all have limitations.

Preferences
One problem with assessing preferences is that patients' decisions about what is important in health care often reflect their individual experience rather than a general view. Interaction between patients in focus groups can help overcome this.

Another issue with measures of preference is deciding what options are presented. Qualitative research methods, such as individual interviews and focus groups, use open ended approaches such as topic lists rather than structured questionnaires. These give the greatest scope for expressing different preferences. Quantitative methods for eliciting preferences include surveys and consensus methods, such as Delphi and nominal group techniques. These techniques ask individuals to rate, rank, or vote for different types of care (such as, general practice or hospital) or attributes of care providers (short waiting list, adequate information). It is unclear whether the different methods of rating produce comparable results. ^{6 7}

Definitions of preferences, evaluations, and reports Preferences are ideas about what should occur in healthcare systems.2 Preference is often used to refer to individual patients' views about their clinical treatment, and the term priorities is used to describe the preferences of a population3 Evaluations are patients reactions to their experience of health carefor example, whether the process or outcome of their care was good or bad4 Reports represent objective observations of organisation or process of care by patients, regardless of their preferences or evaluations.5 Patients can, for instance, register how long they had to wait in the waiting room, irrespective of whether this was too long

Several models have been developed to collect and analyse preference data, including the expectancy-value model, multi-attribute utility models, and conjoint analysis models (discrete choice experiments). ^{8 9} The choice of methods will influence the results.³ Patients should contribute to the development of preference frameworks.

Decisions on prioritisation in healthcare systems inevitably involve a wide array of factors, and instruments have to be able to incorporate this multidimensionality. The most realistic methods present constrained choices, in which trade-offs have to be made between different attributes or choices.

Evaluations
Questionnaires that ask for evaluations of health care in terms of satisfaction or dissatisfaction show less discrimination than questionnaires that use terms such as good and bad or agree and disagree with concrete aspects of care.¹⁰ Some questionnaires measure preferences and experiences and derive evaluations from the two by calculating difference or ratio scores.¹¹ There is some evidence that patients distinguish between preferences and experiences,¹² but there is no validated framework for deriving evaluations from preferences and experiences.¹³

Patients are not always satisfied with their experiences of health care,¹⁴ and qualitative methods can be used to examine their experiences in more depth. Qualitative approaches are particularly useful for exploring patients' views in areas that have not been previously studied.¹⁵ Pragmatic approaches to qualitative analysis, such as logging key themes without full transcription analyses, have been used, but the reliability and validity of such approaches has not been assessed.

Reports
Although reports reflect patients' observations, they do not necessarily imply a patients' perspective on the quality of care. In some situations, patients' reports are the most accurate method of observationfor example, if data are required about a patient's pathway through different healthcare institutions.

	Quality of measurement instruments

Assessments of patients' views cannot be considered representative unless the measure has been properly evaluated. Various aspects need to be considered.

(Credit: SUE SHARPLES)

Validity
In a review of patient satisfaction studies, only 46% reported some validity or reliability data.¹⁶ Ideally, the instrument used should be compared with a criterion measurethat is, a measure with established validity. For instance, patients' reports of their care can be compared with the medical records or clinicians' reports of the care delivered.¹⁵

Criterion measures are often not available, however, so other approaches are needed. In this case, the validity of instruments should be based on a conceptual framework that describes a specific domain (the relevant aspects of health care). Ideally, patients should be consulted about the selection and description of the included aspects. Qualitative studies are particularly suitable for this purpose. Europep, an international instrument for obtaining patients' evaluations of general practice care, was based on systematic literature studies and qualitative and quantitative studies of patients' priorities.¹⁷ Sometimes it is also possible to verify that patients' views are associated with other factors as predicted by theory. This is known as construct validity.

Psychometrics
Quantitative instruments should have adequate psychometric features.¹⁸ High response rates to an item usually indicate that the question is relevant and understandable. However, this does not apply to instruments that examine rare events, such as medical errors (complaint procedures) or side effects of drugs (surveys among people taking the drug).

Instruments designed to measure aspects of quality should also show good variation across patients (discrimination) and variation between measurements at different points in time (responsiveness). If several indicators are supposed to assess one dimension of care, validity is supported by high internal consistency in the responses to indicators in that dimension. Ideally, instruments will also have good test-retest reliability.

The most often used reliability coefficients (such as ) refer to the internal consistency of items within a dimension per patient. In the context of quality improvement, however, aggregated scores per care provider are often important. These figures are based on several indicators and a number of patients or events. Generalisability analyses can be used to calculate reliability coefficients for the aggregated scores.¹⁹

Sampling
Non-responders are more likely to be ill, less satisfied with care provided, and less frequent users of health care than responders, although this isn't always the case. ^{20 21} Surveys or interview methods need to consider the effects of such patients being excluded or dropping out. Response rates for surveys of patients vary considerably. A literature review reported a mean response rate of 60% and a standard deviation of 21%.²² Many factors can influence the response rate of a survey. These include:

• Motivation of the clinician to recruit patients
• Attractiveness of the layout of a questionnaire
• Method of administering the questionnaire to patients
• Use of monetary incentives
• Use of information technology for administering questionnaires.²³

	Effectiveness

Methods to identify and use patient views to improve health care need to be shown to be effective. The best way to show this is by randomised trials. It is important that the outcomes chosen are relevant. We suggest that outcome measures for the evaluation are derived from the underlying objectives of the quality improvement exercise.

	Implementation

As well as studying the effects of specific methods, it is important to know whether they are actually used in health care. Clinicians, patients, and the public may lack the skills to use specific instruments or have negative attitudes about specific approaches. Incentives built into employment frameworks can lead to important shifts in attitudes for example, the new general practice contract in the United Kingdom mentions the evaluation of "patient experience" as one of three areas for measuring quality.²⁶ Such strategies should be evaluated in terms of uptake of the instruments.

	Conclusions

Increased participation of patients and the public in health care is desirable. Considering patients' views can improve processes and outcomes as well as satisfaction. However, many of the methods used have not been shown to be valid or effective. The evaluation of specific methods to obtain the views of patients therefore requires urgent action.

	Acknowledgments

This is a shortened version of a paper previously published in Quality and Safety in Health Care 2002;11:153-7.

	Footnotes

Competing interests: None declared.

	References

1.	Wensing M. Evidence-based patient empowerment [editorial]. Qual Health Care 2000; 9: 200-201[Free Full Text].
2.	Uhlmann RF, Inui TS, Carter WB. Patient requests and expectations. Definitions and clinical applications. Med Care 1984; 22: 681-685[CrossRef][ISI][Medline].
3.	Mullen PM. Public involvement in health care priority setting: an overview of methods for eliciting values. Health Expectations 1999; 2: 222-234[CrossRef][Medline].
4.	Pascoe GC. Patient satisfaction in primary health care: a literature review and analysis. Evaluation and Program Planning 1983; 6: 185-210[CrossRef][Medline].
5.	Brown JB, Adams ME. Patients as reliable reporters of medical care process. Recall of ambulatory encounter events. Med Care 1992; 30: 400-411[ISI][Medline].
6.	Grol R, Wensing M, Mainz J, Ferreira P, Hearnshaw H, Hjortdahl P, et al. Patients' priorities with respect to general practice care: an international comparison. Fam Pract 1999; 16: 4-11[Abstract/Free Full Text].
7.	Souchek J, Stacks JR, Brody B, Ashton CM, Giesler RB, Byrne MM, et al. A trial for comparing methods for eliciting treatment preferences from men with advanced prostate cancer. Results from the initial visit. Med Care 2000; 38: 1040-1050[CrossRef][ISI][Medline].
8.	Froberg DG, Kane RL. Methodology for measuring health-state preferences. 1. Measurement strategies. J Clin Epidemiol 1989; 42: 345-354[CrossRef][ISI][Medline].
9.	Ryan M. Using conjoint analysis to take account of patient preferences and go beyond health outcomes: an application to in vitro fertilisation. Soc Sci Med 1999; 48: 535-546[CrossRef][Medline].
10.	Wensing M, Grol R, Smits A. Quality judgements by patients on general practice care: a literature analysis. Soc Sci Med 1994; 38: 45-53[CrossRef][ISI][Medline].
11.	Sixma HJ, Van Campen C, Kerssens JJ, Peters L. Quality of care from the perspective of elderly people: the QUOTE-Elderly instrument. Age Ageing 2000; 29: 173-178[Abstract/Free Full Text].
12.	Jung HP, Wensing M, Grol R. Comparison of patients' preferences and evaluations regarding aspects of general practice care. Fam Pract 2000; 17: 236-242[Abstract/Free Full Text].
13.	Baker R. Pragmatic model of patient satisfaction in general practice: progress towards a theory. Qual Health Care 1997; 6: 201-204[Abstract].
14.	Williams B. Patient satisfaction: a valid concept? Soc Sci Med 1994; 38: 509-516[CrossRef][ISI][Medline].
15.	Schers H, Wensing M, Huijsmans S, Van Tulder M, Grol R. Low back pain management in primary care. Spine 2001; 26: E348-E353[CrossRef][Medline].
16.	Sitzia J. How valid and reliable are patient satisfaction data? An analysis of 195 studies. Int J Qual Health Care 1999; 11: 319-328[Abstract/Free Full Text].
17.	Grol R, Wensing M, for the Europep group. Patients evaluate general/family practice: the Europep instrument. Nijmegen: World Organisation of Family Doctors (WONCA)/European Association for Quality in Family Practice, 2000.
18.	Streiner DL, Norman GR. Health measurement scales. A practical guide to their development and use. Oxford: Oxford University Press, 1989.
19.	O'Brien RM. Generalizability coefficients are reliability coefficients. Qual Quantity 1995; 29: 421-428[CrossRef].
20.	Rubin HR. Can patients evaluate the quality of hospital care? Med Care Rev 1990; 47: 267-325[Medline].
21.	Etter JF, Perneger TV. Analysis of non-response bias in a mailed health survey. J Clin Epidemiol 1997; 50: 1123-1128[CrossRef][ISI][Medline].
22.	Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol 1997; 50: 1129-1136[CrossRef][ISI][Medline].
23.	Edwards P, Roberts I, Clarke M, DiGuiseffi C, Pratap S, Wentz R, et al. Increasing response rate to postal questionnaires: systematic review. BMJ 2002; 324: 1183-1185[Abstract/Free Full Text].
24.	Guadagnoli E, Ward P. Patient participation in decision-making. Soc Sci Med 1998; 47: 329-339[CrossRef][ISI][Medline].
25.	Edwards A, Elwyn G, Smith C. Consumers' views of quality: identifying the consultation outcomes of importance to consumers, and their relevance to `shared decision-making' approaches. Health Expectations 2001; 4: 151-161[CrossRef][Medline].
26.	General Practitioners' Committee. Your contract, your future. London: BMA, 2002.