Are measures of patient satisfaction hopelessly flawed?BMJ 2010; 341 doi: https://doi.org/10.1136/bmj.c4783 (Published 12 October 2010) Cite this as: BMJ 2010;341:c4783
- Jeannie L Haggerty, McGill research chair in family and community medicine
Measures of patient satisfaction and the patient experience—as instituted in the UK Quality and Outcomes Framework for primary care—supply feedback that helps health professionals provide patient centred care; they also give insight about the interpersonal dimension of quality of care as a complement to the technical quality of care. In the linked study (doi:10.1136/bmj.c5004), Salisbury and colleagues explore whether responses to questions in patient surveys that claim to assess the performance of general practices or doctors reflect differences between the practices, the doctors, or the patients themselves.1 The analysis separates the variance in patient satisfaction and patient experience into that attributed to differences between practices and those between doctors. The study found that when patients were asked a single question about how satisfied overall they were with their practice, only 4.6% of the variance in their satisfaction ratings was a result of differences between practices; the remaining variance resulted from differences between patients plus random error. In contrast, when asked to report on their experience with usual time they had to wait for an appointment, more than 20% of the variance in responses was a result of differences between practices. The authors conclude that for the purpose of discriminating performance between practices, it is better to ask patients to report on their experience rather than ask for satisfaction ratings[f1].
Still, the observation that measures of patient satisfaction and patient experience vary widely even among patients with the same doctor or practice raises questions about their use for evaluating practice performance. How can we make sense of such variance between patients? Is the use of satisfaction ratings a hopelessly flawed approach to evaluating practice performance? And what are the implications for both measurement and performance evaluation?
Firstly, the variance in satisfaction scores is not surprising given the multidimensional nature of health care and patient satisfaction. Although satisfaction is seen as a judgment about whether expectations were met, it is influenced by varying standards, different expectations, the patient’s disposition, time since care, and previous experience.2 None the less, qualitative research shows that patients will give positive satisfaction ratings even in the face of a negative experience unless they believe that the poor care is under the direct control of the person they are evaluating.3 4 For example, they may be unhappy about hurried communication with their doctor but still give an adequate rating because they attribute this to time constraints not a lack of intrinsic skills. Consequently, positive satisfaction ratings include both true positives and false positives. This compromises sensitivity in a diagnostic test and by the same token reduces the precision of satisfaction ratings. In contrast, negative satisfaction ratings tend to be truly negative (or highly specific in the analogy of diagnostic accuracy) and reflect important incidents, such as a lack of respect or medical errors.4 5 The implication is that the representation of satisfaction and satisfaction ratings needs to be changed. It is better to report the proportion of patients who are less than totally satisfied rather than the average satisfaction. High satisfaction ratings indicate that care is adequate not that it is of superior quality; low ratings indicate problems and should not be masked by reporting average scores.
Secondly, a defining characteristic of primary care is its high degree of variety and variance, even within the practice of one doctor.6 7 On a technical note, it is important to remember that analytical modelling that separates the variance into practice, doctor, and patient levels cannot separate variance between patients from random error. Part of this random error comes from the variation within practices and within doctors, which is to be expected, given the complexity of primary care. It is not surprising that such complexity can be only partially captured by a short questionnaire about experience and satisfaction. Despite this, patient assessments of health care work surprising well. Salisbury and colleagues show that assessments of access explain more variance between practices than they do between doctors, which makes sense for an attribute related to organisational arrangements. Conversely, assessments of communication explain more variance between doctors than between practices. Other studies have also found that patient assessments appropriately detect more variance between practices for organisational attributes and between doctors for personal care attributes.8 9 The implication is that the differences between practices and between doctors seen in the current analytical models underestimate the true differences that occur at the practice and doctor levels, and although Salisbury and colleagues are right in advocating prudence in interpreting small differences between practices, we can be confident that statistically significant differences are real and clinically relevant.
Thirdly, these results have implications for improving the science of measurement. Although it is difficult to measure patients’ perceptions of health care, it is most appropriate that patients should assess the interpersonal dimension of quality of care because they are the ones to whom we are ultimately accountable. It is therefore crucial that patient surveys are refined to maximise precision and minimise bias. The research community needs to develop and refine robust and comparable measures, bearing in mind that deficiencies in the measurement of satisfaction are more common in newly devised instruments.4
Measures of patient satisfaction need to be refined, but they are not hopelessly flawed. When they detect problems, these are real and important. They should be presented in a way that highlights the informative negative assessments, and they need to be combined with reports (such as experience) of components that can be benchmarked to recognised best practices.
Cite this as: BMJ 2010;341:c4783
Competing interests: The author has completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declares: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Provenance and peer review: Commissioned; not externally peer reviewed.
Log in using your username and password
Log in through your institution
Register for a free trial to thebmj.com to receive unlimited access to all content on thebmj.com for 14 days.
Sign up for a free trial