Understanding why some ethnic minority patients evaluate medical care more negatively than white patients: a cross sectional analysis of a routine patient survey in English general practicesBMJ 2009; 339 doi: http://dx.doi.org/10.1136/bmj.b3450 (Published 17 September 2009) Cite this as: BMJ 2009;339:b3450
- 1NIHR School for Primary Care, University of Manchester, Manchester M13 9PL
- 2General Practice and Primary Care Research Unit, University of Cambridge, Cambridge CB2 0SR
- Correspondence to: Martin Roland
- Accepted 17 August 2009
Objective To examine why patients from ethnic minorities give poorer evaluations of primary health care than white patients.
Design Cross sectional analysis of patient surveys collected in general practice.
Setting 1098 English general practices that undertook a routine survey of patients using the General Practice Assessment Questionnaire between April 2005 and March 2006.
Participants 188 572 survey respondents, 95.8% of whom identified themselves as “white,” “black/black British,” “Asian/Asian British,” or “Chinese.” Analyses were restricted to between 133 441 (71%) and 147 686 (78%) respondents with complete data on relevant variables.
Main outcome measures Patient evaluations of waiting times for general practitioner appointments, time spent waiting in surgeries for consultations to start, and continuity of care.
Results All aspects of care were rated substantially lower by respondents from the three ethnic minority groups than by white patients. Poorer evaluations of time spent waiting for consultations to begin (rated lowest by Asian patients) and continuity of care (rated lowest by Chinese patients) appeared to reflect worse reported experiences by ethnic minority groups. Substantial differences between white and ethnic minority patients’ ratings of appointment waiting times persisted, however, even after adjusting for the actual time patients reported waiting. This effect disappeared for Chinese and black respondents after adjusting for evaluations of reception staff and doctors’ communication skills, but Asian patients’ ratings remained considerably lower than those of white respondents.
Conclusions Important differences in assessments of care exist in different ethnic minority groups. Some negative evaluations may reflect communication issues. Among Asian patients, lower ratings of waiting times for appointments may also reflect different expectations of care. Adjusting survey results for ethnicity may be justified when comparing healthcare providers; however, health services also have a responsibility to meet legitimate patient expectations.
Patient evaluations are increasingly being used as a way of measuring the quality of medical care. Studies in the United States and the United Kingdom have consistently shown that ethnic minority patients evaluate their care more negatively than do white patients, even after analyses have been adjusted for potential confounders.1 2 3 4 5 6 7 A report from the UK Department of Health in 2008 advised that specific measures needed to be taken to address the high levels of dissatisfaction expressed by patients from ethnic minority communities.8
There are several possible explanations for the lower ratings assigned by ethnic minority groups:
Demographic factors: there may be differences between white and ethnic minority patients in demographic factors such as socioeconomic status and employment status
Health need: ethnic minority patients may have different health needs from those of white patients, leading them to evaluate their care differently
Quality of care: ethnic minority patients might experience lower standards of care than white patients; for example, in terms of access, technical quality of care, or interpersonal care
Response set: ethnic minority patients may have a tendency to give less favourable evaluations even when receiving the same standards of care as white patients, which might reflect different expectations of care or differences in the way questionnaire items are interpreted.
The implications of these alternatives for policy makers, service managers, and healthcare professionals are very different, so it is important to determine which factor is the most likely cause of poor service evaluations by ethnic minority patients compared with white patients.
To address this question, we analysed patient survey data on access and continuity of care in an instrument routinely used in general practice in England, the General Practice Assessment Questionnaire (GPAQ).9 This survey collects data on sociodemographic characteristics, self reported health, and actual experiences of care.
The key research questions in this study were whether patients from ethnic minority groups evaluate general practice care more negatively than do white patients, whether differences in ratings are consistent across different ethnic groups and different aspects of care, and what factors account for lower ratings.
Between 2004 and 2009, English general practitioners (GPs) received a financial incentive to administer a patient survey as part of the quality and outcomes framework.10 The GPAQ was one of two approved questionnaires, although both have recently been replaced by a new GP patient survey introduced by the Department of Health in January 2009.11 Licensed suppliers and primary care trusts offering GPAQ services to general practices made anonymised data available to the National Primary Care Research and Development Centre to support ongoing research and development, and these data form the basis of the current analyses.
The GPAQ is designed for self completion by patients aged 16 years and over. Two versions are available: a post-consultation version designed to be completed by patients in the surgery immediately after seeing a GP; and a postal version designed to be administered by mail to a random sample of the practice population. In reality, however, the postal version is often administered to patients waiting in the surgery to see a clinician. Both versions contain common core items evaluating access, continuity of care, the manner of reception staff, and GP communication skills, along with seven health and sociodemographic items. These items are the focus of the current analysis.
The GPAQ is unusual in that respondents are invited to provide two different assessments for certain aspects of their care (table 1⇓). The first is a report of their actual experience of care (for example, how long they usually wait for an appointment with a particular doctor), and the second is their evaluation of that aspect of care rated on a six point scale (from “very poor” to “excellent”). Although actual experience and evaluation of care are related, research has indicated that there is often significant variation in the way experiences are evaluated.12
The full dataset included 188 572 individual questionnaire responses collected between April 2005 and March 2006 from the patients of 1098 practices, which encompasses approximately two thirds of the English primary care trusts in existence at that time. Sixty two per cent of responses were provided in the post-consultation version of the GPAQ, 36% in the postal version completed by patients attending surgeries, and 2% in the postal version mailed to randomly sampled patients. The data are drawn from a larger dataset of approximately 300 000 responses, but we included only responses that could be matched to individual general practices.
Questionnaires were included in the analyses if respondents had identified themselves as belonging to one of four ethnic groups (“white,” “black/black British,” “Asian/Asian British,” or “Chinese”) and had complete data on all items that were included as explanatory factors in the regression modelling: age, gender, housing tenure (a proxy for socioeconomic status), employment, presence or absence of longstanding illness, and reported number of GP visits in the previous 12 months. Actual sample size therefore varied between 133 441 (71%) and 147 686 cases (78%) depending on the analysis. Information collected in the questionnaires is shown in table 2⇓.
Initial descriptive analyses assessed the representativeness of the respondent sample by comparing proportions from each ethnic group with those reported in the 2001 UK census and the 2004 General Household Survey of Great Britain.13 14 We then carried out a series of regression analyses using as dependent variables patients’ ratings of the four aspects of care that are also measured in the GPAQ with report items (table 1)—namely, waiting time for appointments with a particular GP in the practice; waiting time for appointments with any GP in the practice; time spent waiting in the surgery for appointments to begin; and continuity of care (that is, the proportion of consultations with the patient’s “usual” GP or one they know best). As noted, patients evaluated each of these aspects of care by using a six point scale. These ratings were rescaled to give a score of 0-100 (with 100 representing “excellent” care).
Regressions were initially conducted with dummy independent variables representing each ethnic group. The regression coefficient associated with an ethnic minority group variable represents the difference between that group’s evaluation (that is, average percentage score) and the evaluation given by white respondents for each of the four dependent variables of interest. Comparison of coefficients between minority ethnic groups provides an estimate of the degree to which differences in evaluations (compared with white respondents) vary across groups and across aspects of care. Given that responses to the questionnaire were clustered at the level of the practice, a multi-level analysis was undertaken in Stata v9.2 to provide robust confidence interval estimates.
Further regression analyses were then conducted that incorporated additional independent variables relating to the different possible explanations for lower ethnic minority scores outlined in the introduction. These were:
Demographic factors: age, sex, housing tenure (as a proxy for socioeconomic status), and employment status
Health need: self reported longstanding illness or disability and number of GP consultations in the past 12 months
“Experienced quality of care”: patients’ reports of the actual level of care received on the dependent variable of interest, plus their evaluations of interpersonal care within the practice (both doctor-patient communication and interaction with reception staff).
In addition, scores generated using the post-consultation version of the GPAQ are between 2% and 6% higher than those for the postal version (regardless of whether the latter is administered in surgeries or by mail),15 so mode of survey administration was adjusted for in all analyses.
The independent variables were added to the regression in separate steps: (a) mode of administration; (b) demographic factors; (c) health need; and (d) “experienced quality of care.” The magnitude of the coefficient associated with each ethnic minority group (representing the difference in evaluation score compared with white respondents) was examined at each step. If an ethnic minority group’s mean score remained substantially different from that of white patients, then the independent variables included at that step and at previous steps could not account for the observed difference. If there was no longer a substantial difference, however, we concluded that the independent variables added in that step might account for the difference in ratings compared with white respondents.
The size of the GPAQ sample was such that very small differences in mean scores were statistically significant, so statistical significance alone was a poor guide to the substantive importance of relations between variables. We therefore used the magnitude of the difference in mean scores between ethnic minority respondents and white patients as a guide to importance. A difference of 10 points on a GPAQ scale is considered significant in terms identifying practices with outlying scores,9 but for the purpose of these analyses we defined a difference of 2.5 percentage points or more between white and ethnic minority respondents’ ratings as being an important difference.
Characteristics of the sample
Ethnicity data were available for 98.6% of respondents in the full dataset. Comparisons with the 2001 UK census and the 2004 General Household Survey of Great Britain showed that ethnic minority individuals were slightly over-represented in the GPAQ survey sample, particularly those identifying themselves as Asian/Asian British (table 3⇓). Baseline characteristics of the sample are shown in table 4⇓. Table 5⇓ presents data on the “experienced quality of care variables” from the GPAQ that were included as explanatory factors in the regression analyses. Lower standards of care were reported by all three minority ethnic groups compared with white respondents, except in relation to waiting times for appointments with a particular GP.
Detail of the analyses is presented in tables 6⇓ and 7⇓, and summarised in table 8⇓. Before inclusion of demographic, health need, or “experienced quality of care” variables, the three minority ethnic groups rated all four aspects of care under investigation substantially lower than did white patients, irrespective of mode of questionnaire administration (table 7, model 1).
After adjusting for differences in mode of survey administration, demographic factors, health need, and how many days patients reported having to wait for an appointment, black, Asian, and Chinese patients’ evaluations of waiting times for appointments (both with a particular GP and any GP) remained substantially lower than those of white respondents (table 7, model 4). Once patients’ evaluations of receptionists and GP communication were additionally adjusted for, however, Chinese and black patients’ ratings were no longer lower than those of white patients (table 7, model 6). Although adjusting for communication variables also reduced the difference between Asian and white respondents’ ratings of waiting times for GP appointments, a substantial difference persisted after all independent variables were included in the model.
Substantial differences between the ratings of all three minority ethnic groups and those of white patients with respect to time spent waiting in surgeries for consultations to start disappeared when the analysis was adjusted for actual waiting times reported by respondents (table 7, model 4). This suggests that poorer evaluations, in this instance, are largely explained by the longer waiting times experienced by ethnic minority patients.
Disparity between black and white patients’ ratings of continuity of care appears to be explained by sociodemographic differences in the respondent samples (table 7, model 2). For Asian and Chinese patients, however, lower ratings seem to be explained largely by differences in reported experience of continuity of care (table 7, model 4).
Finally, to allow for clustering of ethnic minority patients within particular practices, we adjusted additionally for the proportion of patients from each minority ethnic group in the respondent sample from each practice (not shown). Although this adjusting made some difference to individual coefficients, it did not alter the summary of results shown in table 8.
In line with previous studies, ethnic minority patients included in our sample of primary care patients rated their care more negatively than did white patients. This finding held true for all four aspects of care and was not to the result of differences in the method of survey administration. However, reasons for poorer evaluation varied by ethnic group and by the aspect of care being assessed. Among black and Chinese patients, negative evaluations seemed to be largely accounted for by differences in demographic profile and in actual experiences of continuity of care and waiting times in surgeries. In addition, issues relating to communication with practice staff played a part in explaining lower evaluations of appointment waiting times among black and Chinese patients compared with white respondents. Similarly, actual experiences of continuity and waiting times in surgeries seemed to account for lower evaluations of those particular aspects of care among Asian patients. However, none of the explanatory variables entered into the final regression model fully accounted for Asian patients’ lower evaluations of appointment waiting times.
Interpretation and policy implications
It is not clear why, after adjusting for so many potential confounding factors, Asian patients should continue to evaluate appointment waiting times substantially more negatively than do white respondents. This finding may reflect different interpretations of the questionnaire items, or higher expectations of access to care among Asians so that equivalent performance is judged more harshly. Bangladeshi patients have previously been found to express particularly high levels of dissatisfaction with their care,6 but we were not able to distinguish this group specifically in our data.
For all three ethnic minority groups, lower mean evaluation scores on each of the four aspects of care are partly explained by interpersonal experiences during care. Changes in absolute scores after adjusting for ratings of receptionists’ and GP communication were greatest among Chinese individuals and least for black respondents (table 7).The fact that black and Chinese patients’ ratings of appointment waiting times were no longer substantially lower than those of white respondents after this adjustment suggests that cultural dissonance and language problems increase the propensity for ethnic minorities to rate care more negatively than do white patients. This is consistent with Weech-Maldonado et al’s finding that linguistic minorities rate healthcare experiences more negatively than do other ethnic minorities.4 Difficulties with language cause particular problems with certain types of appointment system.16 Linguistic barriers also impact on patient reported outcomes of care. Freeman et al found that despite having shorter consultations, patients who spoke South Asian languages and who consulted a GP in their own language reported better outcomes (in terms of patient enablement) than did patients who consulted in English.17
The analyses presented here provide only preliminary insights into the mechanisms underlying disparities in evaluations of primary care between ethnic minority groups. In each analysis, the final regression model explained less than 60% of the variance in the dependent variable, suggesting that other unaccounted factors might explain the difference in evaluations. For example, we were unable to include measures of technical quality of care, known to be of key importance to patients but difficult to assess from patient reports.18 19 20 21 A recent study found little evidence of inequalities between ethnic minority groups in clinical outcomes of care for three conditions commonly managed in general practice in England (hypertension, hypercholesterolaemia, and diabetes); however, the same study found evidence of inequalities in use of hospital services.22 Given that access to UK hospital services is largely via GP referral, poorer evaluations of primary care by ethnic minority patients could be influenced, to some extent, by disparities in access to secondary care.
There are competing models of factors that influence patients’ evaluations of healthcare services. Such factors include beliefs about health, illness, and medical care; previous experiences; values; and expectations.23 24 25 Qualitative work with ethnic minority groups is needed to further explore the decision making processes underlying patient evaluations and to provide a more detailed exploration of variation within and between ethnic groups.26
A key question in measuring the quality of medical care is whether assessments should be compared against predefined standards or whether the aim should be to meet patients’ expectations. Research suggests that patient expectations of care may be higher than the quality standards set by the NHS, underlining the importance of this distinction.12 27 This issue has implications for deciding whether surveys should be adjusted for ethnic minority composition when comparing the scores for different providers. The argument for adjusting is that it would be unfair to compare practices providing the same standard of care without taking into account differences in the expectations of their populations. The alternative argument is that it is the job of the NHS to meet the expectations of local populations. The Healthcare Commission argues that “it is more appropriate to report the actual percentages of these variables rather than adjusting them for variation among trusts.”28 However, practices and primary care trusts in areas with a large ethnic minority population may think that their care is being unfairly represented if no allowance is made for differences in population expectations. This perspective is highlighted in a recent Department of Health report, which suggested that “many [ethnic minority] patients have a limited understanding of the services offered by the NHS . . . [and] can have unrealistic expectations about services.”8
Limitations of the study
Caution is required in generalising the study findings because the vast majority of surveys were completed on a voluntary basis by patients attending GP surgeries. Previous analyses have shown that respondents to the post-consultation version of the GPAQ are broadly similar (in terms of age, sex, and ethnicity) to general practice consulters identified in the 2004 General Household Survey of Great Britain,29 but the current study sample cannot be assumed to be representative of the wider primary care population in England. Furthermore, although the GPAQ is available in a number of languages, only data from surveys completed in English were included in this analysis.
Patterns of response found for ethnic minority groups in England cannot be assumed to be replicable in other countries. Recent examination of disparities in patients’ experiences of primary care in the United States, however, found evidence that the mechanisms underlying poorer evaluations vary between different ethnic minority groups,30 echoing the broad findings of the present study.
There were few constraints on how questionnaires were administered by general practices to fulfil their contractual requirements, thus no details of response rates are available and the data are subject to uncertain levels of response bias. The unstandardised method of administration of the GPAQ survey and lack of data about response rates are, therefore, significant limitations of this study, as it is possible that non-respondents would show different patterns of response. However, the main bias introduced by these problems will be on absolute levels of satisfaction, which were not the principal focus of this paper, and the effect of any bias is likely to be reduced when examining associations between items within the questionnaire. Although the data do not allow us to determine differences in response rate between ethnic groups, the comparison with data in the 2001 UK census and the 2004 General Household Survey of Great Britain suggests that ethnic minorities were slightly over-represented in our sample compared with the general population.
Ethnicity is a constantly evolving sociopolitical construct relating to issues of race, geographical or tribal origin, national identity, migratory status, culture, traditions, language, and religion. As such, it is difficult to define and measure. Criteria for classifying ethnicity vary between countries and are a subject of continuing debate.31 32 33 For brevity (to maximise completion rates) and to enable direct comparison with UK population data, the GPAQ uses a single six category self classification scheme for measuring ethnicity (table 2). This scheme is effectively a condensed version of the 16 category scheme included in the 2001 UK census.34 The relatively crude categorisation used in the GPAQ thus masks important sociocultural differences within categories that may influence the use and evaluation of health services. For example, respondents classifying themselves as “white” could be recent immigrants from eastern European countries who may have particular healthcare needs or may interpret survey questions differently from “white British” or “white Irish” patients. Similarly, the “Asian/Asian British” category fails to differentiate respondents of Indian, Pakistani, and Bangladeshi heritage.35 Although the present analysis includes some adjustment for the heterogeneity in health need and socioeconomic status that exists between ethnic groups, collection of a wider range of information pertaining to ethnicity (such as religion, country of birth, or primary language) would provide a more accurate picture of differences between and within groups. The new primary care patient survey is being sent by post to 5.6 million patients annually and is the largest survey of its kind ever undertaken in the UK.11 This new survey includes both the full 16 category 2001 UK census ethnic group classification and an additional item on religion, which should allow a more detailed examination of ethnic response patterns and potential response bias.
The validity of our analyses depends on the reliability and validity of the items and subscales in the GPAQ. Although there is some evidence of the validity of the GPAQ,36 independent data to verify patient reports are not available (for example, data on the actual availability of appointments in practices to compare with patients’ reports of appointment waiting times). However, other studies suggest that despite some inaccuracy in patient reports, levels of agreement between self reported experiences and external sources of information are reasonable.37 38 Nevertheless, it is important to emphasise that the variables relating to “quality of care” in the current study are based on patients’ perceptions and cannot be assumed to directly reflect actual quality of care provided.
Overall, our results suggest that for aspects of care such as access and waiting times it is important to focus on actual experiences of care. However, it is also important to understand that patients’ assessments of their care may be influenced by factors other than the actual care received.39 In our view, reporting both the absolute results of patient surveys and the results adjusted for important sociodemographic factors such as ethnicity would be valuable. This approach would have important implications for the reporting of data from the new primary care patient survey.11
What is already known on this topic
Patient evaluations are increasingly used as a way of measuring the quality of medical care
Patients from ethnic minorities evaluate their care more negatively than do white patients even after adjusting for potential confounding factors
The causes of these lower evaluations are poorly understood
What this study adds
This study examined differences in white and ethnic minority patients’ evaluations of four aspects of primary care, adjusting for factors that might account for observed differences including demographic factors, health need, and variation in reported standards of received care
Relatively negative evaluations by ethnic minority patients of waiting times to see general practitioners and of continuity of care are partly explained by reported poor experiences of these aspects of care, but may also reflect communication problems and different expectations
There is an argument for presenting the results of patients’ questionnaires both unadjusted and adjusted for ethnic differences when comparing practice scores
Cite this as: BMJ 2009;339:b3450
The authors thank Mark Hann and Pete Bower for advice on the statistical analyses and comments on earlier drafts, and the reviewers for their helpful comments concerning interpretation of the data.
Contributors: Both authors developed the research questions and planned the analyses. NM conducted the analyses, and both authors were responsible for writing the paper. NM had full access to all the data and takes responsibility for the integrity of the data and the accuracy of the analyses. NM is the guarantor.
Funding: This article presents independent research conducted by the National Primary Care Research and Development Centre and supported by core funding from the Department of Health. NM is funded by the National Institute for Health Research. The views expressed in this publication are those of the authors and not those of the NHS, the National Institute for Health Research, or the Department of Health.
Competing interests: The GPAQ is free to use for NHS staff, but commercial companies selling patient evaluation services were required to pay a licence fee at the time these data were collected. Funds derived from selling the GPAQ in these instances were received by the National Primary Care Research and Development Centre and used to fund support for the GPAQ and related research. No individual gains directly from use of the GPAQ.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.