Setting standards based on patients' views on access and continuity: secondary analysis of data from the general practice assessment surveyBMJ 2003; 326 doi: http://dx.doi.org/10.1136/bmj.326.7383.258 (Published 01 February 2003) Cite this as: BMJ 2003;326:258
- Peter Bower (), senior research fellowa,
- Martin Roland, directora,
- John Campbell, professorb,
- Nicola Mead, research fellowa
- a National Primary Care Research and Development Centre, University of Manchester, Manchester M13 9PL
- b Department of General Practice and Primary Care, Peninsula Medical School, Exeter EX2 5DW
- Correspondence to: P Bower
- Accepted 25 November 2002
Objectives: To examine patients' views on access and continuity in general practice to derive quality standards.
Design: Secondary analysis of data from general practice research studies and routine quality assessment activities undertaken by practices and primary care trusts.
Setting: General practice.
Participants: General practice patients.
Results: Satisfactory standards of access were next day appointments with general practitioners and a 6–10 minute wait for consultations to begin. A satisfactory level of continuity was seeing the same general practitioner “a lot of the time.” Standards varied with the analytic method used and by sociodemographic group.
Conclusions: Standards expected by patients in primary care can be derived from linked report-assessment pairs. Patients may have expectations of access that are in excess of government targets. Patients also have high expectations of continuity of care. It is unclear the degree to which such standards are reliable or valid, how conflicts between access and continuity should be resolved, or how these standards relate to other priorities of patients such as high quality interpersonal care.
What is already known on this topic
What is already known on this topic Standards are increasingly being set for the provision of health services
Surveys and consultation exercises before the NHS plan helped set the standard for a maximum waiting time of 48 hours for appointments to see general practitioners
The optimal methods by which patients should be involved in setting standards and the utility of such standards are unclear
What this study adds
What this study adds Satisfactory standards of access were next day appointments, a 6–10 minute wait for consultations to begin, and seeing the same general practitioner a lot of the time
Patients may have expectations for access to primary care in excess of current government targets
The UK government has emphasised the measurement of performance in the NHS, highlighting the need to meet standards for appropriateness, effectiveness, and efficiency of care.1 Standards are defined as “descriptions of specific aspects of healthcare practices to which are attached prescriptive values” and may legitimately include the views of patients.2
Patients have already influenced the setting of standards for access to primary care. The recent national survey of NHS patients (64.5% response rate from 100 000 patients, sampled from electoral registers) found that 34% of patients had got an appointment with a general practitioner of their choice on the same or next day, 29% had to wait two or three days, and 25% had to wait four or more days. Overall, 19% stated that the appointment should have been sooner.3 Waiting times were also an issue during consultation with patients about the NHS plan.4 This plan set a specific standard for waiting times by 2004 of seeing a primary care professional within 24 hours and a general practitioner within 48 hours.
This standard was not explicitly derived from the national survey, which only inquired about how much longer patients waited than they wanted to and whether they thought they should have been seen more quickly.3 An alternative approach to setting standards is provided by the general practice assessment survey, a self report questionnaire examining patients' views of several aspects of general practice.5 Some scales in the questionnaire use two types of items: report (the patient's experience of care) and assessment (the patient's evaluation of that experience). These report-assessment pairs relate to waiting times for appointments with a particular general practitioner, with any general practitioner, and for consultations to begin, and the proportion of consultations with the patient's usual general practitioner (continuity of care).
These questions provide a method for deriving standards for patients by examining the proportion of respondents who are satisfied or dissatisfied with particular levels of service. A second issue is whether such standards vary by sociodemographic group—for example, acceptability may relate to expectations or needs, which may vary with income, ethnicity, or health status.6
We aimed to examine relations between reports of access and continuity in general practice and assessments of acceptability to derive patient based standards and to examine differences in standards between patients from different sociodemographic groups.
The general practice assessment survey comprises multiple subscales, but we restricted our analysis to access and continuity items with report-assessment pairs.7 The relevant items are listed on bmj.com (a copy of the questionnaire is available at http://www.gpas.co.uk/).
The data (21 905 patients) derive from a survey of quality of care, a questionnaire validation study, an evaluation of pilots for personal medical services, and data analysed for primary care groups and trusts by the National Primary Care Research and Development Centre. 5 8 9 Some datasets used version 1 of the general practice assessment survey, whereas others used an updated version 2, but the relevant items are identical in both versions. Table 1 details the source of the data.
The general practice assessment survey is completed by patients attending surgeries or sent by post to those on the practice list. However, items refer to care in general, not to specific consultations.
We excluded patients from the main analysis when data were missing on any of the eight key report or assessment items. We examined sociodemographic correlates of missing data on these variables. In addition, report items on the availability of a particular doctor and of any doctor each have a “not applicable” response option, so we excluded patients using this option. We also excluded patients from the sociodemographic analyses if the relevant sociodemographic variable was missing.
We used cross tabulation to examine patterns of missing data and for the main analysis of relations between reports and assessments. For simplicity, we dichotomised assessments of satisfaction into dissatisfied (very poor, poor, or fair) and satisfied (good, very good, excellent).
To set standards, a minimum proportion of patients (for example, three quarters) might be prescribed who must be satisfied with a given aspect of the service. Such criteria are unambiguous but also arbitrary and are sensitive to relatively small differences in the proportion of satisfied patients (for example, between 74% and 76%). Alternatively, standards might be based on large discontinuities in the data. For example, if a large proportion of patients are satisfied at one level of service (with an appointment the next day, for example) and far fewer are satisfied with the next level (waiting two or three days), then this might suggest a degree of agreement among patients as to an acceptable level of service. Such an approach is less arbitrary and more sensitive to the actual distribution of data. However it is dependent on the existence of such discontinuities and on agreement as to what represents a large discontinuity.
We used both methods in our analyses. The first criterion was that three quarters of patients should report being satisfied, which we then relaxed to two thirds of patients. In addition, a large discontinuity was defined as an absolute percentage change of greater than 25% in the proportion of satisfied patients between different levels of service. Where there was more than one such discontinuity, we took the largest.
To examine effects of demographic characteristics, we stratified raw cross tabulations by age (16-30, 31-45, 46-59, ≥60), sex, ethnicity (white or other), chronic illness (yes or no), employment (employed full or part time or other), and accommodation (owner occupied or other). Although regression analysis using interaction terms would be the usual method of examining the moderating effects of variables, this would consider differences in regression slopes, whereas we focused on identifying discontinuities in the data or specific criteria related to the proportion of satisfied patients.10 Our analysis also ignored the fact that scores are clustered at practice level.8
Missing data in the four report-assessment pairs used in the main analysis were associated with all sociodemographic characteristics. The most consistent predictors of missing data were presence of a chronic illness, no full or part time work, and increasing age. The increase in missing data associated with these variables was in the order of 1-5%, with the largest difference being 8.9%.
Table 2 shows the raw cross tabulations of report and assessment items. We found at least one identifiable discontinuity in all analyses and more than one discontinuity in some. Table 3 shows the standards identified by the different methods.
Associations with sociodemographic variables produced relatively minor variations in the identified standards (the stratified cross tabulations are shown on bmj.com). Table 4 shows, for each method, which standards were moderated by sociodemographic variables and how the standard differed in each subgroup. Acceptable waiting time for consultations to begin was the issue most sensitive to sociodemographic factors. Age and ethnicity were the most consistent moderating factors, with patients from ethnic minorities generally having higher standards and patients in the two older age groups (46-59, ≥ 60 years) having lower standards.
Standards for primary care services can be derived from linked report-assessment pairs in the general practice assessment survey. Our methods suggest that patients may have expectations of access that are in excess of government targets and also have high expectations of continuity of care. Two key issues are raised. The first is the methodological adequacy of this approach to standard setting. The second, dependent on the first, concerns policy implications of the analysis.
Our analysis was suggested by the nature of the items in the general practice assessment survey, but the questionnaire was not designed explicitly as an instrument for setting standards. The processes by which patients make judgments in situations designed to elicit standards may differ from those used in completion of routine questionnaires. It is not clear, however, whether direct methods are necessarily more valid than the indirect method used here.
The results depend on the validity and reliability of the questionnaire. The questionnaire is reliable and has an interpretable factor structure. 5 11 However, patient reports of waiting times have not been validated against objective measures, and validation of subjective assessments of acceptability is problematic. Concerns about validity might arise because a small proportion of patients remained dissatisfied with high standards of care (for example, some were dissatisfied with no waiting time for consultations to begin).
The standards identified obviously depend on the particular definition of satisfaction and the thresholds applied (66%, 75%, or discontinuities). The binary definition of satisfaction is similar to published recommendations, but ratings of “fair” might be considered indicative of some degree of satisfaction. 12 13 If the data are reanalysed in such a way, different results occur (for example, both 66% and 75% of patients are satisfied with waiting two or three days for an appointment with a specific general practitioner, waiting 11–20 minutes for consultations to begin, and seeing the same general practitioner “some of the time”). In the identification of standards, the 66% and 75% criteria are obviously arbitrary. The presence of discontinuities in the data suggests that thresholds of acceptability do exist, but it should be noted that this method was suggested by preliminary analysis, and a discontinuity was not defined a priori.
However, the important issue may not be whether the methods are correct (as they will always be arbitrary to a degree) but whether they are intuitively reasonable and a useful reflection of patients' views.
The validity of the standards also depends on the representativeness of the sample. The data derive from several sources including practices using the general practice assessment survey for routine service evaluation, although 69% of the present sample derives from three research studies. 5 8 9 One of these studies had a response rate of only 38%, whereas another achieved a rate of 66%, similar to the national survey (although additional cases were lost because of missing data). 5 8 However, when our analysis was restricted to data from the study with the higher response rate, the results remained unchanged. The national survey was completed on a community sample, whereas the current data also include patients attending surgeries.
Interpretation of standards
Unfortunately the government's 48 hour standard can only be compared to the “two or three day” response option in the general practice assessment survey. It should, however, be noted that standards for access to any general practitioner varied between “next day” and “same day” (see table 3) and did not extend to “two or three days” in any analysis. The questionnaire does not include items relating to access to primary care professionals other than general practitioners.
Questions relating both to waiting times and to satisfaction are framed somewhat differently in the national survey and our data, complicating direct comparisons. If respondents to the national survey are considered dissatisfied when they responded that they should have been seen a bit sooner or a lot sooner, then 35% are dissatisfied with waiting one day longer than wanted, increasing to 57% dissatisfied with waiting two days longer than wanted, 66% with three days longer than wanted, 72% with 4–7 days longer than wanted, and 83% with eight or more days longer than wanted. If it is assumed that patients wanted an appointment the next day, then waiting one or two days longer would relate to the 63% dissatisfaction for two or three days in our study (see table 1). This compares with around 35% and 57% dissatisfaction for waiting one day or two days longer, respectively, among respondents in the national survey, which indicates even higher levels of dissatisfaction in our study.
The methods of standard setting used by us did not prove highly sensitive to sociodemographic factors (although the dichotomising of sociodemographic variables such as ethnicity may have concealed important variations). Also of interest are factors that were not associated with variation. For example, it might be expected that patients with chronic health problems would value higher standards of continuity of care, but this was not evident, although this may reflect the relatively high levels of satisfaction expressed with continuity. Patients may value continuity over convenience of access in relation to certain problems such as mental health, as opposed to more minor health problems or where immediacy is perceived to be paramount. 13 14 However, we could not compare the relative importance of the different standards and it is unlikely that standards (or health systems) can ever be sensitive to the full range of patient characteristics and preferences.
Finally, the high standards relating to access and continuity derived from our analysis may reflect that they have not been explicitly compared with other aspects of primary care for their overall priority.15 Quality of care in primary care is a combination of access to care and effectiveness of the care provided, and surveys of primary care patients in Europe suggest that interpersonal aspects may be more important than access issues such as waiting times for consultations (ranked 34th of 38 aspects of primary care in the United Kingdom). 16 17 It should, however, be noted that a quick service for emergencies was ranked first in the same survey (our study did not include a measure of perceived urgency), and rapid access to appointments has also been ranked highly in previous studies. 17 18 In addition to the priorities of patients, there is also a possible tension between the priorities of different bodies such as patients, professionals, and government. 18 19
Expectations potentially conflict, such that there is a tension between highly accessible care and continuity. Economists would generally focus on explicit choice based methods in determining patient priorities, and there is evidence from such methods that interpersonal issues are more highly valued than either waiting times for consultation or consultation length.20–22 However, interpersonal attributes such as being able to talk to the doctor are far more complex concepts than waiting times. Although patient based assessments of interpersonal care have some validity, they also have conceptual and practical limitations and do not lend themselves to setting clear, demonstrable standards of care compared with more concrete issues such as waiting times.23–25
Report-assessment pairs in the general practice assessment survey provide a method for examining patient views of general practice that may be of use in setting standards and monitoring performance. Patients have high expectations relating to access to care, which may support current government targets and the development of alternative models of service delivery such as walk-in centres.13 The interpretation of such standards must be sensitive to the methods used to derive them, however, and to issues of priority in other aspects of general practice, such as the effectiveness of interpersonal care. Although access to services is an important issue in itself, some definitions of access include notions of effectiveness, and thus evidence concerning the clinical and cost benefits of rapid access is also of importance in the wider debate.26
We thank Sophie Jerrim for assistance with the database of questionnaires and all those who provided data.
Contributors: MR and NM had the idea for the study and JC provided significant data for the study. PB analysed the data and wrote the paper; he will act as guarantor for the paper. All authors commented on drafts.
Funding This work was conducted as part of the programme of the National Primary Care Research and Development Centre, supported by the Department of Health. The views expressed are those of the authors and are not intended to represent the views of National Primary Care Research and Development Centre or its funders.
Competing interests None declared.
Additional tables appear on bmj.com