Introduction

Top Abstract Introduction Participants and methods Results Discussion References

The UK government, as part of a bid to modernise health services and to make them more convenient to use, has introduced NHS walk-in centres and commissioned their evaluation by an independent team. Our study forms one component of that evaluation. We aimed to determine whether walk-in centres, staffed mainly by nurses, provide adequate and safe clinical care to a range of patients, and how quality of care in these centres compares with that provided in general practice and by NHS Direct (box 1).

	Participants and methods

Top Abstract Introduction Participants and methods Results Discussion References

Recruitment of clinical sites
We approached sites in three geographical areas, in and around Bristol, Birmingham, and London. We invited general practices involved in research or teaching to participate, and we approached walk-in centres and NHS Direct sites on the basis of their geography. Twenty out of 25 (80%) walk-in centres, 11 out of 12 (92%) NHS Direct sites, and 24 out of 62 (39%) practices agreed to participate: we included the first 20 practices to respond.

Selection of clinical scenarios
We chose five clinical scenarios, largely to represent problems likely to be presented by patients to walk-in centres (box 2). The scenario on postcoital contraception was intended to assess management of a common, straightforward problem, the scenario on chest pain to assess ability to exclude a potentially serious diagnosis and reassure accordingly, and the scenario on sinusitis to assess issues around antibiotic prescribing. The scenario on headache was devised to assess ability to explore psychosocial issues, and the scenario on asthma to assess the history taking on drugs and awareness of the side effects of the drugs.

Derivation of assessment criteria
We assessed clinical care against prospectively determined standards. We constructed lists of essential criteria for the adequate management of each scenario by a stepwise procedure, based on the Delphi process.⁴

Standardised patient consultations
Standardised patients are people trained to portray a clinical scenario for teaching or research purposes. ^{5 6} Five role players, each portraying one scenario, worked in each locality. Each role player visited a particular walk-in centre or general practice once, but owing to the smaller number of sites for NHS Direct, contacted one NHS Direct site up to three times. Overall, 305 contacts were planned, 100 in walk-in centres, 100 in practices, and 105 with NHS Direct. Consultations took place from July to September 2001. The accuracy of portrayal of standardised patients, the reliability of assessment by standardised patients, and the validity of standardised patients is given on bmj.com

Analysis
For each consultation we calculated a score representing the proportion of essential criteria fulfilled for all items and separately for the three subgroups of items (history taking, examination, and diagnosis, advice, and treatment). We calculated the means of the all item and subgroup scores for each of the three primary care settings, with 95% confidence intervals calculated with design weighted survey estimators. We then undertook estimation of differences between mean scores for the three settings by using multivariable regression models (see bmj.com).

	Results

Top Abstract Introduction Participants and methods Results Discussion References

Data were collected on 297 of the planned 305 consultations, 99 in each setting. Most (91%) consultations in walk-in centres were exclusively with a nurse, whereas most (96%) in general practice were with a doctor. A minority (3%) of consultations with NHS Direct involved a call handler rather than a nurse.

Quality of care

Walk-in centres versus general practice
Considering all scenarios together, walk-in centres achieved a significantly greater mean score for all essential items than did general practices, but there were between scenario differences. Quality of care for postcoital contraception and asthma was significantly better in walk-in centres than in general practices, that for sinusitis and headache was similar in the two settings, and that for chest pain was better in general practice, although not significantly so (table 1).

Walk-in centres versus NHS Direct
Considering all scenarios together, walk-in centres achieved a significantly greater mean score for all items than did NHS Direct, again with between scenario differences. Quality of care for postcoital contraception and asthma was significantly better in walk-in centres, with no significant difference for other scenarios (table 1). Overall, and for postcoital contraception and asthma, walk-in centres scored significantly better on history taking, with the same pattern for diagnosis, advice, and treatment (table 2).

Referral
About a quarter (26%) of consultations in walk-in centres and four fifths (82%) with NHS Direct resulted in referral. Patients were advised to go to an emergency department in 5% of consultations in walk-in centres and 13% with NHS Direct, but not from consultations in general practice.

Between site variability
Considering all scenarios, the variability of quality scores, expressed as standard deviation, was highest for NHS Direct and lowest for general practice. However, variability of history taking for all scenarios was higher in general practice. Variability of quality in the three settings varied across scenarios. NHS Direct's high variability was largely the result of the postcoital contraception scenario, where referral may have substituted for consistent performance on essential items.

Access
Contacting NHS Direct was sometimes time consuming and unsatisfactory. About one quarter (25 of 99) of completed calls involved "call back" (mean wait 33 minutes). Three consultations with NHS Direct were not completed owing to length of wait for "call back." Fewer problems were reported accessing general practice and walk-in centres.

	Discussion

Top Abstract Introduction Participants and methods Results Discussion References

The accuracy of portrayal (90%) by standardised patients compared well with previous studies, as did their reliability (=0.7 to 0.9). ^{7 8} Detection (1.7%) and positive predictive value (21.7%) also compared favourably.^8-11

Although checklists were meant to consist of essential items, the proportions of items achieved in all three settings were low, a finding consistent with another study that assessed general practitioners against peer determined standards.¹² Performance may not reflect competence, as clinicians exhibit efficiency by only carrying out what is necessary at a particular moment.¹²

Methodological issues
The study's main limitations are non-random sampling of participating sites, use of a limited number of scenariossome more discriminating than othersand use of novel assessment checklists. Participating sites, particularly general practices, were likely to be more interested in the research question and may have provided a higher quality of care, possibly attenuating the study findings. Scenarios were chosen as typical of those seen in walk-in centres and because they were appropriate for portrayal by standardised patients. Scenarios necessitating the presence of abnormal findings or potentially involving certain types of physical examination or referral to third parties could not be included. The finding that walk-in centres offer safe care cannot necessarily be extrapolated to all clinical conditions.

Scenarios did not involve children and elderly patients, who attend general practice most often. Also, because the methods required standardised patients to visit practices as temporary residents, it did not assess some of the supposed strengths of general practicefor example, continuity and availability of past medical records. Thus, the findings do not suggest care in general practice is inferior to walk-in centres, rather that walk-in centres perform adequately and safely compared with controls.

The development of assessment checklists used by standardised patients in previous studies has received little attention, although their construction is crucial to the reliability and validity of assessment.¹³ Checklists used in this study seem to have high face validity and content validity, although few data were collected on other properties, such as reproducibility.

Conclusion
Standardised patients have a useful role in the assessment of quality in the UK primary care setting. Also, walk-in centres provide adequate, safe clinical care to a range of patients, compared with general practice and NHS Direct. However, the cost effectiveness of walk-in centres and their impact on workload of other healthcare providers requires further assessment.