General Practice

Creating a shared vision of out of hours care: using rapid appraisal methods to create an interagency, community oriented, approach to service development

BMJ 1996; 312 doi: (Published 11 May 1996) Cite this as: BMJ 1996;312:1206
  1. Jeremy Dale, senior lecturer in primary carea,
  2. Cathy Shipman, research fellowb,
  3. Lynne Lacock, development managerb,
  4. Mark Davies, clinical lecturerb
  1. a Departments of Accident and Emergency Medicine and General Practice, King's College Hospital, London SE5 9PJ
  2. b Department of General Practice and Primary Care, King's College School of Medicine and Dentistry, London SE5 9PJ
  • Accepted 22 March 1996


Objectives: To undertake a district wide review of out of hours primary health care services and identify the views of users and providers about current arrangements and options for development.

Design: A range of qualitative and quantitative survey methods based on rapid appraisal methods, modified to apply to an inner city district.

Setting: Socially deprived, multiethnic district in south east London with a population of over 700 000.

Main outcome measures: Strengths and weaknesses of current out of hours services and suggestions for developments.

Results: Widespread dissatisfaction with current arrangements was identified, with specific problems relating to access, availability, demand for services, and interagency communication. Several areas for development were identified, including the establishment of an out of hours cooperative, multiagency primary care emergency centres, and telephone advice-triage. Many of these are now being planned or piloted.

Conclusions: Rapid appraisal provided a helpful method, enabling partnerships to be established between local agencies and users in relation to service development. The shared understanding and commitment to improving services that resulted is now having a major impact on out of hours care in the district.

Key messages

  • Considerable potential exists for improving the integration and communication between existing service providers

  • Implementing new arrangements is likely to involve considerable change.

  • Rapid appraisal provides a methodology for eliciting and interpreting the views of a broad range of local stakeholders and involving them in decision making about new services

  • Many suggestions for service developments were suggested in this study, and many are now being piloted or implemented


Out of hours primary care is one of the most poorly integrated aspects of the health service, with much overlap between services provided by different agencies. While much has been written about out of hours services provided by general practitioners, the important part played by other agencies and the opportunities that exist for developing more seamless provision have been little acknowledged.

The demand for out of hours care from general practitioners has risen steadily over many years, but general practitioners perceive 5-59% of the calls they receive as being “inappropriate.”1 2 This has contributed to high levels of dissatisfaction.3 Changes now introduced to general practitioners' contractual arrangements are intended to encourage more flexible systems of service delivery.4 New models of care, such as out of hours cooperatives and emergency centres, are being implemented in many areas.5

Establishing new out of hours services involves substantial change in local health services and provides a unique opportunity for planning more integrated care. Detailed needs assessment and evaluation of service initiatives should guide developments to ensure sensitivity and responsiveness to local needs.6

We describe here a needs assessment exercise for out of hours services in a diverse, multiethnic, socially deprived inner city district of over 700 000 people in south east London (box 1). This was the first phase of a project commissioned as part of the district's programme for the development of primary care in London. Rapid appraisal methods7 8 9 were adapted to enable a district wide approach to be taken to out of hours service development.

Box 1—Objectives of the needs assessment phase of the out of hours project

  • To identify the range and extent of health care and social services provided during out of hours periods

  • To identify the views of users, provider organisations, general practitioners, community nurses, and health care professionals about current arrangements and future service options

  • To inform local policy making and the commissioning of services

  • To facilitate and inform interagency collaboration


A research based approach was adopted underpinned by the “Health for All 2000” philosophy to emphasise the importance attached to the local community participating actively in making decisions about priority setting and resource allocation.10 In line with Annett and Rifkin's concept of the information pyramid,11 the first stage of the project involved identifying the composition and organisation of the community. The project team compiled a list of all local health and social service providers and community groups. Agenda setting meetings and preliminary discussions were held with key agencies and community representatives. Each organisation was asked to nominate someone who would act as a link person with the project.

An outline of the project (including background information, aims and objectives, proposed methods, draft time plan, and expected outcomes) was sent to all key local stakeholders. Subcommunities of users and potential users of out of hours services were identified—for example, according to geographical, cultural, and common needs criteria. Organisational structures, links, and contexts which affect the capacity of both providers and users to respond to developments were identified.

Given the need to complete the work in less than six months, we chose methods that could be applied swiftly and concurrently across the entire district (box 2). A range of methods (interviews, postal questionnaires, focus groups, and community development) was used to minimise the risk of unrepresentative sampling, particularly for minority groups—for example, homeless people and ethnic and refugee groups.

Box 2—Data collection methods

  • Questionnaire sent to all 432 general practitioners in Lambeth, Southwark, and Lewisham; 307 (71%) replied

  • Individual visits to practices and interviews with general practitioners

  • Interviews with service managers and discussions with individuals and groups of health professionals

  • Questionnaires sent to all relevant health and social service providers; responses received from 13 out of 14 agencies

  • Questionnaire survey of all known community nurses (308) who work during out of hours periods; 166 (54%) replied

  • Presentations and discussions at the four general practitioner forums and the local medical committee

  • Questionnaires distributed to all (1800) known local community and voluntary groups (although health commission managers consider that no more than 500 groups are probably currently operational); responses were received from 93, giving a response rate of 5-19%

  • Questionnaire circulated in a newsletter to all carers registered with Lambeth, Southwark, and Lewisham carers associations; 133 responses (sample size uncertain)

  • Community development work, particularly with local black and ethnic groups

  • Consultation with the three local community health councils

  • Distribution to all local general practitioners of a quarterly newsletter on the work of the project, The Out of Hours Bulletin

  • Presentations to local conferences (such as the primary care strategy conference) and local meetings (such as a carers' forum, pensioners' group, and tenants' associations)

  • Articles published in local health service newsletters describing the aims of the project.

Questionnaires sent to general practitioners, community nurses, and other health and social service providers examined how out of hours care was currently being organised and elicited views on the strengths and weaknesses of current provision, ideas for future developments, and issues relating to interagency working. Questionnaires to user and community representatives asked about the strengths and weaknesses of current services and views about future provision. Both sets of questionnaires included a mix of open and closed questions. The aim was to identify and explore the range of issues relating to out of hours services and to produce insights and encourage innovative suggestions, rather than fixed recommendations.

Qualitative data were also gained through exploratory interviewing, meetings, focus groups, and community development work.12 Data analysis, including specification of conceptual links, was undertaken by several researchers concurrently. Triangulation of methods was used to validate findings, but this approach was not allowed to exclude variations in interpretation that arose from different contexts.13

A one day conference was held at the end of this process to share with all interested parties an overview of current provision and to move towards consensus about piloting of new initiatives. Over 170 local people participated, representing the full range of health and social service providers, as well as many voluntary and community groups. Workshops were held focusing on the interface between general medical services and other agencies' services, with speakers invited to contribute relevant agency and user perspectives. Following this a report was completed presenting the findings together with proposed implementation plans and timetables.14 It was disseminated to all agencies that had participated.



A wide range of health and social services were identified that are being directly accessed by the local population between the hours of 6 pm and 9 am on weekdays, at weekends, and over bank holidays (box 3). Non-statutory community and voluntary organisations were also identified as providing many services at these times, representing the interests of several segments of the local population. Examples included advocacy support for people with learning difficulties, a support group and befriending scheme for African and Caribbean families and children, telephone advice and help from churches, mental health users befriending schemes, and help from tenants' and residents' associations for elderly people living alone.

Box 3—Examples of out of hours health and social services available in Lambeth, Southwark, and Lewisham

  • General practice

  • Answering services

  • Practice rotas

  • Deputising services

  • Emergency 999 services

  • Acute hospitals

  • Accident and emergency departments—adult, paediatric

  • Pathology departments—tests and clinical advice

  • On call teams— inpatient direct access specialist care

  • Medical oncology—palliative care

  • Dental emergency service

  • Poisons unit—information

  • Community midwifery

  • Community nursing

  • District nurses

  • Specialist nursing and counselling—HIV and AIDS

  • Paediatric home care team

  • Hospital at home schemes

  • Evening/Saturday morning clinics—family planning, youth advisory, child health, antenatal

  • On call palliative care service

  • Social services

  • Crisis intervention, risk assessment, and protection relating to:

    • Mental health sections

    • Elderly people at risk

    • Teenage runaways/offenders

    • Domestic problems

    • Disabilities/elderly

    • Placement breakdowns

    • Child protection

  • General advice and counselling


Overall there was widespread dissatisfaction with existing out of hours arrangements (box 4). Many general practitioners described out of hours work as being the most important stress in their professional lives. Views were often expressed in emotive terms, with several stating that unless a solution could be found they were unlikely to remain in general practice (box 5).

Box 4—Main problems identified with current out of hours service provision

  • Variation and extent of local needs

  • Appropriateness, in terms of provision and use made of out of hours services

  • Access and availability of services

  • Communication between services and information needs of both service providers and users

  • Professional isolation and personal safety

  • Quality and accountability of services

  • Training and education needs in relation to out of hours working

Box 5—Comments made by general practitioners who responded to the questionnaire survey (n=306)

  • “With a young family…I cannot cope with working all day and having my nights disturbed as well. I am a conscientious doctor and know I am fit for nothing if I have been up several times in the night”

  • “GPs should be discouraged from doing their own on-call—for their health, their family's health, and their patients' health”

  • “I recently audited my out of hours visits using a criterion of whether the presence of a GP had improved the outcome; only 5% fell into this category”

  • “A large number of GPs, especially women, are unwilling to do night calls in the inner city areas”

  • “The current furore over out of hours work is a particular focus of a general discontent over the pressure of work in general practice”

Accident and emergency department staff and general practitioners, in particular, felt that the demand for non-emergency out of hours services was increasing, reflecting rising public expectations. This was generally recognised by both providers and users as reflecting specific characteristics of the local population (including high levels of social deprivation, fragmented social networks, homelessness, unemployment, psychiatric illness, alcohol and drug related disease, lone parent families, teenage pregnancy, and large ethnic groups, including many recently arrived refugees).

Issues were raised about the availability of out of hours services. Services offering the greatest access to the public (accident and emergency departments, ambulance services, and general practitioner services) tended to perceive that they were under-resourced to cope with rising demands. Underprovision of certain out of hours services—for example, community pharmacy, emergency dental care, and social services— was described as leading to inappropriate demands being made on more available services.

Differences and inequalities in overall service provision between different parts of the district were identified as a key issue by both providers and users. Many respondents thought that there was poor communication between services and a lack of understanding of roles. General practitioners, hospital and community trusts, and social service departments in particular thought that this contributed to ineffective service provision and either a lack of referral or inappropriate referral between services.

Underuse of some services was also raised as an issue—for example, by district nurses, who cited the reluctance of some elderly patients to call on general practitioner services. This sometimes resulted in delays in receiving care and in consequence avoidable hospital admissions. People expressed concern that the government's campaign designed to limit out of hours calls to general practitioners might influence the wrong audience, so deterring patients who are in need of care. One over 60s group commented that most of their members “think that their GPs will not be happy if they are called out of hours, so .… they would rather call for an ambulance.”

Fifty seven respondents to the community and voluntary groups' questionnaire stated that they had experienced some difficulty in accessing help out of hours, and just over half had experienced particular difficulty in accessing help from general practitioners. Difficulties encountered related to a range of specific needs (box 6), particularly for those with language differences; 26% of the population of Lambeth, Southwark, and Lewisham are from minority ethnic groups (1991 census). Similar problems with access may occur for those with physical and learning disabilities.

Box 6—Specific difficulties with access encountered by users of out of hours primary care services

  • Limited knowledge about the availability of services and how to access them

  • Specific problems for those with physical, learning, and mental disabilities and difficulties, or literacy, language, and cultural differences

  • Not being registered with a general practitioner

  • Problems accessing services during normal working hours because of employment patterns

  • Variation in the availability of some services in different parts of the district

Some community and voluntary groups were concerned that there appeared to be a general lack of understanding of the needs of their members in relation to out of hours care—for example, those with learning disabilities, the homeless, carers, drug and alcohol misusers, and those with HIV infection or AIDS. Carers identified the considerable financial and personal costs they bear, particularly out of hours, when other support services may be lacking.

The desire for readily available telephone advice was repeated by many community and voluntary groups to help patients and carers decide how serious symptoms might be and whether urgent attention was needed. The ambulance service reported that the public often uses the 999 emergency service for telephone advice which might be more appropriately dealt with by a non-emergency service. Many general practitioners were in favour of expanding the provision of telephone advice to patients; 43% of those responding to the questionnaire supported the development of a telephone service staffed by nurses for providing triage and advice about self care out of hours.

Information on services available out of hours, particularly from pharmacists and dentists, was requested by a number of community groups to enable their members to access help more easily. A common complaint was that members often did not know whom to contact, especially late at night.

Several practical difficulties encountered by health care professionals during out of hours periods were identified. General practitioners and district nurses drew attention to street crime, poor signposting, poor naming and numbering of streets and estates, and poor lighting. Local maps, including the layout and numbering of estates, were identified as a need by some respondents. Personal safety was of considerable concern, and some called for security support when visiting patients at night.


This study has identified a broad range of issues relating to out of hours services and emergency primary care. Because of limited time and resources we adopted a pragmatic approach to gathering and analysing information, and the issues that emerged showed considerable overlap between the concerns of service providers and users. These related to access, availability, appropriateness, and integration of services.

Not surprisingly, the study also identified conflicting views and wants between different agencies and between providers and users of services—for example, between users wanting more easily accessible services and general practitioners wanting to control demand and move the locus of consultation away from the patient's home.

All key stakeholders seemed strongly committed to supporting collaborative working, although the form that this would take was sometimes unclear. The priority placed on out of hours service developments varies between agencies, and the differences identified in organisational structures and arrangements are likely to complicate the implementation of development plans. In addition, limited resources were cited by virtually every agency as the main constraint on developing services. The local health commission will have a key role in prioritising developments, especially those that involve more than one sector or have resource implications.

Many suggestions emerged for service developments, interagency collaboration and liaison, improving relationships with community representatives and users, and developing commissioning and service specifications.14 The project team is now helping to plan and pilot initiatives in response to these proposals (box 7). In particular, work is being undertaken to coordinate service provision, including developing a multiagency point of access to out of hours services with telephone triage and advice giving. This should enable assessment of patients' needs and their direction to the most appropriate agency and also ease cross referral between agencies. The team has also facilitated the establishment of an out of hours cooperative between more than 200 local general practitioners.

Box 7—Developments of out of hours service being worked on by Lambeth, Southwark, and Lewisham out of hours project

  • Improving existing services—for example, developing performance standards and referral proto-cols; improving out of hours information; serviceexpansion; and increasing use and standardisationof client held records

  • New out of hours arrangements for generalpractitioners; setting up a district wide generalpractitioner cooperative; and establishing primarycare emergency centres

  • Establishing a central telephone answering-advice service as a single point of access tostatutory services

  • Establishing multiagency primary care emergency centres at which on call health and socialservice professionals will be based

  • Improving interagency collaboration—forexample, establishing protocols for referral andsupport with clear identification of roles betweendifferent groups like general practitioners, districtnurses, accident and emergency departments

  • Improving collaboration and liaison betweennon-statutory and statutory out of hours services

An important finding was the lack of common measures of out of hours activity, which made comparison between services difficult. The project is currently helping to develop jointly agreed criteria that can be used for setting performance standards and audit.

Methodological considerations

The use of rapid appraisal methods proved effective as a means of broadly delineating issues and of understanding service use and the implications of potential service developments. It offered a rapid, reliable, and collaborative method of accessing community perspectives without having to resort to large, more costly surveys. Our study aimed to identify themes and issues, rather than to produce generalisable results. This was of particular importance to ensure that needs were not overlooked within the complex, heterogeneous district in which the study was conducted. The network of key stakeholders that was established provides a district wide infrastructure within which consultation, negotiation, and planning of new developments can occur in a coordinated manner.

More quantitative data collection methods were used for parts of the needs assessment relating to specific groups when it was important to gain an idea of the representativeness of elicited views. While it was beyond our resources directly to investigate users' perspectives, we did draw on some related work recently undertaken within the district.15 We are also undertaking an in depth study of users of out of hours general medical services through patient telephone interviews. This will enable us to test further the issues identified in the rapid appraisal and will provide data against which the impact of future service developments can be measured.

Developing a dialogue with local people required a flexible way of working. Considerable problems were encountered in attempting accurately to articulate the varied perspectives of users given the limited time and resources available. Given the heterogeneity of the local population obtaining representative views is inevitably difficult.

The process of working with local groups shaped the methods that could be used. For example, carers were concerned about confidentiality and so would not let the project team directly distribute questionnaires to their members. As a result the numbers of individuals who received a questionnaire was unclear, reminders could not be sent, and response rates could not be calculated.

Similarly, it was unclear how many community and voluntary groups were active in the district at the time of the study, and the 93 responses obtained reflected a probable response rate of around 19% (box 2). Although limited in quantitative terms, this size of response rate is common in community consultation, and the responses were none the less valuable in raising issues. The low response rate appeared to reflect variations in perceptions about the relevance of the subject to organisations' members, difficulty for groups to construct a representative response in the time available, cynicism about user perspectives influencing planning, and the rapid turnover of group members.

The strength of rapid appraisal as a method is in its flexibility and its propensity to facilitate change.9 16 Nevertheless, enabling participation risks raising expectations about health services, both about the services that can be delivered and about the role that users can play in decision making. During the planning and implementation of new services information will need to be shared with the public about the rationale behind service developments and the processes and constraints affecting their implementation. This in turn will require user access to all levels of decision making.


This project has shown within one inner city district how rapid appraisal can be used to develop a broad perspective on out of hours primary care arrangements and so facilitate the creation of a shared vision for service developments. Not only does this provide a baseline against which potential service developments can be considered, but the partnerships established should increase sensitivity to stakeholders' concerns and encourage shared ownership of initiatives that follow. This is likely to be of particular importance given the opportunities that currently exist for innovative service developments.4 Over the coming months the impact of this exercise on service development, interagency collaboration and liaison, and improved relationships with community representatives and users will become clear.

We thank the many representatives of providers and users of services who contributed much time, insights, and ideas on which the development plans in this study are based.


  • Funding Lambeth, Southwark and Lewisham Health Commission Tomlinson development programme.

  • Conflict of interest None.


  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
View Abstract