Background details of the two informaticist service projects

Summary of questions asked of two clinical informaticist services
 

Box A: The RiPPLE group’s approach to evaluation The two informaticist service projects described in this paper were part of a larger initiative aimed at promoting the use of research evidence in primary care, which was established by the North Thames Region R & D Implementation Group. An independent evaluation of the overall initiative was commissioned to run concurrently with the individual projects. The aims of the evaluation, which became known as RiPPLE (Research into Practice in Primary Care London Evaluation), were to document each project’s progress and the organisational context within which decisions were made, to identify common themes in the activities and experiences of all the projects and draw lessons about the process of implementing evidence in primary care, and to make policy recommendations at regional level. To this end, the RiPPLE group (JH, TG, SR, CH) initially set out to establish a partnership with the project groups, offering their evaluator (JH) as an independent resource to assist with planning, communication, documentation, and evaluation. Emphasis was placed on assisting project teams to define their needs and design appropriate evaluation strategies, supporting and encouraging appropriate data collection, and identifying opportunities for collaborative work across the projects. The evaluator’s close involvement with the projects allowed the RiPPLE group to gain a thorough understanding of each one’s aims, circumstances, and progress, thus facilitating a case study approach. The main methods of data collection used to evaluate the informaticist service projects were · Attendance at project steering group meetings (or equivalent) · Attendance at other key meetings held by the project groups · Building relationships with both project workers and steering group members and assisting with data collection for the projects’ internal evaluations · Face to face or telephone interviews with participating clinicians who had used the informaticist service · End of project interviews with key staff. JH undertook the field work, sometimes accompanied by other members of the RiPPLE group, and made contemporaneous field notes of all meetings and contacts. Evaluation interviews were guided by a schedule developed collaboratively by the RiPPLE group but also allowed respondents to discuss issues they considered important. These interviews were audiotaped and subsequently transcribed. The RiPPLE group met at two-monthly intervals, at which the evaluator (JH) presented her accumulated raw data along with an interim interpretation, which was then considered and discussed by the group. Interim reports were also fed back periodically to the project groups. The evaluator encouraged both project groups to take an interest in what the other was doing, exchange information informally, and explore possibilities for collaboration. Differences and similarities in approach were frequently discussed, which helped the project groups develop and articulate their distinctive approaches to providing a clinical informaticist service. Eighteen months into the projects the RiPPLE group facilitated a meeting between the two project groups, with the aim of sharing their experiences and exploring further the emerging key similarities and contrasts in their approaches. This prompted reflection by all parties, and the project groups subsequently provided written responses to the preliminary case study reports offered by RiPPLE. These were incorporated into the final analysis, and the paper presented here is the joint work of the RiPPLE group and representatives of both project groups.

 
 

Box B: Background details of the two informaticist service projects Imperial College project Origin of project bid · Academic department of general practice Aim as stated in original proposal · To assess the feasibility and impact of using a vocationally trained general practitioner further trained as a clinical informaticist to promote evidence based practice in a defined group of general practices Main objectives as stated in original proposal · To establish guidelines for using the clinical informaticist · To produce a costed package to be used by others · To explore the knowledge and behaviour gap · To develop a database of evidence based answers to clinical queries · To train a cadre of primary care clinicians to pose questions Staffing · Part time general practitioner informaticist plus part time researcher and administrator. Both based in academic department of general practice. Training and supervision of informaticist · Intensive, one-to-one training by academic librarian in search strategies. Short courses on critical appraisal. Supervised by senior lecturer in general practice within highly rated academic department. Key features of project · Two stage project with pilot phase in two "innovator" health centres followed by definitive study with volunteers · Strong research element from outset, to include several questionnaires completed by participants and stakeholders · Relatively fixed project plan, which did not adapt well to changing circumstances (such as loss of "product champion") · Few local links Basildon project Origin of project bid · Joint bid from individual general practitioner, health authority, and community trust Aim as stated in original proposal · To explore the effect of employing a clinical informaticist to work in primary care Main objectives as stated in original proposal · To stimulate questioning behaviour in primary care clinicians · To research the questions posed · To provide information in a concise and understandable format · To help clinicians implement changes in practice Staffing · Initially, one part time general practitioner informaticist based in a local practice and provided with administrative support. Subsequently, four local general practitioners and a nurse working two sessions each in different localities Training and supervision of informaticist · Short courses on searching, critical appraisal, basic information technology, and presentation skills. After six months, he was described as having attained "a reasonable level." Supervised by project leader (local general practitioner) Key features of project · Planned to use a developmental approach working with enthusiasts and using these to get others on board · Specific aim to engage nurses as well as doctors · No formal academic links · Strong local links and knowledge via project leader, plus active steering group, allowed project to respond to changing environmental conditions and take advantage of opportunities as they emerged

 

Summary of questions asked of two clinical informaticist services* (values are numbers of questions unless stated otherwise)
 

	Clinical informaticist service
	Basildon project	Imperial College project
No of practices or clinicians approached	50 practices	100 clinicians
No of individuals who asked questions:	58	22
General practitioners	37	20
Nurses (practice, district, health visitor)	17	2
Other (manger, pharmacist)	2	0
Primary Care Group	2	0
Total No of questions submitted	119	60
Median time to provide answer	NR	9 days
Question topics:
Interventions (efficacy of non-drug treatments)	17	21
Interventions (efficacy or cost effectiveness of drugs)	28
Interventions (other drug issues B e.g. safety)	14	19
Clinical diagnosis	15	14
Diagnostic tests	9
Screening	3
Prevention	14	0
Aetiology of disease	6	6
Management or organisation of care	5	0
Risk assessment or risk management	5	0
Epidemiology (such as prevalence)	3	0
Other	7	0
Level of certainty of answers:
High quality evidence; confident and precise advice	25	NR
Moderate quality evidence with or without expert opinion	57	NR
Poor quality evidence; advice mainly on expert opinion	23	NR
Poor quality or absent evidence; unable to offer advice because expert opinion divided	10	NR
Sources used by informaticist to answer question:
Secondary databases of evidence	NR	12
Secondary databases plus primary literature	NR	10
Primary literature only	NR	24
No good literature available	NR	7
Other	NR	2
What would questioner have done if service did not exist?:
Taken no action	NR	17
Consulted a colleague	NR	17
Consulted a book	NR	5
Referred patient	NR	1
Other	NR	12
Extent to which questioners perceived their practice changed as a result of answer received:
Changed in relation to this patient	NR	15
Changed in relation to other patients with same problem	NR	29

NR=Not recorded.

*For further details see the project reports: Martin et al¹⁰ (Basildon project), Swinglehurst et al¹¹ (Imperial College project).