Research

Effectiveness of educational interventions in improving detection and management of dementia in primary care: cluster randomised controlled study

BMJ 2006; 332 doi: http://dx.doi.org/10.1136/bmj.332.7543.692 (Published 23 March 2006) Cite this as: BMJ 2006;332:692
  1. Murna Downs, professor in dementia studies (m.downs{at}bradford.ac.uk)1,
  2. Stephen Turner, senior researcher2,
  3. Michelle Bryans, D Clin Psych student3,
  4. Jane Wilcock, research fellow4,
  5. John Keady, professor of admiral nursing5,
  6. Enid Levin, practice development manager6,
  7. Ronan O'Carroll, professor of psychology7,
  8. Kate Howie, statistical consultant and teaching fellow8,
  9. Steve Iliffe, reader in general practice4
  1. 1 Bradford Dementia Group, Division of Dementia Studies, School of Health Studies, University of Bradford, Bradford BD5 0BB
  2. 2 Dental Health Services Research Unit, University of Dundee, Dundee
  3. 3 Department of Psychiatry, Royal Edinburgh Hospital, Edinburgh
  4. 4 Department of Primary Care and Population Sciences, Royal Free and University College Medical School, London
  5. 5 School of Health Education and Community Studies, Northumbria University, Benton, Newcastle upon Tyne
  6. 6 Social Care Institute for Excellence, London
  7. 7 Department of Psychology, University of Stirling, Stirling
  8. 8 Department of Computing, Science, and Mathematics, University of Stirling
  1. Correspondence to: M Downs
  • Accepted 23 February 2006

Abstract

Objective To test the effectiveness of educational interventions in improving detection rates and management of dementia in primary care.

Design Unblinded, cluster randomised, before and after controlled study.

Setting General practices in the United Kingdom (central Scotland and London) between 1999 and 2002.

Interventions Three educational interventions: an electronic tutorial carried on a CD Rom; decision support software built into the electronic medical record; and practice based workshops.

Participants 36 practices participated in the study. Eight practices were randomly assigned to the electronic tutorial; eight to decision support software; 10 to practice based workshops; and 10 to control. Electronic and manual searches yielded 450 valid and usable medical records.

Main outcome measures Rates of detection of dementia and the extent to which medical records showed evidence of improved concordance with guidelines regarding diagnosis and management.

Results Decision support software (P = 0.01) and practice based workshops (P = 0.01) both significantly improved rates of detection compared with control. There were no significant differences by intervention in the measures of concordance with guidelines.

Conclusions Decision support systems and practice based workshops are effective educational approaches in improving detection rates in dementia.

Introduction

Dementia is one of the most pressing challenges facing health and social care. About 5% of those aged > 65 and a fifth of those aged 80 have some kind of dementia, with similar prevalences across Europe.1 Ferri and colleagues estimate that the number of people with dementia will double every 20 years from 24.3 million today to 81.1 million by 2040. Between 2001 and 2040 the number in developed countries is expected to rise by 100% and by 300% in developing countries.2

Primary care is the point of first medical contact and hence the cornerstone of ensuring early detection, timely intervention, and effective ongoing management.3 Inadequate detection and referral and poor management have been documented nationally4 5 and internationally,68 leading to people with dementia and their families being denied optimal pharmacological9 and psychosocial intervention.10

We previously developed three educational interventions— an electronic tutorial on a CD Rom, decision support software, and practice based workshops1115—using literature reviews,16 17 previous experience of training programmes,1820 evidence based guidelines,21 and input from a multidisciplinary expert group. The decision support software has been incorporated into Egton Medical Information Systems' (EMIS) electronic medical record for its subscribers and is now available to about 5000 practices in the UK. A curriculum for practice based workshops for general practitioners and the electronic tutorial are available for download from the Alzheimer's Society website (www.alzheimer-s.org.uk).

We compared the effects of these three educational interventions on detection rates and concordance with guidelines regarding the diagnosis and management of dementia in primary care.

Participants and methods

Design

We used an unblinded, cluster randomised before and after controlled design. The sample size was calculated to give a power of at least 80% of detecting a difference of 1 SD at 5% significance in mean diagnosis concordance score.

Participating practices

We approached practices in one complete and one part health board area (central Scotland) and in two health authority (London) areas through a combination of local group meetings, letters with follow-up phone calls, and visits to individual practices. Practices were offered financial reimbursement to cover the costs of training and data collection up to £900 (€1312, $1562), depending on the number of partners. Practices had to be using EMIS or GPASS (General Practice Administration System for Scotland) software for patient records (because our decision support software was written for these software packages) and to document clinical encounters in the electronic record (that is, not just use it for prescribing).

Of the 124 practices approached, 36 entered and 35 (28%) completed the study (fig 1). These practices yielded valid data from 450 patients' medical records.

FIG 1
FIG 1

Flow of practices through study

Table 1 compares the 35 study practices with all practices in the relevant health authority or health board areas. Participating practices in London under-represented the area's average for older and male general practitioners and those working alone— groups which are over-represented in London compared with the national workforce of general practitioners.

Table 1

Representativeness of participating practices in London and Scotland

View this table:

Medical records

All practices identified registered patients aged 3 75 who had been diagnosed as having dementia or, in the opinion of the general practitioner or specialist, or both, had “probable dementia.” Practices conducted electronic searches of their clinical record system for the terms dementia, confusion, memory loss, and cognitive impairment. Medical and nursing staff updated this electronic search manually. Cases were identified before and nine months after the introduction of the educational interventions. Table 2 shows the demographic characteristics of patients at baseline.

Table 2

Baseline characteristics of patients identified as having dementia by study arm

View this table:

While profiles of age and sex were similar across all four arms, the proportion of patients in residential care was lower in the decision support software and control arm practices.

Consent

We obtained consent to access medical records from the patients or informal carers, next of kin, residential care home supervisors, or general practitioners. Members of the research team examined records in the surgery, and no personalised information was retained afterwards.

Educational interventions

We tested three educational interventions: an electronic tutorial on CD Rom, decision support software, and practice based workshops with a standard curriculum designed by a multidisciplinary expert group. The educational interventions reflected different approaches to adult learning: the electronic tutorial for self directed learning; decision support software for real time, real case learning; and workshops for peer reflection about real cases.

The electronic tutorial allowed for a mode of learning from case analysis, with the emphasis on reflecting on knowledge and revisiting particularly difficult and complex clinical problems. The tutorial is an “electronic book,” with an indexing system that allows easy access to different themes, and hypertext links that allow the reader to move easily from one subject to another.

The decision support software was written inside the existing electronic medical record software and produces prompts for the investigation and management of dementia. This mode of learning was driven by real cases, using computer decision support software to prompt and assist clinical reasoning and care planning in real clinical time.

Two experienced general practitioners with backgrounds in postgraduate education facilitated the small group workshops with general practitioners and practice nurses. Case scenarios were discussed to illustrate appropriate management. This mode of learning involved case discussion in small, multidisciplinary groups.

Control practices were visited only to collect data. Further details about the development, format, and piloting of the interventions are reported elsewhere.1115

Randomisation

An individual outside the research team used a computer generated program to randomise participating practices to receive one of the three interventions or to act as control. Eight practices were randomly assigned to the electronic tutorial on CD Rom, eight to decision support software, 10 to workshops, and 10 as controls. The research team and practices remained blinded to randomisation until after baseline data had been collected.

Outcome measures

Detection rates

Detection rates were based on the case finding exercise described above, which was conducted before and about nine months after the intervention. Data were extracted from the records after the second case finding exercise, unless the patient had died or left the practice.

Concordance with guidelines

We transcribed and scrutinised manual and electronic records for the recording of actions considered to be best practice in the diagnosis and management of dementia in primary care, after a critical review by the research team and an expert advisory group of the evidence based guidelines developed by Eccles et al.21 Additional items were added, including prescription of antidementia (cognition enhancing) drugs in line with guidance from the National Institute for Health and Clinical Excellence.22

The concordance scores for diagnosis and management were created by counting whether particular items were recorded. We extracted all data at the end of the study, calculating scores for before the intervention up to the time of the baseline search and scores for after the intervention from records from after the baseline date. The two forms were used to record symptoms at the index consultation with at least one clinical action in response to a diagnostic query. An additional section recording further action leading to diagnosis was included, covering such actions as testing (for example, cognitive) and “depression considered.”

Analysis

We assessed the effect of the interventions at the practice level because the data were cluster based, and we conducted analyses on an intention to treat basis. Analyses of all quantitative responses were performed with a general linear model with the arm and time as fixed effects and practice identity as a random effect.

We analysed differences in detection rates by using binary logistic regression to include the cluster effect. These were calculated before and after the intervention, excluding cases previously diagnosed in another practice.

FIG 2
FIG 2

Flow of patients through study

Concordance scores for diagnosis and management before and after the intervention were not directly comparable as they comprised counts of actions taken over two different lengths of time: the period before the intervention could be as long as 12 years, whereas after the intervention it was about nine months. Thus we used Vickers' method23 and examined differences in baseline concordance scores across the four arms of the study and then repeated the analysis for scores after the intervention. This analysis also incorporated the cluster effect.

Diagnosis concordance score (10 items)

  • Request for blood tests at index consultation

  • Request for blood tests after index consultation and before diagnosis

  • Referral to consultant, nursing, or secondary care at index consultation

  • History of patient's symptoms taken at index consultation

  • History of patient's symptoms taken after index consultation and before diagnosis

  • Cognitive testing completed at index consultation

  • Cognitive testing after index consultation and before diagnosis

  • Depression considered after index consultation and before diagnosis

  • Scan conducted after index consultation and before diagnosis

  • Diagnosis disclosed to carer or patient, or both

Management concordance score (seven items)

  • Concerns of carer

  • Behaviour problems

  • Depression (assessment or treatment, or both)

  • Referral to, or involvement of, social services

  • Referral to, or involvement of, voluntary organisations

  • Antidementia (cognitive enhancing) drugs

  • Review of medication

Results

The 35 practices reported 683 eligible patients—equivalent to a prevalence of dementia of 5.2% among the 13 068 registered patients aged 3 75. We could not obtain consent for 104 (15%) and records were completely missing for 34 (5%). Of the remaining 545 records, 95 (14%) were incomplete or lacked mention of dementia or possible dementia. This left a total of 450 valid and usable records (fig 2).

Detection rates

The number of people identified as having dementia after the interventions represents 31% of all cases diagnosed in the practice based workshops arm, 20% in the electronic tutorial arm, 30% in the decision support software arm, and 11% in the control arm. We investigated differences between arms using binary logistic regression, including clustering as a factor where study arm was the only significant factor, with Wald statistic of 9.02 (3 df, P = 0.02) with contrasts (using Bonferroni correction) comparing “active” arms with control yielding significant differences in the percentage diagnosed between practice based workshops (P = 0.01) and decision based software (P = 0.01) and control (table 3). These findings are not explained by any differences across study arms in the period from intervention to data collection, or by variation between study arms in the number of patients aged 3 75 registered with each practice.

Table 3

Number (percentage) of patients aged ≥75 diagnosed with dementia before and after intervention (n=280)

View this table:

Concordance with recommended approach to establishing a diagnosis and recommended management

There was no significant difference in mean concordance scores for diagnosis (table 4) or management (table 5) for patients diagnosed by the current practice before or after intervention.

Table 4

Mean (SD) diagnosis concordance scores by study arm before and after intervention

View this table:
Table 5

Mean (SD) management concordance scores by study arm before and after intervention

View this table:

Discussion

Principal findings

In this study of the effectiveness of educational interventions relating to dementia in primary care in terms of improvements in detection, diagnosis, and management we found a significant improvement in rates of reported cases of dementia with decision support software and practice based workshops compared with control. The positive effect of the decision support software on reported cases of dementia is particularly encouraging given that it is a relatively simple and practical intervention to implement. We found no difference in concordance with guidelines regarding the diagnosis or management of dementia.

We also looked at the effect of educational programmes on clinical practice, and developed a method for reviewing case records to test concordance with guidelines.

Explanation of results

The lack of evidence of improved concordance with guidelines regarding the diagnosis may be due to the relatively low number of cases identified after the intervention and the relatively few cases in the control arm. This led to reduced power, although we could still identify important differences with which to assess changes in approaches to diagnosis before and after the intervention.

The lack of significant changes in concordance with guidelines regarding diagnosis and management may be a function of relying on the medical record for evidence of practice. Practitioners may have improved their practice but did not record it—for example, elements of concordance, especially those that were less directly medical in nature (such as referral to support groups), may have become more common without being noted in the medical records. There may have been advantages in providing practitioners with the concordance form at the outset of the study comparable with the use of practice prompts adopted in the study by Feder et al.24 Quality of medical records has been an issue for some time, and a recent review suggests that the more distinct the diagnosis the better the quality of record, a finding that does not bode well for the quality of dementia related record keeping in primary care.25 De Lusignan and colleagues found an improvement in record keeping after a focused educational intervention on data quality in the management of ischaemic heart disease.26

The lack of change in concordance with guidelines regarding the diagnosis and management of dementia is consistent with findings on the broader use of decision support systems in medicine.27 Research on the effectiveness of decision support systems in the management of asthma and angina in primary care similarly failed to find effects.28 The lack of change in concordance with guidelines with decision support systems, despite improved rates of detection, might also support the argument that adherence to generic algorithms is not appropriate to,2931 or requires adaptation to ensure,32 patient centred consultations. Such concerns can lead to underuse on the part of general practitioners. Fugelli argues that the true art of general practice is to be found in the nature of the relationship between general practitioner and patient,29 something we did not examine. Finally, there may have been insufficient training in the use of the decision support systems. Previous research suggests a greater need for training than anticipated.31

Strengths and weaknesses of the study

The problem of transfer of knowledge gain is a familiar finding in research on education. Our results show the value of including multiple sources and types of outcomes in the assessment of effectiveness, as recommended by Kirkpatrick33 and Mann and Tylee.34 A potential weakness is that our approach was randomised at the level of practice, thus removing any effect of local opinion leaders from influencing the diffusion of innovation. The effect may have been greater had “academic detailing” been targeted at those in the primary care trust with lead responsibility for care of people with dementia. While the curriculum drew on a rich source of evidence, it may have benefited from involvement of patients and carers.

Implications for clinicians and policy makers

Recent changes to the general practitioner contract have focused on improving the quality of care for long term conditions by remunerating for better documentation of care given and for concordance with guidelines. This policy shift would enhance care if dementia were one of the long term conditions in the remuneration package. Primary care trusts would be able to draw on educational interventions with a known impact to support practices in improving their performance in this clinical domain.

The proposal for subspecialisation within the new contract may allow for more focused training to key individuals.35 This kind of distinctiveness may serve the needs of people with dementia and their families well. With respect to the use of decision support systems one obvious challenge is to ensure their integration into patient centred primary care.32

What is already known on this topic

General practitioners face difficulties in diagnosing and managing dementia and need training and education

Various educational approaches are available

What this study adds

Decision support systems and workshop formats are effective in improving detection of dementia in primary care

Unanswered questions and future research

Our research tested each intervention in isolation. While interventions in isolation failed to show significant effects in improving concordance with recommended diagnosis or management, the extent of their combined effectiveness should be explored. Further research might usefully explore the potential additive effect of combining locality initiatives with practice based initiatives. To date, the effectiveness of the educational interventions has been tested largely with white practitioners caring for predominantly white patients. In a multicultural society there is a need to test the effectiveness of interventions with the full range of ethnic groups.36 Effects of educational interventions also need to be evaluated from the perspective of patients and their carers.37 Within our patient led NHS it is timely that they contribute not only to the evaluation of such interventions but also to their development.

We thank the practices and carers who participated in this study and the people with dementia who consented to their records being scrutinised; Richard Simpson for his early involvement with the study; Ken Collins; members of the research advisory group; and Campbell Software Solutions. Additional resources were supplied by the North Central Thames Research Network and the Scottish Executive. We also thank Jane Mallinson for preparing the manuscript and Eryk Grant for his professional support.

Footnotes

  • Contributors MD and SI conceived the study, developed the design, obtained funding, managed the research, undertook the analysis and interpretation of findings, drafted this paper, and are guarantors. ST, JW, and MB developed and implemented the research, participated in analysis and in drafting the paper, and approved the final version. EL, JK, and RO'C contributed to design, management, implementation, interpretation of findings, and drafting of the paper and approved the final version. KH advised on statistical analysis, participated in drafting the paper, and approved the final version.

  • Funding Alzheimer's Society through the Alexander and Christina Dykes Project Grant.

  • Competing interests SI has received research funding from pharmaceutical companies producing drugs used in the treatment of Alzheimer's disease. Ethical approval: Obtained from local ethics committees.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
View Abstract