Do clinical guidelines introduced with practice based education improve care of asthmatic and diabetic patients? A randomised controlled trial in general practices in east LondonBMJ 1995; 311 doi: http://dx.doi.org/10.1136/bmj.311.7018.1473 (Published 02 December 1995) Cite this as: BMJ 1995;311:1473
- Gene Feder, senior lecturera,
- Chris Griffiths, senior lecturera,
- Clare Highton, research associatea,
- Sandra Eldridge, statisticianb,
- Matthew Spence, research assistanta,
- Lesley Southgate, professora
- aDepartment of General Practice and Primary Care, St Bartholomew's and Royal London Hospital Medical College, London EC1M 6BQ,
- bDepartment of Epidemiology and Medical Statistics, Queen Mary and Westfield College, University of London, London
- Correspondence to: Dr Feder.
- Accepted 2 November 1995
Objective:To determine whether locally developed guidelines on asthma and diabetes disseminated through practice based education improve quality of care in non-training, inner city general practices.
Design:Randomised controlled trial with each practice receiving one set of guidelines but providing data on the management of both conditions.
Subjects:24 inner city, non-training general practices.
Main outcome measures:Recording of key variables in patient records (asthma: peak flow rate, review of inhaler technique, review of asthma symptoms, prophylaxis, occupation, and smoking habit; diabetes: blood glucose concentration, glycaemic control, funduscopy, feet examination, weight, and smoking habit); size of practice disease registers; prescribing in asthma; and use of structured consultation “prompts.”
Results:In practices receiving diabetes guidelines, significant improvements in recording were seen for all seven diabetes variables. Both groups of practices showed improved recording of review of inhaler technique, smoking habit, and review of asthma symptoms. In practices receiving asthma guidelines, further improvement was seen only in recording of review of inhaler technique and quality of prescribing in asthma. Sizes of disease registers were unchanged. The use of structured prompts was associated with improved recording of four of seven variables on diabetes and all six variables on asthma.
Conclusions:Local guidelines disseminated via practice based education improve the management of diabetes and possibly of asthma in inner city, non-training practices. The use of simple prompts may enhance this improvement.
The use of structured consultation prompts for the recording of clinical information recommended by the guidelines improves implementation of the guidelines in practice
Relatively underdeveloped inner city practices can respond positively to this form of dissemination of guidelines and external audit
Clinical guidelines pervade primary and secondary care. It is now recognised that the development of guidelines based on research evidence must be complemented by dissemination and implementation strategies that encourage clinicians to use guidelines in practice.1 2 In a systematic review of 91 studies Grimshaw and Russell concluded that guidelines had the greatest chance of changing clinical behaviour when they were developed by the clinicians for whom they were intended, disseminated through a specific educational programme and implemented via patientspecific reminders during consultations.3 Only six of the studies reviewed were from British general practice. The largest study, in 62 English training practices4 was disappointing: those receiving “external” guidelines improved neither process nor outcome of care for children with common acute disorders. How can we make guidelines effective in general practice?
The Hackney collaborative clinical guidelines project was a local initiative started in 1991 in east London that developed primary care guidelines and tested methods of dissemination and implementation. The project was based in the local academic department of general practice and supported by the Hackney General Practice Forum.5 The East London and City Health Authority has the highest Jarman underprivileged area index in England6 and a highly mobile population reflected in a 30% average annual turnover of patients on practice lists.7 During our study most practices in the area were single handed or two handed and based in poor quality premises.
Morbidity from asthma in Hackney is high: admission rates in east London are between 80% and 100% above the national average for all age groups.8 For patients with diabetes admission rates in east London are high for both amputation and ketoacidosis.9 We aimed principally to determine whether guidelines on asthma and diabetes disseminated via an educational package to non-training inner city practices affected the quality of care.
Our hypotheses were that practices receiving guidelines for either asthma or diabetes (a) improved their recording of key data, reflecting good quality care; (b) increased the size of the register for the relevant disease; and (c) improved their pattern of prescribing (asthma guidelines only). We also examined the effect of “prompts” (structured records) derived from the guidelines on the quality of care and the effect of the guidelines on consultation rates.
Our dissemination method was practice based, multidisciplinary educational outreach (or “academic detailing”). Studies of educational outreach have been confined to North America, and the measurable effect on prescribing and preventive care by clinicians has been small.10
Subjects and methods GUIDELINE DEVELOPMENT
The guidelines were developed by local general practitioners working through informal consensus with local hospital specialists and relevant professionals and were not explicitly evidence based.11 The guidelines on asthma were based on the British Thoracic Society's first national guidelines12 and the diabetes guidelines on the St Vincents' declaration.13
PRACTICE RECRUITMENT AND DIAGNOSTIC CRITERIA
In autumn 1992 we invited all 49 non-training practices in Hackney to join the study. Twenty seven (55%) practices agreed; only seven of these practices had disease registers before the start of the study. Participating practices were visited by general practitioner members of the research team who proposed uniform diagnostic criteria for asthma and diabetes14 15 and prompted creation or updating of disease registers for these conditions. These disease registers were the source of our samples of patients. The diagnostic criteria were derived from the guidelines (both simple summaries and “complex” documents were disseminated). More details of our intervention package are available on request.
In two practices, only one of two partners agreed to participate. These practices essentially ran personal lists with little interaction between partners and were entered as singlehanded practices. One singlehanded general practitioner withdrew owing to illness, one two partner practice was excluded as it could not adequately develop disease registers, and a further two partner practice served as a pilot for data collection and educational intervention, leaving 24 practices comprising 39 principals for analysis (table 1).
After the initial recruitment session described above, practices were stratified by partnership size, list size per general practitioner, employment of a practice nurse, deprivation, and prior existence of asthma and diabetes clinics. They were then randomised to two groups to receive the guidelines either for asthma or for diabetes. A postal questionnaire on other practice characteristics was completed by all practices (table 1). Three months after recruitment, practices were revisited and introduced to their respective guidelines. The rolling programme of practice sessions ran from January to June 1993.
Every practice provided data from the notes of both patients with asthma and those with diabetes, but they only received guidelines for one condition, acting as a control for the other condition. “Asthma practices” refers to those receiving the guidelines on asthma and “diabetes practices” refers to those receiving the guidelines on diabetes.
DISSEMINATION OF GUIDELINES VIA PRACTICE OUTREACH
Our educational intervention consisted of three lunchtime sessions—approved for the postgraduate education allowance—to which all relevant members of the practice team were invited. The four educators (GF, CG, CH, and a specialist nurse) worked in pairs and visited equal numbers of asthma and diabetes practices. They standardised the content and delivery of sessions.
The first session introduced the allocated guidelines and discussed how the practice's current management could be developed into a practice protocol in line with the recommendations in the guidelines, with an emphasis on patient recall for annual review. Each clinician was given a stamp for reviewing asthmatic patients (fig 1) and a stamp or booklet for reviewing diabetic patients.
These “prompts” reflected the content of an annual review consultation for patients with asthma or diabetes as recommended in the guidelines. The session concluded with a practical discussion of home urine monitoring or peak flow measurement. The second session reviewed the practice's organisational decisions and then focused on the clinical content of the guidelines. It concluded with a demonstration of measuring visual acuity or inhaler technique.
The third session took place about six months later and focused on audit data from the notes of patients with diabetes or asthma. We also reviewed how the practice was coping with implementing the guidelines. All contacts with practices were logged on a database to estimate the costs of our educational intervention.
QUALITY OF CARE VARIABLES
Quality of care variables are based on data which the guidelines recommended that clinicians should collect in annual review consultations for both conditions. The variables correspond with evidence based audit standards.16 17
Prescribing costs for drugs for asthma were obtained from Pactline (the Prescription Pricing Authority's online service) for the year preceding the study and the year after the guidelines were introduced. Costs were derived from the British National Formulary and expressed as cost per prescribing unit per year. The ratio of the prescribing costs of prophylaxis to bronchodilators was calculated for each practice for the year before and year after intervention. This prescribing index has been validated as a marker of the quality of prescribing in asthma in east London practices.18 A measure of changes in quality of prescribing during the study was derived for each practice by dividing the index before intervention by the index after intervention.
Size of disease registers is a measure of case finding by practices. All the asthma registers of the practices in our study (table 2) were smaller at baseline than even conservative estimates of adult asthma in east London, whereas the diabetes registers were close to expected prevalence in east London (1.7% (95% confidence interval 1.1 to 2.6) (I Jones, personal communication)). Changes in size of disease register were calculated as the ratio of the size after the introduction of guidelines to that before.
COLLECTION OF PROCESS AND PRESCRIBING DATA
Sample size was based on practice audits in Hackney: peak flow and blood glucose concentrations had been recorded within the previous year in about 40% of the notes of asthmatic and diabetic patients respectively. Detection of a 50% relative increase in the recording of these two variables (from 40% to 60%) with a power of 95% at a significance level of 5% required a total sample of 310 patients. A sample size of 390 patients (10 patients per principal) was sufficient to detect a clinically relevant difference even with a trend towards increased recording in “control” practices and reduction of power when the practice rather than the patient was taken as unit of analysis.
Data were gathered from the clinical records of 10 patients with asthma and 10 with diabetes per general practitioner principal, selected from disease registers by a method using random numbers. Only permanently registered NHS patients aged 16 and over were included. “Ghost” patients (those who had had no contact with the surgery for the past two years)19 and patients whose asthma or diabetes had been diagnosed less than 12 months previously were excluded and replacement patients randomly selected. For baseline data, these records were scrutinised for all entries relating to asthma and diabetes made during the 12 months up to recruitment of the practice—except for occupation, which was searched for in the three years up to recruitment. Data collection was repeated one year after the introduction of the guidelines, with a new random sample from the disease registers. For notes scrutinised one year after intervention, two further variables were gathered: the number of patients in whom the “prompt” stamp or booklet was used and the number of consultations for the relevant condition during the year before and the year after intervention.
The sources of patient data were written patient records, computerised patient recordings, test results, and hospital letters. The most recent general practice record of any data was entered directly onto a database. If there was no record in the general practice notes within the past year the medical record was searched for letters from hospital clinics or discharge letters within the past year that contained this information.
The accuracy of the gathering and coding of data was validated at baseline by an independent comparison of recording accuracy with the notes of 10 patients with asthma and 10 with diabetes; 96% of the coding was accurate. The consistency of data extraction was assessed by examining coding before the guidelines and one year later using the same 10 sets of notes for asthma and for diabetes. For all but four variables there was perfect agreement. For three variables on asthma (smoking habit, symptom review, inhaler technique) there was substantial agreement ((kappa)=0.7). The only variable for which there was poor agreement was examination of feet in diabetic patients ((kappa)=0.4).
To determine the effect of the introduction of guidelines on the quality of care, analysis of covariance was used to model a practice's level of recording after intervention as a function of the level before intervention and intervention, weighted by the number of patients sampled in each practice. To test hypotheses about prescribing in asthma and size of disease registers, non-parametric tests were used as appropriate owing to the noticeably non-normal distributions of some of the variables.
Practices randomised to receive asthma or diabetes guidelines were similar with respect to stratifying variables and practice characteristics (table 1).
RECORDING OF QUALITY OF CARE VARIABLES
Recording of these variables varied greatly both at baseline and after intervention—for example, funduscopy (fig 2. Asthma and diabetes practices had similar distributions of variables at baseline, except for the recording of smoking habit, which was significantly greater in the diabetes practices (table 3). Differences at baseline were taken into account in the regression models testing the effect of the guidelines.
Analysis of covariance showed that diabetes practices significantly improved their recording of all variables. The difference in percentages and 95% confidence intervals shown in table 3 are based on these analyses. Analysis of place of recording showed that recording within practices improved for all variables except funduscopy; improvement in recording of funduscopy was mostly due to this procedure being performed in hospital. In both groups of practices significant improvements over baseline values were found in the recording of three asthma variables: review of inhaler technique, smoking habit, and symptom review. In asthma practices further improvement was detected in the recording of only one of six variables: review of inhaler technique (table 3).
Practices which already had a disease register before intervention had a higher baseline recording of variables on asthma and diabetes. Proportional improvements in recording were similar irrespective of the previous existence of a disease register.
SIZE AND GROWTH OF DISEASE REGISTER
Before the introduction of the guidelines the median sizes of disease register (as percentage of a practice's total list size) were 1.5% and 1.6% for asthma and diabetes respectively. No significant differences existed in register sizes for each condition between asthma and diabetes practices (Wilcoxon's two sample test: asthma, z= - 0.81, P>0.4; diabetes, z= - 1.41, P>0.1). Although register sizes increased after the guidelines were introduced in most practices, particularly the asthma practices, there was no significantly greater increase in intervention practices.
DRUG PRESCRIBING IN ASTHMA
In the year preceding the study, the median cost per prescribing unit for bronchodilators was pounds sterling1.51 (interquartile range pounds sterling1.29, pounds sterling1.70) and for prophylaxis was pounds sterling1.40 (pounds sterling1.04, pounds sterling2.25), with no significant differences between the asthma and diabetes practices (Wilcoxon's two sample test: bronchodilators, z= - 0.84, P=0.4; prophylaxis, z= - 1.3, P=0.26). The variation in prophylaxis costs between practices was striking, with a 10-fold difference separating the prescribers with the lowest and the highest costs.
In the year after intervention costs for bronchodilators rose by a median of 3p (− 12p, 16p) in the asthma practices and 15p (8p, 29p) in the diabetes practices. Costs for prophylaxis rose by 57p (23p, 78p) in the asthma practices and 40p (16p, 64p) in the diabetes practices. The ratio of prescribing indices before and after intervention was calculated for each practice. The median value of this ratio for the asthma practices was 1.43 (1.1, 1.55) and for the diabetes practices was 1.06 (0.99, 1.29). The value for the asthma practices was significantly greater than that for the diabetes practices (z=2.14, P=0.03).
EFFECT OF “PROMPT” STAMP OR BOOKLET ON RECORDING
In addition to the significant effect of our intervention on diabetes practices, the use of a structured “prompt” for a quarter of patients in our sample was associated with increased proportional recording of three out of six variables (smoking habit, blood pressure, and blood glucose (table 4). For the asthma practices, use of the annual review stamp significantly increased the recording of all six variables on asthma. Although the stamp was used with only 41 patients in the asthma practices, the effect was unambiguous.
Consultation rates in our sample for asthma and diabetes were generally low for all the practices. In the diabetes practices median consultation rates increased for diabetes by 30% (1.6 to 2.1 consultations per patient per year) and for asthma by 22% (1.2 to 1.4). In the asthma practices consultation rates increased for asthma by 50% (1.0 to 1.5) and for diabetes by 23% (1.1 to 1.4).
COST OF EDUCATIONAL INTERVENTION
Each practice required about 30 minutes for correspondence, 20 minutes on the telephone, and three hours in personal visits. With two doctors visiting each practice, this equates to pounds sterling144 per practice of general practice time (at a clinical lecturer's salary of pounds sterling24 500 a year). Postgraduate education allowance costs on average pounds sterling439 per practice, giving a total cost of pounds sterling583 per practice. Although this does not take into account the development of the guidelines or the salary costs of other practice team members, it gives an approximate cost of the educational intervention.
This study addresses a practical question: how can we effectively disseminate clinical guidelines to primary care? Our intervention improved the recording of key data associated with good care for patients with diabetes. For patients with asthma, the effect was marginal, with improvements in recording rates for one out of six variables and improved prescribing. These improvements were not limited to practices who had already developed some form of structured care but were seen even in practices which at the start of the study did not have a disease register. Furthermore, we detected improvements despite the absence of a control group not receiving any guidelines; our study design controlled for but did not test for a Hawthorne effect (improvements in performance by virtue of participation in a study). Our results contrast with those in the north of England study in which guidelines improved the quality of care only in those practices in which a practitioner contributed to their development.4
The education programme through which we disseminated the guidelines aimed to change practitioners' behaviour by small group methods,20 which are particularly appropriate for primary care teams. We incorporated some of the most important features of educational outreach: focusing on a specific group of clinicians; defining clear educational and behavioural objectives; establishing credibility; stimulating active participation in the educational sessions; using concise graphic educational material; highlighting and repeating essential messages; and providing positive reinforcement in follow up visits.21 The excellent rate of participation among many underdeveloped practices reflected widespread acceptability of a programme of guidelines led by peers.20
Our results reflect the management of patients on practice disease registers. Diagnostic accuracy and completeness of disease register22 probably varied between practices. We cannot extrapolate our findings to the management of all patients with asthma or diabetes registered with these practices. However, as systematic bias is unlikely, the variation in quality of disease registers does not invalidate their use in a randomised controlled study.
Why did the diabetes practices make greater improvements in the recording of care than the asthma practices when the educational method and format of the guidelines was the same? There are two possible explanations. Firstly, there was a trend towards improved recording of asthma variables in the diabetes practices. While this could be a result of external factors, such as increased publicity about asthma management, it could also reflect differences in the power of the Hawthorne effect in the two groups of practices. Thus, the more complex nature of the diabetes review and the need for a structured recall system is a larger hurdle for the practice and may require a specific educational intervention, whereas the relatively simpler nature of the asthma review means that intervention around another chronic condition—namely, diabetes—will have an indirect effect on asthma care. A study with a control group of practices receiving no guidelines could test this hypothesis.
Secondly, the diabetes practices might have referred more patients for hospital review, resulting in more information being available from clinic letters. However, the improvements in the recording of diabetes variables persisted, with the exception of funduscopy, even when data from hospital letters were excluded. Thus our intervention prompted an appropriate division of labour between primary and secondary care (the latter being more appropriate for retinopathy screening23).
SAMPLING OF DISEASE REGISTERS
Our resources required us to sample from disease registers. With samples of 10 patients per doctor, irrespective of register size, the practices with smaller registers contributed a greater proportion of the register to the total patient sample. This sampling method avoids overrepresentation of patients from practices with large registers. It also ensured enough patients per practice to feed back meaningful baseline data as part of the educational intervention. The method does not take account of variation in the size of lists or registers. Alternative sampling methods—proportional to register size or total list size—would have made comparison between practices problematic as the practices varied greatly in these features. Although our sampling method means that sampling errors are not simple random but complex errors, any bias to the substantive results is likely to be small.
RECORDING OF ASTHMA AND DIABETES VARIABLES
The recording of variables varied enormously between practices and was generally poor. At the start of the study some practices had no record of funduscopy or peak expiratory flow rate during the past year for any of the patients sampled. Although these results seem poor compared with previous reports of care of asthmatic and diabetic patients, our study did not exclude patients with poor attendance,24 and we assessed practices whose patient turnover approached 30% annually.7 Many of the study practices were only beginning to develop chronic disease management and were doing so under difficult conditions.
There are two caveats about our interpretation of these data. Firstly, we set a harsh standard by counting data only recorded within the past year. Secondly, medical records in general practice do not accurately reflect clinical performance, although laboratory investigations and drug treatment are more likely to be noted than history or physical examination.25 These limitations were addressed by the controlled nature of our study as long as no systematic bias existed in the distribution of diagnostic or recording inaccuracy.
DRUG PRESCRIBING IN ASTHMA
The 10-fold variation in prescribing costs of asthma prophylaxis is unlikely to be fully explained by variations in generic prescribing, list inflation, or case mix. Most practices at the start of our study were spending less than one third as much on prophylaxis as general practitioners with an interest in asthma surveyed in a recent nationwide study.26 Our intervention resulted in more appropriate prescribing by practices receiving guidelines on asthma. This is the first demonstration of changes in prescribing resulting from a guidelines programme in British general practice.
EFFECT OF “PROMPT” STAMP OR BOOKLET
The use of prompts was associated with improved recording of variables for both conditions. This is further evidence that patient specific prompts may enhance the effect of guidelines on clinicians' behaviour,27 although—in the absence of randomisation—we cannot be certain that improved recording was due to the effect of the prompt itself rather than more diligent clinicians choosing to use the prompt.
The use of prompts was relatively evenly spread across practices, which suggests an independent effect of prompts, but this requires further investigation. Manual prompts are a particularly appropriate method in an area where few practices use computers in consultations.
ACCEPTABILITY OF EXTERNAL AUDIT METHOD
External audit was acceptable to study practices that joined our study. Some practices expressed concern about confidentiality but were reassured that data would be available to outside bodies only in an anonymised form. Strengths of this method are convenience for practices, consistency, and therefore comparability of results. External audit is a potentially powerful tool for assessing the quality of chronic disease management.28
Our study shows that local guidelines disseminated with practice based education can improve the management of diabetic patients and probably of asthma patients in inner city, non-training practices. The use of simple recording prompts enhances this improvement. This form of dissemination was acceptable to a wide range of practices, many of which were underdeveloped. Our crude estimation of costs suggests that a modest investment can have a meaningful effect on chronic disease management. In a one year project we could not judge whether these improvements persist with time. Even if they do, our educational method of guidelines dissemination still needs to be tested against other methods of quality improvement and in relation to patient outcomes.29
We thank all the participating practices; Moira Spence for inspiration and managerial advice; Jeremy Grimshaw for advice on design; Jon Deeks and Stephen Evans for statistical advice; Judith Duddle for participation in the practice education sessions; Jeanette Murphy for help with design of the educational intervention; and Joanne Turner, Katie Featherstone, Sarah Mott, and Michelle Ricken for support.
Funding The North East Thames Regional Health Authority and the Department of Health.
Conflict of interest None.