Primary care

Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat

Signe Flottorp, researcher ^a, Andrew D Oxman, director ^a, Kari Håvelsrud, research associate ^a, Shaun Treweek, researcher ^a, Jeph Herrin, research analyst ^b. 

^a Department of Health Services Research, Norwegian Directorate for Health and Social Welfare, PO Box 8054 Dep, N-0031 Oslo, Norway, ^b Flying Buttress Associates, PO Box 2254, Charlottesville VA 22902, USA

Correspondence to: S Flottorp signe.flottorp{at}shdir.no

	Abstract

Top Abstract Introduction Methods Results Discussion References

Objective: To assess the effectiveness of tailored interventions to implement guidelines for urinary tract infections in women and sore throat.
Design: Unblinded, cluster randomised pretest-post-test trial.
Setting: 142 general practices in Norway.
Participants: 72 practices received interventions to implement guidelines for urinary tract infection and 70 practices received interventions to implement guidelines for sore throat, serving as controls for each other. 59 practices in the urinary tract infection group and 61 practices in the sore throat group completed the study. Outcomes were measured in 16 939 consultations for sore throat and 9887 consultations for urinary tract infection.
Interventions: Interventions were developed to overcome identified barriers to implementing the guidelines. The main components of the tailored interventions were patient educational material, computer based decision support and reminders, an increase in the fee for telephone consultations, and interactive courses for general practitioners and practice assistants.
Main outcome measures: Changes in rates of use of antibiotics, laboratory tests, and telephone consultations.
Results: Patients in the sore throat group were 3% less likely to receive antibiotics after the intervention. Women with symptoms of urinary tract infection in the intervention group were 5.1% less likely to have a laboratory test ordered. No significant differences were found between the groups for the other outcomes. Large variation was found across the included practices in the rates of antibiotic prescription, use of laboratory tests and telephone consultations, and in the extent of change for all three outcome measures.
Conclusions: Passively delivered, complex interventions targeted at identified barriers to change had little effect in changing practice.

What is already known on this topic Interventions to change professional practice have small to moderate effects at best Multifaceted interventions targeted at identified barriers to change are more likely to be effective for implementing guidelines than a simple intervention selected by chance What this study adds Large variation exists in the extent of change before and after the delivery of tailored interventions to support implementing guidelines Rigorous methods are needed to evaluate interventions to change practice With passively delivered, complex interventions targeted at identified barriers there was only a 3% decrease in antibiotic prescribing for sore throat and a 5% decrease in test use for urinary tract infection in women

	Introduction

Top Abstract Introduction Methods Results Discussion References

Many theories but limited evidence exist on the effectiveness of interventions to change professional practice. ^{1 2} Although more complex interventions tend to be most effective, their effectiveness varies, they require more resources, and it is difficult to know which interventions to use. Identifying barriers to change and tailoring interventions to address these is a logical approach to selecting appropriate interventions.³ The effectiveness of tailored interventions remains uncertain.^4-10 We aimed to assess the effectiveness of tailored interventions to support the implementation of guidelines for the management of urinary tract infections in women and sore throat.

We developed evidence based guidelines for urinary tract infections in women and sore throat. ^{11 12} The main recommendation for sore throat is that most patients do not need antibiotics. Clinical examination and laboratory tests are therefore generally not necessary, and patients can be given advice by telephone. The main recommendation for urinary tract infections is that non-pregnant women aged 16-55 years with typical symptoms can be treated with antibiotics without any testing. Women who have had a previous urinary tract infection can be offered treatment by telephone.

	Methods

Top Abstract Introduction Methods Results Discussion References

Design
We tested the main hypotheses with a cluster randomised pretest-post-test controlled trial. General practices were randomised to receive tailored interventions to support implementing guidelines for either urinary tract infections in women or sore throat (fig 1). Practices receiving one set of guidelines served as controls for practices receiving the other. Data were collected 18 weeks before and after the intervention.

View larger version (35K): [in this window] [in a new window]   Fig 1.   Flow of practices through trial

Participants
We randomly selected 292 of the 323 general practices in Norway with the WinMed electronic medical record system, and we sent a letter inviting them to participate. Overall, 142 practices were randomised by computer (fig 1).¹³

We included consultations for sore throat if they concerned patients over 3 years old with diagnosis codes relevant for sore throat from the international classification for primary care.¹⁴ We included consultations for urinary tract infection analysis if they concerned non-pregnant women aged 16 to 55 years with diagnosis codes relevant for urinary tract infections from the international classification for primary care.

Interventions
We tailored the interventions to overcome identified barriers. Both guidelines were published in the June 2000 issue of the Journal of the Norwegian Medical Association, which all Norwegian general practitioners receive. ^{11 12} Box 1 shows the main interventions for urinary tract infections and sore throat.

The interventions were initiated in May 2000 and continued until January 2001. Because of the nature of the interventions, participating practices knew the group to which they were assigned. One member of each practice signed an informed consent form. The study protocol was approved by the Data Inspectorate, the Norwegian Board of Health, and the regional committee for medical research ethics.

Data
We extracted outcome data with two separate methodsMediata and QTools, software that we developed in collaboration with Mediata.¹³ The software was sent to the practices on disk and installed by their staff. The staff ran the data extraction routines that transferred relevant data to disks, which were then posted back to us.

The Mediata software also included an interactive decision support application and a tool to collect additional data from pop-up screens that were triggered when a diagnosis code for sore throat or urinary tract infection was entered into a patient's record.

The Qtools software extracted relevant data, which is routinely collected, from the electronic medical record. We used these data for the main analysis because we only obtained data from Mediata for a subset of the consultations. We compared data from the baseline period, 1 January 2000 to 15 May 2000, with data from the follow up period, 15 September 2000 to 31 January 2001.

Outcomes
The primary outcomes for both conditions were changes in the rates of use of antibiotics, laboratory tests, and telephone consultations. We expected that implementation of the guidelines would reduce the proportion of patients who were prescribed antibiotics for sore throat but not change the proportion prescribed antibiotics for urinary tract infections, reduce the use of laboratory tests for both, and increase the proportion of telephone consultations for both.

We identified Anatomical Therapeutic Chemical Classification System codes¹⁵ for prescriptions and information from the prescription name field in WinMed. We classified the drugs as relevant or not relevant for urinary tract infections and sore throat. A standardised coding system for laboratory tests is not used in Norway. We classified all tests extracted from the field for laboratory tests in WinMed as relevant or not relevant for urinary tract infections and sore throat. We categorised a consultation as a telephone consultation if it was coded as such in WinMed or if a fee code for a telephone consultation was registered.

Sample size
We calculated sample size with a method that takes into account the intracluster correlation coefficient, the number of events, the expected effect, and the power of the study.¹⁶ We assumed an intracluster correlation coefficient of =0.2, a minimum of 25 patients for each practice, and a worst case control group rate of 50%. Under these assumptions we anticipated a power of 87% to detect a difference of 15% in rates between the two groups with =0.05 with 60 practices for each intervention group. We anticipated a loss to follow up and therefore planned to randomise 70 practices in each group.

Statistical methods
We used cluster specific methods because practices rather than patients were randomised, and we expected that variance in how patients were managed would be partly explained by the practice.¹⁷ Because some patients had more than one consultation, we added a third level to the analysis to account for the likelihood that variance in what was done at each consultation would be partly explained by the patient. For each of the outcome measures we tested the null hypothesis that the likelihood of the outcome was the same in both study groups after accounting for baseline rates, multiple patients for each practice, and repeated consultations for each patient.

We used hierarchical logistic regression to test for an effect of each intervention on the outcome measures (box 2).^17-19 This technique allowed us to test for a positive interaction between time and intervention, which would indicate an effect of the intervention on outcomes. All hierarchical models were estimated with HLM5.²⁰ All other analyses were done with Stata 7. 

	Results

Top Abstract Introduction Methods Results Discussion References

Thirteen practices in the urinary tract infection arm and nine practices in the sore throat arm dropped out after randomisation (fig 1). All the practices installed the Mediata and Qtools software. We did not get data from the QTools program for two urinary tract infection practices and five sore throat practices. This left 57 of 72 practices in the urinary tract infection arm and 56 of 70 practices in the sore throat arm for the main analyses.

The sore throat practices had more practitioners and registered a greater number of consultations than did the urinary tract infection practices (tables 1-3). The arms were similar for patient characteristics and baseline measurements (tables 2-4).

Outcome data were collected for 16 939 consultations for sore throat and 9887 consultations for urinary tract infection (tables 2 and 3). The intracluster correlation coefficients for the primary outcomes varied from 0.05 to 0.21 (table 4), indicating that cluster specific analytical methods were appropriate.

Use of antibiotics
After adjusting for baseline rates, patients with sore throat in the intervention group were less likely to receive antibiotics than patients with sore throat in the control group (table 4). The absolute reduction in the proportion of consultations where antibiotics were prescribed for sore throat was 3.0% greater in the intervention group. For patients with urinary tract infection there was little change in the proportion of consultations where antibiotics were prescribed in both the intervention group (-0.2%) and the control group (0.2%).

Use of laboratory tests
After adjusting for baseline rates, women with symptoms of urinary tract infection in the intervention group were less likely to have a laboratory test ordered than were women in the control group. The absolute reduction in the proportion of consultations for urinary tract infection where a laboratory test was ordered for urinary tract infections was 5.1% greater in the intervention group. No significant differences were found between the groups for use of laboratory tests for sore throat (table 4).

Use of telephone consultations
After adjusting for baseline rates, the absolute increase in the proportion of telephone consultations for sore throat was 1.2% greater in the control group than it was in the intervention group (table 4). The proportion of telephone consultations for urinary tract infections was decreased in both the intervention group and the control group. The reduction was 0.9% greater in the control group. These differences were not significant.

Other analyses
Large variations occurred across the included practices in the use of antibiotics, laboratory tests, and telephone consultations and in the extent of change for all three outcome measures (fig 2).

View larger version (26K): [in this window] [in a new window]   Fig 2.   Rates of antibiotic use in consultations for sore throat before and after the tailored interventions from all practices with more than 10 consultations in each period

	Discussion

Top Abstract Introduction Methods Results Discussion References

Interventions that were tailored to address identified barriers to implementing guidelines for urinary tract infections in women and sore throat had little impact on changes in the use of antibiotics, laboratory tests, and telephone consultations.

Although only practices using WinMedone of three electronic medical record systems used in Norwaywere included in our study, there is no reason why these practices should differ from other practices.

We might have underestimated the rates for antibiotic prescriptions, ordering of laboratory tests, and telephone consultations because the practitioners may not have registered this information correctly in some instances. This is unlikely to have differed between the groups and is therefore unlikely to have affected the results.

It is possible that we underestimated the reduction in the prescription of antibiotics for sore throat. Some of the general practitioners in the study reported that they now inform patients that antibiotics normally are not necessary but prescribe them for throat infections with Streptococcus for use "if needed." We do not know how often patients were told this or how often prescriptions that were given with this message were not used.

There are several possible explanations for the small effect that we found. We might have failed to identify important barriers to change, although neither the participants nor we identified additional barriers during the course of the study or in discussing the results. We did not tailor the interventions to the needs of individual practices or of individual general practitioners. Tailoring the intervention at this level might have had a greater effect, although we do not have any data to support this hypothesis.

Two more likely explanations are the passive character of the interventions and the lack of time. We did not have the resources to actively support the practices with outreach visits. Many practices did not have routines to support internal discussion and implementation of the guidelines. We identified some barriersfor example, problems with accessibility by telephonethat we were not able to adequately address with our interventions.

It is possible that our follow up period was not long enough to detect changes in practice. Women are used to taking a urine sample to their doctor when they have symptoms of a urinary tract infection, and patients may have expected to have laboratory tests taken and receive antibiotics for sore throats. In addition, the demands of daily practice prevented many participants from devoting much time to the project.

Few trials have been done on implementing guidelines for urinary tract infections or sore throat. A non-randomised controlled before and after study of the implementation of a telephone based guideline for managing presumed cystitis in women aged 18 to 55 years found significant decreases in the proportions of patients who received urinalysis, had a urine culture, or had an initial visit to their doctor.²¹ The design of this study was not robust, but the results are promising and the interventions used were intensive. A randomised controlled trial with groups of general practitioners using an educational intervention to influence prescribing for urinary tract infections found that prescription of first choice drugs increased from 52% to 70% in the intervention group and remained constant in the control group, but the intervention had no effect on the duration of treatment.²² We have not identified trials of the implementation of a guideline for sore throat similar to ours. Studies of scoring rules for sore throat have failed to show that they lower the rates for antibiotic prescription.²³ It has been shown that it is possible to improve doctors' judgments of the probability of streptococcal pharyngitis, but this knowledge did not affect decisions to prescribe antibiotics.²⁴

Figure 2 shows the importance of using adequately sized cluster randomised controlled trials to evaluate interventions to support the implementation of guidelines. Large variation exists in practice and in the extent of change among practices. Uncontrolled or inadequately controlled before and after evaluations in selected practices are likely to have spurious results that are, at best, difficult to interpret.

Conclusions
Despite the small effect that we found, it still seems logical to select strategies to support the implementation of guidelines that are tailored to address identified barriers. However, it is difficult to change practice, and large changes over short periods are not typical. Therefore it is important to use rigorous methods to measure the effects of the interventions that are used and to ensure that they are cost effective before widely employing them. It is also important to invest sufficient resources to ensure that active interventions can be used to engage clinicians and to ensure that they find the time that is needed to change their routines.

	Acknowledgments

We thank Arild Bjørndal, Cheryl Carling, Jan Arve Dyrnes, Tor Arne Bertheussen, and Atle Fretheim for their help and support, the advisory committee for the project and the reference group for the development of the guidelines for support and advice, Mediata, which helped develop the software for data collection and decision support, and the participating practices.

Contributors: SF and ADO developed the study protocol and interventions. SF conducted the study. KH assisted in recruitment of practices and was responsible for the communication with and follow up of the participating practices. ST helped with development and implementation of the software in the practices. JH randomised the practices and was responsible for the statistical analyses. All contributors discussed core ideas and obstacles during the study. SF drafted the manuscript with support from the other authors. SF and ADO will act as guarantors for the paper.

	Footnotes

Funding: Quality Assurance Fund of the Norwegian Medical Association and the National Institute of Public Health.

Competing interests: None declared.

	References

Top Abstract Introduction Methods Results Discussion References

1.	Grol R. Beliefs and evidence in changing clinical practice. BMJ 1997; 315: 418-421[Free Full Text].
2.	Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care 2001; 39: II2-45[Web of Science][Medline].
3.	Oxman AD, Flottorp S. An overview of strategies to promote implementation of evidence-based health care. In: Silagy C, Haines A, eds. Evidence-based practice in primary care. London: BMJ Publishing Group, 2001:101-119.
4.	Cheater F, Baker R, Hearnshaw H, Robertson N, Hicks N, Oxman A, et al. Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes (protocol for a Cochrane Review). In: Cochrane Library. Issue 4. Oxford: Update Software, 2001.
5.	Cranney M, Barton S, Walley T. Addressing barriers to change: an RCT of practice-based education to improve the management of hypertension in the elderly. Br J Gen Pract 1999; 49: 522-526[Web of Science][Medline].
6.	Hux JE, Melady MP, DeBoer D. Confidential prescriber feedback and education to improve antibiotic use in primary care: a controlled trial. CMAJ 1999; 161: 388-392[Abstract/Free Full Text].
7.	Soumerai SB, McLaughlin TJ, Gurwitz JH, Guadagnoli E, Hauptman PJ, Borbas C, et al. Effect of local medical opinion leaders on quality of care for acute myocardial infarction: a randomized controlled trial. JAMA 1998; 279: 1358-1363[Abstract/Free Full Text].
8.	Santoso B, Suryawati S, Prawaitasari JE. Small group intervention vs formal seminar for improving appropriate drug use. Soc Sci Med 1996; 42: 1163-1168.
9.	Van der Weijden T, Grol RP, Knottnerus JA. Feasibility of a national cholesterol guideline in daily practice. A randomized controlled trial in 20 general practices. Int J Qual Health Care 1999; 11: 131-137[Abstract/Free Full Text].
10.	Baker R, Reddish S, Robertson N, Hearnshaw H, Jones B. Randomised controlled trial of tailored strategies to implement guidelines for the management of patients with depression in general practice. Br J Gen Pract 2001; 51: 737-741[Web of Science][Medline].
11.	Flottorp S, Oxman AD, Cooper JG, Hjortdahl P, Sandberg S, Vorland LH. Retningslinjer for diagnostikk og behandling av sår hals. Tidsskr Nor Laegeforen 2000; 120: 1754-1760[Medline].
12.	Flottorp S, Oxman AD, Cooper JG, Hjortdahl P, Sandberg S, Vorland LH. Retningslinjer for diagnostikk og behandling av akutte vannlatingsplager hos kvinner. Tidsskr Nor Laegeforen 2000; 120: 1748-1753[Medline].
13.	Treweek S, Flottorp S. Using electronic medical records to evaluate healthcare interventions. Health Inf J 2001; 7: 96-102.
14.	Lamberts H, Wood M, Hofmans-Okkes I, eds. The international classification of primary care in the European Community, with a multi-language layer. Oxford: Oxford Medical, 1993.
15.	WHO Collaborating Centre for Drug Statistics Methodology. ATC index with DDDs. Oslo: WHO Collaborating Centre for Drug Statistics Methodology, 1999.
16.	Donner A, Klar N. Statistical considerations in the design and analysis of community intervention trials. J Clin Epidemiol 1996; 49: 435-439[CrossRef][Web of Science][Medline].
17.	Murray DM. Design and analysis of group-randomized trials. New York: Oxford University Press, 1998.
18.	Bryk AS, Raudenbush SW. Hierarchical linear models. Newbury Park: Sage, 1992.
19.	Omar RZ, Thompson SG. Analysis of a cluster randomized trial with binary outcome data using a multi-level model. Stat Med 2000; 19: 2675-2688[CrossRef][Web of Science][Medline].
20.	HLM5 2000. Hierarchical linear and nonlinear modelling. Lincolnwood IL: Scientific Software International, 2000.
21.	Saint S, Scholes D, Fihn SD, Farrell RG, Stamm WE. The effectiveness of a clinical practice guideline for the management of presumed uncomplicated urinary tract infection in women. Am J Med 1999; 106: 636-641[CrossRef][Web of Science][Medline].
22.	Lundborg CS, Wahlstrom R, Oke T, Tomson G, Diwan VK. Influencing prescribing for urinary tract infection and asthma in primary care in Sweden: a randomized controlled trial of an interactive educational intervention. J Clin Epidemiol 1999; 52: 801-812[CrossRef][Web of Science][Medline].
23.	McIsaac WJ, Goel V. Effect of an explicit decision-support tool on decisions to prescribe antibiotics for sore throat. Med Decis Making 1998; 18: 220-228.
24.	Poses RM, Cebul RD, Wigton RS. You can lead a horse to waterimproving physicians' knowledge of probabilities may not affect their decisions. Med Decis Making 1995; 15: 65-75[Abstract/Free Full Text].

(Accepted 24 May 2002)