Rapid Responses to:

PRIMARY CARE: Signe Flottorp, Andrew D Oxman, Kari Håvelsrud, Shaun Treweek, and Jeph Herrin Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat BMJ 2002; 325: 367 [Abstract] [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Reporting trials of implementation research

Michael Power   (21 August 2002)

Educating for change

Aloysius N. Siriwardena, Professor Aly Rashid and Professor Mark Johnson   (28 August 2002)

Active tailored interventions are indeed more successful

Petra Denig, Flora Haaijer-Ruskamp   (3 September 2002)

Re: Reporting trials of implementation research and educating for change

Signe Flottorp, Andrew D Oxman, Kari Håvelsrud, Shaun Treweek, and Jeph Herrin   (4 September 2002)

Reporting trials of implementation research 21 August 2002
Michael Power, Clinical Author Sowerby Centre for Health Informatics, University Of Newcastle, Newcastle upon Tyne, NE2 4AB Send response to journal: Re: Reporting trials of implementation research	The paper by Flottorp et al[1] reports a rigorous trial of a suite of interventions intended to facilitate the implementation of two clinical guidelines. I would like to ask two questions of detail that are of interest to implementation researchers. More importantly, I would also like to raise two methodological issues associated with trials of systems to implement guidelines. In the methods section the interventions are said to be tailored to overcome identified barriers. What were the barriers, how were they identified, and, which interventions were tailored to address which barriers? The final sentence of the paper delivers two messages: that active interventions should be used to support guideline implementation and that clinicians should be given time to change their routines. Is there theory or evidence to support these recommendations? The first methodological issue arises from the difficulty of describing ways of implementing guidelines. The study’s data collection and sophisticated analysis is fully described. In contrast, the complex of 7 main interventions are reported with important details missing. An analogy with a clinical trial might make this issue clearer. The intervention would be described as a multivitamin, but no details would be given on indication, formulation, presentation, strengths of components, prescribed dose, frequency, compliance, and adverse effects. Consequently the paper fails to meet the test of adequacy of the methods section of a scientific paper: Does it allow the reader to replicate the study? Of course, rigorous description is much easier to achieve for a pharmaceutical intervention than for a suite of guideline implementation interventions. Space is not a limitation now that extra material can be published electronically. The main problem is the lack of agreed conventions for classifying and reporting on interventions to support guideline implementation. Categories currently used in implementation research (e.g. electronic educational material, computer-based decision support, interactive courses) include very diverse individual interventions. Reporting at this level of detail in a clinical trial would result in a vague description of the intervention as “surgery” or “pharmacotherapy”. Implementation research lacks a vocabulary that would allow guideline implementation methods to be described as precisely as specific surgical technique and individual drug describe clinical interventions. The second methodological issue is illustrated by the main conclusion drawn by the authors: it is difficult to change practice and therefore it is important to measure rigorously the effects of interventions before advocating their widespread use. However, science needs to provide both proof and understanding. In this case, an intervention thought by the authors to be “logical” failed to meet expectations. Surely, first priority now is to understand what was wrong with the logic? Were the interventions ineffective because they were passive and did not reach their intended recipients? Or, were they ineffective because the recipients were resistant to change? And, if the recipients were resistant, perhaps they were so because the proposed change in behaviour is perceived to have (at individual consultation level) little consequence in terms of clinical outcome or cost — in the study populations both UTI and sore throat would be expected to resolve uneventfully with or without any treatment in most cases, and an antimicrobial prescription, urine test and throat swab are likely to have low perceived opportunity costs. Would interventions be more successful in changing practice if they were tailored to individual patients, or tailored to specific clinicians with behaviours that may need to be improved (those in the right-hand part of figure 2), or addressed problems that clinicians are unfamiliar with, or that they perceive to be more serious in terms of clinical outcome, or more important in terms of opportunity cost? Implementation research needs models of behaviour change that would provide a framework to structure these sorts of questions and direct research programmes. Two main conclusions for implementation research can thus be drawn from the report: One: Methodologists should be addressing the need to develop the vocabulary (i.e. classifications of indications, categories of interventions, models of behaviour change, and structures for reports) that would facilitate appraisal of implementation research, synthesis of evidence from several trials, and generalisation of results to other settings. Two: Studies of complex interventions should be designed to contribute, not only to evidence of effect, but also to understanding of mechanism. 1. Flottorp S, Oxman AD, Håvelsrud K, Treweek S, Herrin J. Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat. BMJ 2002; 325: 367-370 (17 August) competing interests: none
Educating for change 28 August 2002
Aloysius N. Siriwardena, Honorary Clinical Research Fellow, De Montfort University, Leicester Minster Practice, Cabourne Ave, Lincoln LN2 2UP, Professor Aly Rashid and Professor Mark Johnson Send response to journal: Re: Educating for change	Dear Sir We were interested to read the study from Norway[1] designed to improve management of urinary infections and sore throat but a little surprised by the methods used to improve performance. The authors stated that they had used tailored strategies to overcome barriers to improving performance in the management of urinary tract infections in women and sore throats but there was little evidence that they had clearly identified the barriers to change or as a consequence that appropriate strategies were sought to change (prescribing and test ordering) behaviour. It has been known for some time that doctors respond poorly to passive dissemination of recommendations and yet this was the main strategy used in this study.[2] Other educational strategies mentioned, such as interactive courses, were poorly described and the extent to which practitioners engaged unclear. The authors also failed to clearly identify professional or patient barriers to implementing evidence.[3] The notion that using tailored strategies, involving multiple (so- called complex) interventions, especially where these overcome barriers is the key to changing performance in health care has been a recurring theme in the literature,[4] but we have doubts about the value of this overly mechanistic model to change the behaviour of healthcare professionals, especially as the results are often so variable and unpredictable. This mechanistic model fails to take into account a number of other factors that are crucial in altering behaviour. These include the importance of identifying, understanding and modifying tacit expert knowledge and promoting the ownership of change amongst professionals.[5] Uncertainty about the evidence may also affect doctor’s behaviour. Was there robust evidence for the use of telephone consultations, had this evidence been disseminated and were the practitioners convinced of this? Who was to initiate telephone consultations and would general practitioners encourage this change in behaviour when they would have been paid a third to a half of the fee of an appointment. The combination of perverse financial incentives, uncertainty about the evidence, and tacit knowledge as well as the failure to actively engage the professionals may have been hidden factors or ‘strange attractors’ that prevented change in this study.[6] Studies of educational interventions need to use strategies that have face validity and which are likely to alter behaviour. Practitioners and health systems need to be able adapt to changes in knowledge, a concept that is central to the idea of the ‘learning organisation’. We feel that taking a more real world means to examining and altering a ‘complex adaptive system’ of health professionals would be more useful than using a ‘one size fits all’ approach. 1. Flottorp S, Oxman AD, Havelsrud K, Treweek S, Herrin J. Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat. BMJ 2002;325:367-70. 2. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995;153:1423-31. 3. Freeman AC,.Sweeney K. Why general practitioners do not implement evidence: qualitative study. BMJ 2001;323:1100-2. 4. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L et al. Changing provider behavior: an overview of systematic reviews of interventions. Med.Care 2001;39:II2-45. 5. Ferlie E, Fitzgerald L, Wood M. Getting evidence into clinical practice: an organisational behaviour perspective. J Health Serv.Res.Policy 2000;5:96-102. 6. Kelley MA,.Tucci JM. Bridging the quality chasm. BMJ 2001;323:61- 2. Competing interests: none
Active tailored interventions are indeed more successful 3 September 2002
Petra Denig, Assistant Professor, University of Groningen Department of Clinical Pharmacology, A.Deusinglaan 1, 9713 AV Groningen (NL), Flora Haaijer-Ruskamp Send response to journal: Re: Active tailored interventions are indeed more successful	In the paper by Flottorp et al [1] reporting on a cluster randomised controlled trial of a multifaceted intervention two messages are brought forward. First, the authors conclude that a passively delivered intervention has little effect on changing the management of UTI in women and sore throats. Second, they advocate the use of adequately sized cluster randomised controlled trials to evaluate interventions aimed at implementating guidelines. As it happens, a number of rigorous trials using hierachical models has been conducted to evaluate interventions to implement guidelines for urinary tract infections as part of the European Drug Education Project.[2-4] Only one of these studies is mentioned and its findings are not adequately discussed, thereby missing the opportunity to compare the intervention methods and to learn more about why some interventions work better than others. In the European Drug Education Project an active intervention strategy was used to improve the management of urinary tract infections in women. Small groups of general practitioners (GPs) received feedback on their management decisions and discussed this with each other and a (local) pharmacist in the light of specific guideline recommendations. This type of educational programme mainly addresses the barriers relating to competence, motivation and attitudes of the GPs, and not the logistic or organisational barriers for implementing guidelines. Since the GPs are actively involved in the programme and receive individual feedback, it is tailored to their needs. The main focus of the educational programme differed somewhat in the participating countries, since the clinically relevant issues were found not to be identical. In Sweden [2] the focus was on prescribing first choice drugs, whereas in the Netherlands [3] and Norway [4] it was on the duration of treatment. The programme had effects in the intervention group relative to the control group ranging from 12% in Norway to 31% the Netherlands.[2-4] In comparison, this active approach focussing on specific barriers and needs at the level of the GPs seems more effective than the passive broad approach that tries to overcome several possible barriers at many different levels at once. It is important to note that our active approach was a group approach in which support is given to encourage internal discussion and implementation of guidelines in peer groups. Supporting local peer groups needs less resources than supporting all practices with individual outreach visits. In the Netherlands, there is a nationwide network of local groups of GPs and pharmacists that discuss pharmacotherapy on a regular basis. This familiarity with the peer group approach may explain why the effects of the European Drug Education Project were largest is this country.[2-5] 1. Flottorp S, Oxman AD, Havelsrud K, Treweek S, Herrin J. Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat. BMJ 2002; 325: 367-370 2. Stalsby Lundborg C, Wahlström R, Oke T, Tomson G, Diwan VK. Influencing prescribing for urinary tract infection and asthma in primary care in Sweden: a randomized controlled trial of an interactive educational intervention. J Clin Epidemiol 1999; 52: 801-812 3. Veninga CCM, Denig P, Zwaagstra R, Haaijer-Ruskamp FM. Improving drug treatment in general practice. J Clin Epidemiol 2000; 53: 762-772 4. Lagerlov P, Loeb M, Andrew M, Hjortdahl P. Improving doctors' prescribing behaviour through reflection on guidelines and prescription feedback: a randomised controlled study. Qual Health Care 2000; 9: 159-165 5. Veninga CCM, Lagerlov P, Wahlström R, et al. Evaluating an educational intervention to improve the treatment of asthma in four European countries. Am J Resp Crit Care Med 1999; 160: 1254-1262.
Re: Reporting trials of implementation research and educating for change 4 September 2002
Signe Flottorp, Researcher Department of Health Services Research, Norwegian Directorate for Health and Social Welfare, *, Andrew D Oxman, Kari Håvelsrud, Shaun Treweek, and Jeph Herrin Send response to journal: Re: Re: Reporting trials of implementation research and educating for change	*Place of work (Flottorp, Oxman, Håvelsrud, Treweek): Department of Health Services Research, Norwegian Directorate for Health and Social Welfare, PO Box 8054 Dep, N-0031 Oslo, Norway Herrin: Flying Buttress Associates, PO Box 2254, Charlottesville VA 22902, USA We agree with Michael Power in his two main conclusions: methodologists should develop a vocabulary for implementation research, and studies of complex interventions should be designed to contribute to understanding of mechanism of effect (or lack of effect). The Cochrane Effective practice and organisation of care group (EPOC) and others are contributing to the development of a vocabulary for this research field (1,2). We conducted a process evaluation to describe how the interventions were carried out in participating practices and to explore why our interventions had so little effect. A report of this evaluation has been submitted for publication. We disagree that we used a mechanistic “one size fits all” approach, as stated by Siriwardena et al. The idea of tailoring interventions is rather the opposite: we aimed at increasing the likelihood of success of our interventions by trying to understand the challenges of implementing these specific guidelines in primary care in Norway. Although we did not tailor the intervention to each individual practice, we did tailor the package of interventions to address as many as possible of the barriers to change that we had identified across practices. We have described how we identified these barriers and developed the interventions in another paper submitted for publication. Unfortunately, it was not possible to fully describe how we tailored the interventions and the results of the process evaluation in the paper published in the BMJ within the space we were allowed. 1. Bero LA, Grilli R, Grimshaw JM, Mowatt G, Oxman AD, Zwarenstein M (eds). Cochrane Effective Practice and Organisation of Care Group. In: The Cochrane Library, Issue 3, 2002. Oxford: Update Software. 2. Thorsen T, Mäkelä M (eds). Changing Professional Practice. Theory and Practice of Clinical Guidelines Implementation. Copenhagen: Danish Institute for Health Services Research and Development, 1999.