General Practice

Designing trials of interventions to change professional practice in primary care: lessons from an exploratory study of two change strategies

Stephen Rogers, senior lecturer in primary health care,  Charlotte Humphrey, senior lecturer in medical sociology,  Irwin Nazareth, senior lecturer in primary health care,  Sue Lister, lecturer in practice development,  Zelda Tomlin, research fellow in qualitative research,  Andy Haines, professor of primary health care. 

Department of Primary Care and Population Sciences, Royal Free and University College Medical School, University College London, Archway Resource Centre, London N10 3UA

Correspondence to: S Rogers s.rogers{at}ucl.ac.uk

Various strategies have been evaluated for their ability to support the adoption of clinical evidence into everyday practice. ^{1 2} There is increasing interest in interventions aimed at groups of healthcare staff and promoting organisational change. Observational studies of these types of interventions have produced encouraging findings, ^{3 4} but the results of randomised controlled trials have sometimes been disappointing.^5-7 These differences may be due to the methodological and practical difficulties of evaluating such interventions in randomised trials rather than to lack of efficacy of the interventions.⁸

We carried out an exploratory trial to examine the independent and combined effects of teaching evidence based medicine and facilitated change management on the implementation of cardiovascular disease guidelines in primary care (box). The trial was accompanied by a formative evaluation, drawing on information collected by the evidence based medicine tutor, the change management facilitator, and a qualitative researcher who observed workshops and meetings and conducted a series of semistructured interviews with study participants.¹⁴ Progress was reviewed at monthly steering group meetings, and the thesis of this paper emerged from the deliberations and discussions of this group.

	Issues arising

Top Issues arising Reconciling interests of... Discussion Conclusions References

Selection of suitable practices
Choice of sampling frameOur exploratory study was carried out in collaboration with the Medical Research Council General Practice Research Framework. Although we recognised advantages in working with a well supported and enthusiastic group of practices with a track record in collaborative research, the practices may be atypical.¹¹ For example, some practices had participated in research on cardiovascular disease before and may have adopted the findings of their research.¹⁵ We subsequently found a wide variation in performance across the cardiovascular disease management topics, but the organisational characteristics of the practices could still limit the generalisability of our conclusions.

Participating practices

Consent to participate

Development and adaptation of interventions
Theoretical credibilityOur teaching of evidence based medicine was based on workshops developed at McMaster University,¹² and the change management programme drew on principles of continuous quality improvement as taught at the Institute of Health Improvement.¹³ This strategy provided external validity for the interventions delivered, but some primary care staff felt that the change management workshop was insufficiently grounded in the language, concerns, and perspectives of primary care.

Acceptability

Replicability and transferability

Evaluability

Choice of experimental design
Multiple interventionsThe decision to adopt a factorial design had implications for the time scale of the trial, the delivery of the interventions, and the demands on the practices. In particular, delays were introduced because practices allocated to both interventions could not accommodate two, one day workshops in close succession. Practice based workshops were convenient for participants but labour intensive for the research staff. They enabled us to avoid contamination between practices allocated to different interventions, but interactions with practices were sometimes compromised as the tutor was asked not to advise on implementation and the facilitator was asked not to comment on research evidence.

Multiple guidelines

	Reconciling interests of research, development, and practice

Top Issues arising Reconciling interests of... Discussion Conclusions References

We had to make various choices in designing our study: which practices to select for participation, how to develop and adapt the interventions, and which experimental design to use. Tensions arose because people whose principal concern was with the scientific rigour of the investigation, those whose main focus was the development and adaptation of the interventions (as a theoretically based model of behaviour change or as a pragmatic service intervention²⁰), and those who were participants in the research had different views on what was important. The table is an analytical framework that summarises the desirable characteristics of trial design from the perspective of the three constituencies. The issues arising are discussed in detail below.

The interests of research, development, and practice would be addressed by a trial design that was able to satisfy the following conditions:

• A practice selection strategy which reconciles issues around the acceptability and appropriateness of interventions to practices and the generation of externally valid, generalisable trial results
• Interventions that remain true to their theoretical foundations, are of sufficient focus, intensity, or duration to have an impact, but are adapted to the needs and conditions of participating practices. The interventions also need to be replicable, transferable, and evaluable
• A statistically efficient experimental design that can ensure scientific rigour without compromising the ability of staff to participate and without damaging the integrity of the interventions being tested.

	Discussion

Top Issues arising Reconciling interests of... Discussion Conclusions References

Interventions requiring the active participation of health professionals in organisational change are likely to require a high degree of motivation from most of the practice team if they are to have an impact. The willingness of practices to participate in trials of professional behaviour change will depend on the interests of members and the organisational characteristics of practices. A trial which focuses on practices expected to derive substantial benefits (an explanatory trial)²¹ would ensure that practices and interventions were well matched but might not provide appropriate information on the application of such interventions in service settings. An alternative approach would be to recruit practices that were typical of those that might come forward for interventions offered in a service context (pragmatic trial).²¹ The designs converge as recruitment becomes increasingly selective. In a practice preference design,²⁰ a randomised controlled trial of practices that have no preferences for particular interventions is nested in an observational study in which practices with strong preferences receive the intervention of their choice. This design could meet the concerns of research, development, and practice but is complex and expensive and requires more effort from the researchers.

In the interests of theoretical credibility, we reproduced established models for teaching the principles of evidence based medicine¹² and continuous quality improvement¹³ while allowing practitioners to explore specific applications. For interventions delivered at this level, measures of organisational effectiveness might be more appropriate than disease specific measures based on implementation of guidelines. ^{17 18} Also, the approach would need to be directed at generating substantial change at practice level for measurable effects to be seen. The linking of theory to a specific task would be acceptable to practices, and a focus on guidelines could be associated with measurable outcomes. This would simplify measurement issues in a trial, but some commentators might express concerns about the application of continuous quality improvement methods to tasks that the practices had not necessarily identified as priorities. ^{3 4}

The factorial design is a powerful approach to studying two interventions simultaneously, though the execution of this design may be demanding for researchers and study participants. It may not be feasible for practices to implement a group of guidelines simultaneously, but a sequence of guidelines could be presented over time (in random sequence to balance order effects). This split plot design⁹ is likely to be acceptable to trial participants, to those whose concern is to maintain the integrity of the interventions, and to researchers provided that funding could be found for the extended trial period. Alternatively, a randomised incomplete block design⁹ could be used to assess the efficacy of a single intervention across two or more guidelines, but as every practice is subject to intervention the design could not provide information on the generic effects of interventions on professional behaviour.

	Conclusions

Top Issues arising Reconciling interests of... Discussion Conclusions References

Clinicians and those involved in service development sometimes dismiss academic research because of an "ivory tower" approach that pays too little attention to issues around service delivery.²² The nature of negotiated access to research subjects in primary care settings can have a direct effect not only on participation rates in research but also on the quality of the research data.²³ If a trial is to be executed successfully, and its findings are to be applicable in a service setting, it is important to identify a trial design that can best reconcile the interests of research, development, and practice. Our analytical framework provides an approach by which it is possible to explore how particular characteristics of trial design appear from each perspective and thereby to assess the most satisfactory design options. The approach cannot assure that trial design will be straightforward and problem free, but early consideration of the perspectives of research, development, and practice might help to prevent fundamental problems arising later.

	Acknowledgments

The study was carried out in collaboration with the MRC General Practice Research Framework, and we are grateful to participating practice staff and Dr M Vickers.

Contributors: AH and IN had the original idea for a trial. SR was overall coordinator of both design and execution of the exploratory trial, organised recruitment of the study practices, and developed packs of evidence based guidelines. CH and ZT designed and executed the qualitative component of the study. IN and SL developed and implemented the intervention strategies used in the feasibility study. SR, C Hill, and S Goubet developed the quantitative outcome measures. The steering group for the study comprised the authors, S Goubet, and C Hill. L Klinger, J Hickling, and M Griffin assisted with collection, checking, or analysis of data. Advisors to the project included M Vickers, M Lawrence, S Thompson, P Greenhalgh, S Ebrahim, and J Roberts. M Modell, R Morris, and D Mant (who was an external reviewer) gave valuable comments on earlier drafts of this paper. SR is the guarantor for the paper.

	Footnotes

Funding: NHS Research and Development Implementation Methods Programme.

Competing interests: None declared.

	References

Top Issues arising Reconciling interests of... Discussion Conclusions References

1.	Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. Can Med Assoc J 1995; 153: 1423-1431[Abstract].
2.	Effective Health Care. Getting evidence into practice. York: University of York, 1999.
3.	Lawrence M, Packwood T. Adapting total quality management for general practice: evaluation of a programme. Qual Health Care 1996; 5: 151-158[Abstract].
4.	Hearnshaw H, Reddish S, Carlyle D, Baker R, Robertson N. Introducing a quality improvement programme to primary healthcare teams. Qual Health Care 1998; 7: 200-208[Abstract].
5.	Wyatt JC, Paterson-Brown S, Johanson R, Altman DG, Bradburn MJ, Fisk NM. Randomised trial of educational visits to enhance use of systematic reviews in 25 obstetric units. BMJ 1998; 317: 1041-1046[Abstract/Free Full Text].
6.	Solberg LI, Kottke TE, Brekke ML. Will primary care clinics organise themselves to improve the delivery of preventive services? A randomised controlled trial. Preventive Med 1998; 27: 623-631[CrossRef][Medline].
7.	Goldberg HI, Wagner EH, Fihn SD, Martin DP, Horowitz CR, Christensen DB, et al. A randomized controlled trial of CQI teams and academic detailing: can they alter compliance with guidelines. Jt Comm J Qual Improv 1998; 24: 130-142[Medline].
8.	Shortell SM, Bennett CL, Byck GR. Assessing the impact of continuous quality improvement on clinical practice: what will it take to accelerate progress? Milbank Quarterly 1998; 76: 593-624.
9.	Cochran and Cox. Experimental designs. New York: Wiley, 1957.
10.	Altman DG. Practical statistics for medical research. London: Chapman and Hall, 1991.
11.	Medical Research Council General Practice Research Framework. A network of general practices throughout the UK managed by the MRC Epidemiology and Medical Care Unit. London: EMCU, Wolfson Institute of Preventive Medicine, 1997.
12.	Neufield VR, Woodward CA, McLeod SM. The McMasters MD programme: a case study in renewal in medical education. Acad Med 1989; 64: 423-432[Medline].
13.	Berwick D. A primer on the improvement of systems. BMJ 1996; 312: 619-622[Free Full Text].
14.	Tomlin Z, Humphrey C, Rogers S. General practitioners' perceptions of effective health care. BMJ 1999; 318: 1532-1535[Abstract/Free Full Text].
15.	Stephens R, Gibson D. The impact of clinical trials on the treatment of lung cancer. Clin Oncol 1993; 5: 211-219.
16.	Brewin CR, Bradley C. Patient preferences and randomised clinical trials. BMJ 1989; 299: 313-315.
17.	Anderson NR, West MA. The team climate inventory: manual and users guide. Windsor: Assessment Services for Employment/National Foundation for Educational Research-Nelson Press, 1994.
18.	Pritchard P, Pritchard J. Teamwork for primary and shared care. Oxford: Oxford University Press, 1994.
19.	Sibley J, Sackett DL, Neufeld V, Gerrard B, Rudnick KV, Fraser W. A randomized trial of continuing medical education. N Engl J Med 1982; 306: 511-515[Abstract].
20.	Bradley F, Wiles R, Kinmonth A-L, Mant D, Gantley M. Development and evaluation of complex interventions in health services research: case study of the Southampton heart integrated care project. BMJ 1999; 318: 711-715[Free Full Text].
21.	Schwarz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutic trials. J Chronic Dis 1967; 20: 637-648[CrossRef][Medline].
22.	Dawson S. Inhabiting different worlds: how can research relate to practice? Qual Health Care 1997; 6: 177-178[Medline].
23.	Murphy E, Spiegal N, Kinmonth AL. "Will you help me with my research?" Gaining access to primary care settings and subjects. Br J Gen Pract 1992; 42: 162-165[Medline].

(Accepted 14 April 2000)