BMJ 2000;321:694-696 ( 16 September )

Education and debate

Framework for design and evaluation of complex interventions to improve health

Michelle Campbell, senior policy analyst ^a, Ray Fitzpatrick, professor of public health and primary care ^b, Andrew Haines, professor of primary health care ^c, Ann Louise Kinmonth, professor of general practice ^d, Peter Sandercock, professor of medical neurology ^e, David Spiegelhalter, senior scientist ^f, Peter Tyrer, professor of community psychiatry ^g. 

^a Office of the President, Medical Research Council of Canada, 1600 Scott Street, Ottawa, Ontario, Canada K1 OW9, ^b Division of Public Health and Primary Health Care, Institute of Health Sciences, University of Oxford, Oxford OX3 7LF, ^c Department of Primary Care and Population Sciences, Royal Free and University College Medical School, London NW3 2PF, ^d General Practice and Primary Care Research Unit, Department of Public Health and Primary Care, Institute of Public Health, Cambridge CB2 2SR, ^e Neuroscience Trials Unit, Department of Clinical Neurosciences, Western General Hospitals NHS Trust, Edinburgh EH4 2XU, ^f MRC Biostatistics Unit, Institute of Public Health, Cambridge CB2 2SR, ^g Department of Public Mental Health, Imperial College of Science, Technology, and Medicine, St Mary's Campus, London W2 1PD

Correspondence to: R Fitzpatrick raymond.fitzpatrick{at}nuffield.ox.ac.uk

Randomised controlled trials are widely accepted as the most reliable method of determining effectiveness, but most trials have evaluated the effects of a single intervention such as a drug. Recognition is increasing that other, non-pharmacological interventions should also be rigorously evaluated.^1-3 This paper examines the design and execution of research required to address the additional problems resulting from evaluation of complex interventionsthat is, those "made up of various interconnecting parts."⁴ The issues dealt with are discussed in a longer Medical Research Council paper (www.mrc.ac.uk/complex_packages.html). We focus on randomised trials but believe that this approach could be adapted to other designs when they are more appropriate.

Summary points Complex interventions are those that include several components The evaluation of complex interventions is difficult because of problems of developing, identifying, documenting, and reproducing the intervention A phased approach to the development and evaluation of complex interventions is proposed to help researchers define clearly where they are in the research process Evaluation of complex interventions requires use of qualitative and quantitative evidence

	Challenges of trials of complex interventions

There are specific difficulties in defining, developing, documenting, and reproducing complex interventions that are subject to more variation than a drug. A typical example would be the design of a trial to evaluate the benefits of specialist stroke units. Such a trial would have to consider the expertise of various health professionals as well as investigations, drugs, treatment guidelines, and arrangements for discharge and follow up. Stroke units may also vary in terms of organisation, management, and skill mix. The active components of the stroke unit may be difficult to specify, making it difficult to replicate the intervention. The box gives other examples of complex interventions.

Examples of complex interventions Service delivery and organisation: Stroke units Hospital at home Interventions directed at health professionals' behaviour: Strategies for implementing guidelines Computerised decision support Community interventions: Community based programmes to prevent heart   disease Community development approaches to improve   health Group interventions: Group psychotherapies or behavioural change   strategies School based interventionsfor example, to reduce   smoking or teenage pregnancy Interventions directed at individual patients: Cognitive behavioural therapy for depression Health promotion interventions to reduce alcohol   consumption or support dietary change

	Framework for trials of complex interventions

Problems often arise in the evaluation of complex interventions because researchers have not fully defined and developed the intervention. It is useful to consider the process of development and evaluation of such interventions as having several distinct phases. These can be compared with the sequential phases of drug development (fig 1) or may be seen as more iterative (fig 2). Either way a phased approach separates the different questions being asked.

Progression from one phase to another may not be linear. In many cases an iterative process occursfor example, if an exploratory trial finds that a complex intervention is unacceptable to potential recipients, the theoretical basis and components of the intervention may have to be re-examined. Preliminary work is often essential to establish the probable active components of the intervention so that they can be delivered effectively during the trial. Identifying which stage of development has been reached in specifying the intervention and outcome measures will give researchers and funding bodies reasonable confidence that an appropriately designed and relevant study is being proposed.

	Preclinical or theoretical phase

The first step is to identify the evidence that the intervention might have the desired effect. This may come from disciplines outside the health sciences (such as theories of organisational change). Review of the theoretical basis for an intervention may lead to changes in the hypothesis and improved specification of potentially active ingredients. In addition, previous studies may have provided some empirical evidencefor example, an intervention may have been found effective for a closely related condition or in another country with a different organisation of health care.⁵

View larger version (31K): [in this window] [in a new window]   Fig 1.   Sequential phases of developing randomised controlled trials of complex interventions

View larger version (16K): [in this window] [in a new window]   Fig 2.   Iterative view of development of randomised controlled trials of complex interventions

	Phase I: defining components of the intervention

Modelling or simulation techniques can improve understanding of the components of an intervention and their interrelationships. Qualitative testing through focus groups, preliminary surveys, or case studies can also help define relevant components. Descriptive studies may help to delineate variants of a service. For example, hospital at home schemes vary in purpose. Some are designed to hasten hospital discharge, others to avoid hospital admissions, and yet others to provide palliative care in the home.⁶

Qualitative research can also be used to show how the intervention works and to find potential barriers to change in trials that seek to alter patient or professional behaviour.⁷ For example, if health professionals see the main barrier to changing their practice as being lack of time or resources, an intervention that focuses only on improving their knowledge will not work.

	Phase II: defining trial and intervention design

Acceptability and feasibility
In phase II the information gathered in phase I is used to develop the optimum intervention and study design. This often involves testing the feasibility of delivering the intervention and acceptability to providers and patients. Different versions of the intervention may need to be tested or the intervention may have to be adapted to achieve optimal effectivenessfor example, if the proposed intensity and duration of the intervention are found to be unacceptable to participants.

It is also important to test for evidence of a learning curve, leading to improved performance of the intervention over time. If a learning curve exists a run-in period might be needed before formal recruitment to the trial to ensure that the intervention is provided effectively.

The exploratory trial is also an opportunity to determine the consistency with which the intervention is delivered. Consultations could be audio or video taped to give feedback of performance to providers together with training to promote consistency.

Defining the control intervention
The content of the comparative arm (control group) of the main trial will be decided during the preparatory phase. It may be an alternative package of care, standard care, or placebo. Although standard practice is often an appropriate control, it can be as complex as the intervention being evaluated and may change with time. It is thus important to monitor the care that is being delivered to the control group. The use of a no treatment control group may be unacceptable to patients. One possible solution is a randomised waiting list study in which all participants ultimately receive the intervention.

Designing the main trial
The exploratory phase should ideally be randomised to allow assessment of the size of the effect. This initial assessment will provide a sound basis for calculating sample sizes for the main trial. Other design variables can also be established in an exploratory trial.

Outcomes
Outcome measures for the main trial will also generally be piloted during the exploratory phase. Investigators should include outcomes that not only are relevant to patients with the disease or condition being studied but also encompass measures of wider relevance to the health system, including economic measures.⁸ Collection of data to assess a full range of costs to patients, carers, and society adds considerably to the workload and costs of researchers and may challenge the feasibility of a trial. Strategic choice of outcomes is therefore needed.⁹

An important decision in trials of complex interventions is whether health outcome needs to be assessed. For studies such as those evaluating strategies to change professional behaviour, it may be sufficient to show that the intervention changed behaviour, provided that clear evidence exists that the changed behaviourfor example, prescribing particular treatmentsis effective.

	Phase III: methodological issues for main trial

The main trial will need to address the issues normally posed by randomised controlled trials, such as sample size, inclusion and exclusion criteria, and methods of randomisation, as well as the challenges of complex interventions. Individual randomisation may not always be feasible or appropriate. For example, cluster randomisation is often used for trials of interventions directed at a practice or hospital team. ^{10 11} Randomised incomplete block designs have also been used to evaluate different approaches to promoting change in professional behaviour.¹²

It is often not possible to conceal allocation of treatment from the patient, practitioner, and researcher in complex intervention trials. The potential biases of unblinded trials therefore have to be taken into account. Dissimilar levels of patient commitment between intervention and control groups may cause differential dropout, making interpretation of results difficult. When patients have strong preferences, a preference trial design may be used; patients without strong preferences are randomised as usual but those with strong preferences receive their preferred treatment.¹³ The results of such trials can, however, be difficult to interpret.

The findings of trials of complex interventions are more generalisable if they are performed in the setting in which they are most likely to be implemented. Eligibility criteria must not lead to the exclusion of patientsfor example, on the grounds of agewho constitute a substantial portion of those to whom the intervention is likely to be offered when implemented in the health system. Poor recruitment to a trial can also raise doubts about generalisability.

Qualitative study of the processes of implementation of interventions in study arms of the main trial can further show the validity of findings.¹⁴

	Phase IV: promoting effective implementation

The purpose of the final phase is to examine the implementation of the intervention into practice, paying particular attention to the rate of uptake, the stability of the intervention, any broadening of subject groups, and the possible existence of adverse effects. As in the case of drug trials, this might be carried out by long term surveillance, although currently there is no established mechanism for funding such activities.

	Conclusions

Trials of complex interventions are of increasing importance because of the drive to provide the most cost effective health care. Although these trials pose substantial challenges to investigators, the use of an iterative phased approach that harnesses qualitative and quantitative methods should lead to improved study design, execution, and generalisability of results.

	Acknowledgments

We thank the participants at a MRC workshop on complex interventions for their contribution. This article represents the views of the authors and does not represent the official view of the Medical Research Council.

	Footnotes

Competing interests: None declared.

	References

1.	Friedli K, King MB. Psychological treatments and their evaluation. Int Rev Psychiatry 1998; 10: 123-126[CrossRef].
2.	Stephenson J, Imrie J. Why do we need randomised controlled trials to assess behavioural interventions? BMJ 1998; 316: 611-613[Free Full Text].
3.	Buchwald H. Surgical procedures and devices should be evaluated in the same way as medical therapy. Controlled Clinical Trials 1997; 18: 478-487[CrossRef][Medline].
4.	Collins English dictionary. London: Collins, 1979.
5.	Haines A, Iliffe S. Innovations in services and the appliance of science. BMJ 1995; 310: 815-816[Free Full Text].
6.	Shepperd S, Iliffe S. The effectiveness of hospital at home compared with in-patient hospital care: a systematic review. J Pub Health Med 1998; 20: 344-350[Abstract/Free Full Text].
7.	Haynes RB, Haines A. Barriers and bridges to evidence-based practice. BMJ 1998; 317: 273-276[Free Full Text].
8.	Fitzpatrick R, Davey C, Buxton MJ, Jones DR. Evaluating patient based outcome measures for use in clinical trials. Health Technology Assessment 1998; 2(14): 1-74[Medline].
9.	Mannheim L. Health services research clinical trials: issues in the evaluation of economic cost and benefits. Controlled Clinical Trials 1999; 19: 149-158.
10.	Edwards S, Braunholtz D, Stevens A, Lilford R. Ethical issues in the design and conduct of cluster RCTs. BMJ 1999; 318: 1407-1409[Free Full Text].
11.	Ukoumunne OC, Gulliford MC, Chinn S, Sterne JAC, Burney PGJ. Methods for evaluating area-wide and organisation-based interventions in health and health care: a systematic review. Health Technology Assessment 1999; 3(5): 1-110.
12.	Grimshaw J, Freemantle N, Wallace S, Russell I, Hurwitz B, Watt I, et al. Developing and implementing clinical practice guidelines. Qual Health Care 1995; 4: 55-64[Medline].
13.	Brewin CR, Bradley C. Patient preferences and randomised controlled trials. BMJ 1989; 289: 313-315.
14.	Bradley F, Wiles R, Kinmonth A, Mant D, Gantley M. Development and evaluation of complex interventions in health services research: case study of the Southampton heart integrated care project (SHIP). BMJ 1999; 318: 711-715[Free Full Text].

(Accepted 31 May 2000)

---

© BMJ 2000
CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Complex interventions or complex systems? Implications for health economic evaluation

Alan Shiell, Penelope Hawe, and Lisa Gold
BMJ 2008 336: 1281-1283. [Extract] [Full Text] [PDF]

Internal and external validity of cluster randomised trials: systematic review of recent trials

Sandra Eldridge, Deborah Ashby, Catherine Bennett, Melanie Wakelin, and Gene Feder
BMJ 2008 336: 876-880. [Abstract] [Full Text] [PDF]

Effectiveness of the diabetes education and self management for ongoing and newly diagnosed (DESMOND) programme for people with newly diagnosed type 2 diabetes: cluster randomised controlled trial

M J Davies, S Heller, T C Skinner, M J Campbell, M E Carey, S Cradock, H M Dallosso, H Daly, Y Doherty, S Eaton, C Fox, L Oliver, K Rantell, G Rayman, K Khunti on behalf of the Diabetes Education and Self Management for Ongoing and Newly Diagnosed Collaborative
BMJ 2008 336: 491-495. [Abstract] [Full Text] [PDF]

Structured education for people with type 2 diabetes

Sean F Dinneen
BMJ 2008 336: 459-460. [Extract] [Full Text] [PDF]

Support for self care for patients with chronic disease

Anne Kennedy, Anne Rogers, and Peter Bower
BMJ 2007 335: 968-970. [Extract] [Full Text] [PDF]

Realist review to understand the efficacy of school feeding programmes

Trisha Greenhalgh, Elizabeth Kristjansson, and Vivian Robinson
BMJ 2007 335: 858-861. [Extract] [Full Text] [PDF]

Designing and evaluating complex interventions to improve health care

Neil C Campbell, Elizabeth Murray, Janet Darbyshire, Jon Emery, Andrew Farmer, Frances Griffiths, Bruce Guthrie, Helen Lester, Phil Wilson, and Ann Louise Kinmonth
BMJ 2007 334: 455-459. [Extract] [Full Text] [PDF]

Strategies to prevent falls and fractures in hospitals and care homes and effect of cognitive impairment: systematic review and meta-analyses

David Oliver, James B Connelly, Christina R Victor, Fiona E Shaw, Anne Whitehead, Yasemin Genc, Alessandra Vanoli, Finbarr C Martin, and Margot A Gosney
BMJ 2007 334: 82. [Abstract] [Full Text] [PDF]

Self management for men with lower urinary tract symptoms: randomised controlled trial

Christian T Brown, Tet Yap, David A Cromwell, Lorna Rixon, Liz Steed, Kathleen Mulligan, Anthony Mundy, Stanton P Newman, Jan van der Meulen, and Mark Emberton
BMJ 2007 334: 25. [Abstract] [Full Text] [PDF]

Assessment of generalisability in trials of health interventions: suggested framework and systematic review

C Bonell, A Oakley, J Hargreaves, V Strange, and R Rees
BMJ 2006 333: 346-349. [Full Text] [PDF]

Balancing potential risks and benefits of using confidential data

Christina Davies and Rory Collins
BMJ 2006 333: 349-351. [Full Text] [PDF]

Process evaluation in randomised controlled trials of complex interventions

Ann Oakley, Vicki Strange, Chris Bonell, Elizabeth Allen, Judith Stephenson RIPPLE Study Team
BMJ 2006 332: 413-416. [Full Text] [PDF]

Evaluating and implementing new services

Ann McDonnell, Richard Wilson, and Steve Goodacre
BMJ 2006 332: 109-112. [Full Text] [PDF]

Complementary therapies and the NHS

Trevor Thompson and Gene Feder
BMJ 2005 331: 856-857. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• Sharkey, S., Sharples, A. (2008). From the Beginning: Negotiation in Community Evaluation. Evaluation 14: 363-380 [Abstract]  
• Busse, M. E, Khalil, H., Quinn, L., Rosser, A. E (2008). Physical Therapy Intervention for People With Huntington Disease. ptjournal 88: 820-831 [Abstract] [Full text]  
• Shiell, A., Hawe, P., Gold, L. (2008). Complex interventions or complex systems? Implications for health economic evaluation. BMJ 336: 1281-1283 [Full text]  
• van Bokhoven, M. A, Koch, H., Dinant, G.-J., Bindels, P. J., Grol, R. P., van der Weijden, T. (2008). Exploring the black box of change in improving test-ordering routines. Fam Pract 0: cmn022v1-cmn022 [Abstract] [Full text]  
• Brown, C, Hofer, T, Johal, A, Thomson, R, Nicholl, J, Franklin, B D, Lilford, R J (2008). An epistemology of patient safety research: a framework for study design and interpretation. Part 1. Conceptualising and developing interventions. Qual Saf Health Care 17: 158-162 [Abstract] [Full text]  
• Brown, C, Hofer, T, Johal, A, Thomson, R, Nicholl, J, Franklin, B D, Lilford, R J (2008). An epistemology of patient safety research: a framework for study design and interpretation. Part 2. Study design. Qual Saf Health Care 17: 163-169 [Abstract] [Full text]  
• Hiscock, A., Miller, S., Rothwell, J., Tallis, R. C., Pomeroy, V. M. (2008). Informing Dose-Finding Studies of Repetitive Transcranial Magnetic Stimulation to Enhance Motor Function: A Qualitative Systematic Review. Neurorehabil Neural Repair 22: 228-249 [Abstract]  
• Slade, M., Gask, L., Leese, M., McCrone, P., Montana, C., Powell, R., Stewart, M., Chew-Graham, C. (2008). Failure to improve appropriateness of referrals to adult community mental health services--lessons from a multi-site cluster randomized controlled trial. Fam Pract 25: 181-190 [Abstract] [Full text]  
• Sanderson, S. C., Humphries, S. E., Hubbart, C., Hughes, E., Jarvis, M. J., Wardle, J. (2008). Psychological and Behavioural Impact of Genetic Testing Smokers for Lung Cancer Risk: A Phase II Exploratory Trial. J Health Psychol 13: 481-494 [Abstract]  
• Eldridge, S., Ashby, D., Bennett, C., Wakelin, M., Feder, G. (2008). Internal and external validity of cluster randomised trials: systematic review of recent trials. BMJ 336: 876-880 [Abstract] [Full text]  
• Araujo-Soares, V., McIntyre, T., Sniehotta, F. F. (2008). Predicting changes in physical activity among adolescents: the role of self-efficacy, intention, action planning and coping planning. Health Educ Res 0: cyn005v1-cyn005 [Abstract] [Full text]  
• Heederik, D., van Rooy, F. (2008). Exposure assessment should be integrated in studies on the prevention and management of occupational asthma. Occup. Environ. Med. 65: 149-150 [Full text]  
• Armstrong, R., Waters, E., Moore, L., Riggs, E., Cuervo, L. G., Lumbiganon, P., Hawe, P. (2008). Improving the reporting of public health intervention research: advancing TREND and CONSORT. J Public Health (Oxf) 30: 103-109 [Abstract] [Full text]  
• Davies, M J, Heller, S, Skinner, T C, Campbell, M J, Carey, M E, Cradock, S, Dallosso, H M, Daly, H, Doherty, Y, Eaton, S, Fox, C, Oliver, L, Rantell, K, Rayman, G, Khunti, K, on behalf of the Diabetes Education and Self Manag, (2008). Effectiveness of the diabetes education and self management for ongoing and newly diagnosed (DESMOND) programme for people with newly diagnosed type 2 diabetes: cluster randomised controlled trial. BMJ 336: 491-495 [Abstract] [Full text]  
• Dinneen, S. F (2008). Structured education for people with type 2 diabetes. BMJ 336: 459-460 [Full text]  
• Eisenstein, E. L, Collins, R., Cracknell, B. S, Podesta, O., Reid, E. D, Sandercock, P., Shakhov, Y., Terrin, M. L, Sellers, M. A., Califf, R. M, Granger, C. B, Diaz, R. (2008). Sensible approaches for reducing clinical trial costs. Clin Trials 5: 75-84 [Abstract]  
• Beynon, S., Soares-Weiser, K., Woolacott, N., Duffy, S., Geddes, J. R. (2008). Psychosocial interventions for the prevention of relapse in bipolar disorder: systematic review of controlled trials. Br. J. Psychiatry 192: 5-11 [Abstract] [Full text]  
• Protheroe, J., Bower, P., Chew-Graham, C. (2007). The use of mixed methodology in evaluating complex interventions: identifying patient factors that moderate the effects of a decision aid. Fam Pract 24: 594-600 [Abstract] [Full text]  
• Kennedy, A., Rogers, A., Bower, P. (2007). Support for self care for patients with chronic disease. BMJ 335: 968-970 [Full text]  
• Beich, A., Gannik, D., Saelan, H., Thorsen, T. (2007). Screening and brief intervention targeting risky drinkers in danish general practice A pragmatic controlled trial. Alcohol Alcohol 42: 593-603 [Abstract] [Full text]  
• Greenhalgh, T., Kristjansson, E., Robinson, V. (2007). Realist review to understand the efficacy of school feeding programmes. BMJ 335: 858-861 [Full text]  
• Brereton, L., Carroll, C., Barnston, S. (2007). Interventions for adult family carers of people who have had a stroke: a systematic review. Clin Rehabil 21: 867-884 [Abstract]  
• Protheroe, J., Bower, P., Chew-Graham, C., Peters, T. J., Fahey, T. (2007). Effectiveness of a Computerized Decision Aid in Primary Care on Decision Making and Quality of Life in Menorrhagia: Results of the MENTIP Randomized Controlled Trial. Med Decis Making 27: 575-584 [Abstract]  
• Absetz, P., Valve, R., Oldenburg, B., Heinonen, H., Nissinen, A., Fogelholm, M., Ilvesmaki, V., Talja, M., Uutela, A. (2007). Type 2 Diabetes Prevention in the "Real World": One-year results of the GOAL Implementation Trial. Diabetes Care 30: 2465-2470 [Abstract] [Full text]  
• Boon, H., MacPherson, H., Fleishman, S., Grimsgaard, S., Koithan, M., Norheim, A. J., Walach, H. (2007). Evaluating Complex Healthcare Systems: A Critique of Four Approaches. Evid Based Complement Alternat Med 4: 279-285 [Abstract] [Full text]  
• Khan, N., Bower, P., Rogers, A. (2007). Guided self-help in primary care mental health: Meta-synthesis of qualitative studies of patient experience. Br. J. Psychiatry 191: 206-211 [Abstract] [Full text]  
• Seidl, R. O, Nusser-Muller-Busch, R., Hollweg, W., Westhofen, M., Ernst, A. (2007). Pilot study of a neurophysiological dysphagia therapy for neurological patients. Clin Rehabil 21: 686-697 [Abstract]  
• Crawford, M. J, Patterson, S. (2007). Arts therapies for people with schizophrenia: an emerging evidence base. Evid. Based Ment. Health 10: 69-70 [Full text]  
• Murtagh, M J, Thomson, R G, May, C R, Rapley, T, Heaven, B R, Graham, R H, Kaner, E F, Stobbart, L, Eccles, M P (2007). Qualitative methods in a randomised controlled trial: the role of an integrated qualitative process evaluation in providing evidence to discontinue the intervention in one arm of a trial of a decision support tool. Qual Saf Health Care 16: 224-229 [Abstract] [Full text]  
• Power, R., Langhaug, L., Cowan, F. (2007). "But there are no snakes in the wood": risk mapping as an outcome measure in evaluating complex interventions. Sex. Transm. Infect. 83: 232-236 [Abstract] [Full text]  
• Channon, S. J., Huws-Thomas, M. V., Rollnick, S., Hood, K., Cannings-John, R. L., Rogers, C., Gregory, J. W. (2007). A Multicenter Randomized Controlled Trial of Motivational Interviewing in Teenagers With Diabetes. Diabetes Care 30: 1390-1395 [Abstract] [Full text]  
• Murchie, P., Hannaford, P. C, Wyke, S., Nicolson, M. C, Campbell, N. C (2007). Designing an integrated follow-up programme for people treated for cutaneous malignant melanoma: a practical application of the MRC framework for the design and evaluation of complex interventions to improve health. Fam Pract 24: 283-292 [Abstract] [Full text]  
• Seenan, P., Long, M., Langhorne, P. (2007). Stroke Units in Their Natural Habitat: Systematic Review of Observational Studies. Stroke 38: 1886-1892 [Abstract] [Full text]  
• Feigin, V. L. (2007). Herbal Medicine in Stroke: Does It Have a Future?. Stroke 38: 1734-1736 [Full text]  
• Barrett, B., Byford, S. (2007). Collecting service use data for economic evaluation in DSPD populations: Development of the Secure Facilities Service Use Schedule. Br. J. Psychiatry 190: s75-s78 [Abstract] [Full text]  
• Campbell, N. C, Murray, E., Darbyshire, J., Emery, J., Farmer, A., Griffiths, F., Guthrie, B., Lester, H., Wilson, P., Kinmonth, A. L. (2007). Designing and evaluating complex interventions to improve health care. BMJ 334: 455-459 [Full text]  
• Scott, I. (2007). The evolving science of translating research evidence into clinical practice. Evid. Based Med. 12: 4-7 [Full text]  
• Taylor, S, Eldridge, S, Chang, Y-M, Sohanpal, R, Clarke, A (2007). Evaluating hospital at home and early discharge schemes for patients with an acute exacerbation of COPD. Chronic Respiratory Disease 4: 33-43 [Abstract]  
• Hoddinott, P., Pill, R., Chalmers, M. (2007). Health professionals, implementation and outcomes: reflections on a complex intervention to improve breastfeeding rates in primary care. Fam Pract 24: 84-91 [Abstract] [Full text]  
• Oliver, D., Connelly, J. B, Victor, C. R, Shaw, F. E, Whitehead, A., Genc, Y., Vanoli, A., Martin, F. C, Gosney, M. A (2007). Strategies to prevent falls and fractures in hospitals and care homes and effect of cognitive impairment: systematic review and meta-analyses. BMJ 334: 82-82 [Abstract] [Full text]  
• Brown, C. T, Yap, T., Cromwell, D. A, Rixon, L., Steed, L., Mulligan, K., Mundy, A., Newman, S. P, van der Meulen, J., Emberton, M. (2007). Self management for men with lower urinary tract symptoms: randomised controlled trial. BMJ 334: 25-25 [Abstract] [Full text]  
• Simons, L. (2007). Moving from collision to integration: Reflecting on the experience of mixed methods. Journal of Research in Nursing 12: 73-83 [Abstract]  
• Humphrey, C. (2006). Ways of seeing: Biomedical perspectives on the social world. JRSM 99: 602-606 [Full text]  
• Bower, P., Gilbody, S., Richards, D., Fletcher, J., Sutton, A. (2006). Collaborative care for depression in primary care: Making sense of a complex intervention: systematic review and meta-regression. Br. J. Psychiatry 189: 484-493 [Abstract] [Full text]  
• SLADE, M., McCRONE, P., KUIPERS, E., LEESE, M., CAHILL, S., PARABIAGHI, A., PRIEBE, S., THORNICROFT, G. (2006). Use of standardised outcome measures in adult mental health services: Randomised controlled trial. Br. J. Psychiatry 189: 330-336 [Abstract] [Full text]  
• Wilson, B. J, Torrance, N., Mollison, J., Watson, M S., Douglas, A., Miedzybrodzka, Z., Gordon, R., Wordsworth, S., Campbell, M., Haites, N., Grant, A. (2006). Cluster randomized trial of a multifaceted primary care decision-support intervention for inherited breast cancer risk. Fam Pract 23: 537-544 [Abstract] [Full text]  
• Thomas, P. (2006). General medical practitioners need to be aware of the theories on which our work depend.. Ann Fam Med 4: 450-454 [Abstract] [Full text]  
• Patton, G. C., Bond, L., Carlin, J. B., Thomas, L., Butler, H., Glover, S., Catalano, R., Bowes, G. (2006). Promoting Social Inclusion in Schools: A Group-Randomized Trial of Effects on Student Health Risk Behavior and Well-Being. Am. J. Public Health 96: 1582-1587 [Abstract] [Full text]  
• Bonell, C, Oakley, A, Hargreaves, J, Strange, V, Rees, R (2006). Assessment of generalisability in trials of health interventions: suggested framework and systematic review. BMJ 333: 346-349 [Full text]  
• Wammes, B., Breedveld, B., Kremers, S., Brug, J. (2006). The 'balance intervention' for promoting caloric compensatory behaviours in response to overeating: a formative evaluation. Health Educ Res 21: 527-537 [Abstract] [Full text]  
• Stronks, K., Mackenbach, J. P. (2006). Evaluating the effect of policies and interventions to address inequalities in health: lessons from a Dutch programme. Eur J Public Health 16: 346-353 [Abstract] [Full text]  
• Strange, V., Allen, E., Oakley, A., Bonell, C., Johnson, A., Stephenson, J., The Ripple Study Team, (2006). Integrating Process with Outcome Data in a Randomized Controlled Trial of Sex Education. Evaluation 12: 330-352 [Abstract]  
• Stange, K. C., Crabtree, B. F., Miller, W. L. (2006). Publishing Multimethod Research. Ann Fam Med 4: 292-294 [Full text]  
• Rosen, L., Manor, O., Engelhard, D., Zucker, D. (2006). In Defense of the Randomized Controlled Trial for Health Promotion Research. Am. J. Public Health 96: 1181-1186 [Abstract] [Full text]  
• Baker, G R (2006). Strengthening the contribution of quality improvement research to evidence based health care.. Qual Saf Health Care 15: 150-151 [Full text]  
• Underwood, M. R., Harding, G., Klaber Moffett, J., in collaboration with the UK BEAM trial team, (2006). Patient perceptions of physical therapy within a trial for back pain treatments (UK BEAM) [ISRCTN32683578]. Rheumatology (Oxford) 45: 751-756 [Abstract] [Full text]  
• White, M., Bush, J., Kai, J., Bhopal, R., Rankin, J. (2006). Quitting smoking and experience of smoking cessation interventions among UK Bangladeshi and Pakistani adults: the views of community members and health professionals. J. Epidemiol. Community Health 60: 405-411 [Abstract] [Full text]  
• Salisbury, C., Fahey, T. (2006). Overcoming clinical inertia in the management of hypertension.. CMAJ 174: 1285-1286 [Full text]  
• (2006). EAPC Abstracts. Palliat Med 20: 229-379  
• TYRER, P. (2006). . Br. J. Psychiatry 188: 400-400 [Full text]  
• Zimny, N. J (2006). Invited Commentary. ptjournal 86: 446-447 [Full text]  
• Vreeman, D. J, Taggard, S. L, Rhine, M. D, Worrell, T. W (2006). Author Response. ptjournal 86: 448-449 [Full text]  
• Oakley, A., Strange, V., Bonell, C., Allen, E., Stephenson, J., RIPPLE Study Team, (2006). Process evaluation in randomised controlled trials of complex interventions.. BMJ 332: 413-416 [Full text]  
• Paxton, R., Kennedy, P., Carpenter, J. (2006). Research in the real world. Psychiatr. Bull. 30: 43-45 [Full text]  
• McDonnell, A., Wilson, R., Goodacre, S. (2006). Evaluating and implementing new services. BMJ 332: 109-112 [Full text]  
• Hardeman, W., Sutton, S., Griffin, S., Johnston, M., White, A., Wareham, N. J., Kinmonth, A. L. (2005). A causal modelling approach to the development of theory-based behaviour change programmes for trial evaluation. Health Educ Res 20: 676-687 [Abstract] [Full text]  
• Thompson, T., Feder, G. (2005). Complementary therapies and the NHS. BMJ 331: 856-857 [Full text]  
• Lingard, L, Espin, S, Rubin, B, Whyte, S, Colmenares, M, Baker, G R, Doran, D, Grober, E, Orser, B, Bohnen, J, Reznick, R (2005). Getting teams to talk: development and pilot implementation of a checklist to promote interprofessional communication in the OR. Qual Saf Health Care 14: 340-346 [Abstract] [Full text]  
• Sanderson, S. C., Wardle, J. (2005). Will Genetic Testing for Complex Diseases Increase Motivation to Quit Smoking? Anticipated Reactions in a Survey of Smokers. Health Educ Behav 32: 640-653 [Abstract]  
• Leurs, M. T. W., Schaalma, H. P., Jansen, M. W. J., Mur-Veeman, I. M., St. Leger, L. H., de Vries, N. (2005). Development of a collaborative model to improve school health promotion in the Netherlands. HEALTH PROMOT INT 20: 296-305 [Abstract] [Full text]  
• Wade, D. T (2005). Describing rehabilitation interventions. Clin Rehabil 19: 811-818 [Abstract]  
• Cartwright, T., Torr, R. (2005). Making Sense of Illness: The Experiences of Users of Complementary Medicine. J Health Psychol 10: 559-572 [Abstract]  
• Leviton, L. C., Horbar, J. D. (2005). Cluster Randomized Trials for the Evaluation of Strategies Designed to Promote Evidence-Based Practice in Perinatal and Neonatal Medicine. The ANNALS of the American Academy of Political and Social Science 599: 94-114 [Abstract]  
• Marsden, J, Greenwood, R (2005). Physiotherapy after stroke: define, divide and conquer. J. Neurol. Neurosurg. Psychiatry 76: 465-466 [Full text]  
• Greenhalgh, T., Collard, A., Begum, N. (2005). Sharing stories: complex intervention for diabetes education in minority ethnic groups who do not speak English. BMJ 330: 628- [Abstract] [Full text]  
• Rogers, A., Kennedy, A., Nelson, E., Robinson, A. (2005). Uncovering the Limits of Patient-Centeredness: Implementing a Self-Management Trial for Chronic Illness. Qual Health Res 15: 224-239 [Abstract]  
• Young, R. J., Taylor, J., Friede, T., Hollis, S., Mason, J. M., Lee, P., Burns, E., Long, A. F., Gambling, T., New, J. P., Gibson, J. M. (2005). Pro-Active Call Center Treatment Support (PACCTS) to Improve Glucose Control in Type 2 Diabetes: A randomized controlled trial. Diabetes Care 28: 278-282 [Abstract] [Full text]  
• Byng, R., Norman, I., Redfern, S. (2005). Using Realistic Evaluation to Evaluate a Practice-level Intervention to Improve Primary Healthcare for Patients with Long-term Mental Illness. Evaluation 11: 69-93 [Abstract]  
• Woolfe, J., Stockley, L. (2005). Nutrition Health Promotion in Schools in the UK: Learning from Food Standards Agency Funded Schools Research. Health Education Journal 64: 229-246 [Abstract]  
• Griffin, S. J., Kinmonth, A.-L., Veltman, M. W. M., Gillard, S., Grant, J., Stewart, M. (2004). Effect on Health-Related Outcomes of Interventions to Alter the Interaction Between Patients and Practitioners: A Systematic Review of Trials. Ann Fam Med 2: 595-608 [Abstract] [Full text]  
• Bateman, A. W., Tyrer, P. (2004). Services for personality disorder: organisation for inclusion. Adv. Psychiatr. Treat. 10: 425-433 [Abstract] [Full text]  
• Oakley, A., Strange, V., Stephenson, J., Forrest, S., Monteiro, H. (2004). Evaluating Processes: A Case Study of a Randomized Controlled Trial of Sex Education. Evaluation 10: 440-462 [Abstract]  
• Power, R., Langhaug, L. F., Nyamurera, T., Wilson, D., Bassett, M. T., Cowan, F. M. (2004). Developing complex interventions for rigorous evaluation--a case study from rural Zimbabwe. Health Educ Res 19: 570-575 [Abstract] [Full text]  
• Tyrer, P. (2004). Psychiatry in the future: The future of mental health services research. Psychiatr. Bull. 28: 353-354 [Full text]  
• Hawe, P., Shiell, A., Riley, T., Gold, L. (2004). Methods for exploring implementation variation and local context within a cluster randomised community intervention trial. J. Epidemiol. Community Health 58: 788-793 [Abstract] [Full text]  

Rapid Responses:

Read all Rapid Responses

FOCUSING ON BEING PRECISELY WRONG RATHER THAN VAGUELY RIGHT

David Kernick
bmj.com, 2 Oct 2000 [Full text]