BMJ 1999;318:711-715 ( 13 March )

General Practice

Development and evaluation of complex interventions in health services research: case study of the Southampton heart integrated care project (SHIP)

Fiona Bradley, lecturer, a Rose Wiles, senior research fellow, b Ann-Louise Kinmonth, professor, c David Mant, professor, d Madeleine Gantley, senior lecturer, e for the SHIP Collaborative Group.

a Department of Community Health and General Practice, Trinity College, Dublin 2, Ireland, b Health Research Unit, School of Occupational Therapy and Physiotherapy, University of Southampton, Southampton SO17 1BJ, c General Practice and Primary Care Research Unit, Institute of Public Health, Cambridge CB2 2SR, d Department of Primary Health Care, Oxford University, Institute of Health Sciences, Headington, Oxford OX3 7LF, e Department of General Practice and Primary Care, St Bartholomew's and Royal London School of Medicine and Dentistry, Queen Mary and Westfield College, London E1 4NS

Correspondence to: Professor Kinmonth alk25{at}medschl.cam.ac.uk

The development and evaluation of complex interventions within randomised controlled designs is a challenging area in health services research. The process usually entails a pilot phase to confirm the feasibility and potential effectiveness of the design before embarking on large and costly trials. However, the focus is often more on the study design and measures than on the theoretical base and extent to which the intervention can be appropriately applied. In this article, we use a case study to describe an approach to pilot work that addresses this gap.


Summary points

Interventions are often defined pragmatically and lack any clear theoretical basis, which limits generalisability
Implementation is rarely described, which limits understanding of why an intervention is or is not locally successful
Integration of qualitative methods within pilot trials can help interpret the quantitative result by clarifying process and testing theory
This approach defines three levels of understanding: the evidence and theory which inform the intervention, the tasks and processes involved in applying the theoretical principles, and people with whom, and context within which, the intervention is operationalised
A case study shows how this novel method of programme development and evaluation can be applied



    Background
Top
Background
Case study
Comment
References

Compared with drug trials or trials of surgical procedures, the design and development of a health service intervention is highly complex. In practice such interventions are often defined pragmatically, according to local circumstance, rather than building on any specific theoretical approach.1 Even if an approach or technology can be clearly grounded in theory and evidence, it must still be operationalised and evaluated among specific practitioners and patients. There is thus a tension between evaluation of complex interventions and generalisability of results. Randomised trials alone can not tell us why an intervention was or was not successful, or whether the theory and evidence informing the intervention were appropriate or needed revision.

To clarify this, we propose three levels for defining a complex intervention: the evidence and theory which inform the intervention, the tasks and processes involved in applying the theoretical principles, and the people with whom, and context within which, the intervention is operationalised.

    Case study
Top
Background
Case study
Comment
References

The Southampton heart integrated care programme brought together the principal researchers from the family heart and OXCHECK studies---two trials of primary care interventions for risk reduction in cardiovascular disease. 2 3 Debate after publication of the trials highlighted the importance of focusing on the design and process of interventions as much as on the evaluation of outcomes, to understand the process of application of an intervention in a way that would allow success to be understood and replicated, and unsuccessful approaches to be abandoned.4

The comparatively disappointing results from primary preventive programmes for cardiovascular disease also moved the focus of research to secondary prevention programmes. The Southampton heart integrated care programme was such a programme. It was led by specialist nurses who coordinated and supported follow up care in general practice of patients who had a hospital diagnosis of myocardial infarction or angina.

Definition of levels
The table describes the three levels of intervention as applied to the Southampton heart integrated care programme. Level 1 summarises the theory and evidence underpinning, in this case, the choice of a target population, service provision, and management of behaviour change within the programme; it deals with the gap between evidence of efficacy and provision of treatments for people with established ischaemic heart disease.4-7 It also deals with best practice in enabling behaviour change among practitioners and patients, using guidelines and psychological models.19-22 Level 2 defines the essential tasks and processes required for operationalisation in these areas, at a generalisable level. Level 3 defines who would do what locally---elements which are specific to a local setting.


                              
View this table:
[in this window]
[in a new window]
 

Defining a complex intervention at three levels for health services research: case of secondary prevention of ischaemic heart disease

Quantitative approach
The pilot trial was designed to assess the impact of the programme on lifestyle and cardiovascular risk. The two arms of the trial compared the new approach with the usual care of patients with myocardial infarction or angina.8 Overall, 597 adult patients (from all 67 general practices in Southampton and south west Hampshire) with myocardial infarction or angina were randomised to intervention (33 practices) or control (34) groups. Follow up was 90% complete. The intervention increased follow up in general practice at 4 months and 1 year, and improved attendance for rehabilitation. No important difference was, however, observed between the intervention and control groups in any of the primary outcome measures of cardiovascular risk.8

Integration of a qualitative approach
The trial quantified the effect on contacts in primary care, but not the quality of those contacts. It did not allow interpretation of negative findings at the three levels of individual people, processes, or theory. To understand how the intervention was delivered, whether some of the elements seen as generalisable by the research team were particularly important or problematic, and the appropriateness of underlying theory, a qualitative approach is necessary.

The methods and results from the qualitative part of the programme have been reported in detail. 9 10 Twenty five patients from the intervention group and, where appropriate, their partners were interviewed in depth. Participants were selected on the basis of maximum variety sampling.11 Patients were interviewed twice: shortly after discharge from hospital and again around 3 months later. In addition, 22 practitioners (hospital and practice nurses) involved with the intervention were interviewed or participated in a focus group once. A semistructured schedule of topics was used to guide the interviews and focus groups. Interviews and focus groups were audiotaped, fully transcribed, and subject to thematic analysis to identify emergent themes. Analysis was based on the grounded theory approach, and preliminary analysis of early interviews was used to inform further data collection and analysis.12

Explanation of numerical results

Level 3: people and context
Early data from interviews were used to refine the programme. To optimise implementation of the approach, the findings were fed back to the development group 4 months into the 18 month intervention. Initial analyses concentrated on the experiences and understanding that patients, their partners, and providers had of the role of the liaison nurse, primary care team, and rehabilitation services. The analyses examined the way these individuals perceived the intervention as being implemented. This phase of the analysis showed that patients were confused about the nature of the rehabilitation programme, that progression through the system was often slower than practitioners led them to expect, and that conflicting messages about the need for follow up were sometimes provided by primary and secondary care.

These findings were used to optimise the intervention. One improvement involved each specialist liaison nurse recruiting patients from specific practices for which they were solely responsible, rather than from all practices as before. This change encouraged communication between liaison nurses and practice nurses and between practice nurses and patients.

Level 2: tasks and processes
The later in-depth phase of the qualitative analysis was concerned with the interviewees' experiences, perceptions, and understandings over time after myocardial infarction or angina was diagnosed. The purpose of this analysis was to understand in more detail the tasks and processes needed to successfully operationalise the intervention. This analysis identified issues that may clarify the negative findings of the programme. For example, the focus groups for practice nurses suggested that in order for them to effectively follow up patients with established heart disease, greater sophistication in specification of essential tasks and processes was required than we had anticipated (box).


Suggestions from focus groups with practice nurses, about changes to their role in follow up of patients with ischaemic heart disease

1. Status within the primary healthcare team must be developed

2. Training must address knowledge and skills of cardiac assessment, and drug use and adherence, as well as facilitating behaviour change in relation to lifestyle

3. Opportunity must be given for nurses to give continuity of care

4. Improved integration at the primary-secondary care interface needs to take place, with secondary care staff clearly recognising the role of the practice nurse

Level 1: testing theory
The second purpose of the in-depth analysis was to confirm or question the evidence and theory on which the intervention was based. At this level, the qualitative inquiry also identified potentially important insights. For example, the findings suggest that understanding of heart attack as an acute but short term event will inhibit the adoption of subsequent long term lifestyle change, and that the intervention failed to address this possibility. There were several reasons. Initially patients described their astonishment at surviving a heart attack, which they had previously understood to be a fatal event, and therefore defined their own event as necessarily mild. In the period immediately after the heart attack, the information provided to patients by practitioners apparently encouraged this view of heart attack as a self limited episode from which complete recovery was probable, with little reference to the continuing underlying disease processes. At this stage lifestyle change seemed to be understood by patients as being linked to recovery in the short term rather than a long term preventive measure.

At a later stage patients' understandings about heart attack were subject to change particularly in cases where experience of recovery did not reflect the information given. For example, information encouraged patients to believe that they would be able to have sex in 2-3 weeks, would be back to work in 6 weeks, and would be back to normal within 3 months, when this was often not the case.13 At this later stage, faith could be lost in "official" information from practitioners and evidence drawn instead from personal experience. Conflict between the two was associated with questioning the explanatory power of information from practitioners, and viewing the adoption of long term lifestyle change as action that would not guarantee protection from a further heart attack.

The programme thus found that patients' understandings of heart attack are closely linked to their attitudes to the potential of lifestyle change to keep them well. The failure of the intervention to acknowledge that the occurrence of heart attack, the severity of heart attack, and the natural history of recovery from heart attack cannot be accounted for entirely by lifestyle seemed to be a central feature in patients' understanding.9

    Comment
Top
Background
Case study
Comment
References

It is increasingly recognised that using qualitative research methods can "reach the parts other methods cannot reach."14 Qualitative methods can be used on their own as a preliminary to a quantitative study in order, for example, to establish meaningful wording for a questionnaire or to develop the elements of an intervention. 15 16 They can be used to explain findings after quantitative research has been completed.17 Quantitative and qualitative methods are also used in parallel with substantive trials. 18 23 24 In the pilot trial of the Southampton heart integrated care programme, we have moved a step further by explicitly integrating qualitative methods within a pilot trial design. We argue here that parallel application of qualitative methods in a pilot trial can contribute significantly and efficiently to both optimising and evaluating a new health services intervention.

The pilot randomised controlled trial tests a hypothesis formulated from existing theory and evidence before data collection, examines inputs (resource use) and outcome (effect size), and provides evidence of whether the approach is feasible within a specific locality and worth substantive evaluation. Complementary to this, the qualitative research is concerned with the perspectives of the people involved in delivering and receiving the intervention, and the context in which the data are produced. Analysis of these phenomena can provide information about the process of implementing an intervention and can lead to suggestions about logistical changes needed to optimise the completion of tasks and processes within a local context.

There is always a learning curve in applying new interventions, but it is usually hidden. Our approach makes it explicit and formalises it through systematic feedback, allowing both description and optimisation of the application of an intervention at local level. This begins to move intervention development in the direction of what industrialists call "evolutionary operations."25 This process involves performing rolling analyses over time---integrating both quantitative and qualitative findings to systematically optimise a production process.

At the deeper level of analysis for emergent themes, qualitative research can help understanding of the process whereby particular outcomes come about. It can thus enable a more sophisticated definition of what needs doing, as was shown with the practice nurses.9 It can also examine and test the theoretical basis of an intervention and question or affirm the principles on which the tasks and processes have been based. In this case the qualitative analysis raised questions about the information on natural history that is currently presented to patients by practitioners. It may be that in an effort to minimise fears and anxieties, practitioners are inadvertently providing an over- optimistic view of the natural history of convalescence for some patients and minimising the chronic nature of the underlying disease. These theoretical ideas are open to future hypothesis testing.

In summary, an integrated quantitative and qualitative approach to developing and evaluating complex interventions in health service research is both efficient and generalisable. Within one pilot study it formalises the usually hidden learning curve of implementation and optimisation. It allows better judgment of transferability of potentially effective programmes to other settings for confirmatory trials. It helps interpret quantitative findings, and questions underlying theory and assumptions to better inform future hypotheses and intervention designs.

    Acknowledgments

We thank the patients and staff of the participating general practices. The Southampton heart integrated care programme was coordinated by the Primary Medical Care Group, University of Southampton in collaboration with: the Institute for Health Research and Development and the Institute for Health Policy Studies, University of Southampton; General Practice and Primary Care Research Unit, University of Cambridge; Department of Medical Statistics and Evaluation, Imperial College School of Medicine, London; Medical Statistics Unit, London School of Hygiene and Tropical Medicine; Health Economics Research Group, Brunel University; Department of Psychology, University of St Andrews; Department of Cardiac Medicine, National Heart and Lung Institute; and Department of Community Health and General Practice, Trinity College, Dublin. Members of the Southampton Heart Integrated Care Programme Collaborative Group are DA Wood (steering group chair), FB (former medical coordinator), M Buxton, A Davies, K Done, K Enright, M Johnston, D Johnston, K Jolly (medical coordinator), A-LK, DM (principal investigator), S Sharp, H Smith, V Speller, S Thompson, D Waller, RW, and L Wright. Members of the parallel qualitative research group are A-LK (chair), M Blaxter, MG, J Robison, A Spackman, and RW.

Contributors: FB led the writing group and was lead trial executive while she worked in Southampton. RW carried out and wrote up the qualitative research on which the paper depends. A-LK chaired the Qualitative Research Group and was responsible for articulating the three levels of definition of the programme; she will act as guarantor for the paper. DM chaired the trial group and MG contributed to the design and process of the qualitative research and to the idea of writing up the development and evaluation of the complex intervention. All the authors have contributed to the drafting and critical review of the paper on behalf of the Southampton Heart Integrated Care Programme Collaborative Group.

Funding: Research and development national programme grant from the National Health Service Executive, with service support from Southampton and South West Hampshire Health Authority. RW was in receipt of a NHS South and West Region research and development research training fellowship and FB was in receipt of a European Union research training fellowship. Subsequent development of the ideas in this paper were not externally funded.

Competing interests: None declared.

    References
Top
Background
Case study
Comment
References

  1. Shepperd S, Jenkinson C, Morgan P. Randomised controlled trials and health services research (letter). BMJ 1995; 310: 125-126[Free Full Text].
  2. Family Heart Study Group. Randomised controlled trial evaluating cardiovascular screening and intervention in general practice: principal results of British family heart study. BMJ 1994; 308: 313-320[Abstract/Free Full Text].
  3. Imperial Cancer Research Fund OXCHECK Study Group. Effectiveness of health checks conducted by nurses in primary care: final results of the OXCHECK study. BMJ 1995; 310: 1099-1104[Abstract/Free Full Text].
  4. King's Fund. The way forward following the publication of the OXCHECK and FHS studies. London: King's Fund , 1994.
  5. Action on Secondary Prevention through Intervention to Reduce Events Steering Group. A British Cardiac Society survey of the potential for the secondary prevention of coronary disease: ASPIRE (Action on Secondary Prevention through Intervention to Reduce Events) principal results. Heart 1996; 75: 334-342[Abstract/Free Full Text].
  6. Whitford D, Southern A. Audit of secondary prophylaxis after myocardial infarction. BMJ 1994; 309: 1268-1269[Free Full Text].
  7. Bradley F, Morgan S, Smith H, Mant D. Preventive care for patients following myocardial infarction. Fam Pract 1997; 14: 220-226[Abstract/Free Full Text].
  8. Southampton Heart Integrated Care Programme Collaborative Group. Randomised controlled trial of follow up care in general practice of patients with myocardial infarction and angina: final results of the Southampton heart integrated care project (SHIP). BMJ 1999; 318: 706-711[Abstract/Free Full Text].
  9. Wiles R. Patients' perceptions of heart attack. Soc Sci Med 1998; 46: 1477-1486.
  10. Wiles R. Empowering practice nurses in the follow-up of patients with established heart disease: lessons from patients' experiences. J Adv Nurs 1997; 26: 729-735[Medline].
  11. Patton M. Qualitative evaluation and research methods. Newbury Park, CA: Sage , 1990.
  12. Strauss A, Corbin J. Basics of qualitative research: grounded theory procedures and techniques. London: Sage , 1990.
  13. Southampton heart integrated care programme. Information for patients. Southampton: Primary Medical Care Group, University of Southampton , 1994.
  14. Pope C, Mays N. Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. BMJ 1995; 311: 42-45[Free Full Text].
  15. Johnson A, Wadsworth J, Wellings K, Field J. Sexual attitudes and lifestyles. Oxford: Blackwell Scientific , 1994.
  16. Little P, Williamson I, Warner G, Gould C, Gantley M, Kinmonth A-L. Open randomised trial of prescribing strategies in managing sore throat. BMJ 1997; 314: 722-777[Abstract/Free Full Text].
  17. Bloor MJ, Venters GA, Samphier ML. Geographical variation in the incidence of operations on the tonsils and adenoids: an epidemiological and sociological investigation. J Laryngol Otology 1976; 92: 791-801, 883-95.
  18. Brannen J, Moss P. Managing mothers: dual earner households after maternity leave. London: Unwin Hyman , 1991.
  19. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet 1993; 342: 1317-1322[Medline].
  20. Schwarzer R. Self-efficacy in the adoption and maintenance of health behaviours: theoretical approaches and a new model. In: Schwarzer R, ed. Self-efficacy: thought control of action. London: Hemisphere, 1992:217-243.
  21. Prochaska JO, DiClemente CC. Stages and processes of self-change of smoking: toward an integrative model of change. J Consult Clin Psychol 1983; 51: 390-395[Medline].
  22. Rollnick S, Heather N, Bell A. Negotiating behaviour change in medical settings: the development of brief motivational interviewing. J Ment Health 1992; 1: 25-37.
  23. Oakley A. Social support and motherhood. Oxford: Blackwell , 1992.
  24. Stott NCH, Rollnick S, Rees M, Pill RM. Innovation in clinical method, diabetes care and negotiating skills. Fam Pract 1995; 12: 413-418[Abstract/Free Full Text]
  25. Box GEP. Evolutionary operation: a method for increasing industrial productivity. Appl Stat 1956; vi(2): 81-101.

(Accepted 10 February 1999)

© BMJ 1999
Add to CiteULike CiteULike   Add to Complore Complore   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?

Relevant Articles

Process evaluation in randomised controlled trials of complex interventions
Ann Oakley, Vicki Strange, Chris Bonell, Elizabeth Allen, Judith Stephenson RIPPLE Study Team
BMJ 2006 332: 413-416. [Full Text] [PDF]

Secondary prevention of heart disease needs to be systematic
BMJ 1999 318: 0. [Full Text]

This article has been cited by other articles:

  • Ononeze, V., Murphy, A.W., MacFarlane, A., Byrne, M., Bradley, C. (2009). Expanding the value of qualitative theories of illness experience in clinical practice: a grounded theory of secondary heart disease prevention. Health Educ Res 24: 357-368 [Abstract] [Full text]  
  • O'Cathain, A. (2009). Editorial: Mixed Methods Research in the Health Sciences: A Quiet Revolution. Journal of Mixed Methods Research 3: 3-6  
  • Murtagh, M J, Thomson, R G, May, C R, Rapley, T, Heaven, B R, Graham, R H, Kaner, E F, Stobbart, L, Eccles, M P (2007). Qualitative methods in a randomised controlled trial: the role of an integrated qualitative process evaluation in providing evidence to discontinue the intervention in one arm of a trial of a decision support tool. Qual Saf Health Care 16: 224-229 [Abstract] [Full text]  
  • Murchie, P., Hannaford, P. C, Wyke, S., Nicolson, M. C, Campbell, N. C (2007). Designing an integrated follow-up programme for people treated for cutaneous malignant melanoma: a practical application of the MRC framework for the design and evaluation of complex interventions to improve health. Fam Pract 24: 283-292 [Abstract] [Full text]  
  • Scott, I. (2007). The evolving science of translating research evidence into clinical practice. Evid. Based Med. 12: 4-7 [Full text]  
  • Bower, P., Gilbody, S., Richards, D., Fletcher, J., Sutton, A. (2006). Collaborative care for depression in primary care: Making sense of a complex intervention: systematic review and meta-regression. Br. J. Psychiatry 189: 484-493 [Abstract] [Full text]  
  • Ononeze, V., Murphy, A. W, Byrne, M., Bradley, C., Macfarlane, A. (2006). Patients and health professionals' perspectives on the sociocultural influences on secondary cardiac behaviour: a qualitative study of the implications in policy and practice. Fam Pract 23: 587-596 [Abstract] [Full text]  
  • Underwood, M. R., Harding, G., Klaber Moffett, J., in collaboration with the UK BEAM trial team, (2006). Patient perceptions of physical therapy within a trial for back pain treatments (UK BEAM) [ISRCTN32683578]. Rheumatology (Oxford) 45: 751-756 [Abstract] [Full text]  
  • Oakley, A., Strange, V., Bonell, C., Allen, E., Stephenson, J., RIPPLE Study Team, (2006). Process evaluation in randomised controlled trials of complex interventions.. BMJ 332: 413-416 [Full text]  
  • Shaw, S., Boynton, P. M, Greenhalgh, T. (2005). Research governance: where did it come from, what does it mean?. JRSM 98: 496-502 [Abstract] [Full text]  
  • Rogers, A., Kennedy, A., Nelson, E., Robinson, A. (2005). Uncovering the Limits of Patient-Centeredness: Implementing a Self-Management Trial for Chronic Illness. Qual Health Res 15: 224-239 [Abstract]  
  • Rowlands, G., Sims, J., Kerry, S. (2005). A lesson learnt: the importance of modelling in randomized controlled trials for complex interventions in primary care. Fam Pract 22: 132-139 [Abstract] [Full text]  
  • Hawe, P., Shiell, A., Riley, T., Gold, L. (2004). Methods for exploring implementation variation and local context within a cluster randomised community intervention trial. J. Epidemiol. Community Health 58: 788-793 [Abstract] [Full text]  
  • Aldrich, R., Kemp, L., Williams, J. S., Harris, E., Simpson, S., Wilson, A., McGill, K., Byles, J., Lowe, J., Jackson, T. (2003). Using socioeconomic evidence in clinical practice guidelines. BMJ 327: 1283-1285 [Full text]  
  • Lancaster, T., Moher, M. (2002). Review: multidisciplinary coronary heart disease management programmes improve the process of care and reduce hospital admissions. Evid. Based Med. 7: 62-62 [Full text]  
  • Thomas, P., Griffiths, F., Kai, J., O'Dwyer, A. (2001). Networks for research in primary health care. BMJ 322: 588-590 [Full text]  
  • Campbell, M., Fitzpatrick, R., Haines, A., Kinmonth, A. L., Sandercock, P., Spiegelhalter, D., Tyrer, P. (2000). Framework for design and evaluation of complex interventions to improve health. BMJ 321: 694-696 [Full text]  
  • Wilson, S., Delaney, B. C, Roalfe, A., Roberts, L., Redman, V., Wearn, A. M, Hobbs, F D R. (2000). Randomised controlled trials in primary care: case study. BMJ 321: 24-27 [Full text]  
  • Rogers, S., Humphrey, C., Nazareth, I., Lister, S., Tomlin, Z., Haines, A. (2000). Designing trials of interventions to change professional practice in primary care: lessons from an exploratory study of two change strategies. BMJ 320: 1580-1583 [Full text]  
  • Feder, G., Griffiths, C., Foster, G., Ahmed, S., Maclaren, D., Carter, Y., Premaratne, U N, Sterne, J A C, Burney, P G J, Marks, G B, Webb, J R (2000). Greenwich asthma study. BMJ 320: 580-580 [Full text]  
  • Grande, G E, Todd, C J (2000). Why are trials in palliative care so difficult?. Palliat Med 14: 69-74  
  • Robson, J., Mant, D. (1999). Follow up care in general practice of patients with myocardial infarction and angina. BMJ 319: 380-380 [Full text]  
Access jobs at BMJ Careers
Whats new online at Student BMJ