Evaluation of complex interventions
Hawe et al.’s paper on the evaluation of complex interventions makes
a valuable contribution to the discourse on conventional evaluation
methodologies for complex interventions (1, 2, 3). We support their
argument that the way in which complex interventions have been
conceptualised in the past is not conducive to their effectiveness or
their (meaningful) evaluation but disagree with their emphasis on the RCT.
Hawe et al. suggest that if complex interventions are standardised by
their functions then the RCT remains a legitimate evaluation technique.
They say contextual-adaptation increases effectiveness of complex
interventions, but this is difficult to reconcile with their support for
the RCT to demonstrate such effectiveness. RCTs align the experimental and
control groups and disregard the social context in which programmes are
applied. Will this crucial account of context not be lost with an RCT, no
matter how the intervention is standardised? This may have been one reason
why marginal or no effects are seen from RCTs of complex interventions,
some of which Hawe et al. refer to (2, 3).
The examples of evaluation of complex interventions from the MRC and
WHO are interesting and useful. However, we were surprised at the lack of
reference to the wider body of literature in this field which has observed
the limitations of the experimental and quasi-experimental approach (3,
4). Hawe et al. make no reference to pluralism, where a range of
approaches such as qualitative methods, case studies and ecological
analyses can be used.
Hawe et al. highlight the consequences of evidence hierarchies
overlooking certain areas where effectiveness is not commonly demonstrated
by RCTs. However, they then find a way to continue using the RCT for the
evaluation of complex interventions. The complexity of the intervention
is a necessary influence on the evaluation technique. We are concerned
that the RCT will be maintained as the only acceptable form of evaluation
for any intervention.
Lucia Dambruoso, Emma Pitchforth, Julia Hussein on behalf of the
Dugald Baird Centre for Research on Women’s Health Journal Club,
University of Aberdeen.
1. Hawe P, Sheill A, Riley T. Complex interventions: how "out of
control" can a randomised controlled trial be? British Medical Journal, 26
June 2004; 328: 1561-1563.
2. Medical Research Council. A framework for the development and
evaluation of randomised controlled trials for complex interventions to
improve health. London: MRC, 2000.
3. Pawson R, Tilley N. Realistic Evaluation. Sage Publishers,
4. Milne L, Scotland G, Tagiyeva-Milne N, Hussein J. Safe
Motherhood Program Evaluation: Theory and Practice. Journal of Midwifery
and Women’s Health, July/August 2004; 49 (4): 338 -344.
Competing interests: No competing interests