Analysis of quality of interventions in systematic reviews

Introduction

Complex health interventions, such as surgery or physiotherapy, can be administered well or badly. Variation in the quality of administration of interventions may explain some of the variability in estimates of effects between trials in systematic reviews. We argue that systematic reviews of complex interventions should assess the quality of interventions, and we suggest how to make such assessments.

Best evidence

Randomised controlled trials provide the best test of the efficacy of preventive or therapeutic interventions because they can separate the effects of the intervention from those of extraneous factors such as natural recovery and statistical regression. When more than one trial has examined a particular intervention, systematic reviews potentially provide the best summaries of the available evidence.1

Systematic reviewers can summarise findings of randomised trials using an impressionistic approach (qualitative synthesis) or they can produce quantitative syntheses by statistically combining the results from several studies (meta-analysis). Regardless of the method used, most systematic reviewers seek to reduce data from clinical trials into simple statements about treatment. Systematic reviews that provide succinct statements about the effects of an intervention are particularly useful to clinicians. But simple summaries are possible only when the studies address similar questions in similar ways.

Heterogeneity

We can differentiate several types of heterogeneity between trials. Clinical heterogeneity can be identified before analysis of data. It may be due to variations across trials in the inclusion and exclusion criteria, the way in which the intervention is administered or the outcomes that are measured. Statistical heterogeneity occurs when estimates of effects …