Education and debate

Why do we need randomised controlled trials to assess behavioural interventions?

Judith Stephenson, senior lecturer in epidemiology,  John Imrie, research fellow. 

Department of Sexually Transmitted Disease, University College London Medical School, London WC1 6AU

Correspondence to: Dr Stephenson jstephen{at}gum.ucl.ac.uk

The value of the randomised controlled trial still generates debate.¹ Although some of the earliest examples of these trials can be found in behavioural and psychosocial research, this is not an area that has adopted readily the randomised controlled trial to assess interventions.² Two recent developments have intensified debate about the role of randomised controlled trialsthe urgent need to find effective behavioural interventions against HIV ^{3 4} and the advance of evidence based medicine, which is moving the randomised controlled trial beyond clinical trials into areas such as health promotion. This article considers the merits and limitations of randomised controlled trials in the behavioural area compared with clinical medicine, and asks how these trials can be applied successfully to assess behavioural interventions.

	Merits and limitations

The merits and limitations of randomised controlled trials in general have been widely discussed ^{5 6} ; only key points are repeated here. In clinical medicine, the randomised controlled trial is considered the best way of measuring the efficacy of interventions because of its ability to minimise bias and avoid false conclusions. Random assignment of individuals to different treatment groups is the best way of achieving a balance between groups for the known and unknown factors that influence outcome. This may seem to run counter to the traditional medical model of the doctor deciding which treatment is best for each patient, but it is considered ethical only when there is genuine uncertainty about which treatment to offer. By the same token, failure to tackle genuine uncertainty about treatments through randomised controlled trials can be considered unethical because it allows ineffective or harmful treatments to continue unchecked.

Limitations
Aside from ethical issues, the limitations of randomised controlled trials are relative, and shared to some degree by other study designs. These include cost, feasibility, and relevance to the real world. The effect of an intervention in an ideal research setting (efficacy) may well differ from its effect in the real world (effectiveness). This is particularly true of "explanatory" trials, which are designed to establish a cause and effect relation, but less so of "pragmatic" trials, which aim to mimic real life situations.⁷ Efficacy tends to differ from effectiveness because people who give informed consent to enter trials usually differ, in ways that affect outcome, from those who are eligible but decline or are not invited. Furthermore, taking part in research often involves procedures and commitments that are different from routine practice. In this sense, effectiveness cannot be judged from tightly controlled research, but without prior evidence of efficacy, it can be hard to attribute events in the real world to the effectiveness of an intervention (see below).

	What is the debate?

Debate about randomised controlled trials generally takes one of two forms. If it is accepted that the randomised controlled trial is the method of choice for estimating the efficacy of interventions, then debate is confined to the conditions which permit the trial on ethical and practical grounds and make the findings useful beyond the trial itself.⁷ If a randomised controlled trial is not possible for ethical or practical reasons, observational studies may be the only way of assessing an intervention, and some of these have undoubtedly been useful. To accept the randomised controlled trial as the best way of gauging efficacy is not necessarily to dismiss other study designs that have the same objective. ^{1 7 8} For example, no randomised controlled trial of the efficacy of condoms in preventing sexual transmission of HIV has been carried out. Given the seriousness of HIV disease and the consistency of early, albeit inconclusive, studies supporting condom use, such a trial would have been unethical. However, in a well designed prospective study of sexual partners whose HIV status differed, the seroconversion rate was zero between couples who used condoms consistently and about 5% per year in those who did not.⁹ Given this evidence of condom efficacy, we can be more confident that the decrease in the rates of sexually transmitted diseases and HIV infection in Thailand is due to the effectiveness of a nationwide campaign that dramatically increased condom use in brothels.¹⁰

View larger version (84K): [in this window] [in a new window]   Is it the cognitive-behavioural component of the therapy that works, or is non-specific group support more important?

The second form of debate involves more fundamental opposition to randomised controlled trials. In the behavioural and psychosocial field, ethical objections have been raised about withholding interventions that are believed or assumed to be beneficial. In addition, it is argued that randomised trials are not applicable in this field because they ignore the importance of external influences, participant choice, qualitative research methods, and the complexity of behavioural and psychosocial interventions.¹¹

	Assessing behavioural interventions

Behaviours and outcomes related to health are clearly influenced by complex social and economic factors.¹² Randomised controlled trials may have little to contribute in terms of explaining how and why these factors affect health and behaviour, but this does not deny their usefulness in testing applied interventions with specific objectives. In fact, randomisation seeks to balance out external influences between groups so that the true effect of an applied intervention is detectable. For example, advertising and pricing policies undoubtedly have a major impact on smoking levels, but the effectiveness of a specific intervention to stop smokingfor example, hypnosisis best examined by a randomised controlled trial of smokers.¹³

Interventions that target behaviour are often demanding and costly. They generally require several sessions run by highly skilled staff. Without evidence of efficacy, scarce resources might be better spent elsewhere, and the possibility of causing harm should be considered. One study of counselling after an HIV test found that the incidence of gonorrhoea in people who tested negative was twice as high in the six months after testing and counselling than in the preceding six months.¹⁴ Without a control group these findings are hard to interpret, and there are few good trials in this area. The point is that well meaning measures may not work as intended.

Behavioural interventions are often evaluated through uncontrolled, before and after comparisons. ^{4 15} Dissatisfaction with these comparisons in clinical medicine is partly related to the statistical law known as regression to the mean. If extreme values (for example, of blood cholesterol) are singled out from a distribution, they are likely, for purely statistical reasons, to fall closer to the usual level if measurement is repeated. In the absence of a control group, lower cholesterol concentrations at follow up might merely reflect the laws of statistics but be wrongly attributed to the effect of an intervention.

It is easy to imagine a similar, although non-statistical, phenomenon occurring in a behavioural setting. For example, people are most at risk of contracting a sexually transmitted disease when they are taking greater sexual risks than usual, and are likely to return to their usual level of risk behaviour afterwards. If people with a sexually transmitted disease were recruited to an uncontrolled study of a behavioural intervention, lower rates of sexually transmitted diseases or increased condom use might be expected at follow up, even if the intervention had no effect (particularly if participants were reluctant to disclose high risk behaviour or if the process of diagnosing a sexually transmitted disease and treatment alone had had an impact). Effects of this kind could not be detected without a comparable (randomised) control group receiving standard care but not the intervention of interest.

	Randomised controlled trials have limitations

An important limitation to randomised controlled trials in behavioural or psychosocial research relates to "blinding" and participant choice. Blinding to treatment allocation in clinical trials (for example, by using placebos that are indistinguishable from active drugs) is intended to prevent the expectations of patients or researchers from influencing the outcome. In behavioural trials, blinded allocation to treatment may be impossible, but blinded assessment of outcome need not be. For example, in a randomised controlled trial of psychotherapy versus supportive listening in patients with irritable bowel syndrome, both the psychiatrist and the patients knew which treatment group they were in, but the outcomes (psychological and bowel symptoms) were assessed by another psychiatrist and a gastroenterologist who were blind to treatment allocation.¹⁶

Does choice improve outcome?
Excluding choice by allocating patients randomly to one or other treatment is seen as a great strength in clinical trials. In behavioural trials, this has brought criticism. Choosing your preferred intervention, it is argued, increases motivation and thereby the success of the intervention.¹⁷ Even if this is true, can such a motivational effect be measured? Some researchers claim it can be measured by comparing outcomes between those who are randomly assigned to a particular intervention and those who choose it.¹⁷ The problem with this is that people who choose an intervention probably differ from those who do not in ways that affect the outcome. For example, in the trial of patients with irritable bowel syndrome, psychotherapy was better than supportive listening at three months, at which time the supportive listening group was offered psychotherapy.¹⁶ The 77% who accepted the offer clearly differed in psychological symptomsat the point of decisionfrom the 23% who declined. Other (unknown) factors related to irritable bowel syndrome may also have differed between the two subgroups.

Inappropriate for exploratory research
As noted above, randomised controlled trials are not appropriate for exploratory research into factors that determine behaviour related to health. This is the domain of qualitative research.¹⁸ Identifying the cultural context, values, beliefs, and community norms of target groups through good qualitative research is the key to the design and implementation of promising interventions. Examples can be found in the development of behavioural interventions led by peers to reduce the risk of infection with HIV, particularly among hidden or hard to reach groups such as injecting drug users who are not in contact with treatment services, and young gay men. ^{19 20} Clearly the strengths of qualitative research should not be pitted against those of randomised controlled trials.

Standardisation is a major challenge
Standardising the content and delivery of a complex intervention in a randomised controlled trial is a major challenge. More extensive training and monitoring of those delivering the intervention may be needed than in clinical trials. Monitoring can be done through supervision and feedback or, more objectively, by audiotaping or videotaping random sessions by independent assessors. Careful monitoring and qualitative research may explain why complex interventions fail or, conversely, shed light on the factors that lead to behaviour changethe active ingredients.²¹ For example, if a support group based on cognitive-behavioural therapy proves effective, is it the cognitive-behavioural component of the treatment that works, or is non-specific group support more important? To be confident about which is most effective would require each component to be tested in additional experiments or a trial with several arms. Both options are usually too costly and too lengthy to be realistic. In terms of assessing healthcare interventions, pragmatic randomised controlled trials may therefore be more appropriate and manageable than explanatory ones, even if they do not identify the active ingredient. By comparison with the effort required to standardise and monitor a complex intervention, the additional effort of justifying randomisation to potential participants is small, while the gain in reducing bias is great.

	Conclusion

The value of the randomised controlled trial in behavioural research does not generally differ fundamentally from its value in clinical medicine. Issues that occasionally arise in other areas, such as lack of opportunity for blinding and complexity of intervention, are particular features of behavioural and psychosocial research. Standardising the content and delivery of a complex intervention may prove more limiting than random allocation. When interventions are complex, pragmatic trials may be more likely to succeed than explanatory ones.

	Acknowledgments

Funding: JS is funded by a Medical Research Council programme grant and JI by a North Thames research and development project grant.

Conflict of interest: None.

	References

Top References

1. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996; 312: 1215-1218[Free Full Text].
2. Powers E, Witmer H. An experiment in the prevention of delinquencythe Cambridge-Somerville youth study. New York: Columbia University Press , 1951.
3. Coates TJ, Aggleton P, Gutzwiller F, Des Jarlais D, Kinata M, Kippax S, et al. HIV prevention in developed countries. Lancet 1996; 348: 1143-1148[Medline].
4. Oakley A, Fullerton D, Holland J. Behavioural interventions for HIV/AIDS prevention. AIDS 1995; 9: 479-486[Medline].
5. Oakley A, Fullerton D. The lamp-post of research: support or illumination? The case for and against randomised controlled trials. In: Oakley A, Roberts H, eds. Evaluating social interventions. A report of two workshops. Ilford: Barnardo's , 1996.
6. Pocock SJ. The justification for randomised controlled trials Clinical trials. A practical approach. , Chichester: Wiley, 1983:50-65.
7. Stephenson JM, McKeigue P. Pitfalls in patient selection and statistical procedures in diabetes research. In: Mogensen CE, Standl E, eds. Research methodologies in human diabetes. Part 2.  Berlin: Walter de Gruyter , 1995.
8. Smith PG. Evaluating interventions against tropical diseases. Int J Epidemiol 1986; 16: 159-166[Free Full Text].
9. De Vincenzi I, for the European Study Group on Heterosexual Transmission of HIV. Human immunodeficiency virus (HIV) transmission from HIV infected persons to their stable heterosexual partners: follow-up of a European cohort of 304 couples. N Engl J Med 1994;331:341-6.
10. Rojanapithayakorn W, Hanenberg R. The 100% condom program in Thailand. AIDS 1996; 10: 1-7[Medline].
11. Oakley A. Who's afraid of the randomized trial? Some dilemmas of the scientific method and "good" research practice. In: Roberts H, ed. Women's health counts. London: Routledge , 1990.
12. Susser M. The tribulations of trialsintervention in communities. Am J Public Health 1995; 85: 156-159[Free Full Text].
13. Law M, Tang JL. An analysis of the effectiveness of interventions intended to help people stop smoking. Intern Med 1995; 155: 1933-1941.
14. Zenilman JM, Erickson B, Fox R, Reichart CA, Hook III EW. Effect of HIV post test counseling on STD incidence. JAMA 1992;267:843-5.
15. Choi K, Coates TJ. Prevention of HIV infection. AIDS 1994; 8: 1371-1389[Medline].
16. Guthrie E, Creed R, Dawson D, Tomenson B. A randomised controlled trial of psychotherapy in patients with refractory irritable bowel syndrome. Br J Psychol 1993; 163: 321.
17. Brewin CR, Bradley C. Patient preferences and randomised clinical trials. BMJ 1996; 299: 313-315.
18. Pope C, Mays N. Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. BMJ 1995; 311: 42-45[Free Full Text].
19. Power R, Jones S, Kearns G, Ward J. An ethnography of risk management amongst illicit drug injectors and its implications for the development of community-based interventions. Soc Health Illness 1996; 18: 86-106.
20. Kelly J, St. Lawrence J, Brasfield T. AIDS/HIV risk reduction: the effects of endorsements by popular people in three cities. Am J Public Health 1992; 80: 1483-1489.
21. O'Leary A, DiClemente RJ, Aral SO. Reflections on the design and reporting of STD/HIV behavioral interventions research. AIDS Educ Prev 1997; 9: 1-15[Medline].

(Accepted 21 July 1997)