Randomised controlled trials - a flawed approach to behavioural research
I was disappointed to find that the BMJ continues to publish articles
describing randomised controlled trials of interventions to change human
behaviour. We seem to have reached a position in which the statement
“systematic reviews of rigourous studies provide the best evidence of the
effectiveness of different strategies for promoting behavioural change.”
[1] is accepted without question. We have been mesmerized by the aura of
spurious scientific respectability surrounding the initials “RCT”. The
phrase “hierarchy of evidence” is trotted out on all occasions, with
little or no thought for the true nature and purpose of an RCT.
In the natural sciences, experimenters can be confident that one
sample of calcium carbonate will respond in exactly the same way as any
other when treated in the same way. Chemists therefore only need to
perform their experiments on one sample. When investigating the effects of
drugs on the human body, we cannot make the same assumption. The human
body is complex, and so in order to gain some idea of what the average
effect of a drug will be, we carry out the same test on large numbers of
people. In order to minimize unwanted effects, we make the trial
randomized and, if at all possible, double blind as well. The results give
us statistical information about the probable effects of a particular
drug. We can accept such results because we can assume that if the same
person were given the same drug on several occasions, their body will
respond in the same way. The body cannot choose how to respond to a drug.
When investigating human behaviour, as van Eijk et. al. set out to do, we
can make no such assumption. Human beings can choose how to respond to an
intervention. An educational input which catches my imagination on one
occasion, may not do so on another occasion if I am bored, tired or
emotionally distressed. The effects of an intervention can only be assumed
to hold true on one occasion, with one set of individuals. Randomly
assigning people to different groups will not result in control for all
other factors, as it is impossible to know how what influences an
individuals preferred learning style.
All this is important, because large amounts of time and money are
spent on studies such as this. I agree that it is vital that we try and
understand the process of professional behaviour change. If we are to
obtain useful information we must stop subjecting large groups of
individuals to interventions and assuming that the results obtained will
hold true for totally different individuals – or even the same individuals
at a different point in time, and start asking people about their
responses to interventions. Only by understanding how people learn and
decide how to behave through well-conducted qualitative studies will we
begin to make progress.
1. Bero, L., et al., Getting research findings into practice. Closing
the gap between research and practice: an overview of systematic reviews
of interventions to promote the implementation of research findings.
British Medical Journal, 1998. 317: p. 465.
Rapid Response:
Randomised controlled trials - a flawed approach to behavioural research
I was disappointed to find that the BMJ continues to publish articles
describing randomised controlled trials of interventions to change human
behaviour. We seem to have reached a position in which the statement
“systematic reviews of rigourous studies provide the best evidence of the
effectiveness of different strategies for promoting behavioural change.”
[1] is accepted without question. We have been mesmerized by the aura of
spurious scientific respectability surrounding the initials “RCT”. The
phrase “hierarchy of evidence” is trotted out on all occasions, with
little or no thought for the true nature and purpose of an RCT.
In the natural sciences, experimenters can be confident that one
sample of calcium carbonate will respond in exactly the same way as any
other when treated in the same way. Chemists therefore only need to
perform their experiments on one sample. When investigating the effects of
drugs on the human body, we cannot make the same assumption. The human
body is complex, and so in order to gain some idea of what the average
effect of a drug will be, we carry out the same test on large numbers of
people. In order to minimize unwanted effects, we make the trial
randomized and, if at all possible, double blind as well. The results give
us statistical information about the probable effects of a particular
drug. We can accept such results because we can assume that if the same
person were given the same drug on several occasions, their body will
respond in the same way. The body cannot choose how to respond to a drug.
When investigating human behaviour, as van Eijk et. al. set out to do, we
can make no such assumption. Human beings can choose how to respond to an
intervention. An educational input which catches my imagination on one
occasion, may not do so on another occasion if I am bored, tired or
emotionally distressed. The effects of an intervention can only be assumed
to hold true on one occasion, with one set of individuals. Randomly
assigning people to different groups will not result in control for all
other factors, as it is impossible to know how what influences an
individuals preferred learning style.
All this is important, because large amounts of time and money are
spent on studies such as this. I agree that it is vital that we try and
understand the process of professional behaviour change. If we are to
obtain useful information we must stop subjecting large groups of
individuals to interventions and assuming that the results obtained will
hold true for totally different individuals – or even the same individuals
at a different point in time, and start asking people about their
responses to interventions. Only by understanding how people learn and
decide how to behave through well-conducted qualitative studies will we
begin to make progress.
1. Bero, L., et al., Getting research findings into practice. Closing
the gap between research and practice: an overview of systematic reviews
of interventions to promote the implementation of research findings.
British Medical Journal, 1998. 317: p. 465.
Competing interests: No competing interests