The systematic review of treatments for sleeping problems in young
children (1) and a recent review of treatment of colic (2) provided
evidence for the efficacy of behavioural treatments. This, despite the
generally lower quality ratings that behavioural trials received compared
to drug or dietary trials.
It is worrisome that quality criteria derived from drug trials are
used as "off the shelf" gold standards (3) to judge the quality of
behavioural treatments in recent reviews in the BMJ. Double blinding is
used in trials to guard against performance bias, i.e. systematic
differences in the care provided apart from the intervention being
evaluated (4). Double blinding is not possible in behavioural or many
other medical treatment studies. Imagine the therapist or surgeon who does
not know why or what s/he is doing with this patient? Thus a range of safe
guards have been proposed and tested to guard against performance bias
such as the investigation of generalised placebo effects, e.g. comparison
of a treatment to a different treatment and a no-treatment group. The
difference between a different (placebo) treatment and no treatment
indicates generalised placebo effects (e.g. how just attention leads to
improvements) (5). Randomisation is applied to exclude selection bias.
However, in small samples (e.g. < 30) effect sizes may become inflated
due to outliers and thus checking for a range of pre-treatment differences
may be as important than randomisation itself (6). The reviewed drug or
dietary trials (1,2) were mostly small sample studies.
It should be remembered that the aim of quality ratings is to judge
how optimal the internal validity of a trial is (i.e. the confidence that
the cause of any observed differences is due to the treatment conditions)
rather than to apply a set of criteria that are "off the shelf" and were
developed without behavioural treatments in mind.
1. Ramchandani P, Wiggs L, Webb V, Stores G. A systematic review of
treatments for settling problems and night waking in young children.
British Medical Journal 2000;320:209-213.
2. Lucassen PLBJ, Assendelft WJJ, Gubbels JW, van Eijk JTM, van
Gelfrop WJ, Knuistingh Neven A. Effectiveness of treatments for infantile
colic: systematic review. British Medical Journal 1998;316:1563-1569.
3. Jaddad AR, Moore RA, Carroll D, Jenkinson C, Reynolds JM, Gavaghan
DJ, et al. Assessing the quality of randomized clinical trials: is
blinding necessary? Controlled Clinical Trials 1996;17:1-12.
4. Cook TD, Campbell DT. Quasi-experimentation: Design and analysis
for field settings. Chicago: Rand McNally, 1979.
5. Howard KI, Moras K, Brill PL, Martinovich Z, Lutz W. Evaluation of
Psychotherapy. American Psychologist 1996;51:1059-1064.
6. Lipsey MW, Wilson DB. The efficacy of psychological, educational,
and behavioral treatment. American Psychologist 1993;48:1181-1209.
Competing interests:
No competing interests
23 January 2000
Dieter Wolke
Professor of Psychology
University of Hertfordshire, Department of Psychology, DWRU, GB-Hatfield, Herts AL10 9AB
Rapid Response:
Quality Ratings of Behavioural Trials
The systematic review of treatments for sleeping problems in young
children (1) and a recent review of treatment of colic (2) provided
evidence for the efficacy of behavioural treatments. This, despite the
generally lower quality ratings that behavioural trials received compared
to drug or dietary trials.
It is worrisome that quality criteria derived from drug trials are
used as "off the shelf" gold standards (3) to judge the quality of
behavioural treatments in recent reviews in the BMJ. Double blinding is
used in trials to guard against performance bias, i.e. systematic
differences in the care provided apart from the intervention being
evaluated (4). Double blinding is not possible in behavioural or many
other medical treatment studies. Imagine the therapist or surgeon who does
not know why or what s/he is doing with this patient? Thus a range of safe
guards have been proposed and tested to guard against performance bias
such as the investigation of generalised placebo effects, e.g. comparison
of a treatment to a different treatment and a no-treatment group. The
difference between a different (placebo) treatment and no treatment
indicates generalised placebo effects (e.g. how just attention leads to
improvements) (5). Randomisation is applied to exclude selection bias.
However, in small samples (e.g. < 30) effect sizes may become inflated
due to outliers and thus checking for a range of pre-treatment differences
may be as important than randomisation itself (6). The reviewed drug or
dietary trials (1,2) were mostly small sample studies.
It should be remembered that the aim of quality ratings is to judge
how optimal the internal validity of a trial is (i.e. the confidence that
the cause of any observed differences is due to the treatment conditions)
rather than to apply a set of criteria that are "off the shelf" and were
developed without behavioural treatments in mind.
1. Ramchandani P, Wiggs L, Webb V, Stores G. A systematic review of
treatments for settling problems and night waking in young children.
British Medical Journal 2000;320:209-213.
2. Lucassen PLBJ, Assendelft WJJ, Gubbels JW, van Eijk JTM, van
Gelfrop WJ, Knuistingh Neven A. Effectiveness of treatments for infantile
colic: systematic review. British Medical Journal 1998;316:1563-1569.
3. Jaddad AR, Moore RA, Carroll D, Jenkinson C, Reynolds JM, Gavaghan
DJ, et al. Assessing the quality of randomized clinical trials: is
blinding necessary? Controlled Clinical Trials 1996;17:1-12.
4. Cook TD, Campbell DT. Quasi-experimentation: Design and analysis
for field settings. Chicago: Rand McNally, 1979.
5. Howard KI, Moras K, Brill PL, Martinovich Z, Lutz W. Evaluation of
Psychotherapy. American Psychologist 1996;51:1059-1064.
6. Lipsey MW, Wilson DB. The efficacy of psychological, educational,
and behavioral treatment. American Psychologist 1993;48:1181-1209.
Competing interests: No competing interests