Effect of peer led programme for asthma education in adolescents: cluster randomised controlled trial
BMJ 2001; 322 doi: https://doi.org/10.1136/bmj.322.7286.583 (Published 10 March 2001) Cite this as: BMJ 2001;322:583
All rapid responses
We have concerns about the design and analysis of a cluster
randomised trial of a peer led education programme for asthma.1 Neither
the printed nor electronic version mentioned how clustering was accounted
for in the trial design. The sample size was not justified, either the
number of clusters (six schools) or numbers of children within them. This
may seem unimportant since confidence intervals were provided for the
between-arm comparisons, but this omission is crucial. First, the authors
did not specify the magnitude of differences considered in advance as
clinically significant. Second, the very small intra-cluster correlations
(ICCs) observed could just be fortuitous. With so few clusters, any
estimate of between-cluster variance (and hence ICC) will be extremely
imprecise. Without proper trial design details the danger of publication
bias remains, where a low-powered study is more likely to be published
when statistical significance is attained. The widths of the comparative
confidence intervals are not reassuring here. It is unlikely that a fully
considered trial design would involve only six clusters, with apparently
no attention to stratification given the gender and year imbalances.2
Important information was also omitted regarding the analysis plan,
with between-arm comparisons in Table 2 for quality of life as a total
score and three sub-domains.1 First, only the electronic version states
that the total score was the primary outcome. More fundamentally,
comparisons for each of the four outcomes were presented first for all
children and then repeated for four subgroups (males and females in years
7 and 10). Separate tests for subgroups rather than formal interaction
tests are highly prone to false positive results. There was no indication
that these subgroup analyses were established in advance, nor specifically
whether others were conducted. The analysis section of the electronic
version refers to gender and year as potential confounders, not effect-
modifiers.
If the principle of 'Electronic Long, Paper Short' articles is to be
successful, it is important that short versions do not omit crucial
methodological information. It cannot be assumed that all readers have
access to electronic versions, and one has already commented on this paper
without apparently checking the long version.3 While we appreciate that
the exclusion of cluster-adjusted results was justified here on grounds of
space,4 we contend that issues of design and analysis are vital in cluster
randomised trials and these should have been mentioned in the short
version.
1 Shah S, Peat JK, Mazurski EJ et al. effect of peer led programme
for asthma education in adolescents: cluster randomised controlled trial.
BMJ 2001;322:583-5.
2 Ukoumunne OC, Gulliford MC, Chinn S, Sterne JAC, Burney PGJ.
Methods for evaluating area-wide and organisation-based interventions in
health and health care: a systematic review. Health Technol Assess
1999;3(5).
3 Plummer W. Analysis should reflect the clustered study design. BMJ
2001; March 10th. http://www.bmj.com/cgi/eletters/322/7286/583.
4 Wang H, Peat J, Shah S. Analysis reflects clustered study design.
BMJ 2001; March 14th. http://www.bmj.com/cgi/eletters/322/7286/583.
Dr Tim J Peters, Reader in Medical Statistics, Department of Social
Medicine, University of Bristol, Canynge Hall, Whiteladies Road, Bristol
BS8 2PR.
E-mail: tim.peters@bristol.ac.uk
Dr Anna Graham, Clinical Research Fellow, Division of Primary Health
Care, University of Bristol, Canynge Hall, Whiteladies Road, Bristol BS8
2PR.
Dr Chris Salisbury, Senior Lecturer in General Practice, Division of
Primary Health Care, University of Bristol, Canynge Hall, Whiteladies
Road, Bristol BS8 2PR.
Dr Laurence Moore, Senior Research Fellow, Cardiff School of Social
Sciences, Cardiff University, King Edward VII Avenue, Cardiff CF10 3WT.
Competing interests: None.
Competing interests: No competing interests
In this study, the main outcome measure is change in score
on a validated paediatric asthma quality of life
questionnaire. In previous correspondence, the authors have
reported the intra-class correlation coefficient for this
outcome to be very low. In view of this, it is correct to
analyse the study as if it were not a clustered design, and
confidence intervals calculated in this way can be relied upon.
Unfortunately, the magnitude of the difference in change of
score between the control and treatment group has been
reported as 0.12, but the confidence interval (and p-value)
for this difference has been omitted from the final printed
version of the paper.
I wonder if the authors would be kind enough to report this
confidence interval.
Dr W. Plummer.
Competing interests: No competing interests
Dear Editor,
We thank Dr Plummer1 for emphasising the pitfalls that can arise in
the statistical analysis of studies such as ours in which subjects are
necessarily randomised to study groups in clusters. In fact, we did
calculate intra-class correlations (ICC) to explore any effects due to
clustering although space precluded us from including both these data and
more extensive within-school baseline data. For all outcomes, the ICC
values were exceedingly small, for example the ICC values for the mean
differences in quality of life scores were less than 0.002 for all
domains. Clearly, values such as this have a negligent effect on P values.
We hope that this reassures readers that our findings were free of
bias due to clustering.
Han Wang, Jennifer Peat, Smita Shah.
1. Pummer W. Analysis should reflect the clustered study design. BMJ
2001; March 10th.
Competing interests: No competing interests
I noticed this article because it was described as a cluster
randomised trial and read it specifically to see how the
clustering had been handled in the data analysis. I was very
disappointed to find that the clustering has not been
allowed for at all in the analysis, even though the authors
state that the clustered design of the trial had led to an
imbalance in gender between the control and intervention groups.
In this study, interventions are specifically targetted at
schools and it is very likely that there will be strong
intra-school correlations in outcome. Some authorities (such
as Kerry and Bland)would suggest that the school should be
the unit of analysis in a study like this. Perhaps this is
going a bit far when only six schools are included. However,
providing data on baseline variables and the main outcomes
for each school would at least allow the reader to judge the
variability between schools. It may also have been possible
to give some sort of estimate of the intra-school
correlation for the main outcome variable, and perhaps
adjust significance tests to take account of this.
I recognize the difficulties in carrying out a study like
this, where the cluster randomised design is clearly
appropriate, but feel that the quality of the paper could
have been improved by an analysis which takes account of the
design.
Yours sincerely,
W. P Plummer
Ref: Kerry, S.M., Bland, J.M.(1998) "Analysis of a trial
randomised in clusters" BMJ 1998; 316: 54.
Competing interests: No competing interests
" Missing the wood for the trees" OR "A Cluster of distracting comments"
To the Editor,
There must be a word for people who criticise others and assume the
worst. Maybe it is 'statistician'. We conducted a trial of an important
intervention(asthma health promotion) in a problematic group(adolescents
in a rural high school setting) and measured outcomes that are
acknowledged to be relevant to people with asthma(quality of life,
frequency of asthma attacks and school absenteeism). We found that by
using a peer-led approach to asthma health promotion we could achieve
significant beneficial results. We chose to publish in the BMJ rather than
a specialist journal in order to communicate these important results to a
wider audience. Now we find ourselves answering criticisms that reflect
the space allocated to published papers rather than our research process,
yet in each case the authors of these criticisms assume our research
process to be in error.
Although Drs Peters and Plummer find important omissions from the analysis
plan in the written version of the paper, these are reported in the
electronic version. The sample size calculation and justification are
reported in our study protocol which was the basis for the peer reviewed
funding application. Journals seldom publish these. We based these
calculations on an earlier study where we measured quality of life before
and after the intervention in a smaller number of adolescents(Gibson PG).
We apriori chose the subgroups of male/female and years 7 and 10 because
of our prior results, and because of the design of the intervention where
year 10 students are active participants in the education process, whereas
year 7 students are passive recipients. This is described in more detail
in the references below.
There are clear advantages to electronic publication. As experience with
the principle of 'electronic long, paper short' publishing grows, we
should see increasingly detailed electronic publications that allow full
description of the research process and additional results. This system
works well with the electronic versions of systematic reviews on the
Cochrane library. From this we have learnt that given the opportunity of
electronic publication, we will add rather than subtract detail, and ask
the editors to show us the text version prior to publication. The down-
side is that with no restriction on space, all sorts of critical letters
can be published, which may confuse readers and detract from the valid
message of the research. Peer-led education is a valid and potentially
successful approach. We want people to read our results and see
opportunities for improving the health of adolescents with chronic
illness.
Peter Gibson
Smita Shah
Gibson PG, Shah S. Mamoon HA. Peer-led asthma education for
adolescents. J Adol Hlth 1998;22:66-72
Shah S, Mamoon H, Gibson PG. Peer-led health education for
adolescents:1 development and implementation. Health Prom J Aust,
1998;8:177-182
Competing interests: No competing interests