Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Analysis should reflect the clustered study design.
- William Plummer (10 March 2001)
-
Analysis reflects clustered study design
- Smita Shah (14 March 2001)
-
Missing confidence interval.
- W Plummer (28 March 2001)
-
Inadequate design and reporting of a cluster randomised controlled trial
- Tim J Peters (3 April 2001)
-
" Missing the wood for the trees" OR "A Cluster of distracting comments"
- Peter Gibson (17 April 2001)
|
|
|||
|
William Plummer, Consultant Psychiatrist Canterbury, Kent. UK
Send response to journal:
|
I noticed this article because it was described as a cluster randomised trial and read it specifically to see how the clustering had been handled in the data analysis. I was very disappointed to find that the clustering has not been allowed for at all in the analysis, even though the authors state that the clustered design of the trial had led to an imbalance in gender between the control and intervention groups. In this study, interventions are specifically targetted at schools and it is very likely that there will be strong intra-school correlations in outcome. Some authorities (such as Kerry and Bland)would suggest that the school should be the unit of analysis in a study like this. Perhaps this is going a bit far when only six schools are included. However, providing data on baseline variables and the main outcomes for each school would at least allow the reader to judge the variability between schools. It may also have been possible to give some sort of estimate of the intra-school correlation for the main outcome variable, and perhaps adjust significance tests to take account of this. I recognize the difficulties in carrying out a study like this, where the cluster randomised design is clearly appropriate, but feel that the quality of the paper could have been improved by an analysis which takes account of the design. Yours sincerely, W. P Plummer Ref: Kerry, S.M., Bland, J.M.(1998) "Analysis of a trial randomised in clusters" BMJ 1998; 316: 54. |
|||
|
|
|||
|
Smita Shah, Director Primary Health Care Education and Research Unit, Department of Public Health and Community Medicine
Send response to journal:
|
Dear Editor, We thank Dr Plummer1 for emphasising the pitfalls that can arise in the statistical analysis of studies such as ours in which subjects are necessarily randomised to study groups in clusters. In fact, we did calculate intra-class correlations (ICC) to explore any effects due to clustering although space precluded us from including both these data and more extensive within-school baseline data. For all outcomes, the ICC values were exceedingly small, for example the ICC values for the mean differences in quality of life scores were less than 0.002 for all domains. Clearly, values such as this have a negligent effect on P values. We hope that this reassures readers that our findings were free of bias due to clustering. Han Wang, Jennifer Peat, Smita Shah. 1. Pummer W. Analysis should reflect the clustered study design. BMJ 2001; March 10th. |
|||
|
|
|||
|
W Plummer, Consultant Psychiatrist Canterbury, Kent.
Send response to journal:
|
In this study, the main outcome measure is change in score on a validated paediatric asthma quality of life questionnaire. In previous correspondence, the authors have reported the intra-class correlation coefficient for this outcome to be very low. In view of this, it is correct to analyse the study as if it were not a clustered design, and confidence intervals calculated in this way can be relied upon. Unfortunately, the magnitude of the difference in change of score between the control and treatment group has been reported as 0.12, but the confidence interval (and p-value) for this difference has been omitted from the final printed version of the paper. I wonder if the authors would be kind enough to report this confidence interval. Dr W. Plummer. |
|||
|
|
|||
|
Tim J Peters, Reader in Medical Statistics University of Bristol
Send response to journal:
|
We have concerns about the design and analysis of a cluster randomised trial of a peer led education programme for asthma.1 Neither the printed nor electronic version mentioned how clustering was accounted for in the trial design. The sample size was not justified, either the number of clusters (six schools) or numbers of children within them. This may seem unimportant since confidence intervals were provided for the between-arm comparisons, but this omission is crucial. First, the authors did not specify the magnitude of differences considered in advance as clinically significant. Second, the very small intra-cluster correlations (ICCs) observed could just be fortuitous. With so few clusters, any estimate of between-cluster variance (and hence ICC) will be extremely imprecise. Without proper trial design details the danger of publication bias remains, where a low-powered study is more likely to be published when statistical significance is attained. The widths of the comparative confidence intervals are not reassuring here. It is unlikely that a fully considered trial design would involve only six clusters, with apparently no attention to stratification given the gender and year imbalances.2 Important information was also omitted regarding the analysis plan, with between-arm comparisons in Table 2 for quality of life as a total score and three sub-domains.1 First, only the electronic version states that the total score was the primary outcome. More fundamentally, comparisons for each of the four outcomes were presented first for all children and then repeated for four subgroups (males and females in years 7 and 10). Separate tests for subgroups rather than formal interaction tests are highly prone to false positive results. There was no indication that these subgroup analyses were established in advance, nor specifically whether others were conducted. The analysis section of the electronic version refers to gender and year as potential confounders, not effect- modifiers. If the principle of 'Electronic Long, Paper Short' articles is to be successful, it is important that short versions do not omit crucial methodological information. It cannot be assumed that all readers have access to electronic versions, and one has already commented on this paper without apparently checking the long version.3 While we appreciate that the exclusion of cluster-adjusted results was justified here on grounds of space,4 we contend that issues of design and analysis are vital in cluster randomised trials and these should have been mentioned in the short version. 1 Shah S, Peat JK, Mazurski EJ et al. effect of peer led programme for asthma education in adolescents: cluster randomised controlled trial. BMJ 2001;322:583-5. 2 Ukoumunne OC, Gulliford MC, Chinn S, Sterne JAC, Burney PGJ. Methods for evaluating area-wide and organisation-based interventions in health and health care: a systematic review. Health Technol Assess 1999;3(5). 3 Plummer W. Analysis should reflect the clustered study design. BMJ 2001; March 10th. http://www.bmj.com/cgi/eletters/322/7286/583. 4 Wang H, Peat J, Shah S. Analysis reflects clustered study design. BMJ 2001; March 14th. http://www.bmj.com/cgi/eletters/322/7286/583. Dr Tim J Peters, Reader in Medical Statistics, Department of Social
Medicine, University of Bristol, Canynge Hall, Whiteladies Road, Bristol
BS8 2PR.
Dr Anna Graham, Clinical Research Fellow, Division of Primary Health Care, University of Bristol, Canynge Hall, Whiteladies Road, Bristol BS8 2PR. Dr Chris Salisbury, Senior Lecturer in General Practice, Division of Primary Health Care, University of Bristol, Canynge Hall, Whiteladies Road, Bristol BS8 2PR. Dr Laurence Moore, Senior Research Fellow, Cardiff School of Social Sciences, Cardiff University, King Edward VII Avenue, Cardiff CF10 3WT. Competing interests: None. |
|||
|
|
|||
|
Peter Gibson, Senior Staff Specialist Airways Research Centre, John Hunter Hospital,Newcatle, Australia
Send response to journal:
|
To the Editor, There must be a word for people who criticise others and assume the worst. Maybe it is 'statistician'. We conducted a trial of an important intervention(asthma health promotion) in a problematic group(adolescents in a rural high school setting) and measured outcomes that are acknowledged to be relevant to people with asthma(quality of life, frequency of asthma attacks and school absenteeism). We found that by using a peer-led approach to asthma health promotion we could achieve significant beneficial results. We chose to publish in the BMJ rather than a specialist journal in order to communicate these important results to a wider audience. Now we find ourselves answering criticisms that reflect the space allocated to published papers rather than our research process, yet in each case the authors of these criticisms assume our research process to be in error. Although Drs Peters and Plummer find important omissions from the analysis plan in the written version of the paper, these are reported in the electronic version. The sample size calculation and justification are reported in our study protocol which was the basis for the peer reviewed funding application. Journals seldom publish these. We based these calculations on an earlier study where we measured quality of life before and after the intervention in a smaller number of adolescents(Gibson PG). We apriori chose the subgroups of male/female and years 7 and 10 because of our prior results, and because of the design of the intervention where year 10 students are active participants in the education process, whereas year 7 students are passive recipients. This is described in more detail in the references below. There are clear advantages to electronic publication. As experience with the principle of 'electronic long, paper short' publishing grows, we should see increasingly detailed electronic publications that allow full description of the research process and additional results. This system works well with the electronic versions of systematic reviews on the Cochrane library. From this we have learnt that given the opportunity of electronic publication, we will add rather than subtract detail, and ask the editors to show us the text version prior to publication. The down- side is that with no restriction on space, all sorts of critical letters can be published, which may confuse readers and detract from the valid message of the research. Peer-led education is a valid and potentially successful approach. We want people to read our results and see opportunities for improving the health of adolescents with chronic illness. Peter Gibson Smita Shah Gibson PG, Shah S. Mamoon HA. Peer-led asthma education for adolescents. J Adol Hlth 1998;22:66-72 Shah S, Mamoon H, Gibson PG. Peer-led health education for adolescents:1 development and implementation. Health Prom J Aust, 1998;8:177-182 |
|||