Cluster randomised trials with repeated cross sections: alternatives to parallel group designs
BMJ 2015; 350 doi: https://doi.org/10.1136/bmj.h2925 (Published 08 June 2015) Cite this as: BMJ 2015;350:h2925
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Hooper and Bourke introduce and illustrate the sample size calculations in a novel type of cluster randomised design, the so called dog-leg design (the incomplete cross-forward design) [1]. This design is related, but has distinct features to the stepped wedge cluster randomised design.
Whether a simple parallel cluster trial, or some alternative design, will be the preferable design depends on many considerations, of which efficiency is just one. In cluster trials, comparisons of efficiency, can consider minimising the number of clusters or the number of participants (or preferably both). There has been much debate and confusion in the literature over the relative efficiencies of different designs. Hooper and Bourke make the following two claims:
1. “Stepped wedge designs need fewer clusters than parallel group designs with a single follow-up, simply because they assess the same clusters repeatedly.”
2. “A dog leg design run over two repeated cross sections, for example, needs fewer clusters and fewer participants in total than a trial with a single cross section.”
Both of these claims are debatable and potentially miss-leading.
Firstly, it is not true that the cross-sectional stepped wedge design always requires a fewer number of clusters than the simple parallel design [2]. Whether or not this statement holds depends on whether the cluster sizes are fixed across the designs. If the cluster size is fixed (i.e. the recruitment rate and duration of the study is fixed) then the design choice comes down to a decision of, in which clusters and when, to intervene. In this case the stepped wedge design might need more clusters than the parallel designs (for low intra-cluster correlations). An alternative comparison is to compare a simple parallel design with a smaller cluster size, to a stepped wedge study that is allowed to run for a longer duration and so have a larger total cluster size. It is only when the cluster sizes are not fixed across the designs that the stepped wedge design always requires fewer clusters than the parallel design [3]. We illustrate this below by way of a counter example. Please see the attached PDF for a clearer view of the table:
Design constraints CRT SW-CRT
Total cluster size ICC Number steps DE TSS Number clusters DE TSS Number clusters
30 0.01 2 1.29 1017 34 3.22 2538 85
30 0.25 2 8.25 6501 217 3.23 2544 85
TSS: Total sample size; ICC: Intra-cluster correlation; DE: Design effect (multiplication over sample size needed under individual randomisation). Study designed to detect a moderate standardised effect size (0.2) at 80% power and 5% significance.
Secondly, whilst Hopper and Bourke have shown previously that the dog-leg design does indeed require fewer clusters and fewer participants than the conventional parallel design [4], it is important to note that the commonly used parallel design with a baseline period can offer an even more efficient design than the dog-leg design. And whilst this didn’t hold in the example presented in their paper, this is because efficiency depends on the correlations (e.g. the ICC), a point which deserves recognition. The parallel cluster design with a baseline isn’t necessarily the most efficient design amongst all competing designs, but as a well-used and known design it should be considered a contender.
References
[1] Hooper R, Bourke L. Cluster randomised trials with repeated cross sections: alternatives to parallel group designs. BMJ. 2015 Jun 8;350:h2925. doi:10.1136/bmj.h2925. PubMed PMID: 26055828.
[2] Hemming K, Lilford R, Girling AJ. Stepped-wedge cluster randomised controlled trials: a generic framework including parallel and multiple-level designs. Stat Med. 2015 Jan 30;34(2):181-96. doi: 10.1002/sim.6325. Epub 2014 Oct 24. PubMed PMID: 25346484; PubMed Central PMCID: PMC4286109.
[3] Rhoda DA, Murray DM, Andridge RR, Pennell ML, Hade EM. Studies with staggered starts: multiple baseline designs and group-randomized trials. Am J Public Health. 2011 Nov;101(11):2164-9. doi: 10.2105/AJPH.2011.300264. Epub 2011 Sep 22. Review. Erratum in: Am J Public Health. 2014 Mar;104(3):e12. PubMed PMID:21940928; PubMed Central PMCID: PMC3222403.
[4] Hooper R, Bourke L. The dog-leg: an alternative to a cross-over design for pragmatic clinical trials in relatively stable populations. Int J Epidemiol. 2014 Jun;43(3):930-6. doi: 10.1093/ije/dyt281. Epub 2014 Jan 22. PubMed PMID: 24453236.
Competing interests: No competing interests
Repeated cross-sections in cluster randomised trials
We are glad of Dr Hemming’s interest in our work [1,2]. Readers of our paper will see that we endorse and recommend Hemming’s and her colleagues’ own work in this area without hesitation. We take this opportunity to clarify some of the issues raised in Hemming’s letter, but before addressing her main concerns we make a couple of general observations.
First, the notion of “cluster size” as used here by Hemming may be confusing in the context of a repeated cross-section cluster randomised trial design, partly because a new set of participants is observed in each cluster on each occasion, and partly because this may only be a sample from the eligible participants in that cluster. In our school breakfast example, where the clusters are schools from which we sample 50 children aged 9-11 each year, we do not think it is helpful (or correct) to say that the “size” of a school (or cluster) in a two-period design is 100, while the size of a school in a one-period design is 50. We think the school is the same size in each case (and it is not 50 or 100). In our article we refer instead to the “sample size in each cluster at each cross-section”, and in our framework we assume that this is fixed when comparing alternative designs.
Second, Hemming envisages a process of consecutive sampling of participants over time, one by one, so that by extending the duration of each period the sample size is increased. We, in contrast, have tried to make it clear in our statistical model, in our example, and in our terminology, that we are assuming a series of repeated cross-sections – each sampled in one go and followed at fixed intervals over time. In the school breakfast example we do not recruit schoolchildren over one year at a rate of 50 per year; instead we take a cross-section of 50 children and assess them, and then if the design requires it we take another cross-section a year later. A study using Hemming’s sampling approach could certainly be analysed assuming our models, though an interrupted time series or segmented regression approach might be a more appropriate way to model time effects in such a case. When Hemming speaks of doubling the duration of a parallel group design A vs B in order to double the number of participants, the analogy in our framework (a framework of repeated cross-sections with fixed sample size in each cluster at each time) is the design AA vs BB, that is a parallel group design with two repeated cross-sections under the same treatment conditions.
So, to address Hemming’s two main points (the short answer being that we agree entirely):
1. Yes, if the correlation between repeated cluster means is low enough, then a trial with design AA vs BB requires fewer clusters and fewer individual participants than a trial with design BA vs BB. Numbers of clusters and participants for the former design may be calculated using the formula for parallel group designs with more than one baseline or follow-up, as given in our main Figure (with number of baselines set to zero and number of follow-ups set to two). Regarding the general question of the best design when there is a fixed number of cross-sections in each arm, our paper cites Hemming and colleagues.
2. Yes, if the correlation is high enough, then a trial with design BA vs BB requires fewer clusters and fewer individual participants than a trial with a dog-leg design (A– vs BA vs –B). Readers can see this illustrated in our own Figure, in the web appendix to our article.
We echo the warning that the relative efficiency of different designs depends on the correlation between repeated cluster means. For this reason we have presented general formulae for different designs, along with an example to illustrate the method of sample size calculation. Triallists must also weigh the competing costs and constraints associated with increasing the number of clusters, the number of individual participants, and the duration of the trial. We do not fool ourselves that the dog-leg design is always the best, but we are excited by the discovery that this previously unreported and untried trial design is worth considering in some cases. The other point we wanted to make in our paper was that if you show us a cluster randomised trial involving a single cross-section, we can show you how – by re-visiting the same clusters and taking another cross-section with the same sample size per cluster – you can reduce both the number of clusters and the total number of participants required. We hope readers will share our interest in these findings.
References
[1] Hooper R, Bourke L. Cluster randomised trials with repeated cross sections: alternatives to parallel group designs. BMJ. 2015;350:h2925
[2] Hemming K. The stepped wedge cluster randomised trial does not always need fewer clusters. BMJ 2015 http://www.bmj.com/content/350/bmj.h2925/rr
Competing interests: No competing interests