- Marion K Campbell, programme director ()1,
- Diana R Elbourne, professor of health care evaluation2,
- Douglas G Altman, professor of statistics in medicine3
- 1Health Services Research Unit, University of Aberdeen, Aberdeen, AB25 2ZD
- 2Medical Statistics Unit, London School of Hygiene and Tropical Medicine, London
- 3Centre for Statistics in Medicine, Oxford
- Correspondence to: M K Campbell
- Accepted 10 December 2003
Reports of cluster randomised trials require additional information to allow readers to interpret them accurately
The effective reporting of randomised controlled trials has received useful attention in recent years. Many journals now require that reports conform to the guidelines in the Consolidated Standards of Reporting Trials (CONSORT) statement, first published in 19961 and revised in 2001.2 The statement includes a checklist of items that should be included in the trial report. These items are evidence based whenever possible and are regularly reviewed.3 The statement also recommends including a flow diagram to show the flow of participants from group assignment through to the final analysis.
The CONSORT statement focused on reporting parallel group randomised trials in which individual participants are randomly assigned to study groups. However, in some situations it is preferable to randomly assign groups of individuals (such as families or medical practices) rather than individuals. Reasons include the threat of contamination of some interventions (such as dietary interventions) if individual randomisation is used.4 5 Also, in certain settings randomisation by group may be the only feasible method of conducting a trial.6 Trials with this design are variously known as field trials, community based trials, place based trials, or (as in this paper) cluster randomised trials.7 In an earlier discussion paper we considered the implications of the CONSORT statement for the reporting of cluster randomised trials.8 Here we present updated guidance, based on the 2001 revision of the CONSORT statement.2
Methodological issues in cluster randomised trials
Compared with individually randomised trials, cluster randomised trials are more complex to design, require more participants to obtain equivalent statistical power, and require more complex analysis. The methodological issues in cluster randomised trials have been widely discussed.7 9 In brief, observations on individuals in the same cluster tend to be correlated (non-independent), and so the effective sample size is less than the total number of individual participants.
The reduction in effective sample size depends on average cluster size and the degree of correlation within clusters, known as the intracluster (or intraclass) correlation coefficient (ρ). The intracluster correlation coefficient is the proportion of the total variance of the outcome that can be explained by the variation between clusters. To retain power, the sample size should be multiplied by 1+(m - 1)ρ, called the design effect, where m is the average cluster size. Hayes and Bennett describe a related coefficient of variation, k, between clusters10 and Connelly considers an economic approach.11 Software is available to adjust for the intracluster correlation coefficient in an analysis.12
The conduct of cluster randomised controlled trials may also differ from that of trials that randomise individuals. For instance, clusters are usually randomised all at once (or in batches) rather than one at a time. After randomisation, individuals in the clusters may be approached for consent, which raises the possibility of post-randomisation selection bias,4 or they may not, which raises ethical concerns.13 14 An expanded explanation of the methods of cluster randomised trials is available on the CONSORT website (www.consort_statement.org).
Quality of reporting of cluster trials
Surveys of published cluster trials have found that their conduct and reporting have been poor. Of 21 cluster trials identified in two major public health journals, only four (19%) had accounted for the clustering in the planning of the trial.15 Similarly, in a review of physicians' patient care practices, 70% (38/54) of the identified studies had not appropriately accounted for the clustered nature of their study data in their analysis.16 Of 16 cluster trials reviewed by Donner et al, only four provided any rationale for adopting a clustered design, only three accounted for clustering in the sample size calculations, and only eight accounted for clustering in the analysis.17
Recent studies have shown continuing problems with the design and analysis of cluster trials; 42% (62/149) of trials of implementation research interventions did not account appropriately for the clustering in their design,18 and 42% (10/24) of trials of clinical decision support systems did not appropriately account for clustering in their analysis (none accounted for clustering in their sample size calculations).19 A recent review of 51 cluster randomised trials conducted in sub-Saharan Africa published in the past 30 years showed only 10 trials took clustering into account in sample size or power calculations, and only 19 took clustering into account in the analysis. Intracluster correlation coefficients and design effects were reported in only one and three trials, respectively.20
Extension of CONSORT statement to cluster trials
To accommodate the reporting of the special features of the cluster randomised trial, we have extended the CONSORT statement to include the following information:
The rationale for adopting a cluster design
How the effects of clustering were incorporated into the sample size calculations
How the effects of clustering were incorporated into the analysis
The flow of both clusters and individuals through the trial, from assignment to analysis.
Specific changes to the checklist items and the flow diagram to include these issues are described below. For items not mentioned here the advice is as for individually randomised trials.3
Table 1 shows the modified checklist with the additions to the standard CONSORT list in italics. In this section, we discuss the rationale for these extensions and provide examples of good reporting. In some examples we have added text in square brackets to explain the context.
Title and abstract
Item 1: How participants were allocated to interventions (eg random allocation, randomised, or randomly assigned), specifying that allocation was based on clusters.
Title: “Self help smoking cessation in pregnancy: cluster randomised trial.”21
Abstract (design): “Pragmatic cluster randomised controlled trial with community midwife as the unit of randomisation,”21
The primary reason for identifying the design in the title or abstract is to ensure appropriate indexing of the study as a cluster randomised trial in Medline. This indexing ensures ease of identification of these studies for inclusion in systematic reviews. In addition, readers will not be misled by apparently large sample sizes. Researchers should also consider reporting the number of clusters in the abstract.
Item 2: Scientific background and explanation of rationale, including the rationale for using a cluster design.
Our intention was to enhance the application of evidence by the whole labour ward team so, to minimise contamination, the unit of randomisation and analysis was the obstetric unit.22
Under the principles of the Helsinki declaration, it is unethical to expose people unnecessarily to the risks of research.23 Because a cluster randomised design increases the complexity of the research and requires many more participants than an individually randomised design (to ensure equivalent statistical power), it is particularly important that the rationale for adopting a cluster design is outlined in the introduction.17
The main difference when reporting a cluster trial, as opposed to an individually randomised trial, is that there are two levels of inference rather than one: the cluster level and the individual level.24 Thus, to allow readers to interpret the results appropriately, it is important to indicate explicitly the level at which the interventions were targeted, the hypotheses were generated, the outcomes were measured, and randomisation was done.
Item 3: Eligibility criteria for participants and clusters and the settings and locations where the data were collected.
The study comprised 41 practices in Wessex… Inclusion criteria were ≥ list size > 7000; a diabetes register with > 1% of practice population; and a diabetes service registered with the health authority… Nurses reported all new cases of diabetes to the trial office. Willing patients aged 30-70 were included in the trial. Patients were excluded if they were private patients, housebound, mentally ill, had severe learning difficulties, or were subsequently found to have been diagnosed previously with, or not to have, diabetes, or were found to have type 1 diabetes.25
Because there are two levels of inference, the eligibility criteria for clusters, as well as participants, need to be reported. In a cluster trial, the primary eligibility criterion is often all the clusters in a defined geographical area.
Item 4: Precise details of the interventions intended for each group, whether they pertain to the individual level, the cluster level, or both, and how and when they were actually administered.
We… paired the 14 [urban sectors of Trujillo, Venezuela] according to the incidence of cutaneous leishmaniasis in the 12 months before the baseline household survey. For each of the seven pairs we randomly allocated one sector… to the intervention group and the other to the control group… In the intervention group the windows of all 241 houses (with a total of 1336 inhabitants) were covered with loosely hanging polyester curtains impregnated with the pyrethroid insecticide… In the 222 houses in six of the control sectors the windows were covered with non-impregnated curtains and in one randomly selected control sector with 106 houses no curtains were provided.”26
Again, if the intervention was targeted at the cluster level, specific details of how it was administered should be described.
Item 5: Specific objectives and hypotheses, and whether they pertain to the individual level, the cluster level, or both.
We aimed to compare the effectiveness of three different interventions for improving the secondary preventive care for patients with coronary heart disease delivered at the level of general practice.27
Descriptions of specific objectives and hypotheses need to make it clear whether they pertain to the individual level, the cluster level, or both. Knowing the level of inference will subsequently aid interpretation of the statistical methods.
Item 6: Report clearly defined primary and secondary outcome measures, whether they pertain to the individual level, the cluster level, or both, and, when applicable, any methods used to enhance the quality of measurements (eg multiple observations, training of assessors).
We evaluated the effect of a computer based clinical decision support system and cardiovascular risk chart [both targeted at physicians—that is, clusters] on patient centred outcomes of absolute cardiovascular risk and blood pressure.28
Whether an intervention is evaluated at the cluster level or the participant level has implications for the appropriate analysis of the outcome data. It is therefore important that the level at which outcomes are measured be explicit in the trial report.
Item 7: How total sample size was determined (including method of calculation, number of clusters, cluster size, a coefficient of intracluster correlation (ICC or k), and an indication of its uncertainty) and, when applicable, explanation of any interim analyses and stopping rules.
We calculated sample size with a method that takes into account the intracluster correlation coefficient, the number of events, the expected effect, and the power of the study. We assumed an intracluster correlation of ρ= 0.2, a minimum of 25 patients for each practice, and a worst case control rate of 50%. Under these assumptions we anticipated a power of 87% to detect a difference of 15% in rates between the two groups with α= 0.05 with 60 practices for each intervention group.29
A principal difference between a cluster randomised trial and an individually randomised trial is the calculation of the sample size. As indicated above, to retain equivalent power to an individually randomised trial, the number of individuals in a cluster randomised trial needs to be increased. The key determinants of the increase required are the intracluster correlation and the cluster size. Reports of cluster randomised trials should state the assumptions used when calculating the number of clusters and the cluster sample size.
Item 8: Method used to generate the random allocation sequence, including details of any restriction (eg blocking, stratification, matching).
To help ensure comparability of the intervention and comparison communities with respect to baseline HIV and STD prevalences and risk factors for infection, the communities were matched into six pairs according to the following criteria: roadside, lakeshore, island, or rural location; geographical area (paired communities were generally in the same district and less than 50 km apart); and prior STD attendance rates at the health centre.30
Cluster randomised trials may use a simple, completely randomised design; a matched cluster design; or a stratified design. In individually randomised trials random assignment generally ensures that any baseline differences in group characteristics are the result of chance rather than some systematic bias.31 This cannot be assumed, however, for the cluster randomised trial.
Although the assumption holds for cluster specific characteristics (that is, characteristics of the randomly allocated clusters), the researcher has little control over the individuals within each cluster7 and the number of clusters is usually relatively small. As a result, some form of constraint (matching or stratification) is often imposed on randomisation in a cluster randomised design in an attempt to minimise imbalance across treatment groups. Any constraint imposed on the cluster randomised trial affects the sample size and the analysis and thus should be reported.
Item 9: Method used to implement the random allocation sequence, specifying that allocation was based on clusters rather than individuals and clarifying whether the sequence was concealed until interventions were assigned.
Practices agreeing to participate were… assigned by simple random allocation to use the computer decision support system… Randomisation was performed with a table of random numbers by a researcher not involved in the study and who was blind to the identity of the practices.28
In individually randomised trials, adequate concealment of the treatment allocation is crucial to minimising potential bias. If the person recruiting participants has foreknowledge of the allocation, bias can result.32 In a cluster randomised trial, allocation of treatment is predetermined for each member of the cluster. Hence the potential for selection bias (selective inclusion of patients into the trial) within clusters is particularly high.4 5 It is, therefore, particularly important that authors outline any strategies that were implemented to minimise the possibility of selection bias—for example, whether all patients within a cluster were included, or, if not, whether recruitment of patients was by a person masked to the cluster allocation.
Item 12: Statistical methods used to compare groups for primary outcome(s) indicating how clustering was taken into account; methods for additional analyses, such as subgroup analyses and adjusted analyses.
Because we randomised obstetric units… we analysed rates of marker clinical practices by obstetric units.22
We used cluster specific methods because practices rather than patients were randomised… We used hierarchical logistic regression.29
Identification of the level of inference allows readers to evaluate the methods of analysis. For example, if the intervention was targeted at the cluster level (at general practitioners rather than patients, for example) and outcomes were aggregated at the cluster level, sophisticated cluster adjusted analyses are not needed (as in the first example above). If outcomes were measured at the individual patient level, however, the analysis would need to adjust for potential clustering in the data (as in the second example above).
Item 13: Flow of clusters and individual participants through each stage (a diagram is strongly recommended). Specifically, for each group report the numbers of clusters and participants randomly assigned, receiving intended treatment, completing the study protocol, and analysed for the primary outcome. Describe protocol deviations from study as planned, together with reasons.
Knowing how many clusters and individuals did not receive the intervention as planned, the proportion dropping out, and the proportion for whom follow up data were not available is essential to interpret the study accurately. For example, individuals excluded after random assignment may not be representative of all participants in the study, and different drop-out rates may be directly related to the treatment received (if, say, one treatment has more severe side effects).33 The potential for differential adherence and follow up is exacerbated in the cluster randomised design because there are two levels at which drop-outs can occur: whole clusters or individuals in a cluster. It is therefore important to describe the flow of both clusters and individuals when reporting a cluster randomised trial. A flow diagram is usually the best way to present this information (see below).
Item 15: Baseline information for each group for the individual and cluster levels as applicable.
See table 2.
Random assignment by individual ensures that any differences in group characteristics at baseline are the result of chance rather than some systematic bias.32 This assumption does not hold, however, for cluster randomised trials. It is important to present summary baseline information for both clusters and individuals, most simply as tables of summary data.
Item 16: Number of clusters and participants (denominator) in each group included in each analysis and whether the analysis was by intention to treat. State the results in absolute numbers when feasible (eg 10/20 not 50%).
See table 3.
The number of participants who contribute to the analysis of a trial is essential to interpreting the results. Not all participants, however, may contribute to the analysis of each outcome. In a cluster trial, this fact is compounded by the possibility that not all clusters may contribute to a particular analysis. Because the sample size calculation, and hence the power of the study, is calculated on the assumption that all participants and (especially) all clusters will provide information, the number of participants and clusters contributing to a particular analysis should be reported so that any potential drop in statistical power can be assessed. The flow diagram can include this information if there is only one primary outcome; otherwise the numbers of participants and clusters contributing should be summarised for each outcome.
Outcomes and estimation
Item 17: For each primary and secondary outcome, a summary of results for each group for the individual or cluster level as applicable, and the estimated effect size and its precision (eg 95% confidence interval) and a coefficient of intracluster correlation (ICC or k) for each primary outcome.
See table 3.
When reporting the results of a cluster randomised trial, point estimates with confidence intervals should be reported for primary outcomes. Given the impact of the extent of the intracluster correlation on the power of the study, the intracluster correlation coefficient or k statistic, for each outcome being analysed should also be provided. This information will allow readers to assess the appropriateness of the original sample size calculations as well as the magnitude of the clustering for each outcome. Showing both adjusted and unadjusted estimates would provide another indication of the extent of the clustering. Several authors have advocated publishing intracluster correlation coefficients to allow them to inform the development of future cluster trials in similar settings.9 35–37
Item 21: Generalisability (external validity) to individuals and/or clusters (as relevant) of the trial findings.
Although our trial was completed successfully from both a methodological and practical point of view, our results may not be generalisable. The 21 participating practices tended to be large, with good nursing support, and may have been particularly committed to improving their quality of care… Furthermore, the observed intervention effect would probably have been greater if the trial had not taken place in the context of a health authority audit initiative relating to patients with coronary heart disease, backed by a financial incentive.”27
In the discussion section of any trial report, the external validity of the results should be considered. External validity is more complicated for cluster randomised trials because the results may be generalisable to the clusters, to the individuals in those clusters, or to both, and thus the level at which external validity is addressed should be identified.
We previously presented three options for modifying the CONSORT flow diagram for presenting clustered data: presenting the flow of data based only on clusters, only on individual participants, or both.8 Further experience suggests that the type of diagram should depend on the type of analysis because different approaches to analysis require information at different levels of the clustered design.
For example, if the analysis is aggregated at the level of the cluster, the flow diagram should relate to the cluster level data (fig 1). To allow meaningful interpretation, a measure of the cluster size (and an indication of how variable cluster sizes are) also needs to be included in the diagram. If, however, the analysis is multilevel or hierarchical, the flow diagram should present data flow for both clusters and individual participants (fig 2).
Although we recommend this diagram for communicating the flow of clusters and participants throughout the study, the exact form and content of the flow diagram should be varied to present the specific features of a trial.
Reports of cluster trials should include key information on the design and analysis to allow readers to accurately interpret the results. This information is also particularly important for meta-analysts attempting to extract data from such reports.39 We therefore recommend that journal editors include our guidelines in their instructions to authors.
Accurate reporting of trials is essential to ensure appropriate interpretation of results
The CONSORT statement provides a framework for reporting individually randomised trials
In some situations it is preferable to randomise groups of individuals (so called cluster randomised trials)
Reports of cluster randomised trials require extra information on their special features
The CONSORT statement has been extended to accommodate these features
Inadequate methodological reporting of trials has been shown to be associated with bias in the estimate of treatment effects.40 Use of the CONSORT statement for the reporting of two group parallel trials is associated with improved reporting quality.41 42 43 We believe that the routine use of this proposed extension to the CONSORT statement will result in similar improvements for cluster trials.
Editorial by Campbell
The CONSORT group is also developing modified recommendations to help improve the quality of reporting of clinical trials of other designs, including equivalence trials, crossover trials and multi-arm trials. The most up to date versions of all CONSORT recommendations can be found at http://www.consort-statement.org/. We thank the members of the CONSORT Group, especially David Moher, Ken Schulz, Tom Lang, David Sackett, Peter G⊘tzsche, Matthias Egger, and John Ioannidis for helpful comments on earlier drafts. The Health Services Research Unit is funded by the Chief Scientist's Office of the Scottish Executive Health Department. DGA is supported by Cancer Research UK. The views expressed are not necessarily those of the funding bodies.
Competing interests None declared.