BMJ  2005;330:1057-1058 (7 May), doi:10.1136/bmj.38413.576713.AE (published 10 March 2005)

Paper

Adequacy and reporting of allocation concealment: review of recent trials published in four general medical journals

¹ York Trials Unit, Department of Health Sciences, University of York, York YO10 5DD, ² Medical Research Collaborating Center, Seoul National University Hospital/College of Medicine, Seoul 110-744, Korea

Introduction

In randomised controlled trials, allocation concealment (separating the process of randomisation from the recruitment of participants) is important for rigorously designed trials.^1-4 In 1996 many major medical journals adopted the CONSORT statement (whereby researchers have to include a short checklist of essential items and a flow diagram when reporting trials),⁵ and this move encouraged the reporting of allocation concealment. We reviewed the prevalence of adequate allocation concealment and its association with the statistical significance of trial results.

Methods and results

We searched by hand four general medical journals (the BMJ, JAMA, the Lancet, and the New England Journal of Medicine) to identify randomised controlled trials published from January 2002 to December 2002. We included articles if the authors reported that participants were randomised and if the trial was published as a full report with the results of the main analyses. We categorised articles according to whether allocation concealment was adequate (the person who executed the allocation sequence was different from the person who recruited participants), inadequate (the person who recruited participants also executed the allocation sequence), or unclear (the article failed to describe how the researchers concealed the allocation). We considered the widely used "sealed envelope" method to be inadequate unless performed by an independent third party. We used a kernel density plot to compare the P values of trials that used adequate concealment methods with those that used inadequate methods; we used P values because these were readily available across most of the trials, which used different statistical methods and outcome measures. Our statistical analyses adjusted for clustering effects by journal.

Among the 234 trials that met the inclusion criteria, allocation concealment was adequate in 132 (56%) and inadequate in 41 (18%); in 61 (26%) the concealment method was unclear. Of the trials whose allocation concealment was considered adequate, 118 used independent allocation (which included using a telephone, fax machine, or pager to a randomisation service); five used sealed envelopes opened by a third party; eight used a computer; and one used a combination of adequate methods. Of the 41 trials whose allocation concealment was inadequate, 39 used sealed envelopes, one selected a card from a pile, and one added the name of the next participant to the randomisation list.

What is already known on this topic The effect of adequacy of allocation concealment in randomised controlled trials may influence the degree of effect What this study adds Despite researchers' acceptance that adequate allocation concealment is important, almost a fifth of trials recently published in major medical journals used inadequate concealment and a quarter failed to describe how the allocation was concealed

For 28 (17%) of the 166 trials published in the CONSORT journals (the BMJ, the Lancet, and JAMA) and 33 (49%) of the 68 trials published in the non-CONSORT journal (the New England Journal of Medicine) we could not discern whether allocation was adequately concealed. We excluded 97 of the original 234 trials from the P value analysis because a single P value was not extractable (for example, unclear primary outcome). The figure shows a trend towards lower P values for inadequately concealed trials (mean = 0.022); P values from trials that used adequate methods were more widely spread around a higher mean value (0.052). The difference between these two mean values was significant (P = 0.045). In a logistic regression, adjusted for sample size and journal, we found that the odds of a trial with inadequate concealment yielding a significant result (P 0.05) compared with a trial with adequate concealment was 1.8 (95% confidence interval 0.8 to 3.7), thus suggesting that trials using inadequate concealment tend to show significant differences between the groups in the primary outcome more often than trials using adequate concealment.

View larger version (26K): [in this window] [in a new window]   Distribution of P values by adequacy of allocation concealment. As the P values were highly positively skewed, the data were transformed using the logit function. The vertical lines represent mean P values for trials using adequate or inadequate concealment

 

Comments

We found that despite the CONSORT statement more than 40% of trials published in major medical journals either did not use adequate methods or failed to describe how they concealed the allocation. Our results confirm the association found in previous studies that trials using inadequate allocation concealment are more likely to report significant findings than those using adequate concealment. Readers should critically assess the reported methods of allocation concealment.

---

We thank Doug Altman for reviewing an earlier version of the paper and providing valuable feedback. A list of included studies is available on request from CH.

Contributors: DJT suggested the idea of the review. CH, DJT, and JW did the searches and extracted data from the included papers. CH, with advice from SH and JMB, undertook the data analysis. CH wrote the paper, and JMB, SH, DJT, and JW commented on it. CH is the guarantor.

Funding: Department of Health Sciences, University of York. Competing interest statement: None declared.

Ethical approval: Not needed. doi 10.1136/bmj.38413.576713.AE

This article was posted on bmj.com on 10 March 2005: http://bmj.com/cgi/doi/10.1136/bmj.38413.576713.AE

References

1. Juni P, Altman DG, Egger M. Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 2001;323: 42-6.[Free Full Text]
2. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273: 408-12.[Abstract]
3. Chalmers T, Celano P, Sacks H, Smith H. Bias in treatment assignment in controlled clinical trials. N Engl J Med 1983;309: 1358-61.[Abstract]
4. Emerson JD, Burdick E, Hoaglin DC, Mosteller F, Chalmers TC. An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials. Control Clin Trials 1990;11: 339-52.[CrossRef][ISI][Medline]
5. Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 2001;134: 663-94.[Abstract/Free Full Text]

(Accepted 28 January 2005)

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Is restricted randomisation necessary?

Catherine E Hewitt and David J Torgerson
BMJ 2006 332: 1506-1508. [Full Text] [PDF]

Reporting attrition in randomised controlled trials

Jo C Dumville, David J Torgerson, and Catherine E Hewitt
BMJ 2006 332: 969-971. [Full Text] [PDF]

Systematic reviews in health care: Assessing the quality of controlled clinical trials

Peter Jüni, Douglas G Altman, and Matthias Egger
BMJ 2001 323: 42-46. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• Mathoulin-Pelissier, S., Gourgou-Bourgade, S., Bonnetain, F., Kramar, A. (2008). Survival End Point Reporting in Randomized Cancer Clinical Trials: A Review of Major Journals. JCO 26: 3721-3726 [Abstract] [Full text]  
• Sedrakyan, A. (2007). Response to Letter by Tourmousoglou et al. Stroke 38: e84-e85 [Full text]  
• Hewitt, C. E, Torgerson, D. J (2006). Is restricted randomisation necessary?. BMJ 332: 1506-1508 [Full text]  
• Dumville, J. C, Torgerson, D. J, Hewitt, C. E (2006). Reporting attrition in randomised controlled trials. BMJ 332: 969-971 [Full text]  
• McCulloch, M., See, C., Shu, X.-j., Broffman, M., Kramer, A., Fan, W.-y., Gao, J., Lieb, W., Shieh, K., Colford, J. M. Jr (2006). Astragalus-Based Chinese Herbs and Platinum-Based Chemotherapy for Advanced Non-Small-Cell Lung Cancer: Meta-Analysis of Randomized Trials. JCO 24: 419-430 [Abstract] [Full text]  
• Shu, X., McCulloch, M., Xiao, H., Broffman, M., Gao, J. (2005). Chinese Herbal Medicine and Chemotherapy in the Treatment of Hepatocellular Carcinoma: A Meta-analysis of Randomized Controlled Trials. Integr Cancer Ther 4: 219-229 [Abstract]  

Rapid Responses:

Read all Rapid Responses

Effect of publication bias

Kerenza Hood
bmj.com, 15 Mar 2005 [Full text]