Papers

Size and quality of randomised controlled trials in head injury: review of published studies

Karen Dickinson, medical student ^a,  Frances Bunn, review group coordinator ^a,  Reinhard Wentz, information specialist ^a,  Phil Edwards, research fellow ^a,  Ian Roberts, director ^b. 

^a Cochrane Injuries Group, Child Health Monitoring Unit, Department of Epidemiology and Public Health, Institute of Child Health, London WC1N 1EH, ^b Child Health Monitoring Unit, Department of Epidemiology and Public Health, Institute of Child Health

Correspondence to: I Roberts ian.roberts{at}ich.ucl.ac.uk

	Abstract

Top Abstract Introduction Methods Results Discussion References

Objective: To assess whether trials in head injury are large enough to avoid moderate random errors and designed to avoid moderate biases.
Design: All randomised controlled trials on the treatment and rehabilitation of patients with head injury published before December 1998 were surveyed. Trials were identified from electronic databases, by hand searching journals and conference proceedings, and by contacting researchers. Data were extracted on the number of participants, quality of concealment of allocation, use of blinding, loss to follow up, and types of participants, interventions, and outcome measures.
Results: 279 reports were identified, containing information on 208 separate trials. The average number of participants per trial was 82, with no evidence of increasing size over time. The total number of randomised participants in the 203 trials in which size was reported was 16 613. No trials were large enough to detect reliably a 5% absolute reduction in the risk of death or disability, and only 4% were large enough to detect an absolute reduction of 10%. Concealment of allocation was adequate in 22 and inadequate or unclear in 25 of the 47 (23%) in which it was reported. Of 126 trials assessing disability, 111 reported the number of patients followed up, and average loss to follow up was 19%. Of trials measuring disability, 26 (21%) reported that outcome assessors were blinded.
Conclusions: Randomised trials in head injury are too small and poorly designed to detect or refute reliably moderate but clinically important benefits or hazards of treatment. Limited funding for injury research and unfamiliarity with issues of consent may have been important obstacles.

	Introduction

Top Abstract Introduction Methods Results Discussion References

Worldwide, many millions of people are treated each year for severe head injury. A substantial proportion die, and many more are permanently disabled.¹ Road traffic accidents alone account for an estimated five million head injuries each year.² Reliable assessment of the net benefits and hazards of various interventions for the treatment and rehabilitation of head injuries could be of considerable importance to public health.

If, for such a common problem, a widely practicable treatment could be shown to reduce the absolute risk of death and disability by "just" a few per cent, this might affect the treatment of hundreds of thousands of patients each year and protect thousands from death or long term disability. To detect reductions of this magnitude, however, both moderate random errors and moderate biases must be avoided.³ This means that randomised controlled trials of treatments for head injury should be large enough to avoid moderate random errors and designed in such a way that moderate biases are also avoided.³ To assess the extent to which randomised controlled trials in head injury meet these criteria we conducted a survey of their size and quality.

	Methods

Top Abstract Introduction Methods Results Discussion References

Inclusion criteria We included all randomised controlled trials of interventions in the treatment and rehabilitation of head or brain injury contained within the specialised register of the Cochrane Injuries Group as of May 1999 and published before December 1998. A randomised controlled trial was defined as a trial in which the patients followed were assigned to one of two (or more) interventions with random allocation or some quasi-random method of allocation. There was no language restriction.

Data extraction and data analysis

Statistical analysis

	Results

Top Abstract Introduction Methods Results Discussion References

The combined search strategies identified 370 reports, of which 279 were reports of randomised controlled trials of interventions for head or brain injury. After taking into account multiple reports of the same randomised controlled trial we identified 208 discrete randomised controlled trials. Most of the reports (86%) were published as full journal articles, with the remainder published as abstracts in conference proceedings (13%) or book chapters (2%). Most reports were published in English (92%), with the remainder in French (2%), Italian (2%), Chinese (1%), German (1%), and Spanish (1%).

There was 90% agreement between assessors for the number of randomised participants and for method of concealment of allocation. In the trials in which there was disagreement about the number of randomised participants the numbers differed by only one participant in each trial.

The number of randomised participants was reported in 203 of the 208 trials. The total number of randomised participants in the 203 trials was 16 613. The average number of participants per trial was 82, with no evidence of a trend towards larger trial size over time. Interventions were classified into categories, and table 1 shows the number of trials and number of randomised participants for each category. The largest trial was of additional information, advice, and support to patients after head injury and included 1156 participants. The largest trial of a pharmacological agent was of the aminosteroid tirilazad mesylate and included 1120 participants. The category of agents for which there was the largest number of randomised participants was corticosteroids with 2515 participants.

The figure shows the size of trials with a curve giving the magnitude of intervention effect (absolute risk reduction) that a trial of that size would be able to detect at the 0.05 level of significance (=0.05) with 80% power (=0.2), with an assumed baseline risk of 0.2. None of the existing trials could be expected to detect reliably a difference in event rates of less than 5%, and only 4% of trials would have been large enough to detect a difference in event rates of less than 10%.

View larger version (29K): [in this window] [in a new window]   Trial size and smallest absolute risk reduction detectable

The method of concealment of allocation was reported in only 47 (23%) of the trials. Of these, concealment was judged to be adequate in 22 (11%) and inadequate or unclear in 25. Table 2 shows the methods of allocation used in the trials.

The Glasgow coma scale was used as an inclusion criterion in 56 trials. Of these, 51 included patients with severe head injury, 17 included patients with moderate head injury, and eight trials included patients with mild head injury. A total of 126 trials assessed the effect of the trial intervention on the extent of disability. The most commonly used outcome measure was the Glasgow outcome scale, used in 45% of trials assessing disability. Of the 126 trials assessing disability, 111 reported the number of patients followed up. In those trials, the average loss to follow up was 19%. Of the trials measuring disability, only 26 (21%) reported that the outcome assessors had been blinded.

	Discussion

Top Abstract Introduction Methods Results Discussion References

There is a growing recognition that large absolute reductions in death or disability cannot realistically be expected in the treatment of head injury.⁵ Changes in outcome of "only" a few per cent would nevertheless be important, especially if achievable with widely practicable treatments. To detect such moderate treatment effects, both moderate random errors and moderate biases must be avoided.

Failures of existing research
The results of our review suggest that currently available trials in head injury fail on both counts. Most trials of head injury are small. The average number of randomised participants was 82, and the largest trial included only 1156 participants. None of the trials would have been large enough to detect reliably the difference between a 20% and a 15% risk of death or disability. If baseline risk was higher than 20%, then existing trials would have even lower power to detect any given absolute reduction in the risk of death or disability.

Problems leading to failure
Why is it that those who provide care for patients with head injury have such a meagre evidence base on which to draw? Two factors in particular may be important. Firstly, in comparison with the burden of disability, funding for injury research is less than for almost any other cause of human misery,¹³ perhaps because there are fewer advocates for research in head injury than for cardiovascular disease or cancer. Secondly, people with head injury and impaired consciousness are unable to give informed consent. They present an important exception to the general requirement to obtain informed consent in clinical research.¹⁴ Nevertheless, many ethics committees and investigators are unfamiliar with the idea of randomisation without consent, and the resulting confusion has been an important obstacle to enrolment. It was concern that safer and more effective interventions for life threatening emergencies were not being developed because of the need to obtain informed consent, however, that stimulated the US Department of Health and Human Services to announce in 1995 a waiver of requirements for informed consent in certain emergency research.¹⁵ It will be interesting to see whether, over the next decade or so, this has the hoped for effect on trial size.

	Acknowledgments

We thank Simona Fiore, Olivier Duperrex, and Leah Li for translating the Italian, Spanish, French and Chinese trial reports; and Iain Chalmers, Doug Altman, and Richard Peto for their comments on the manuscript.

Contributors: KD extracted the data and commented on the paper. FB extracted the data, created the trial database, analysed the data, and commented on the paper. RW conducted the searches, obtained the trial reports, translated the German papers, and commented on the paper. PE conducted the statistical analyses, prepared the figures, and commented on the paper. IR proposed the study, coordinated its conduct, and wrote the paper. IR is guarantor.

	Footnotes

Funding: The Cochrane Injuries Group is funded by the NHS Executive.

Competing interests: None declared.

	References

Top Abstract Introduction Methods Results Discussion References

1.	Murray CJL, Lopez AD. Global health statistics: a compendium of incidence, prevalence and mortality estimates for over 200 conditions. Boston: Harvard University Press, 1996.
2.	World Bank. Global road safety partnership. Washington, DC: World Bank, 1999.
3.	Peto R, Collins R, Gray R. Large-scale randomised evidence: large simple trials and overviews of trials. J Clin Epidemiol 1995; 48: 23-40[CrossRef][Medline].
4.	Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273: 408-412[Abstract].
5.	Teasdale G. The treatment of head trauma: implications for the future. J Neurotrauma 1991; 8(suppl 1): 53-60.
6.	Royal College of Surgeons of England. Report of the working party on the management of patients with head injury. London: Royal College of Surgeons, 1999.
7.	Roberts I, Schierhout G, Alderson P. Absence of evidence for the effectiveness of five interventions routinely used in the intensive care management of severe head injury: a systematic review. J Neurol Neurosurg Psychiatry 1998; 65: 729-733[Abstract/Free Full Text].
8.	CAST (Chinese Acute Stroke Trial) Collaborative Group. CAST: randomised placebo-controlled trial of early aspirin use in 20 000 patients with acute ischaemic stroke. Lancet 1997; 349: 1641-1649[CrossRef][Medline].
9.	ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both or neither among 17187 cases of suspected acute myocardial infarction. Lancet 1988; ii: 349-360.
10.	International Stroke Trial Collaborative Group. International stroke trial (IST): a randomised trial of aspirin, subcutaneous heparin, both, or neither among 19 435 patients with acute ischaemic stroke. Lancet 1997; 349: 1569-1581[CrossRef][Medline].
11.	Kunz R, Oxman AD. The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ 1998; 317: 1185-1190[Abstract/Free Full Text].
12.	Liberati A, Himel HN, Chalmers TC. A quality assessment of randomised controlled trials of primary treatment of breast cancer. J Clin Oncol 1986; 4: 942-951[Abstract/Free Full Text].
13.	Ad Hoc Committee on Health Research Relating to Future Intervention Options. Investing in health research and development. Geneva: World Health Organization, 1996.
14.	Smith R. Informed consent: the intricacies. BMJ 1997; 314: 1059-1060[Free Full Text].
15.	Department of Health and Human Services, Food and Drug Administration. Protection of human subjects: informed consent. Fed Reg 1995; 60: 49086-49103.
16.	Yates D, Farrell B, Teasdale G, Sandercock P, Roberts I. Corticosteroids in head injurythe CRASH trial. J Accid Emerg Med 1999; 16: 83-90[Medline].
17.	www.crash.ucl.ac.uk (accessed 10 Mar 2000).

(Accepted 3 February 2000)