Rapid Responses to:

PAPERS: Karen Dickinson, Frances Bunn, Reinhard Wentz, Phil Edwards, and Ian Roberts Size and quality of randomised controlled trials in head injury: review of published studies BMJ 2000; 320: 1308-1311 [Abstract] [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

A Multi-centre study into the management of head injured children

Joel Desmond   (20 May 2000)

Evidence for best practice is still being delayed by poor trials

M D D Bell   (23 May 2000)

Large scale trials of patients with severe head injury would be difficult and expensive

Nicola Latronico   (3 June 2000)

Size and quality of randomised controlled trials in head injury

Chantal W P M Hukkelhoven, Ewout W Steyerberg, Andrew I R Mass   (3 June 2000)

Size and quality of randomised controlled trials in head injury

Gordon D Murray, Graham M Teasdale   (31 July 2000)

A Multi-centre study into the management of head injured children 20 May 2000
Joel Desmond, Emergency Medicine Research Group Manchester Royal Infirmary Send response to journal: Re: A Multi-centre study into the management of head injured children	Dr. Roberts et al. are to be congratulated for highlighting the deficiency in head injury research. The situation is no better for head injured children. The American Academy of Pediatrics issued a report based on extensive literature review of 108 articles on head injury in children (1). Their conclusion was that 'The literature on mild head trauma does not provide a sufficient scientific basis for evidence based recommendations about most of the key issues in clinical management'. In order to address this issue a 'multicentre study to develop a set of clinical decision rules for the management of head injured children' is now in its third month of data collection. All head injured children at nine hospitals in the Northwest are seen by the attending doctor using a tailored study proforma. 40 clinical correlates relating to symptoms, signs and investigations are entered.We are collecting patients at the rate of 1300 per month. Once 15,000 forms have been collected, all admission, Neurosurgical intervention, and mortality data will be collected. A set of clinical decision rules will then be derived using recursive partitioning (as with the Ottowa ankle guidelines). The guidelines will then be validated in a further 15,000 patients. This is the first time that such an approach has been used in the management of head injury, and I agree with the authors of the article that it is only by large, well conducted trials that we are going to advance the evidence base in head injury 1).Technical report: Minor Head injury in children. C.J.Homer, L Kleinman, Pediatrics 1999;104(6). www.pediatrics.org/cgi/content/full/104/6/e78
Evidence for best practice is still being delayed by poor trials 23 May 2000
M D D Bell Send response to journal: Re: Evidence for best practice is still being delayed by poor trials	The Cochrane Group states that previous randomized trials in head injury are too poorly designed to detect the benefits or hazards of treatment.1 The conclusion that large scale trials have the potential to make an important contribution to the prevention of death or disability is unquestionable. The reference to the CRASH trial2 as an example of this, is however worthy of comment, since there is concern as to both the hazards of high-dose steroids and the extent and specificity of the data collection in that study. The metabolic consequences of the steroid may well be significant, with either high glucose levels constituting a secondary cerebral insult, or the use of insulin to control this, altering potassium balance between compartments, with the potential for dysrhythmias. It does not appear that these aspects are under surveillance. The immuno-suppressive impact of the steroid may well prove deleterious for a sub-set of patients who currently suffer from a very aggressive bacterial pneumonia unresponsive to anti-microbial therapy, exemplifying the need to be specific about the cause of death. Concerns about data collection relate not only to these aspects but to the diagnostic sub-groups. The GCS does not appear to be qualified by consideration as to whether a reduction could be attributable to alcohol or drugs or a post-ictal state. The data on CT findings is extremely limited with; a. reliance on either no scan or just one, this despite an early scan under-stating damage, b. no differentiation between a subdural and extra- dural haematoma and c. no record of the site, size or number of any intra -cerebral haematoma / contusion. Without the information to determine which subsets, if any, are capable of benefiting, no proper record of the complications of the trial drug, and no standardization or data collection on other aspects of head injury management, one could conclude that the potential risks outweigh any overall benefits. This consideration also raises questions as to the validity of not seeking consent for this trial. The neurosurgical and neuro-intensive care staff at this institution have felt unable to sanction inclusion in the trial of any patients from within the region until these concerns are addressed. Despite the laudable goals of the project group, it thus appears that evidence for best practice is still being delayed by trials that are either poorly designed or poorly communicated. Conflict of interest: none DR MDD BELL Consultant in Intensive Care/Anaesthesia 1. Dickinson K, Bunn F, Wentz R, Edwards P, Roberts I. Size and quality of randomised controlled trials in head injury: review of published studies. BMJ 2000; 320: 1308-1311. 2. Yates D, Farrell B, Teasdale G, Sandercock P, Roberts I. Corticosteroids in head injury-the CRASH trial. J Accid Emerg Med 1999; 16: 83-90.
Large scale trials of patients with severe head injury would be difficult and expensive 3 June 2000
Nicola Latronico Send response to journal: Re: Large scale trials of patients with severe head injury would be difficult and expensive	EDITOR - Dickinson et al message that large scale randomised controlled trials (RCT) are needed to detect small yet important drug effects is to be shared (1). However, it is questionable that mild, moderate and severe head injury were all included in their review and treated as a unitary subject. Differences among these classes are substantial, and try to find common explanations for the lack of good quality studies seems illogical. Brain injury incidence in the USA is around 200/100,000 per year, and 80% of head trauma patients admitted to hospitals have suffered minor injuries (2). Mortality is almost zero, while morbidity is relevant, therefore large scale RCT are a realistic and highly desirable goal. Severe head trauma patients are a minority (10%) of hospital admissions and mortality is around 30% at the best (3). More importantly, hospital treatment for such patients is highly complex and prolonged, and a 10,000 RCT would involve hundreds of centres. It would likely cost several hundred million dollars, an unaffordable task to date. Clinicians should take Dickinson' recommendations in great consideration when engaging in new studies. At the same time, epidemiologists should maintain a link with clinicians, not to loose the sense of reality. Nicola Latronico, MD Istituto di Anestesia e Rianimazione, University of Brescia, Spedali Civili, 25125 Brescia, ITALY 1. Dickinson K, Bunn F, Wentz R, Edwards P, Roberts I. Size and quality of randomised controlled trial in head injury: review of published studies. BMJ 2000: 320:1308-1311 2. Kraus JF Epidemiology of head injury. In: Head Injury, Cooper PR (Ed), 3rd Ed, Williams & Wilkins, Baltimore, 1993:1-25 3. Unterberg A. Severe head injury: improvement of outcome. Intensive Care Med 1999; 25:348-349
Size and quality of randomised controlled trials in head injury 3 June 2000
Chantal W P M Hukkelhoven, epidemiologist, epidemiologist, neurosurgeon Erasmus Medical Centre Rotterdam, P.O. Box 1738, 3000 DR Rotterdam, the Netherlands, Ewout W Steyerberg, Andrew I R Mass Send response to journal: Re: Size and quality of randomised controlled trials in head injury	We agree with Dickinson et al.1 that larger and better designed randomised controlled trials are necessary to detect benefits of treatment in head injury2. However, increasing the sample size is not the only solution to demonstrate efficacy. The statistical power of a study can also be enhanced by randomising the same number of patients but taking prognostic factors, such as age or Glasgow Coma Scale, into account. First, one might limit the inclusion of patients to those with an intermediate prognosis3 e.g. between 20% and 80% probability of a favourable outcome. This leads to a focus on patients for whom treatment effects can be well determined. For the same power, a 30% reduction in sample size might be achievable3. After showing efficacy in the intermediate risk group, additional funding may be raised more easily to study patients with a poorer or better prognosis. Note that this reasoning assumes that the relative effect of a treatment is constant across risk groups. This in contrast to the assumption of an absolute effect of 5% as discussed by Dickinson et al1. Such an absolute effect is relatively large at a baseline incidence of 20%, as indicated by an odds ratio of 0.71 for the comparison of 15% versus 20% incidence. In contrast, the odds ratio is 0.82 for the same absolute effect at 50% baseline incidence (45% versus 50%). The 5% absolute effect is more easily detected at 20% baseline incidence, while a relative effect such as an odds ratio of 0.71 is more easily detected at 50% incidence. So, the assumption of an absolute or relative effect is crucial in reasoning about power and inclusion criteria. Second, even if inclusion would be limited to patients at intermediate risk, heterogeneity will remain regarding the probability of a favourable outcome. Predictive characteristics which account for this heterogeneity can be adjusted for in the analysis, which will increase the statistical power to detect a treatment effect4. In an analysis of acute MI patients, the potential reduction in sample size was 12% by adjustment for age5. Besides dealing with heterogeneity, we may also consider to restrict data collection to the 'essential' variables, such that larger numbers of patients are accrued at the same costs. We hope that the here proposed strategies will be applied in the study of therapy for head injury, together with an increase in funding. Competing interests: none. 1. Dickinson K, Bunn F, Wentz R, Edwards P, Roberts I. Size and quality of randomised controlled trials in head injury: review of published studies. BMJ 2000; 320: 1308-1311. 2. Maas AIR, Steyerberg EW, Murray GD, Bullock R, Baethmann A, Marshall LF, Teasdale GM. Why have recent trials of neuroprotective agents in head injury failed to show convincing efficacy? A pragmatic analysis and theoretical considerations. Neurosurgery 1999; 44:1286-98. 3. Machado SG, Murray GD, Teasdale GM. Evaluation of designs for clinical trials of neuroprotective agents in head injury. J Neurotraum 1999; 16: 1131-1138. 4. Robinson LD, Jewell NP. Some surprising results about covariate adjustment in logistic-regression models. Int Stat Rev 1991;59:227-240. 5. Steyerberg EW, Bossuyt PMM, Lee KL. Clinical trials in acute myocardial infarction: Should we adjust for baseline characteristics? Am Heart J 2000; 139:745-751.
Size and quality of randomised controlled trials in head injury 31 July 2000
Gordon D Murray, Professor of Medical Statistics, Professor of Neurosurgery University of Edinburgh, University of Glasgow, Graham M Teasdale Send response to journal: Re: Size and quality of randomised controlled trials in head injury	Dear Sir The review published by Roberts & Colleagues1 shows a remarkable lack of insight into the development and assessment of therapeutic interventions in head injury. Indeed, we find it surprising that their expert advisers did not recommend that the authors should reread the paper 'Why do we need some [our italics] large, simple randomized trials?' by S Yusuf, R Collins and R Peto2. Clearly in many clinical areas trials have historically been underpowered, but that does not mean that all trials need to be large. Dickinson et al either ignore or are ignorant of the range of reasons for conducting comparative investigations in head injury, and the need for matching the study design to the research question. We are not told when the papers analysed were published, but it is likely that this was at least a timespan of 3 decades. Clinical head injury investigators have not been unaware of the substantial advances there have been in clinical trial technique over this time. The reference to an article that was written more than a decade ago by one of us (GT) in support of a view that there is 'growing recognition', highlights the reviewers' lack of involvement in and understanding of developments since then. The premise in the authors' report is that the aim in investigations in head injury should be to detect changes in the rate of occurrence of death and disability by 'a few percent'. The trials they analysed were not all performed for this purpose and it is completely inappropriate for the authors to judge all previous research by applying a standard that was not appropriate when the study was done. Thus phase III studies (in which substantial numbers are appropriate), do not just happen but instead are the final step in what may be a long process of preclinical development and successive phase I / phase II testing of safety and potential efficacy. In most areas of clinical medicine, published trials are dominated by phase I / phase II 'explanatory' studies. Such trials will not have as their primary focus clinical outcome measures such as mortality or the Glasgow Outcome Scale. These may be reported, not in the expectation that the trial should have power to detect beneficial effect, but rather as reassurance that a major safety issue had not been identified before embarking on a phase III trial. Moreover, the clinical outcome measures of death and disability which are relevant to phase III studies of treatment in the acute stage of head injury are not necessarily appropriate for studies aimed at later intervention. This seems likely to have been the case in a substantial number of the studies reviewed by Dickinson et al, for example those concerning antidepressants, anti seizure medication and antispasticity drugs, physical therapy, neurotropics, phospholipids, pyritinol, physiotherapy and stimulants. The authors' stress on 'moderate treatment effects' in the context of 'a widely practicable treatment' is self evident but needs careful thought in the context of severe head injury. Some potential interventions in severe head injury are not widely practicable, are potentially toxic and likely to be expensive. Against this background, the aim of research has been to seek substantial benefit rather than a modest one. Preclinical studies of many agents have shown striking consistent efficacy leading to a reasonable expectation of substantial benefit in clinical trials. Support for this expectation was also founded on the finding that an agent of only modest experimental efficacy, (nimodipine), was able to achieve an absolute reduction of 13% in unfavourable outcome in acute haemorrhagic brain damage. A Cochrane systematic review of the same agent in head injury suggests the possibility of a substantial treatment benefit in an important group of patients, with an odds ratio for death of 0.59 and for death or severe disability of 0.676. Concepts of the size of effect that it is desired to detect (i.e. not to miss) have been well known to head injury researchers for two decades but Roberts & Colleagues make no acknowledgement of this nor of the factors that may make it appropriate to seek substantial benefit. One of these is economic and the realisation that in acute hospital budgets, patient benefit from different interventions has to be 'traded off'. The use of the 'number needed to treat' concept to express a treatment effect is ignored by Dickinson and colleagues but illustrates the issue. Thus there is doubt amongst clinicians that budget holders will be prepared to support a new intervention, costing some thousands of pounds, if fifty patients need to be treated for one to benefit. A postulated treatment effect of this magnitude is the basis of the CRASH trial in which the corresponding author, Dr Roberts, is the principal applicant. The focus on a 10% benefit has reflected a perception that more success may be expected in obtaining funding for treatment that benefits one person in ten. However, even this may be optimistic; despite the 13% benefit obtained from nimodipine treatment in subarachnoid haemorrhage, corresponding to a number needed to treat of 8, clinicians had difficulties in gaining funding for its use in routine clinical treatment. The dominance of cost/benefit issues over 'statistical significance' in the case of b- interferon is an even more topical example of this issue. The extent to which the individual's personal state may be improved - eg the merits of being '2%' or '10%' better, is only just beginning to be explored. A particular limitation of the review is that it does not acknowledge that limitations of past trials are widely recognised by those working in the area. Recent experience is being thoughtfully and we hope creatively analysed in order to refine trial design with the aim of having the sensitivity to detect more modest benefits3,4,5. Techniques such as targeting pathophysiological mechanisms and prognosis, relating expected outcome to initial prognosis, and how this will be changed by treatment are actively being explored. Conversely, the authors turn a 'blind eye' to the problems facing large scale studies in head injured patients - case ascertainment in the early stage and a follow-up rate that may not reach, let alone exceed, 70%. One of the problems highlighted by Roberts et al as 'leading to failure' was an unfamiliarity on the part of Ethics Committees and investigators of the idea of randomisation without consent, resulting in confusion and an obstacle to enrolment. This view is erroneous, and displays a dangerously superficial attitude towards the immensely complex ethical and legal issues in this area. What needs to be clarified is not the attitudes of investigators and committees to the ethical issues; instead, it is the legal framework in which research in incompetent adults takes place. Thus, recent legislation in the Scottish Parliament contains no provision for an exception to the requirement to obtain informed consent (Adults with Incapacity (Scotland) Bill, Clause 48 (3) (f)). Efforts over several years to remedy this by clinicians and experienced investigators in Scotland were unsuccessful. A letter dated 2nd May 2000 from the Health Department of the Scottish Executive states 'legal advice was that there was uncertainty that research in an "emergency situation", without consent from a proxy, would be consistent with the European Convention of Human Rights and also with the European Convention on Human Rights & Biomedicine'; a matter for Westminster. A consultation process similar to that which led to legislation in Scotland has been going now for some years in England and it cannot be assumed, until this has been translated into legislation, that this will provide a legal framework for research without consent. The superficial treatment of this issue by Roberts et al is all the more surprising because one of us (who was involved in the events that led to revised regulations in the USA) directed the corresponding author's attention to these issues a year ago. Good reasons for conducting large simple studies in head injury can be put forward without denigrating previous work. In writing an article that displays ignorance of much of the development in the field in the last decade, and by ignoring the possibility of dialogue with experienced workers over the contents of the article, the approach of the authors risks alienation rather than the constructive, respectful dialogue from which progress is likely to emerge. A further 'reason for failure' identified by the authors is a lack of funding and this is coupled with the statement 'large randomised controlled trials of widely practicable treatments for head injuries are needed'. The corresponding author has been in receipt of MRC funding for a pilot study of a trial of this type (CRASH), and is an applicant to the MRC for very substantial funding for continuation of this study into a full phase trial, in part supported by the manufacturer of the agent under trial. In view of this, and his apparently strong position on this issue7, it may be found surprising that no competing interests were declared. Yours faithfully, Gordon D Murray, Professor of Medical Statistics, University of Edinburgh Graham M Teasdale, Professor of Neurosurgery, University of Glasgow. Competing Interests GDM is a director of a charitable organisation, the European Brain Injury Consortium, and in this role has been active in providing statistical advice to a number of pharmaceutical companies on the design, conduct and analysis of clinical trials in head injury. These comprise: Bayer; Cambridge Neuroscience; Novartis; Pharmos; SmithKline Beecham; Synthelabo. GMT: in addition to extensive declared interests in head injury (BMJ 2000; 230: 1631-1635), was a co-applicant to the Medical Research Council and a member of the Steering Committee for the pilot phase of the CRASH study but is not an applicant for funding for full phase and has withdrawn from the Steering Committee References 1. Dickenson K, Bunn F, Wentz R, et al. Size and quality of randomised controlled trials in head injury: a review of published studies. BMJ 2000; 320: 1308-1311. 2. Yusuf S, Collins R, Peto R. Why do we need some large simple randomized trials? Statistics in Medicine 1984; 3: 409-420. 3. Maas AIR, Steyerberg EW, Murray GD, et al. Why have recent trials of neuroprotective agents in head injury failed to show convincing efficacy? A pragmatic analysis and theoretical considerations. Neurosurgery 1999; 44: 1286-1298. 4. Machado SG, Murray GD, Teasdale GM. Evaluation of designs for clinical trials of neuroprotective agents in head injury. Journal of Neurotrauma 1999; 16: 1131-1138. 5. Reinert MM, Bullock R. Clinical trials in head injury. Neurol Res 1999; 21: 330-338. 6. Langham J, Goldfrad C, Teasdale G, et al. Assessing the effect of calcium channel blockers in patients with acute traumatic brain injury on mortality and neurological disability. In: The Cochrane Library, Issue 3, 1998. Oxford: Update Software. 7. Roberts I. Beyond conflict of interest - competing interest still exists. BMJ 1999; 318: 465.