Jump to: Page Content, Site Navigation, Site Search,
You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.
BMJ 2007;334:197 (27 January), doi:10.1136/bmj.39048.407928.BE (published 15 December 2006)
Pablo Perel, research fellow1, Ian Roberts, clinical coordinator CRASH 2 trial1, Emily Sena, PhD student2, Philipa Wheble, medical student2, Catherine Briscoe, medical student2, Peter Sandercock, professor of medical neurology2, Malcolm Macleod, senior lecturer2, Luciano E Mignini, researcher3, Pradeep Jayaram, senior house officer4, Khalid S Khan, professor of obstetrics-gynaecology4
1 Crash Trials Coordinating Centre, London School of Hygiene and Tropical Medicine, London WC1E 7HT, 2 Clinical Neurosciences, University of Edinburgh, 3 Centro Rosarino de Estudios Perinatales, WHO Collaborative Centre in Maternal and Child Health, Rosario 2000, Argentina, 4 Division of Reproductive and Child Health, Birmingham Women's Hospital, University of Birmingham
Correspondence to: P Perel pablo.perel{at}lshtm.ac.uk
Study design Systematic review.
Data sources Medline, Embase, SIGLE, NTIS, Science Citation Index, CAB, BIOSIS.
Study selection Animal studies for interventions with unambiguous evidence of a treatment effect (benefit or harm) in clinical trials: head injury, antifibrinolytics in haemorrhage, thrombolysis in acute ischaemic stroke, tirilazad in acute ischaemic stroke, antenatal corticosteroids to prevent neonatal respiratory distress syndrome, and bisphosphonates to treat osteoporosis.
Review methods Data were extracted on study design, allocation concealment, number of randomised animals, type of model, intervention, and outcome.
Results Corticosteroids did not show any benefit in clinical trials of treatment for head injury but did show a benefit in animal models (pooled odds ratio for adverse functional outcome 0.58, 95% confidence interval 0.41 to 0.83). Antifibrinolytics reduced bleeding in clinical trials but the data were inconclusive in animal models. Thrombolysis improved outcome in patients with ischaemic stroke. In animal models, tissue plasminogen activator reduced infarct volume by 24% (95% confidence interval 20% to 28%) and improved neurobehavioural scores by 23% (17% to 29%). Tirilazad was associated with a worse outcome in patients with ischaemic stroke. In animal models, tirilazad reduced infarct volume by 29% (21% to 37%) and improved neurobehavioural scores by 48% (29% to 67%). Antenatal corticosteroids reduced respiratory distress and mortality in neonates whereas in animal models respiratory distress was reduced but the effect on mortality was inconclusive (odds ratio 4.2, 95% confidence interval 0.85 to 20.9). Bisphosphonates increased bone mineral density in patients with osteoporosis. In animal models the bisphosphonate alendronate increased bone mineral density compared with placebo by 11.0% (95% confidence interval 9.2% to 12.9%) in the combined results for the hip region. The corresponding treatment effect in the lumbar spine was 8.5% (5.8% to 11.2%) and in the combined results for the forearms (baboons only) was 1.7% (1.4% to 4.7%).
Conclusions Discordance between animal and human studies may be due to bias or to the failure of animal models to mimic clinical disease adequately.
The usefulness of animal testing has, however, been questioned.3 4 5 Some believe that the results from animal experiments cannot be applied to humans because of the biological differences between the species and because the results of animal experiments often depend on the type of animal model.3 To date the methods used to assess the value of animal trials include historical analyses, critiques of animal models, surveys of clinicians, and citation analyses. In this paper we compared treatment effects from systematic reviews of clinical trials with those of our own systematic review of the corresponding animal experiments.6 7 8
We searched Medline, Embase, SIGLE (System for Information on Grey Literature), NTIS (National Technical Information Service), Science Citation Index, CAB, and BIOSIS. Details of the search strategies are presented elsewhere.18 Reference lists were checked and we contacted the authors of included studies, relevant drug companies, and the authorities that regulate animal testingthe Home Office in the United Kingdom. No language restrictions were applied. To reduce the number of missed studies two reviewers examined the results for potentially relevant interventions.22 These reports were retrieved in full. Two reviewers independently applied the selection criteria.
Eligible reports were assessed for methodological quality.23 Two reviewers extracted data on allocation concealment and blinding. If the method of allocation concealment was not clearly reported, we tried to contact the authors for clarification. We used Schulz et al's definition for adequate concealment.24 Two reviewers extracted data on study design, allocation concealment, number of randomised animals, type of model, intervention, outcome, and funding source. We contacted authors if relevant outcome measures were not reported but we believed the data to be available. The reviewers were not blinded to the authors or journal.25
For dichotomous measures (for example, mortality) we estimated odds ratios and confidence intervals and for continuous measures (for example, infarct volume) we estimated the effect size:
Effect size=((outcome[control]outcome[treated])/outcome[control])x100%
We calculated pooled odds ratios, effect sizes, and 95% confidence intervals using a random effects model. Heterogeneity was examined using the I2 statistic.26 We investigated the possibility of small study bias by checking for funnel plot asymmetry and by using graphical and statistical methods.27
|
|
Thrombolysis in acute ischaemic stroke
Thrombolysis with recombinant tissue plasminogen activator reduces death or dependency after ischaemic stroke, despite an increase in intracranial haemorrhage.13 14 Overall, 113 reports were found on the effects of using tissue plasminogen activator or related agents for thrombolysis in animal models of acute ischaemic stroke (369 comparisons).w26-w138 The quality of the experiments was poor (table 1
). Infarct volume was reported in 212 comparisons (3301 animals), neurobehavioural scores in 84 (1438 animals), and haemorrhage in 146 (2791 animals). The funnel plot suggested an excess of imprecise studies overstating efficacy (fig 2
) and this was supported by an Egger regression analysis (P<0.001). Tissue plasminogen activator reduced infarct volume by 24% (95% confidence interval 20% to 28%), improved neurobehavioural scores by 23% (17% to 29%), and increased the probability of haemorrhage (odds ratio 1.96, 95% confidence interval 1.63 to 2.35; fig 3
). Substantial heterogeneity was found for infarct volume (I2=78.2%, P<0.0001) and for neurobehavioural scores (I2=75.2%, P<0.0001) but not for haemorrhage (I2=0%).
|
|
|
Respiratory distress syndrome was reduced in the corticosteroid groups in all three experiments. In one experiment two of 15 calves in the corticosteroid group compared with nine in the control group developed respiratory distress syndrome (P=0.01). In another experiment the total (SD) lung capacity in newborn rabbits in the corticosteroid group was 1.8 (0.4) ml/g compared with 1.4 (0.4) ml/g in the control group. In a third experiment, six of 12 monkeys in the corticosteroid treated group compared with 11 in the control group developed severe respiratory distress syndrome (P=0.03). Seven experiments reported the effects of corticosteroids on neonatal mortality: pooled odds ratio for mortality 4.2 (95% confidence interval 0.85 to 20.9). Significant heterogeneity was found (I2=72.7%, P=0.003). The pooled odds ratio for mortality in ewe models was 12.5 (1.9 to 79.2) with no evidence of significant heterogeneity (I2=33.1%, P=0.22).
Bisphosphonates to treat osteoporosis
Bisphosphonates increase bone mineral density in postmenopausal women with osteoporosis.17 Sixteen reports were found of bisphosphonates in animal models (two experiments in baboons and 14 in rats).w213-w228 The quality of the experiments was poor (table 1
). All experiments were carried out in ovariectomised animals. The effect of bisphosphonates on bone mineral density was reported in 11 experiments (fig 5
). When outcome data were available, 11 of 11 (100%) studies showed an increase in bone mineral density and six of six (100%) studies showed improvements in bone mass.
|
The four experiments in our meta-analysis of corticosteroids in animal head injury models used the weight drop model.28 All had good allocation concealment and blinded outcome assessment. Taken together they showed benefit. The experiments were, however, from one laboratory, had little evidence on adverse effects, and did not examine the influence of comorbidities. We also found a difference in results for tirilazad to treat stroke. The data from the animal studies suggested a benefit but the clinical trials showed no benefit and possible harm. It should be noted that the interval between stroke onset and treatment was longer in the clinical studies (median five hours) than in the animal models (median 10 minutes). Some of the clinical trials recruited patients up to 24 hours after stroke onset. For antifibrinolytics in haemorrhage, clinical trials showed clear evidence of benefit despite the lack of any reliable data from animal models.
Methodological strengths and weaknesses
It would be inappropriate to make general statements about the utility of animal research on the basis of only six interventions. Animal studies are often carried out to learn about biological mechanisms and we cannot comment on the value of animal research in these areas nor provide precise estimates of agreement. Although we tried to contact the authors of individual animal studies, we analysed what was reported and cannot rule out that other relevant data were not published. Our systematic review does, however, provide insights into the limitations of animal models, including the extent to which they represent disease in humans. As the number of systematic reviews of animal experiments increases, a quantitative approach to determine similarities between animal models and clinical trials should be possible in the future.
Implications for research
Systematic reviews could facilitate the translation of research findings from animals to humans. The animal studies in our systematic review varied in methodological quality and sample sizes rather than providing a single definitive high quality experiment for each intervention. Randomisation and blinding were rarely reported, which can have important implications as it has been shown that animal experiments carried out without either are five times more likely to report a positive treatment effect.23 In the systematic review of thrombolysis in acute ischaemic stroke we found strong evidence of publication bias. The number of experiments in the other systematic reviews made assessment of this source of bias difficult. In most cases we pooled the results to provide precise estimates of efficacy, although given the extent of heterogeneity the precision is open to question. These methodological issues are important given concerns about the differences between promising animal studies and negative clinical trials across a range of interventions. Because animal experiments are part of the evidence used to decide which interventions are taken forward in clinical trials, efforts to avoid bias and random error are as important when reviewing the results of animal models as when reviewing the results of clinical trials.
Prospective registration of animal experiments might reduce publication bias. Although the agencies that regulate animal research hold records of animal studies we were unable to access these. Animal research in the United Kingdom is regulated by the Home Office. We asked the Home Office for details of any animal experiments relevant to our study but they were unable to provide them. In response they stated: "It is not Home Office policy or practice to gather or retain information derived from work licensed under the Animals (Scientific Procedures) Act 1986 in the way you envisage. Such information is, generally, held by the licensed establishments concerned and made available to Home Office inspectors for inspection on site, should the need arise. I am, therefore, unable to provide you with any of the information you request. Nor am I able to confirm from Home Office records that any relevant trials were conducted under projects licensed under the 1986 act." We did not invoke the Freedom of Information Act. Nevertheless, the Home Office response calls into question the usefulness of its records in relation to efforts to create an accessible register of animal experiments.
Research is needed on the aspects of study design that can bias treatment effects in animal models. Empirical evidence of bias from study design characteristics helped to improve the quality of clinical trials and might do the same for animal experiments. Standards for evidence based reporting, similar to the consolidated standards of reporting trials statement for clinical trials, might ensure that relevant aspects of experiment methodology are reported.29
Systematic reviews can provide insights into the limitations of animal models. For example, the animal models for stroke, where there was agreement with the results from clinical trials, seemed more representative of the condition in humans than the animal models for head injury, where there were differences in the results. In stroke, the time from the occlusive event to the start of treatment was similar in animal and human studies. In head injury, treatment was given within five minutes of injury in the animal models but up to eight hours after injury in the clinical trials. None of the animal experiments used models that mimic the complex situations that usually follow traumatic head injury. Comorbidities are clearly relevant in stroke, which occurs in older people with hypertension and diabetes but also in people with head injuries, often accompanied by other injuries and by hypotension and hypothermia. Comorbidities were examined in the stroke models but not in the head injury models.
That there is a gap between clinical research and clinical practice is well established.30 Our work highlights another gapspecifically the lack of communication between those involved in animal research and clinical trialists. Systematic reviews of animal experiments could promote closer collaboration between the research communities and encourage an iterative approach to improving the relevance of animal models to clinical trial design. When models do not represent the clinical context they could be adapted accordingly. Furthermore, as is the case for human research, systematic reviews could help identify and improve deficiencies in the conduct and reporting of animal research.
|
We thank Sir Iain Chalmers for his constructive comments on the manuscript.
Contributors: IR, PS, MM, LEM, PJ developed the study protocol. All authors carried out the systematic reviews. PP and IR drafted the manuscript, which was revised on the basis of comments from all authors.
Funding: This work was funded by the National Health Service research and development health technology assessment programme. The views expressed in this publication are those of the authors and not necessarily those of the methodology programme, health technology assessment programme, or the Department of Health. Funding of this research by the NHS should not be taken as implicit support for any recommendations made by the authors.
Competing interests: IR was an investigator in the corticosteroid randomisation after significant head injury trial. The trial was funded by the UK Medical Research Council. Pharmacia and Upjohn (Pfizer from 2003) provided the Medical Research Council with the methylprednisolone (free of charge) needed for the trial, a grant in aid for preparation of the placebo, and support for collaborators' meetings. PS is co-chief investigator of the third international stroke trial, testing intravenous recombinant tissue plasminogen activator in acute ischaemic stroke; the start-up phase (completed in 2005) of this trial was supported by Boehringer Ingelheim, the manufacturers of tissue plasminogen activator, a donation of drug and placebo for the first 300 patients. The current phase of the trial is supported by the Medical Research Council and the Health Foundation. None of the authors have any relevant competing financial interests.
Ethical approval: Not required.
A table showing the quality of animal experiments included in the systematic reviews is available at www.crash2.lshtm.ac.uk.
![]()
CiteULike
Complore
Connotea
Del.icio.us
Digg
Reddit
Technorati What's this?
Read all Rapid Responses