How predictive and productive is animal research?
BMJ 2014; 348 doi: https://doi.org/10.1136/bmj.g3719 (Published 05 June 2014) Cite this as: BMJ 2014;348:g3719
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Fiona Godlee, the editor in chief of The BMJ, recently wrote a somewhat surprising Editorial (BMJ 2014;348:g3719) in her ‘Editors Choice’ of an article by Pandora Pound and Michael Bracken (BMJ 2014;348:g3387) that argues medical research is valueless. Surprising because the editorial is unbalanced. It appears to support several flawed statements in the Pound and Bracken article whilst not referring to how basic biomedical research has been and still is crucial to modern clinical practice. It is implied that interspecies differences in molecular and metabolic pathways are a serious obstacle to drug treatment discovery when in fact this is not true whilst the real problem is lack of rigorous analysis of animal research data. The editorial confuses basic research where animal models are of pivotal importance with applied or translational research of actions of drugs in animal models. Furthermore Fiona Godlee appears to ignore that the faults with tests of drugs in animals that have been criticised by Altman (BMJ 1994;308:283), Ioannidis (2005, PLoS Med 2 (8):e124), Pound and colleagues (2004 BMJ,328:514517; 2014) are also apparent in clinical trials. It seems to me that the editorial is unworthy of a Journal like the BMJ that is read by so many in the medical profession.
John H Coote
Professor Emeritus School of Clinical and Experimental Medicine, University of Birmingham.
Visiting Professor, Cardiovascular Science, Glenfield Hospital, University of Leicester. Consultant in Applied Physiology, RAF Institute Of Aviation Medicine.
The author has no competing interests.
Competing interests: No competing interests
LETTER TO THE EDITOR – BRITISH MEDICAL JOURNAL
ANIMAL RESEARCH IS NOT A BROKEN MODEL – REBUTTAL OF POUND & BRACKEN’S UNJUSTIFIED ASSAULT ON DISCOVERY-BASED BIOMEDICAL RESEARCH
David W Walker PhD, D.Sc, Senior Scientist
Stuart B Hooper PhD, Executive Director
The Ritchie Centre, MIMR-PHI Institute of Medical Research,
Monash Medical Centre, Melbourne, Australia
Whitelaw & Thoresen1 have provided a robust rebuttal from a neonatologists’ point of view to Pound & Bracken’s2 assertions that animal-based research is insufficiently based on evidence to be a cornerstone for advancements in medical research. However, we feel compelled to respond further, not only because much more can be said to rebut Pound & Bracken’s position on the value of animal-based research, but because their views seem to have received support from the BMJ in the form of your editorial commentary.
Pound & Bracken claim that the conduct of much animal research borders on the inept and unethical in omitting procedures that ensure randomisation to treatment arms and blinding of the investigators to outcomes. Further, they claim that so little animal-based research has resulted in clinically acceptable new practice that almost all of the >60% of public funding for this type of research should be considered wasted money.
For the reasons set out below we believe this is naïve and ill-informed opinion. It is true that some aspects of Pound & Bracken’s commentary might recognised by animal scientists as ‘inconvenient truths’. While good experimental design dictates that animals are randomised at the entry point to an experiment, it is often not possible or practicable to fully 'blind' investigators to the treatment that any particular animal receives during an experiment. However, this is not a problem restricted to animal research and occurs in many RCTs where devices or procedures are compared. Good experimental design also dictates that any samples collected are coded so that the investigator is blinded to the treatment and the expected outcomes during any subsequent analysis. Further, to suggest that reviews are rarely made that mimic the systematic meta-analyses of independently conducted clinical trials demonstrates a lack of understanding of the advantages of discovery-based research in assessing biological mechanisms, which clinical trials cannot do. Indeed, the implication that animal-based researchers wilfully choose to operate in a less critically-reviewed environment than clinical scientists, whose work is now dominated by meta-analyses of repetitive, expensive RCTs, is an opinion which we forcefully reject, for the following reasons:
Firstly, animal-based research has different priorities and end-points compared to human research. Animals are not people, and they are not ‘participants’ in a research project in the same way that patients or human volunteers are. By necessity, all human subjects must know why the experiment is being done, what outcomes are expected, and that the treatment is expected to produce a benefit and cause no harm. However, within ethically constrained boundaries, the first priority of animal-based research is dictated by the hypothesis being tested. As such, this research can interrogate cellular mechanisms and have end-points such as major illness, disability or even death that are acceptable under well-defined and controlled conditions. These would always be unacceptable endpoints for human research. In animal research factors can potentially be studied, always using appropriate controls, irrespective of the benefit or harm they may cause. In contrast, the first and only priority of any clinical study is the health and well-being of the patient and the hypothesis under investigation is always subservient to that consideration. As such the overwhelming necessity to not cause harm can considerably complicate the study design, leading to the persistence of unwanted variability, and restricting the interrogation of underlying mechanisms as this usually requires further interventions that would be unacceptable. Thus, to suggest that we should limit biomedical research to such restrictive experimental approaches and procedures of human clinical research, as implied by Pound & Bracken, is unrealistic and ill informed. This does not call into question in any way the essential role for properly designed clinical trials in determining the effectiveness of a treatment or device in medical practice, but should serve to highlight the fundamental difference in priorities and approach that leads to understanding the complexity of most health problems and their treatment.
Secondly, it is unreasonable to expect that all animal experiments should always lead directly to new clinical procedures, as Pound & Bracken suggest. Many animal experiments remain exercises in ‘discovery science’ where good ideas are tested and mechanisms that lie behind observable biological phenomena are identified. These mechanisms are almost always complex, require a sequential set of experiments that approach the same question from different perspectives, and usually result in extended debate over details and biological significance. That is why these kinds of experimental reviews, of which there are many contrary to Pound and Bracken’s assertion, are fundamentally unlike the meta-analyses of clinical RCTs. These reviews seek to rationalise differences between studies rather than lump the results into a congealed data set, which is the result of most clinical meta-analyses. The fact that it is illogical or impossible to do this for most areas of experiment biomedical science should be seen as a strength, and not a weakness.
Thirdly, it is difficult for us to accept that animal based science has produced a poor return for clinicians, as asserted by Pound and Bracken under “Lack of benefit for humans”. The example they point to – stroke medicine – remains an intractable problem, like dementia, diabetes, preterm labour, asthma, and infertility. This is precisely because they are intractable problems, which an army of clinicians, big pharma and discovery-based science have so far been unable to resolve. We could equally point out the errors that have arisen when clinical practise moves too quickly to implement treatment in the face of poor scientific evidence – thalidomide and diethylstilboestrol are obvious cases, to which we could add the use of 100% oxygen in preterm infants as well as a variety of other treatments. Furthermore, over the past 10 years there have been a plethora of failed RCTs that have consumed huge amounts of money because they were not underpinned by good basic science. Both experimental approaches are required to effectively drive improvements in health from the theoretical, to practical application, and finally into widespread clinical practice.
Finally, we think it is also helpful to consider where good ideas come from in the first place, and an honest appraisal shows that they are not always derived firstly from clinical medicine. As mentioned by Whitelaw and Thoresen, antenatal glucocorticoid treatment of women in preterm labour, which has markedly reduced the neonatal disease burden caused by preterm birth, came from exploratory studies in sheep investigating the mechanisms of labour. Did the idea of hypoxic preconditioning come from clinical medicine, or from comparative animal physiology? Did the idea that micronutrient deficiency causing metabolic disease come from human medicine or from agricultural research? (the latter, of course). Did the concept of pluripotency of embryonic stem cells come from human embryology or from comparative embryology, and was the driver for this the need to find a disease cure, or simply plain curiosity? (the latter, of course). The increased imperative from funding agencies that all biomedical research should offer the immediate reward of new clinical treatments is unrealistic, stupefying to the creative process, and leads to the ill-informed and unrealistic expectations that provide the basis of Pound & Brackens’ criticism of discovery-based laboratory research.
We trust that this essay will provide the basis of further meaninglful, and productive debate on this issue.
References:
1. Whitelaw A, Thoresen M (2014) Animal research has been essential to saving babies’ lives. BMJ 348:g4174 doi: 10.1136/bmj.g4174
2. Pound P, Bracken M (2014) Is animal research sufficiently evidence based to be a cornerstone of biomedical research? BMJ 348:g3387 doi: 10.1136/bmj.g3387
Competing interests: No competing interests
As a Psychology graduate student many years ago, I did behavioral research on albino rats. When one supplier was unable to fill our order, we ordered rats from another supplier. We specified the genetic strain, age, and weights so they were presumably identical to rats from the original supplier. Except they weren't identical. The new rats were more aggressive and learned more slowly. I learned that our data varied based on what company supplied our rats! Animal researchers (and journalists) should know that not all rodents are the same; study results may not be very generalizable even within the same species.
Competing interests: No competing interests
The race to produce as many scientific publications as possible in high impact factor journals is one of the main causes of poor research. It has led to insufficient quality in planning, execution and reporting of animal studies. Major deficiencies occur in the materials and methods sections of publications. Even though scientists are educated in laboratory animal science courses where they learn how important it is to publish all details, and even though the ARRIVE guidelines for reporting have been adopted by over 300 scientific journals, thus far, this hasn’t helped to improve the quality of reporting of animal studies (Pound and Bracken 2014). As long as scientists are rated by their output of publications in high impact factor journals, it is highly unlikely that the current situation is going to change for the better. It resembles the trend towards the financial crisis. As long as "funding is coming in and papers are published" things seem to be "going well” and no-one takes responsibility to stop the runaway train or change the course of action.
Systematic reviews of animal studies show repeatedly that the quality of reporting is far below good scientific standards. Randomisation and blinding are often unreported, even though they are the starting points of good scientific practice. While high impact factor journals continue to accept mainly positive results (publication bias), this leads to the effects of drugs and treatments being overstated on the basis of what is published, leading to the potential for exposure of patients and research volunteers to unnecessary harms. Ideally, animal studies would be planned and executed according to the Gold Standard Publication Checklist (Hooijmans et al. 2010), and should reach the same standards expected of clinical studies (Muhlhausler 2013) and then the publications would at least fulfill the ARRIVE guidelines.
All animal trials should be prospectively registered and the individual animal data stored in central databases and the raw data made fully and openly accessible to reviewers. Systematic reviews and meta-analyses could then be carried out with more confidence to provide more rigorous evidence for whether animal studies have translational value. Currently, it cannot be determined whether the low rate of translation of results from animal studies to humans is to be attributed to the poor quality of reporting and/or a really low translational value. This represents a serious waste of research.
The Lancet series 'Increasing value and reducing waste in biomedical research' (www.researchwaste.net 2014) offer suggestions for improving research such as methodological rigour and reproducing results. However, all research evaluation schemes seem to depend only on quantitative output of publications in high impact factor journals.
Clinical trials have recently been described as resting precariously on the pillars of flawed basic and toxicological research in 'the temple of biomedical science,' (Hartung 2013) and methodologists have (so far) mapped 235 different forms of bias in biomedical research, many of which scientists are either unaware of or do not account for (Chavalarius and Ioannidis 2010). It is therefore hardly surprising that only 15% of basic and clinical research has utility (Chalmers and Glasziou 2009), that only 11% of medical compounds are approved following years of expensive research (Arrowsmith 2011) and that $100 billion is wasted annually on biomedical research due to poor research conduct and analysis. (Glasziou 2014). Only when a more rigorous approach is followed, will it become possible to determine whether animal studies are worth their cost and how much are they useful for human healthcare.
At the moment patients and research volunteers seem unaware of the current situation which leaves them vulnerable to the negative effects from poor quality research, waste of funding and loss of therapeutic interventions. As an example, in the case of Alzheimer's Disease (AD) the US annual research spending on AD in 2013 was $504 million, but in spite of this spending, which grows each year, there are only four medications available, and as the NIH states 'these drugs don’t change the underlying disease process, are effective for some but not all people, and may help only for a limited time.' There are now calls in the UK for increased spending on AD and other diseases, however, it would be useful at this stage to stop and reappraise all existing animal research with systematic reviews and to consider adopting a more forward thinking programme such as the one launched by the US National Research Council which aims to tackle the lack of translation of animal studies in toxicology. In their strategy (National Academies 2007) which is now being implemented worldwide the plan is a more human-relevant approach which aims to be faster, less expensive and more predictive for human exposures. The aim is to overcome long-standing translational problems caused by, among other issues 'animal-intensive research.'
SYRCLE and SABRE are working hard for improvements in scientific practice, and unless stakeholders (regulators, funders, charities, journals, researchers, politicians and other key players) mobilise soon, it may become too late, as we saw in the financial crisis. Ultimately the public will find it unacceptable when they learn that best scientific practices are not mandatory in biomedical science and that lives are put at risk from bias and that vested interests come before patient's interests. SABRE has therefore taken the lead in formulating a set of recommendations (the 10 Rs+ www.sabre.org.uk: Register, Report, Represent, Replicate, Retract, Record, Restore, Review, Regulate, Reappraise) to be implemented into scientific practice. So the question is who will be the first to take responsibility for seeing that safeguards are put in place to ensure better science for better healthcare?
Merel Ritskes-Hoitinga₁ and Susan Green₂
₁ SYRCLE, Radboudumc, Nijmegen, The Netherlands.
₂SABRE Research UK
Chalmers I, Glasziou P. (2009). Avoidable waste in the production and reporting of research evidence. Lancet. Jul 4;374(9683):86-9.
Chavalarias D, Ioannidis JP. (2010) Science mapping analysis characterizes 235 biases in biomedical research J Clin Epidemiol. Nov;63(11):1205-15.
Glasziou P (2014) The Role of Open Access in Reducing Waste in Medical Research. PLoS Med 11(5): e1001651
Hartung, T. (2013) Look back in anger – what clinical studies tell us about preclinical work. ALTEX 30, 275–291
Hooijmans CR, Leenaars M, Ritskes-Hoitinga M. A gold standard publication checklist for improving quality of animal studies, fully integrate the 3Rs and to make systematic reviews feasible. ATLA (2010, 38, 167-182).
Muhlhausler BS, Bloomfield FH, Gillman MW. (2013) Whole animal experiments should be more like human randomized controlled trials. PLoS Biol.11(2):e1001481
National Academies Committee on Toxicity Testing and Assessment of Environmental Agents and National Research Council eds. (2007) Toxicity Testing in the 21st Century: A Vision and a Strategy, National Academies Press
Pound P, Bracken MB. (2014) Is animal research sufficiently evidence based to be a cornerstone of biomedical research? BMJ. May 30;348:g3387
Competing interests: No competing interests
Dear Dr Godlee,
I read the editorial ‘How predictive and productive is animal research?’ (5th June 2014) with concern.
Study integrity is crucial for producing high-quality biomedical research. Research using animals – as with other fields— has suffered from many of the issues that afflict the robustness and reproducibility of research. This is a greatly troubling issue that needs to be looked at across the biomedical sciences and the training of all biomedical scientists in these matters is paramount. Robustness could even be considered as a fourth ‘R’ alongside the current 3Rs that should be followed in pursuit of responsible animal research.
However, I was disappointed to see the editorial’s concluding assertion arising from Pound and Bracken’s research — namely that preclinical animal research should not be endorsed or funded because results produced by animal models poorly predict human drug responses.
Pound and Bracken’s analysis addresses only a small sub-set of studies using animals. It is not appropriate to extrapolate from their findings to give an indictment of all outputs from animal research. The authors’ publication and the accompanying editorial both fail to account for the wider benefits that animal research provides beyond preclinical drug testing. Research using animals is essential for furthering our knowledge of molecular and biological pathways that underpin human physiology in health and disease. It has preceded, or directly led to, a large proportion of our groundbreaking discoveries in medicine. And of course studies on animals have in many instances been of direct relevance to animal health. Without a combination of animal and human studies, we would be largely ignorant of many existing treatments and many promising frontiers for developing more, including gene therapy, stem cell applications, and neuroprotection. Furthermore, the impressive economic return garnered from investment in medical research is reliant on expenditure in large part on basic research (including the use of animals), in addition to clinical research.
Predicting human outcomes from animal research is an undoubted challenge because of the extreme complexity of biological pathways being studied, as well as their inter-species differences. Efforts should be accelerated to develop model systems that better reflect human physiology and safe in vivo systems to test proof of concept in man. Nonetheless, soundly conducted animal research is a vital precursor to clinical research in many respects, and it is essential that it continues to be supported in our efforts to improve human and animal health.
Yours sincerely,
John Tooke
Competing interests: No competing interests
Dear Editor
Your article highlights the difficulties in getting new medicines and therapies to market. You correctly argue that poor experimental design and selective reporting of results are significant issues in both clinical and animal studies. This however, is not a valid reason for reducing funding for early pre-clinical and basic research. This instead should be a call to focus on good quality and well reported science regardless of whether it is basic, pre-clinical or clinical. To this end, The Physiological Society, along with other scientific and medical charities, and over 300 journals have signed up to the Animal Research, Reporting of In Vivo Experiments (ARRIVE) guidelines, which were developed to improve the reporting of how animal experiments are designed, conducted and analysed.
In the last 20 years much progress has been made in reducing the use of animals, through new technologies, such as remote imaging and diagnostics. These have given researchers a better elucidation of disease and human patho-physiology, and as these technologies continue to evolve, there will be a further decrease in the use of animals in research. However, we are not yet at the point where the complexity of how the systems of the body function can be completely understood without the use of animal research. Recent advances in the understanding and treatment of a wide range of cancers illustrate this point: progress in developing efficacious medicines and therapies have been founded on both the use of GM mice in early research and enhanced clinical research focused on patient outcomes. There are, of course, differences between humans and animal models, but good research is designed and results interpreted in light of the differences in molecular and metabolic pathways between humans and animals.
We firmly believe therefore, that if animal research is reduced or proscribed precipitously it will have long-term and severe deleterious consequences on our ability to understand and treat diseases and ultimately slow the advances that have a real impact on patient outcomes.
Competing interests: No competing interests
Re: How predictive and productive is animal research? Research involving animals is a keystone of medical research.
Research involving animals is a keystone of medical research
JC McGrath, EM McLachlan & MJ Curtis
ian.mcgrath@glasgow.ac.uk
A BMJ editorial (1) poses the question “How predictive and productive is animal research?” in the context of an article entitled “Is animal research sufficiently evidence based to be a cornerstone of biomedical research?” (2). This article criticizes the quality of the conduct and reporting of “animal research” as well as challenging the concept that such research is the “cornerstone of biomedical research” and concludes that this is a broken model that does not deserve public funding. Our answer is that research involving animals is a keystone of medical research.
Both editorial and article discuss “animal” and “clinical” research as if they were disparate. They claim no benefits to medicine from animal research studies and complain of the limited use of systematic reviews of published reports. The suggestion is made that funding would be better spent directly on clinical research.
This view of research is circumscribed and somewhat outmoded. In the first place, understanding the normal biology of mammalian cells, tissues and organs and their complex physiology is an unfinished task. To complain that research on animals that is directed to this end should not be supported unless it is directly translatable into a clinical outcome is narrow-minded and unrealistic. Clinical medicine today can only be improved by attempts made to replace the medical myths developed in the past with the revelations made possible by modern science.
Even for animal research that is focussed primarily on identifying novel treatments for disease, the arguments that have been put forward in these articles confuse studies that are directed at discovering a potential treatment with those that seek to test its feasibility in models of human disease. Sir Colin Dollery recently pointed out “Translational medicine in the (current) era begins with the choice of target, not with first time in human, and does not end until many years later when both physicians and patients have understood how to use it safely and effectively in the real world” (3). The key start of the process is identifying the target for therapy. Without preclinical research, there would be no targets and so no therapies to test in humans. A multicentre clinical trial cannot ever be the entry point for the invention of new drugs.
Before a new therapy is tried out in humans, diverse basic disciplines have been utilized, guided by the best current understanding of physiology, medicine and pathology. Sometimes experiments involve animal or human cells or tissues, sometimes other assay systems such as cell culture, physicochemical analyses and sometimes live animals. Scientists do not choose to use animal tissues or cells simply because they are “proponents” of such research, as labelled by Pound & Bracken (2), but because in their judgment these are the best means to test a novel scientific idea. They will use human cells or tissues where these are suitable and available. However, studies in human cell lines alone (bypassing animal research) could never justify a clinical trial.
A major intractable problem in clinical research results from the variability amongst the sampled subjects. It is important to review systematically data from multiple large scale studies to overcome these limitations and this has become a discipline in itself. Research on animal cells, tissues, organs or organisms is designed to avoid this source of variance, at the same time as defining and controlling as precisely as possible the conditions under which these investigations are conducted. Thus systematic reviews are generally inappropriate.
Nevertheless, some animal experimentation and much of its reporting could be better. However, imperfection in experimental design and conduct is not confined to experiments involving animals but can be found in every field including clinical research. Learned societies and journal editors are well aware of this and there is a strong push to improve standards by enhancing standards for statistical reporting and experimental design (4-8). This is a worldwide issue. Ensuring the quality of research output is the responsibility of funding bodies, Universities and research institutions as well as scientific journals that determine what is published.
To argue that imperfection in one area of the research continuum should sway the balance of research funding is a mistake. If preclinical research is allowed to wither, the drug pipeline, already meagre, will evaporate.
References
1. Godlee, F. Editor's Choice: How predictive and productive is animal research? BMJ 2014;348:g3719
2. Pound, P & Bracken, MB 2014 “Is animal research sufficiently evidence based to be a cornerstone of biomedical research?”
Analysis: Is animal research sufficiently evidence based to be a cornerstone of biomedical research? BMJ 2014;348:g3387
3. Dollery, C. T. (2014), Lost in Translation (LiT): IUPHAR Review 6. British Journal of Pharmacology, 171: 2269–2290. doi: 10.1111/bph.12580
4. ARRIVE Guidelines http://onlinelibrary.wiley.com/doi/10.1111/j.1476-5381.2010.00872.x/full
Kilkenny, C., Browne, W., Cuthill, I. C., Emerson, M. and Altman, D. G. (2010), Animal research: Reporting in vivo experiments: The ARRIVE guidelines. British Journal of Pharmacology, 160: 1577–1579.
5. UK Concordat http://www.understandinganimalresearch.org.uk/policy/concordat-on-openne...
6. Basel Declaration http://www.basel-declaration.org
7. British Journal of Pharmacology Virtual Issue: Best Practice in Statistical Reporting http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1476-5381/homepage/statistical_reporting.htm
8. British Journal of Pharmacology Virtual Issue: Animal Models in Pharmacology Research http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1476-5381/homepage/animal_models.htm
Competing interests: No competing interests