Improving the conduct, reporting, and appraisal of animal research
BMJ 2018; 360 doi: https://doi.org/10.1136/bmj.j4935 (Published 10 January 2018) Cite this as: BMJ 2018;360:j4935
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
While we welcome the application of rigorous standards to animal experimentation, we feel that the position expressed here and, to some extent, in the accompanying editorial, miss the central value of most animal experimentation. Medical research exists on a spectrum from fundamental discovery to clinical validation. Large randomized clinical trials sit on the latter end. These studies must meet very high levels of proof. As such, they generally only can rigorously test a single hypothesis with a binary answer. Strong study design plays a key role in our confidence in the results of any individual trial and in our system of testing for interventions.
The vast majority of animal studies, however, lie at the other end of the spectrum. They are designed to discover new biology. Like almost any laboratory experiment, they involve multiple endpoints that can be changed during the course of the experiment. Not having this kind of flexibility severely limits what we can learn and undervalues the lives of animal subjects. Clearly, the confidence in any given conclusion is less than that from a well-powered clinical study. But these are not low quality data - in fact, the most convincing validation often comes from orthogonal approaches, not large numbers of animals.
The very rare animal experiment might be key for preclinical testing of an effect in humans. That might be true when an animal model is known to reliably predict a response in patients. In those instances, it might be justified to use the type of study design seen in human trials to test the effectiveness of an intervention. But that is certainly not the case for TB vaccines. At the initiation of the MVA85A trial we did not know how any particular response in any given animal species predicted human protection against disease. In cases such as this, the only way to define the correlation between animal and human responses is to conduct clinical trials, in some cases regardless of the animal efficacy data. Animal testing in the case of MVA85A was critical for establishing safety and helped generate hypotheses for human trials. But, while we can second-guess the decision to proceed on the basis of the animal results, there certainly was no simple yardstick to apply. And using a much larger group of nonhuman primates to achieve the statistical significance that some seem to want would have required tens of millions of dollars and several years and would not clearly have altered any decision-making.
There is substantial pressure to apply clinical trial criteria to animal research. Here in the US, the NIH increasingly is requiring sample size calculations and predetermined hypothesis testing when proposing animal experiments. This thinking is echoed here, where the editorialists call for meta-analyses of animal data as a prerequisite for clinical testing. We feel that these ideas, while well intentioned, represent a fundamental misunderstanding of the goals and practice of using animal models.
Competing interests: No competing interests
Re: Improving the conduct, reporting, and appraisal of animal research; PREPARE before you ARRIVE
Despite widespread journal endorsement of reporting guidelines, the inadequate reporting of preclinical research is being increasingly debated by the scientific and laboratory animal scientific communities (1, 2, 3). Studies demonstrating inadequate reporting (which is frequently unrelated to the journals' impact factor) suggest that it is related to the poor reproducibility of many animal experiments. This is at any rate a major threat to the validity of preclinical research (3). In a recent Editorial in BMJ on improving the conduct, reporting and appraisal of animal research, Merel Ritskes-Hoitinga and Kim Wever cite pre-registration, systematic reviews and better reporting as the major requirements for raising standards (4).
We propose that there is an elephant in the room that has been ignored for too long. Reporting guidelines, while well-intentioned, attempt to solve the reproducibility problem after the event - and appear to be failing. It should really be obvious that better reporting cannot improve the intrinsic quality of an experiment which has already been performed. While a good sales pitch may attract more customers, a product does not improve until its constituent parts and manufacturing conditions are upgraded. Systematic improvement of animal research must begin at the beginning, with guidelines for planning experiments. Such an approach is also in keeping with the 3R's principle of Replacement, Reduction and Refinement (5) which is part of modern animal research legislation such as the EU Directive 2010/63 (6).
With this in mind, we have constructed a set of planning guidelines called PREPARE (7), based upon our experiences over the last 30 years in planning and supervising animal experiments. PREPARE contains, of course, many of the elements embodied in reporting guidelines like ARRIVE (8), but, importantly, PREPARE highlights issues which are seldom reported, partly due to lack of space, but which can have dramatic impacts on not only the scientific validity of the research but also on animal welfare and the safety of all those involved.
PREPARE contains a "checklist" as does ARRIVE, which acts more of an aide memoire than a form to complete. It aims to provide a gentle reminder to all scientists of the items which should be addressed pre-study, much in the same way as pilots, however experienced, work their way through a checklist before take-off.
More importantly, we have constructed a website which supports and expands upon the topics in the checklist. PREPARE can thus offer scientists an up-to-date overview, with links to specific guidelines for each topic, adding new ones as they are published. An intelligent search engine on the 6,000-page Norecopa website where PREPARE resides (https://norecopa.no/PREPARE) facilitates the location of other relevant resources as well.
It is our hope that the debate on poor reproducibility will rotate towards planning, rather than reporting animal experiments. Otherwise, we are in danger of wasting time discussing the quality of the lock on the door of the stable from which the horse has already bolted.
References
1. Enserink M. Sloppy reporting on animal studies proves hard to change. Science2017;357:1337-8. doi:doi:10.1126/science.357.6358.1337pmid:28963232
2. Baker M. 1,500 scientists lift the lid on reproducibility. Nature2016;533:452-454. doi:10.1038/533452a
3. Skibba R. Swiss survey highlights potential flaws in animal studies. Nature2016;doi:10.1038/nature.2016.21093
4. Avey MT, Moher D, Sullivan KJ, Fergusson D, Griffin G, Grimshaw JM, Hutton B, Lalu MM, Macleod M, Marshall J, Mei SHJ, Rudnicki M, Stewart DJ, Turgeon AF, McIntyre L. The Devil Is in the Details: Incomplete Reporting in Preclinical Animal Research. PLoSONE2016;11:e0166733. doi:doi:10.1371/journal.pone.0166733
5. Ritskes-Hoitinga M, Wever K. Improving the conduct, reporting, and appraisal of animal research. BMJ2018;360:j4935. doi:doi:10.1136/bmj.j4935
6. Russell WMS, Burch RL. The Principles of Humane Experimental Technique; Universities Federation for Animal Welfare: Wheathampstead, UK, 1959. http://altweb.jhsph.edu/pubs/books/humane_exp/het-toc
7. EU Commission. Directive 2010/63/EU of the European Parliament and of the Council of 22 September 2010 on the Protection of Animals Used for Scientific Purposes. http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2010:276:0033...
8. Smith AJ, Clutton RE, Lilley E, Hansen KEAa Brattelid T. PREPARE: Guidelines for planning animal research and testing. LaboratoryAnimals2017. Prepublished 3 August 2017. doi:10.1177/0023677217724823
9. Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLOSBiol2010;8:e1000412. doi:doi:10.1371/journal.pbio.1000412pmid:20613859
Competing interests: We have read and understood BMJ policy on declaration of interests and declare the following interests: we are the unpaid authors of the PREPARE guidelines which we refer to in our Response. AS is the paid Secretary and employee of Norecopa. The other authors hold paid positions at other institutions and promote PREPARE where appropriate when they lecture.