The Home Office has now responded to the Number 10 e-Petition formulated by the group SABRE calling for more systematic reviews in applications for animal research projects.
The response seems entirely sensible and pragmatic. The Home Office has long taken an approach that it should be reasonable in assessing applications for project licences. It could hardly be reasonable to require a mandatory systematic review every time, especially where none exists.
RDS has contributed an extensive analysis of systematic reviews and animal research to the SABRE website on research methodology (It is difficult to find, and so we have copied it here under the extended text section).
Systematic reviews and project licences in animal research - a contribution from RDS
Recent reviews have exposed the methodological failings of many studies using animals. For example an article examining US animal studies of amyotrophic lateral sclerosis suggested that most of the studies that were published, including some in top-rank journals, had been done with small sample sizes, with no randomisation of treatment and control groups, and without blinded evaluations of outcomes (Nature 2008). If the analysis is correct, then this is clearly unsatisfactory.
There is increasing recognition of the need to improve both the design and analysis of studies using animals. RDS has consistently supported the use of systematic reviews as one such way to improve research. This is highlighted on our website, on our blog, and has been featured in our newsletter. Such reviews could also help further the 3Rs. SABRE does a good job of pointing out where systematic review are appropriate.
The SABRE petition proposes that each licence application includes references to systematic reviews of existing relevant studies. Depending on how exactly this is interpreted, such a proposal could cause difficulties should the referencing of systematic reviews become a ‘requirement’ for the award of a project licence.
The UK project licence is given under the Animals (Scientific Procedures) Act 1986 (ASPA) is a legal document based on a cost-benefit assessment of whether it is permissible to cause harm to an animal. It is a contract between the UK government and an individual researcher. Its award is dependent on a legally-defined process of review designed to ensure that animals are used in research only when there are no other means of obtaining the results and only when the anticipated outcome warrants any harm inflicted on the animals.
Systematic reviews and meta-analyses of animal studies are rare. Only 103 such reviews, comprising 57 systematic reviews, 29 systematic reviews accompanied by a meta-analysis and 17 meta-analyses were identified up to 2005 (Peters et al, 2006). For the vast majority of proposed animal studies, there will be no systematic reviews to refer to when applying for a project licence.
If systematic reviews do not exist in some areas, and cannot be forced into existence, they cannot be made a legal requirement for the award of a project licence. This is particularly the case when novel, groundbreaking research into basic biological mechanisms or structures is proposed, rather than evaluation of an existing drug, compound or procedure. Even where reviews can be carried out, there remains the difficulty of deciding whether the requirement would be for review of the procedures utilised or the substances undergoing test.
Basic animal studies may be designed to study biological mechanisms (rather than outcomes), or species, strain and methodological differences. Animal studies may open completely new fields of research where there is little prior knowledge (for example the creation for the first time of a genetically modified animal model of the human disease). Yet for a systematic review to draw a meaningful conclusion there must be sufficient available primary studies. But repetition in animal studies is strongly discouraged both by regulators and by the publication process. For much basic research, the literature may be neither sufficiently extensive nor sufficiently consistent to permit systematic review.
Additional questions remain about the applicability of systematic reviews to all types of animal research (Lemon and Dunnett, 2005). Some examples may help to illustrate the problems.
Much basic research is shallow and broad in its cover – that is, only one or two research groups world-wide may be studying an area. This is commonly the case in environmental/conservation research. Even flicking a dart into some deer to monitor their populations in the wild, or taking a biopsy or blood sample from a live seal washed up on the shore, may need a project licence, as would the use of guinea pigs as hosts for a study of tick biology.
Furthermore, a project licence is not always awarded for a single experiment or research project, but more often for a series of procedures. Some of the procedures to be carried out may depend on results obtained during an earlier project licence, which have not yet been published. Researchers do not always know in advance which procedures will be carried out within the remit of their project licence. So, for instance, to require application for renewal of a licence to be delayed until a systematic review had been conducted of the results of the current licence, would be totally unworkable.
In the case of toxicology tests, laboratories use well-characterised and relatively standardised techniques. However, it is not known in advance which substances will be tested over the course of the project licence. So it is unclear what would be the nature of the systematic review required. About 10% of all procedures are for the production of biological materials such as tissues, blood products, infectious agents, vectors, neoplasms and antibodies. And in the assay and quality control of biologicals, the models are generally well validated and the studies are already carried out in methodologically consistent ways.
When animals are used as models of humans, systematic reviews can be used in two ways: (i) to study the results of a particular method or intervention in an animal model to give a clear conclusion or (ii) to determine whether the animal model is a good predictor of results in humans. No systematic review can be used for the latter purpose unless the response in humans is known. However, if animal studies indicate that a potential treatment does not work, then that treatment would not normally proceed into human trials. In such cases, it is not possible to determine whether the animal models are predictive of humans, as no human data is available. In the case of nimodipine (Evans et al 2006), a systematic review of animal studies at the pre-clincial stage would not have led to human trials, and would not allow us to know whether an animal model is or is not predictive of the human situation. On the other hand, a comparison of systematic reviews of animal studies against the results of clinical trials can help validate the animal models, but obviously cannot further inform the decision taken to proceed to human trials. These two distinct purposes of carrying out systematic reviews are different, and must be recognised as such.
We are not arguing against systematic reviews here, nor suggesting that the current mechanisms for undertaking animal research are perfect (or even adequate). Animal studies need to be improved, just as improvements are needed for all types of research. Even in clinical trials, where systematic reviews are accepted as the gold standard, there remain improvements to be made.
RDS acknowledges that systematic reviews are at the top of the hierarchy of evidence, and are superseding narrative reviews because of their improved quality. It is said that the point of conducting systematic reviews is to reduce bias as much as possible in the evaluation of animal research (SABRE 2006). If the use of systematic reviews amounts to no more than the use of explicit and reproducible methods to reduce bias, then all reviews should be systematic.
However, most descriptions of systematic reviews (which are usually about assessing the effect of a therapeutic intervention) emphasise the importance of having a clearly defined topic and a specific question to answer, as well as having rigorous inclusion and exclusion criteria. Both present practical difficulties when basic research using animals is founded on a very diverse range of prior research. Sometimes results from in vitro and in silico studies are relevant, as well as information from other fields of animal research, or even knowledge from human epidemiological and clinical studies. This is a concern because it seems the most common reason for discrepancies between systematic reviews is differences in inclusion criteria (Linde 2003). Even proponents of systematic reviews seem to recognise this issue by calling for ‘initial research to discover the best methodological approaches for systematically reviewing animal studies’ (SABRE Charter, May 2008). We support this proposal.
Narrative (critical) reviews are currently common in basic research in which animal-based studies play a part. By definition even narrative reviews require a substantial body of research to have been undertaken, but at least they can overcome diversity and a lack of consistency in approach, or patchy coverage of the initial studies in a field.
A valid criticism of traditional narrative reviews is that they are ‘rarely explicit about how studies are selected, assessed and integrated. Thus the reader is generally unable to assess the likelihood of prior beliefs or other biases clouding the review process’ (Davies). There is no reason for this to continue. RDS considers that good methodology which leads to an objective account of the available data can be applied to both the systematic and narrative review process, and that the most useful analysis of the literature is obtained though use of the most appropriate review method.
Many of the protocols associated with systematic reviews can also be applied to narrative reviews, so that the data is assessed in a reproducible, transparent manner and bias is reduced. This requires that search criteria are broad, inclusion criteria are specified, and methodologies of included studies are evaluated critically. This approach would allow readers to reach more consistent conclusions from narrative reviews.
There are nonetheless many instances when systematic reviews as currently practised could be of great value. For example, they are desirable when a number of papers studying the same intervention have been published without any clear conclusion, or when a candidate medicine is considered suitable to proceed to human clinical trials. RDS continues to support systematic reviews in animal research.
References:
Davies HTO, Crombie, K (2001) What are systematic reviews? http://www.jr2.ox.ac.uk/bandolier/painres/download/whatis/Syst-review.pdf
Evans I, Thornton H, Chalmers I (2006) Testing Treatments: Better Research for Better Healthcare.
Linde K and Willich SN, J R Soc Med (2003) How objective are systematic reviews? Differences between reviews on complementary medicine.96(1) January.
SABRE Charter (2008) http://www.sabre.org.uk/ May.
SABRE personal communication (2006).
Lemon R and Dunnett SB (2005) Surveying the literature from animal experiments. BMJ 330: 977-978.
Nature News feature. Standard model. (2008) 454/7.
Nature Medicine editorial: when less is not more. (2008) 14, Number 8.
Perel P, Roberts I, Sena E, Wheble P, Briscoe C, Sandercock P, Macleod M, Mignini LE, Jayaram P, and Khan KS (2006) Comparison of treatment effects between animal experiments and clinical trials: systematic review. BMJ 15 December.
Peters, J L, Sutton, AJ, Jones, DR, Rushton, L, and Abrams, KR (2006) A systematic review of systematic reviews and meta-analyses of animal experiments with guidelines for reporting. J Environ Sci Health B 41: 1245-1258.
