Assessment of therapeutic safety in systematic reviews: literature reviewBMJ 2001; 323 doi: http://dx.doi.org/10.1136/bmj.323.7312.546 (Published 08 September 2001) Cite this as: BMJ 2001;323:546
- Edzard Ernst, director (, )
- Max H Pittler, research fellow
- Department of Complementary Medicine, School of Postgraduate Medicine and Health Sciences, University of Exeter, Exeter EX2 4NT
- Correspondence to: E Ernst
- Accepted 22 March 2001
Evidence based medicine aims to objectively and continually evaluate all medical interventions with a view to incorporating the knowledge gained into routine healthcare practice. The ultimate goal is to enhance patient care. The most powerful tools used in evidence based medicine are systematic reviews and meta-analyses.1 Generally, these studies focus on the efficacy or effectiveness of therapeutic interventions; indisputably, however, information on safety is equally important for making informed, evidence based decisions on the value of a given treatment. We evaluated two major medical databases to assess the extent to which therapeutic safety is addressed in systematic reviews and meta-analyses.
Methods and results
We performed systematic literature searches to identify all systematic reviews and meta-analyses on the efficacy or effectiveness and the safety of any type of therapeutic intervention. Data sources were Medline (1966-December 2000) and the Cochrane Library (2000, issue 4). The title, abstract, and keywords of papers were searched using the terms efficacy, effectiveness, systematic review, meta-analysis, safety, adverse effect, adverse event, and adverse reaction and their derivatives. No language restrictions were imposed. The title and abstract of all studies were read and data were according to the criteria defined. Studies were included only if they stated that they were systematic reviews or meta-analyses or if they indicated that the available data were systematically searched for, appraised, and summarised. Studies that included no abstract or that provided no data on humans were excluded. Data were assessed according to the year of the study, and the studies were categorised into three categories: systematic reviews on efficacy or effectiveness, systematic reviews that included safety aspects as a secondary outcome measure (safety category A), and systematic reviews with a primary focus on the safety of the therapy (safety category B). Study selection, data extraction, and evaluation were performed independently by the two investigators, and discrepancies were resolved through discussion.
The results (table) showed a continuous increase in the number of systematic reviews and meta-analyses published between 1966 and December 2000. The number of studies that included safety aspects as a secondary outcome measure (category A) amounted to just over one quarter of the total number of studies on efficacy or effectiveness. The number of systematic reviews that assessed safety aspects as the primary focus of the investigation (category B) amounted to only 3-5% (table).
Systematic reviews and meta-analyses provide little information on the safety aspects of therapeutic interventions. Most systematic reviews include data on safety only in as much as the primary studies, usually randomised clinical trials, report on adverse events. This seems inadequate. Randomised clinical trials typically assess only a small number of patients and thus the chances of detecting rare adverse events are small. Randomised clinical trials are usually of short duration and thus cannot identify delayed adverse events.2 The assessment of safety has to go far beyond randomised clinical trials and should use various methods, including post-marketing surveillance studies, spontaneous reporting schemes, and epidemiological investigations.3 Systematic reviews on the safety of therapeutic interventions should take these considerations into account and should combine data from various types of studies.4 Safety aspects have so far not been adequately investigated by systematic reviews. One challenge of evidence based medicine therefore is to address safety issues more systematically than has been done before.
Contributors: EE conceived the idea of the study. EE and MHP planned the design, reviewed the original data, evaluated the results, and wrote up the report. EE is the guarantor of the paper.
Funding None declared.
Competing interests None declared.