Quality of reporting in systematic reviews of adverse events: systematic reviewBMJ 2014; 348 doi: https://doi.org/10.1136/bmj.f7668 (Published 08 January 2014) Cite this as: BMJ 2014;348:f7668
- Liliane Zorzela, PhD candidate1,
- Su Golder, MRC fellow in health services research2,
- Yali Liu, lecturer3,
- Karen Pilkington, senior research fellow4,
- Lisa Hartling, assistant professor5,
- Ari Joffe, clinical professor6,
- Yoon Loke, clinical senior lecturer7,
- Sunita Vohra, professor8
- 1Department of Pediatrics, 4-548 Edmonton Clinic Health Academy, University of Alberta, Edmonton, Canada
- 2Centre for Reviews and Dissemination, University of York, York, UK
- 3Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Chengguan District, Lanzhou, Gansu, People’s Republic of China
- 4School of Life Sciences, University of Westminster, London, UK
- 5Department of Pediatrics, 4-472 Edmonton Clinic Health Academy, University of Alberta, Canada
- 6Stollery Children’s Hospital, Department of Pediatrics, University of Alberta, Edmonton, Canada
- 7University of East Anglia Medical School, Norwich, UK
- 8Department of Pediatrics, University of Alberta, Canada, T5K 0L4
- Correspondence to: S Vohra
- Accepted 17 December 2013
Objectives To examine the quality of reporting of harms in systematic reviews, and to determine the need for a reporting guideline specific for reviews of harms.
Design Systematic review.
Data sources Cochrane Database of Systematic Reviews (CDSR) and Database of Abstracts of Reviews of Effects (DARE).
Review methods Databases were searched for systematic reviews having an adverse event as the main outcome, published from January 2008 to April 2011. Adverse events included an adverse reaction, harms, or complications associated with any healthcare intervention. Articles with a primary aim to investigate the complete safety profile of an intervention were also included. We developed a list of 37 items to measure the quality of reporting on harms in each review; data were collected as dichotomous outcomes (“yes” or “no” for each item).
Results Of 4644 reviews identified, 309 were systematic reviews or meta-analyses primarily assessing harms (13 from CDSR; 296 from DARE). Despite a short time interval, the comparison between the years of 2008 and 2010-11 showed no difference on the quality of reporting over time (P=0.079). Titles in fewer than half the reviews (proportion of reviews 0.46 (95% confidence interval 0.40 to 0.52)) did not mention any harm related terms. Almost one third of DARE reviews (0.26 (0.22 to 0.31)) did not clearly define the adverse events reviewed, nor did they specify the study designs selected for inclusion in their methods section. Almost half of reviews (n=170) did not consider patient risk factors or length of follow-up when reviewing harms of an intervention. Of 67 reviews of complications related to surgery or other procedures, only four (0.05 (0.01 to 0.14)) reported professional qualifications of the individuals involved. The overall, unweighted, proportion of reviews with good reporting was 0.56 (0.55 to 0.57); corresponding proportions were 0.55 (0.53 to 0.57) in 2008, 0.55 (0.54 to 0.57) in 2009, and 0.57 (0.55 to 0.58) in 2010-11.
Conclusion Systematic reviews compound the poor reporting of harms data in primary studies by failing to report on harms or doing so inadequately. Improving reporting of adverse events in systematic reviews is an important step towards a balanced assessment of an intervention.
Contributors: All authors participated in the study conception and design, and analysis and interpretation of data; drafted the article or revisited it critically for important intellectual content; and approved the final version to be published. SV is the study guarantor.
Funding: No specific funding was given for this study.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: No ethics approval was necessary because this was a review of published literature. No patient data or confidential information were used in this manuscript.
Data sharing: The dataset is available from the corresponding author at.
This manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.
Log in using your username and password
Log in through your institution
Register for a free trial to thebmj.com to receive unlimited access to all content on thebmj.com for 14 days.
Sign up for a free trial