Relation of completeness of reporting of health research to journals’ endorsement of reporting guidelines: systematic reviewBMJ 2014; 348 doi: https://doi.org/10.1136/bmj.g3804 (Published 25 June 2014) Cite this as: BMJ 2014;348:g3804
- Adrienne Stevens, clinical research manager1,
- Larissa Shamseer, PhD candidate12,
- Erica Weinstein, medical student3,
- Fatemeh Yazdi, senior clinical research associate1,
- Lucy Turner, biostatistician1,
- Justin Thielman, MSc candidate1,
- Douglas G Altman, professor4,
- Allison Hirst, project manager5,
- John Hoey, retired professor6,
- Anita Palepu, professor78,
- Kenneth F Schulz, distinguished scientist9,
- David Moher, senior scientist12
- 1Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, ON, Canada, K1H 8L6
- 2Department of Epidemiology and Community Medicine, University of Ottawa, K1H 8M5 Ottawa, Canada
- 3Albert Einstein College of Medicine, Yeshiva University, Bronx, NY 10461, USA
- 4Centre for Statistics in Medicine, University of Oxford, Oxford OX3 7LD, UK
- 5Nuffield Department of Surgical Sciences, University of Oxford, Oxford OX3 9DU, UK
- 6Population and Public Health Initiative, Queen’s University, Kingston, ON, Canada, K7L 3N6
- 7Centre for Health Evaluation and Outcome Sciences, St Paul’s Hospital, Vancouver, BC, Canada, V6Z 1Y9
- 8Department of Medicine, University of British Columbia, Vancouver, BC, Canada, V5Z 1M9
- 9International Clinical Sciences Support Center, FHI 360, Durham, NC 27713, USA
- Correspondence to: D Moher
- Accepted 27 May 2014
Objective To assess whether the completeness of reporting of health research is related to journals’ endorsement of reporting guidelines.
Design Systematic review.
Data sources Reporting guidelines from a published systematic review and the EQUATOR Network (October 2011). Studies assessing the completeness of reporting by using an included reporting guideline (termed “evaluations”) (1990 to October 2011; addendum searches in January 2012) from searches of either Medline, Embase, and the Cochrane Methodology Register or Scopus, depending on reporting guideline name.
Study selection English language reporting guidelines that provided explicit guidance for reporting, described the guidance development process, and indicated use of a consensus development process were included. The CONSORT statement was excluded, as evaluations of adherence to CONSORT had previously been reviewed. English or French language evaluations of included reporting guidelines were eligible if they assessed the completeness of reporting of studies as a primary intent and those included studies enabled the comparisons of interest (that is, after versus before journal endorsement and/or endorsing versus non-endorsing journals).
Data extraction Potentially eligible evaluations of included guidelines were screened initially by title and abstract and then as full text reports. If eligibility was unclear, authors of evaluations were contacted; journals’ websites were consulted for endorsement information where needed. The completeness of reporting of reporting guidelines was analyzed in relation to endorsement by item and, where consistent with the authors’ analysis, a mean summed score.
Results 101 reporting guidelines were included. Of 15 249 records retrieved from the search for evaluations, 26 evaluations that assessed completeness of reporting in relation to endorsement for nine reporting guidelines were identified. Of those, 13 evaluations assessing seven reporting guidelines (BMJ economic checklist, CONSORT for harms, PRISMA, QUOROM, STARD, STRICTA, and STROBE) could be analyzed. Reporting guideline items were assessed by few evaluations.
Conclusions The completeness of reporting of only nine of 101 health research reporting guidelines (excluding CONSORT) has been evaluated in relation to journals’ endorsement. Items from seven reporting guidelines were quantitatively analyzed, by few evaluations each. Insufficient evidence exists to determine the relation between journals’ endorsement of reporting guidelines and the completeness of reporting of published health research reports. Journal editors and researchers should consider collaborative prospectively designed, controlled studies to provide more robust evidence.
Systematic review registration Not registered; no known register currently accepts protocols for methodology systematic reviews.
We thank Shona Kirtley for information regarding the EQUATOR network search strategy for reporting guidelines, Andra Morrison for peer reviewing the search strategies developed for this review, Becky Skidmore for designing and conducting literature searches, Mary Gauthier and Sophia Tsouros for assisting with screening, Kavita Singh for assisting with screening and data extraction, Misty Pratt for assisting with screening and data extraction and verification, Raymond Daniel for article acquisition and management of bibliographic records within Reference Manager 12 and Distiller SR, Hadeel AlYacoob for assisting with verification of data extraction and analyses, Iveta Simera for contributions to the design of the project and feedback on the manuscript, and the authors of reporting guidelines and journal editors who kindly responded to our requests for information. We thank Stefania Boccia, Nikola Panic, Julien Peron, Benoit You, and other authors of evaluations who responded to our queries and provided raw data for our analyses.
Contributors: AS, LS, DM, DGA, LT, AH, JP, AP, and KFS contributed to the conception and design of this review via the published protocol. All authors contributed to the screening, data extraction, analysis, or data interpretation phases of the review. AS and DM drafted the review, and all remaining authors revised it critically for important intellectual content. All authors approved the final manuscript. DM is the guarantor.
Funding: This work was supported by a grant from the Canadian Institutes of Health Research (funding research number KSD-111750). The Canadian Institutes of Health Research had no role in the design, data collection, analysis, or interpretation of the data; in the writing of the report; or in the decision to submit this work for publication. DGA is supported by Cancer Research UK, DM by a University of Ottawa research chair, and KFS by FHI360. All researchers are independent from their relevant funding agencies. DGA, DM, and KFS are executive members of the EQUATOR network; AH served as an EQUATOR staff member during this project. The EQUATOR Network is funded by the National Health Service National Library of Health, National Health Service National Institute for Health Research, National Health Service National Knowledge service, United Kingdom Medical Research Council, Scottish Chief Scientist Office, and Pan American Health Organization.
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: all authors maintained their independence from the agency that funded this work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work.
Ethical approval: This systematic review did not require ethics approval in Canada.
Data sharing: Datasets are available on request from the corresponding author.
Transparency: The lead author (guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.