Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional studyBMJ 2005; 330 doi: https://doi.org/10.1136/bmj.38414.515938.8F (Published 05 May 2005) Cite this as: BMJ 2005;330:1053
- Lorenzo P Moja, researcher1,
- Elena Telaro, researcher1,
- Roberto D'Amico, researcher2,
- Ivan Moschetti, researcher1,
- Laura Coe, researcher1,
- Alessandro Liberati, director ()1 on behalf of the Metaquality Study Group
- 1Centro Cochrane Italiano, Istituto Mario Negri, Via Eritrea 62, 20157 Milan, Italy,
- 2Università degli Studi di Modena e Reggio Emilia, Modena, Italy
- Correspondence to: A Liberati, Università degli Studi di Modena e Reggio Emilia, Modena, Italy
- Accepted 2 March 2005
Objectives To describe how the methodological quality of primary studies is assessed in systematic reviews and whether the quality assessment is taken into account in the interpretation of results.
Data sources Cochrane systematic reviews and systematic reviews in paper based journals.
Study selection 965 systematic reviews (809 Cochrane reviews and 156 paper based reviews) published between 1995 and 2002.
Data synthesis The methodological quality of primary studies was assessed in 854 of the 965 systematic reviews (88.5%). This occurred more often in Cochrane reviews than in paper based reviews (93.9% v 60.3%, P < 0.0001). Overall, only 496 (51.4%) used the quality assessment in the analysis and interpretation of the results or in their discussion, with no significant differences between Cochrane reviews and paper based reviews (52% v 49%, P = 0.58). The tools and methods used for quality assessment varied widely.
Conclusions Cochrane reviews fared better than systematic reviews published in paper based journals in terms of assessment of methodological quality of primary studies, although they both largely failed to take it into account in the interpretation of results. Methods for assessment of methodological quality by systematic reviews are still in their infancy and there is substantial room for improvement.
Contributors LPM, ET, RD'A, and AL wrote the protocol of the study and RD'A and AL developed the grant application to complete this project. ET and LPM coordinated the project, participated in the quality assessment, data extraction, database development, and data entry. LC, IM, and RD'A participated in the quality assessment, data extraction, and data entry. RD'A, LPM, and IM completed the data analyses. All authors provided feedback on earlier versions of the paper. LPM, RD'A, and AL took primary responsibility for writing the manuscript, which was revised by coauthors. AL is guarantor for the paper. Metaquality Study Group: the following researchers were members of the Milano master course in systematic reviews and contributed to the study in the pilots, reading and quality assessment of systematic reviews, data extraction, and data entry: Battaggia Alessandro, Bianco Elvira, Calderan Alessandro, Colli Agostino, Ferri Marica, Fraquelli Mirella, Girolami Bruno, Marchioni Enrico, Mezza Elisabetta, Piccoli Giorgina, Vignatelli Luca, Monaco Giuseppe, Morganti Carla, Franceschini Roberta, Bermond Francesca, Cevoli Sabina, Franzoso Federico, Ruggeri Marco, Joppi Roberta, Minesso Elisabetta, Lomolino Grazia, and Bellù Roberto.
Funding Ministero della Istruzione, dell'Università e della Ricerca, Rome, Italy (protocol grant No 2002061749 COFIN 2002)
Conflict of interest All members of the research team are members of the Cochrane Collaboration. ET, IM, RD'A, LC, and AL are authors of Cochrane reviews published in the Cochrane Library. The views and opinions expressed are those of the authors and do not necessarily reflect those of the Cochrane Collaboration
Ethical approval Not required
- Accepted 2 March 2005