ReviewImproving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement
Introduction
Health-care providers and other decision-makers now have, among their information resources, a form of clinical report called the meta-analysis,1, 2, 3, 4 a review in which bias has been reduced by the systematic identification, appraisal, synthesis, and, if relevant, statistical aggregation of all relevant studies on a specific topic according to a predetermined and explicit method.3 The number of published meta-analyses has increased substantially in the past decade.5 These integrative articles can be helpful for clinical decisions, and they may also serve as the policy foundation for evidence-based practice guidelines, economic evaluations, and future research agendas. The value of meta-analysis is evident in the work of the international Cochrane Collaboration,6, 7 the primary purpose of which is to generate and disseminate high-quality systematic reviews of health-care interventions.
Like any research enterprise, particularly one that is observational, the meta-analysis of evidence can be flawed. Accordingly, the process by which meta-analyses are carried out has undergone scrutiny. A 1987 survey of 86 English-language meta-analyses8 assessed each publication on 23 items from six content areas judged important in the conduct and reporting of a meta-analysis of randomised trials: study design, combinability, control of bias, statistical analysis, sensitivity analysis, and problems of applicability. The survey results showed that only 24 (28%) of the 86 meta-analyses reported that all six content areas had been addressed. The updated survey, which included more recently published meta-analyses, showed little improvement in the rigour with which they were reported.9
Several publications have described the science of reviewing research,1 differences among narrative reviews, systematic reviews, and meta-analyses,2 and how to carry out,3, 4, 10 critically appraise,11, 12, 13, 14, 15 and apply16 meta-analyses in practice. The increase in the number of meta-analyses published has highlighted such issues as discordant meta-analyses on the same topic17 and discordant meta-analyses and randomised-trial results on the same question.18
An important consideration in interpretation and use of meta-analyses is to ascertain that the investigators who did the meta-analysis not only report explicitly the methods they used to analyse the articles they reviewed, but also report the methods used in the research articles they analysed. The meta-analytical review methods used may not be provided when a paper is initially submitted: even when they are, other factors such as page limitations, peer review, and editorial decisions may change the content and format of the report before publication.
Several investigators have suggested guidelines for reporting of meta-analyses.3, 19 However, a consensus across disciplines has not developed. After the initiative to improve the quality of reporting of randomised controlled trials (RCTs),20, 21, 22 we organised the Quality of Reporting of Meta-analyses (QUOROM) conference to address these issues as they relate to meta-analyses of RCTs. This report summarises the proceedings of that conference. The issues discussed might also be useful for reporting of systematic reviews (ie, meta-analysis, as defined above, without statistical aggregation), particularly of RCTs.
Section snippets
Methods
The QUOROM steering committee began with a comprehensive review of publications on the conduct and reporting of meta-analyses. The databases searched included MEDLINE and the Cochrane Library,23 which consists of the Cochrane Database of Systematic Reviews, the Cochrane Controlled Trials Register, the York Database of Abstracts of Reviews of Effectiveness, and the Cochrane Review Methodology Database. We examined reference lists of the retrieved articles and individual personal files. Articles
Results
The conference resulted in the QUOROM statement: a checklist (table) and a flow diagram (figure). The checklist of standards for reporting of meta-analyses describes our preferred way to present the abstract, introduction, methods, results, and discussion sections of a report of a meta-analysis. The checklist is organised into 21 headings and subheadings to encourage authors to provide readers with information on searches, selection, validity assessment, data abstraction, study characteristics,
Discussion
In developing the checklist, we identified supporting scientific evidence for only eight of 18 items to guide the reporting of meta-analyses of RCTs.26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39 Some of this evidence is indirect. For example, we ask authors to use a structured abstract format. The supporting evidence for this item was collected by examining abstracts of original reports of individual studies27 and may not pertain specifically to the reporting of meta-analyses. However,
References (48)
- et al.
Methodologic guidelines for systematic reviews of randomized controlled trials in health care from the Potsdam consultation on meta-analysis
J Clin Epidemiol
(1995) Statistical and theoretical considerations in meta-analysis
J Clin Epidemiol
(1995)- et al.
Identification of meta-analyses: the need for standard terminology
Control Clin Trials
(1990) - et al.
Language bias in randomised controlled trials published in English and German
Lancet
(1997) - et al.
Empirical evidence of bias in infertility research: overestimation of treatment effect in crossover trials using pregnancy as the outcome measure
Fertil Steril
(1996) - et al.
Does the quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?
Lancet
(1998) - et al.
Assessing the quality of reports of randomized clinical trials: is blinding necessary?
Control Clin Trials
(1996) Does blinding of readers affect the results of meta-analyses?
Lancet
(1997)- et al.
Publication bias in clinical research
Lancet
(1991) Publication bias and meta-analysis
EvalEduc
(1980)
Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews
Lancet
Do certain countries produce only positive results? A systematic review of controlled trials
Control Clin Trials
Risperidone versus haloperidol: I, meta-analysis of efficacy and safety
Clin Ther
Efficacy and extrapyramidal side-effects of the new antipsychotics olanzapine, quetiapine, risperidone, and sertindole compared to conventional antipsychotics and placebo: a meta-analysis of randomized controlled trials
Schizophrenia Res
The medical review article: state of the science
Ann Intern Med
Systematic reviews: synthesis of best evidence for clinical decisions
Ann Intern Med
Undertaking systematic reviews of research on effectiveness CRD guidelines for those carrying out or commissioning reviews. CRD report no 4
Reporting, updating, and correcting systematic reviews of the effects of health care
The Cochrane Collaboration: preparing, maintaining, and disseminating systematic reviews of the effects of health care
JAMA
The Cochrane Collaboration helping unravel tangled web woven by international research
Can Med Assoc J
Meta-analyses of randomized controlled trials
N Engl J Med
Meta-analysis: an update
Mt Sinai J Med
Users' guides to the medical literature: VI, how to use an overview
JAMA
Cited by (4039)
Cognitive Biases in Fact-Checking and Their Countermeasures: A Review
2024, Information Processing and ManagementA new Normalized Index for Ranking Papers in Systematic Literature Reviews
2024, Decision Analytics JournalPillar Pain After Minimally Invasive and Standard Open Carpal Tunnel Release: A Systematic Review and Meta-analysis
2024, Journal of Hand Surgery Global OnlineEnhancing the quality of reporting of orthodontic clinical research
2024, Seminars in Orthodontics
Other members listed at end of paper