- George H Swingler, associate professor1,
- Jimmy Volmink, professor2,
- John P A Ioannidis, chairman ()3
- 1School of Child and Adolescent Health, Red Cross Children's Hospital, University of Cape Town, 7700 Rondebosch, South Africa
- 2Primary Health Care, Faculty of Health Sciences, University of Cape Town, Cape Town
- 3Clinical Trials and Evidence-Based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece
- Correspondence to: J P A Ioannidis
- Accepted 4 August 2003
Systematic reviews are key to implementing evidence based medicine.1 We wondered if the reviews done to date are related to the burden of disease from various conditions. Ideally, evidence should be prioritised for diseases with the greatest global impact.
Methods and results
We estimated Spearman correlations between the number of systematic reviews in two important databases (the Cochrane database of systematic reviews (CDSR) and the database of abstracts of reviews of effects (DARE)) and the burden of disease (globally and in established market economies) across disease categories. We also estimated the burden of disease for each available review measured in disability adjusted life years (DALYs).2 3 We used 1990 estimates of burden of disease because studies included in systematic reviews would have responded to recent past health needs. Results with estimates from 2000 were similar.
We categorised tar geted diseases in 923 reviews from the CDSR and 1899 reviews from the DARE in issue 4, 2000, of the Cochrane Library using 20 categories of the global burden of disease taxonomy.3 We excluded unclassifiable topics (health systems, pain or anaesthesia, general operative techniques, and smoking cessation). To avoid small contributors to burden of disease, a separate analysis retained only the top 10 groups of disease accounting for > 90% of the global burden of disease. Reviews in the DARE came from high profile general medical journals (173), other general journals (77), specialist journals (1532), or other reports (117). Two independent investigators did categorisations and resolved disagreements by discussion.
We looked for correlation between the number of systematic reviews and the burden of disease. Given the small number of categories, modest differences in estimated correlations between databases and subgroups should not be attributed formal statistical significance.
We categorised 866 reviews from the CDSR and 1639 reviews from the DARE (898 and 1729 disease group entries). Coverage was similar across databases except the CDSR covered maternal and perinatal conditions better. Across disease groups, global DALYs for each review varied between 0.2-33.0 million in the CDSR and 0.1-5.5 million in the DARE. Among the top 10 disease groups, nutritional deficiencies, injuries, respiratory infections, and infectious diseases were most neglected (> 2 million global DALYs for each available review in either database).
Burden of disease was modestly correlated with the number of systematic reviews in the CDSR (global r = 0.54, P = 0.014; established market economies r = 0.46, P = 0.041), the DARE (global r = 0.65, P = 0.002; established market economies r =0.76, P < 0.001) and in subgroups of the DARE.
For the top 10 disease groups, correlations between the number of systematic reviews and the global burden of disease remained unchanged in CDSR (r = 0.52, P = 0.13), but decreased in DARE (r = 0.42, P = 0.23). The burden of disease in established market economies correlated modestly with the number of reviews in the CDSR (r = 0.56; P = 0.09); correlations in the DARE were high (overall r = 0.87, P < 0.001, range 0.63-0.94 across subgroups of reviews).
The number of reviews in the DARE seemed less responsive to global burden of disease than to the burden in established market economies, but the difference was not significant. The CDSR did not show this (figure).
The number of published systematic reviews is still relatively small and unevenly covers different diseases and aspects of health care. Often, millions of DALYs correspond to each available systematic review. Of course, new systematic reviews are continuously produced: there are currently 5877 reviews in the Cochrane Library (issue 3, 2004, including 1754 in CDSR and 4123 in DARE), but the overall picture is probably the same. Differences between the CDSR and the DARE may be because the CDSR is an electronic database without space limitations. Conversely, the content in the DARE depends more on selection forces related to the subject matter with preference for health problems in established market economies. In peer reviewed journals (and the database of abstracts of reviews of effect (DARE)), systematic reviews follow the priorities of established market economies rather than global priorities.
Measuring research and the emphasis of research is difficult.4 The availability of synthesised knowledge is a product of the amount of primary research. For most healthcare practitioners, synthesised knowledge is more relevant than primary research. Also, doing systematic reviews is important even if evidence is lacking.5 Systematic reviews fathom the existing uncertainty and help build an agenda for future research.
Contributors: JPAI and GHS designed the study. GHS coordinated the study. JPAI did the analysis. All authors extracted data, interpreted findings, and wrote the report. JPAI is guarantor.
Funding No additional funding.
Competing interests All three authors have been involved in the Cochrane Collaboration.
Ethical approval Not needed.
The disease categories in the figure are in order on bmj.com