BMJ 2000;320:537-540 ( 26 February )

Papers

Systematic reviews and meta-analyses on treatment of asthma: critical evaluation

Alejandro R Jadad, professor, ^a Michael Moher, Royal College of General Practitioners research training fellow, ^b George P Browman, professor, ^a Lynda Booker, research assistant, ^a Christopher Sigouin, doctoral student, ^a Mario Fuentes, research assistant, ^c Robert Stevens, research assistant. ^c

^a Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada L8N 3Z5, ^b Institute of Health Sciences, University of Oxford, Headington, Oxford OX3 7LF, ^c Foresight Consultants, Dundas, Ontario, Canada L9H 2R5

Correspondence to: A R Jadad jadada{at}fhs.mcmaster.ca

Objective: To evaluate the clinical, methodological, and reporting aspects of systematic reviews and meta-analyses on the treatment of asthma and to compare those published by the Cochrane Collaboration with those published in paper based journals.
Design: Analysis of studies identified from Medline, CINAHL, HealthSTAR, EMBASE, Cochrane Library, personal collections, and reference lists.
Studies: Articles describing a systematic review or a meta-analysis of the treatment of asthma that were published as a full report, in any language or format, in a peer reviewed journal or the Cochrane Library.
Main outcome measures: General characteristics of studies reviewed and methodological characteristics (sources of articles; language restrictions; format, design, and publication status of studies included; type of data synthesis; and methodological quality).
Results: 50 systematic reviews and meta-analyses were included. More than half were published in the past two years. Twelve reviews were published in the Cochrane Library and 38 were published in 22 peer reviewed journals. Forced expiratory volume in one second was the most frequently used outcome, but few reviews evaluated the effect of treatment on costs or patient preferences. Forty reviews were judged to have serious or extensive flaws. All six reviews associated with industry were in this group. Seven of the 10 most rigorous reviews were published in the Cochrane Library.
Conclusions: Most reviews published in peer reviewed journals or funded by industry have serious methodological flaws that limit their value to guide decisions. Cochrane reviews are more rigorous and better reported than those published in peer reviewed journals.

---

© BMJ 2000
CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study

Veronica Yank, Drummond Rennie, and Lisa A Bero
BMJ 2007 335: 1202-1205. [Abstract] [Full Text] [PDF]

Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review

Anders W Jørgensen, Jørgen Hilden, and Peter C Gøtzsche
BMJ 2006 333: 782. [Abstract] [Full Text] [PDF]

Compliance with QUOROM and quality of reporting of overlapping meta-analyses on the role of acetylcysteine in the prevention of contrast associated nephropathy: case study

Giuseppe G L Biondi-Zoccai, Marzia Lotrionte, Antonio Abbate, Luca Testa, Enrico Remigi, Francesco Burzotta, Marco Valgimigli, Enrico Romagnoli, Filippo Crea, and Pierfrancesco Agostoni
BMJ 2006 332: 202-209. [Abstract] [Full Text] [PDF]

Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study

Lorenzo P Moja, Elena Telaro, Roberto D'Amico, Ivan Moschetti, Laura Coe, Alessandro Liberati on behalf of the Metaquality Study Group
BMJ 2005 330: 1053. [Abstract] [Full Text] [PDF]

The quality of systematic reviews

Stephen Senn, Douglas G Altman, Jonathan J Deeks, Mike Clarke, Christopher Cates, N C Barnes, C Hallett, T A J Harris, Carole Wilson, Paula Waugh, Paul Garner, Alejandro R Jadad, Michael Moher, George P Browman, Lynda Booker, Christopher Sigouin, Mario Fuentes, and Robert Stevens
BMJ 2000 321: 297. [Extract] [Full Text] [PDF]

Systematic reviews and meta-analyses are often flawed
BMJ 2000 320: 0. [Full Text]

Advances in managing chronic disease

Ronald M Davis, Edward G Wagner, and Trish Groves
BMJ 2000 320: 525-526. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• Boluyt, N., Tjosvold, L., Lefebvre, C., Klassen, T. P., Offringa, M. (2008). Usefulness of Systematic Review Search Strategies in Finding Child Health Systematic Reviews in MEDLINE. Arch Pediatr Adolesc Med 162: 111-116 [Abstract] [Full text]  
• Garg, A. X., Hackam, D., Tonelli, M. (2008). Systematic Review and Meta-analysis: When One Study Is Just not Enough. CJASN 3: 253-260 [Full text]  
• Yank, V., Rennie, D., Bero, L. A (2007). Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study. BMJ 335: 1202-1205 [Abstract] [Full text]  
• Boluyt, N., van der Lee, J. H., Moyer, V. A., Brand, P. L. P., Offringa, M. (2007). State of the Evidence on Acute Asthma Management in Children: A Critical Appraisal of Systematic Reviews. Pediatrics 120: 1334-1343 [Abstract] [Full text]  
• Lathyris, D. N, Trikalinos, T. A, Ioannidis, J. P. (2007). Evidence from crossover trials: empirical evaluation and comparison against parallel arm trials. Int J Epidemiol 36: 422-430 [Abstract] [Full text]  
• Jorgensen, A. W, Hilden, J., Gotzsche, P. C (2006). Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ 333: 782- [Abstract] [Full text]  
• Waters, E., Doyle, J., Jackson, N., Howes, F., Brunton, G., Oakley, A. (2006). Evaluating the effectiveness of public health interventions: the role and activities of the Cochrane Collaboration.. J. Epidemiol. Community Health 60: 285-289 [Abstract] [Full text]  
• Biondi-Zoccai, G. G L, Lotrionte, M., Abbate, A., Testa, L., Remigi, E., Burzotta, F., Valgimigli, M., Romagnoli, E., Crea, F., Agostoni, P. (2006). Compliance with QUOROM and quality of reporting of overlapping meta-analyses on the role of acetylcysteine in the prevention of contrast associated nephropathy: case study. BMJ 332: 202-209 [Abstract] [Full text]  
• Tattersfield, A E (2005). When should a long acting {beta} agonist be added to an inhaled corticosteroid?. Thorax 60: 710-710 [Full text]  
• Moja, L. P, Telaro, E., D'Amico, R., Moschetti, I., Coe, L., Liberati, A., on behalf of the Metaquality Study Group, (2005). Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ 330: 1053- [Abstract] [Full text]  
• Gilligan, P, Law, H, Lumsden, G, Brenchley, J, Kitching, G, Taylor, A, Khan, A, Shepherd, M, Jones, J, Hegarty, D (2005). SOCRATES 3 (synopsis of Cochrane reviews applicable to emergency services). Emerg. Med. J. 22: 50-52 [Full text]  
• Eng, J., Krishnan, J. A., Segal, J. B., Bolger, D. T., Tamariz, L. J., Streiff, M. B., Jenckes, M. W., Bass, E. B. (2004). Accuracy of CT in the Diagnosis of Pulmonary Embolism: A Systematic Literature Review. Am. J. Roentgenol. 183: 1819-1827 [Abstract] [Full text]  
• Als-Nielsen, B., Chen, W., Gluud, C., Kjaergard, L. L. (2003). Association of Funding and Conclusions in Randomized Drug Trials: A Reflection of Treatment Effect or Adverse Events?. JAMA 290: 921-928 [Abstract] [Full text]  
• Jewitt, K., Hughes, R. (2003). SYSTEMATIC REVIEWS TO HELP GUIDE CLINICAL PRACTICE IN NEUROMUSCULAR DISEASE. J. Neurol. Neurosurg. Psychiatry 74: ii43-44 [Full text]  
• Lexchin, J., Bero, L. A, Djulbegovic, B., Clark, O. (2003). Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 326: 1167-1170 [Abstract] [Full text]  
• Astin, J. A., Shapiro, S. L., Eisenberg, D. M., Forys, K. L. (2003). Mind-Body Medicine: State of the Science, Implications for Practice. J Am Board Fam Med 16: 131-147 [Abstract] [Full text]  
• Linde, K., Willich, S. N (2003). How objective are systematic reviews? Differences between reviews on complementary medicine. JRSM 96: 17-22 [Abstract] [Full text]  
• McDonald, E., Cook, D., Newman, T., Griffith, L., Cox, G., Guyatt, G. (2002). Effect of Air Filtration Systems on Asthma: A Systematic Review of Randomized Trials. Chest 122: 1535-1542 [Abstract] [Full text]  
• Shea, B., Moher, D., Graham, I., Pham, B., Tugwell, P. (2002). A Comparison of the Quality of Cochrane Reviews and Systematic Reviews Published in Paper-Based Journals. Eval Health Prof 25: 116-129 [Abstract]  
• FIELD, S K, SUTHERLAND, L R, GIBSON, P (2001). Gastro-oesophageal reflux and asthma. Thorax 56: 898-898 [Full text]  
• Olsen, O., Middleton, P., Ezzo, J., Gotzsche, P. C, Hadhazy, V., Herxheimer, A., Kleijnen, J., McIntosh, H. (2001). Quality of Cochrane reviews: assessment of sample from 1998. BMJ 323: 829-832 [Abstract] [Full text]  
• Hopayian, K. (2001). The need for caution in interpreting high quality systematic reviews. BMJ 323: 681-684 [Full text]  
• T-L. Choi, P., Halpern, S. H., Malik, N., Jadad, A. R., Tramer, M. R., Walder, B. (2001). Examining the Evidence in Anesthesia Literature: A Critical Appraisal of Systematic Reviews. Anesth. Analg. 92: 700-709 [Abstract] [Full text]  
• Senn, S., Greig, A D, Ram, F. S F, Shrewsbury, S., Pyke, S., Britton, M. (2000). Meta-analysis of increased inhaled steroid or addition of salmeterol in asthma. BMJ 321: 1016a-1016 [Full text]  
• Senn, S., Altman, D. G, Deeks, J. J, Clarke, M., Cates, C., Barnes, N C, Hallett, C, Harris, T A J, Wilson, C., Waugh, P., Garner, P., Jadad, A. R, Moher, M., Browman, G. P, Booker, L., Sigouin, C., Fuentes, M., Stevens, R. (2000). The quality of systematic reviews. BMJ 321: 297-297 [Full text]  
• (2000). A Systematic Review of Systematic Reviews. JWatch General 2000: 10-10 [Full text]  
• Davis, R. M, Wagner, E. G, Groves, T. (2000). Advances in managing chronic disease. BMJ 320: 525-526 [Full text]  

Rapid Responses:

Read all Rapid Responses

A never-ending trail...

Eyal Shahar
bmj.com, 25 Feb 2000 [Full text]

Biased review

Stephen Senn
bmj.com, 26 Feb 2000 [Full text]

Systematic review

Alan M Edwards
bmj.com, 2 Mar 2000 [Full text]

The quality of systematic reviews

Douglas G Altman
bmj.com, 17 Mar 2000 [Full text]

Erratum

Cecilia Hammarquist
bmj.com, 28 Mar 2000 [Full text]

Meta-analysis: if it's not stated, was it done?

C Hallett
bmj.com, 11 Apr 2000 [Full text]

Use of HEPA filter vacume cleaner in house dust mite control

J W Fletcher
bmj.com, 28 Jun 2000 [Full text]