Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review

doi:10.1136/bmj.38973.444699.0B

BMJ 2006;333:782 (14 October), doi:10.1136/bmj.38973.444699.0B (published 6 October 2006)

Research

Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review

Anders W Jørgensen, physician¹, Jørgen Hilden, associate professor², Peter C Gøtzsche, director¹

¹ Nordic Cochrane Centre, Rigshospitalet, DK-2100 Copenhagen Ø, Denmark, ² Department of Biostatistics, Panum Institute, University of Copenhagen, Denmark

Correspondence to: P C Gøtzsche pcg{at}cochrane.dk

Abstract

Objective To compare the methodological quality and conclusionsin Cochrane reviews with those in industry supported meta-analysesand other meta-analyses of the same drugs.

Design Systematic review comparing pairs of meta-analyses thatstudied the same two drugs in the same disease and were publishedwithin two years of each other.

Data sources Cochrane Database of Systematic Reviews (2003,issue 1), PubMed, and Embase.

Data extraction Two observers independently extracted data andused a validated scale to judge the methodological quality ofthe reviews.

Results 175 of 1596 Cochrane reviews had a meta-analysis thatcompared two drugs. Twenty four meta-analyses that matched theCochrane reviews were found: eight were industry supported,nine had undeclared support, and seven had no support or weresupported by non-industry sources. On a 0-7 scale, the medianquality score was 7 for Cochrane reviews and 3 for other reviews(P < 0.01). Compared with industry supported reviews andreviews with undeclared support, Cochrane reviews had more oftenconsidered the potential for bias in the review—for example,by describing the method of concealment of allocation and describingexcluded patients or studies. The seven industry supported reviewsthat had conclusions recommended the experimental drug withoutreservations, compared with none of the Cochrane reviews (P= 0.02), although the estimated treatment effect was similaron average (z = 0.46, P = 0.64). Reviews with undeclared supportand reviews with not for profit support or no support had conclusionsthat were similar in cautiousness to the Cochrane reviews.

Conclusions Industry supported reviews of drugs should be readwith caution as they were less transparent, had few reservationsabout methodological limitations of the included trials, andhad more favourable conclusions than the corresponding Cochranereviews.

Add to CiteULike CiteULike Add to Complore Complore Add to Connotea Connotea Add to Del.icio.us Del.icio.us Add to Digg Digg Add to Reddit Reddit Add to StumbleUpon StumbleUpon Add to Technorati Technorati What's this?

Relevant Articles

The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration: Alessandro Liberati, Douglas G Altman, Jennifer Tetzlaff, Cynthia Mulrow, Peter C Gøtzsche, John P A Ioannidis, Mike Clarke, P J Devereaux, Jos Kleijnen, and David Moher
BMJ 2009 339: b2700. [Abstract] [Full Text]

Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study: Veronica Yank, Drummond Rennie, and Lisa A Bero
BMJ 2007 335: 1202-1205. [Abstract] [Full Text] [PDF]

Authors' reply on Cochrane reviews v industry supported meta-analyses: Anders W Jørgensen, Peter C Gøtzsche, and Jørgen Hilden
BMJ 2006 333: 1072-1073. [Extract] [Full Text] [PDF]

Cochrane reviews v industry supported meta-analyses: We should read all reviews with caution: James C Coyne
BMJ 2006 333: 916. [Extract] [Full Text] [PDF]

Cochrane reviews v industry supported meta-analyses: Has Cochrane really achieved its goals?: Marko Tostad
BMJ 2006 333: 916. [Extract] [Full Text]

Read industry supported drug reviews with caution
BMJ 2006 333: 0. [Full Text]

Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study: Lorenzo P Moja, Elena Telaro, Roberto D'Amico, Ivan Moschetti, Laura Coe, Alessandro Liberati on behalf of the Metaquality Study Group
BMJ 2005 330: 1053. [Abstract] [Full Text] [PDF]

Pharmaceutical industry sponsorship and research outcome and quality: systematic review: Joel Lexchin, Lisa A Bero, Benjamin Djulbegovic, and Otavio Clark
BMJ 2003 326: 1167-1170. [Abstract] [Full Text] [PDF]

Quality of Cochrane reviews: assessment of sample from 1998: Ole Olsen, Philippa Middleton, Jeanette Ezzo, Peter C Gøtzsche, Victoria Hadhazy, Andrew Herxheimer, Jos Kleijnen, and Heather McIntosh
BMJ 2001 323: 829-832. [Abstract] [Full Text] [PDF]

The need for caution in interpreting high quality systematic reviews: Kevork Hopayian
BMJ 2001 323: 681-684. [Extract] [Full Text] [PDF]

Systematic reviews in health care: Assessing the quality of controlled clinical trials: Peter Jüni, Douglas G Altman, and Matthias Egger
BMJ 2001 323: 42-46. [Extract] [Full Text] [PDF]

Systematic reviews and meta-analyses on treatment of asthma: critical evaluation: Alejandro R Jadad, Michael Moher, George P Browman, Lynda Booker, Christopher Sigouin, Mario Fuentes, and Robert Stevens
BMJ 2000 320: 537-540. [Abstract] [Full Text] [PDF]

Meta-analysis: Unresolved issues and future developments: George Davey Smith and Matthias Egger
BMJ 1998 316: 221-225. [Extract] [Full Text]

Publication bias: evidence of delayed publication in a cohort study of clinical research projects: Jerome M Stern and R John Simes
BMJ 1997 315: 640-645. [Abstract] [Full Text]

This article has been cited by other articles:

Okike, K., Kocher, M. S., Wei, E. X., Mehlman, C. T., Bhandari, M. (2009). Accuracy of Conflict-of-Interest Disclosures Reported by Physicians. NEJM 361: 1466-1474 [Abstract] [Full text]
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P.A., Clarke, M., Devereaux, P. J., Kleijnen, J., Moher, D. (2009). The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration. ANN INTERN MED 151: W-65-W-94 [Abstract] [Full text]
Liberati, A., Altman, D. G, Tetzlaff, J., Mulrow, C., Gotzsche, P. C, Ioannidis, J. P A, Clarke, M., Devereaux, P J, Kleijnen, J., Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 339: b2700-b2700 [Abstract] [Full text]
Luty, J, Arokiadass, S M R, Easow, J M, Anapreddy, J R (2009). Preferential publication of editorial board members in medical specialty journals. J. Med. Ethics 35: 200-202 [Abstract] [Full text]
Lambers, M. J., Mijatovic,, V., Hompes, P. G.A. (2009). Low dose aspirin and IVF: 'Is it time for a meta-analysis'? Continued: the consequences of the choices made. Hum Reprod Update 15: 262-263 [Full text]
Nasser, M., van Binsbergen, J., Javaheri, H., Yassiny, K. (2008). Diet and nutrition advice from the Cochrane Library: is it useful for the consumers and family physicians?. Fam Pract 25: i60-i66 [Abstract] [Full text]
Rosenheck, R.A., Leslie, D.L., Busch, S., Rofman, E. S., Sernyak, M. (2008). Rethinking Antipsychotic Formulary Policy. Schizophr Bull 34: 375-380 [Abstract] [Full text]
Garg, A. X., Hackam, D., Tonelli, M. (2008). Systematic Review and Meta-analysis: When One Study Is Just not Enough. CJASN 3: 253-260 [Full text]
Yank, V., Rennie, D., Bero, L. A (2007). Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study. BMJ 335: 1202-1205 [Abstract] [Full text]
Krumholz, H. M., Masoudi, F. A. (2007). The Year in Epidemiology, Health Services Research, and Outcomes Research. J Am Coll Cardiol 50: 2254-2262 [Full text]
Boluyt, N., van der Lee, J. H., Moyer, V. A., Brand, P. L. P., Offringa, M. (2007). State of the Evidence on Acute Asthma Management in Children: A Critical Appraisal of Systematic Reviews. Pediatrics 120: 1334-1343 [Abstract] [Full text]
BYRNE, A., WODAK, A. (2007). Data Do Not Support Buprenorphine as a First-Line Treatment for Addiction. Am. J. Psychiatry 164: 1757-1757 [Full text]
HEILIG, M. (2007). Dr. Heilig Replies. Am. J. Psychiatry 164: 1757-1758 [Full text]
Milbrandt, E. B., Ishizaka, A., Angus, D. C. (2007). Update in Critical Care 2006. Am. J. Respir. Crit. Care Med. 175: 638-648 [Full text]
(2006). Drug Meta-Analyses: Who Supports Them Makes a Difference. JWatch General 2006: 5-5 [Full text]
Bastian, H. (2006). 'They would say that, wouldn't they?' A reader's guide to author and sponsor biases in clinical research. JRSM 99: 611-614 [Full text]
Deeks, J. J (2006). Word limits best explain failings of industry supported meta-analyses. BMJ 333: 1021-1021 [Full text]
(2006). Interpret Industry-Supported Meta-Analyses with Caution. Journal Watch Cardiology 2006: 3-3 [Full text]
(2006). Drug Company-Sponsored Research: Buyer Beware!. JWatch Emergency Med. 2006: 5-5 [Full text]
Coyne, J. C (2006). Cochrane reviews v industry supported meta-analyses: we should read all reviews with caution.. BMJ 333: 916-916 [Full text]
Tostad, M. (2006). Cochrane reviews v industry supported meta-analyses: has Cochrane really achieved its goals?. BMJ 333: 916-916 [Full text]

Rapid Responses:

Read all Rapid Responses

Balanced comparison mandatory for new drugs launches: Rajan TD
bmj.com, 10 Oct 2006 [Full text]
A short ten-step-questionnaire about possible bias in medical research: Manfred Gogol
bmj.com, 13 Oct 2006 [Full text]
Industry sponsered Drug trials : It is commerce masquerading as science ! Let us stop the abuse !: venkatesan sangareddi
bmj.com, 14 Oct 2006 [Full text]
Has Cochrane really achieved its goals?: Marko Tostad
bmj.com, 14 Oct 2006 [Full text]
Cochrane bias: Stephen J Senn
bmj.com, 16 Oct 2006 [Full text]
Doctored meta-analyses are not the only way drug companies manipulate physician drug use.: Rekha N Pillai, et al.
bmj.com, 16 Oct 2006 [Full text]
Before We Put Too Much Stock In Cochrane Systematic Reviews�Or Anyone�s�: James C. Coyne, et al.
bmj.com, 18 Oct 2006 [Full text]
Journals must share the blame - word restrictions, not the funder, best explain these findings: Jonathan J Deeks
bmj.com, 19 Oct 2006 [Full text]
Cochrane bias article: Douglas Keller
bmj.com, 23 Oct 2006 [Full text]
Pharmaceutical Lies: Deepak Malhotra, et al.
bmj.com, 28 Oct 2006 [Full text]
Inherent biases in publication process key confounders: Kostas Zacharias, et al.
bmj.com, 28 Oct 2006 [Full text]
Cochrane reviews and other meta-analyses, authors� reply: Anders W J�rgensen, et al.
bmj.com, 5 Nov 2006 [Full text]
Introduce Evidence Based Medicine into Medical Education Curriculum: Babatunde Adetunji, MD, MA, MS, FASAM, et al.
bmj.com, 30 Nov 2006 [Full text]