BMJ  2006;333:1021 (11 November), doi:10.1136/bmj.39024.372662.1F

Letters

Word limits best explain failings of industry supported meta-analyses

Although few doubt that industry funded systematic reviews sometimes use poor methods and misrepresent findings, Jørgensen et al overestimate this bias and misattribute differences in methods and reporting to it.¹ Median quality scores of the included reviews were 7 for Cochrane reviews, and 2, 2, and 3 for industry funded, undeclared funding and non-profit or no funding journal reviews. These results are best explained by word restrictions, not financial support. For example, the BMJ paper and online versions of the included celecoxib review were restricted to 2211 and 3425 words,² whereas the unrestricted Cochrane review has 6002 words.³

The reliability of unblinded quality assessments raises concern, at least for the celecoxib reviews. Contrary to the findings of Jørgensen et al, the reviews gave equivalent detail concerning allocation concealment, and the industry funded review contained four paragraphs with reservations about results. Interestingly, the BMJ deleted two of these paragraphs to shorten the review for the paper journal. Had Jørgensen's study only included published reviews, the comparison by funding source could have been blinded and controlled for word length.

The industry celecoxib review was produced by an experienced Cochrane reviewer, protected from industry interference by a contract allowing freedom to publish (including results of previously unpublished trials). Data were extracted from full industry reports, avoiding problems extracting detail from abridged journal articles, such as the JAMA report of the CLASS trial ⁴. The Cochrane review did not have access to this level of information.

Assessment of the likelihood of bias in reviews, including Cochrane reviews, should always be based on the methods and completeness of results, not on prejudices about the organisations from which they emanate. The lesson from the paper by Jørgensen et al may be more that journals should reverse the trend of reducing word lengths, and give authors the opportunity to explain methods in the detail afforded by the Cochrane review format.

¹ Department of Public Health and Epidemiology, University of Birmingham, Birmingham B15 2TT j.deeks@bham.ac.uk

---

Competing interests: JJD has contributed to the Cochrane Collaboration for over a decade, is currently a member of the steering group of the Cochrane Collaboration and Treasurer. He has co-authored 14 Cochrane reviews, and is the lead editor of the statistical section of the Cochrane Handbook. The views expressed here are his own and not necessarily those of the Cochrane Collaboration. His previous unit (Centre for Statistics in Medicine, Oxford) received funding from Pfizer and Searle to undertake the review mentioned in reference 2 and JJD received consultancy fees from Pfizer from 2001-2. Both his previous and current units have received royalties from the BMJ on sales of reprints of reference 2.

References

1. Jørgensen AW, Hilden J, Gøtzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic reviews. BMJ 2006:333:782-5. (14 October.)[Abstract/Free Full Text]
2. Deeks JJ, Smith LA, Bradley MD. Efficacy, tolerability, and upper gastrointestinal safety of celecoxib for treatment of osteoarthritis and rheumatoid arthritis: systematic review of randomised controlled trials. BMJ 2002;325:619.[Abstract/Free Full Text]
3. Garner SE, Fidan DD, Frankish RR, Judd MG, Shea BJ, Towheed TE, et al. Celecoxib for rheumatoid arthritis. Cochrane Database Syst Rev 2002;4:CD003831.
4. Silverstein FE, Faich G, Goldstein JL, Simon LS, Pincus T, Whelton A, et al. Gastrointestinal toxicity with celecoxib vs nonsteroidal anti-inflammatory drugs for osteoarthritis and rheumatoid arthritis: the CLASS study: a randomized controlled trial. celecoxib long-term arthritis safety study. JAMA 2000;284:1247-55.[Abstract/Free Full Text]

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Are US flu death figures more PR than science?

Peter Doshi
BMJ 2005 331: 1412. [Extract] [Full Text] [PDF]

Efficacy, tolerability, and upper gastrointestinal safety of celecoxib for treatment of osteoarthritis and rheumatoid arthritis: systematic review of randomised controlled trials

Jonathan J Deeks, Lesley A Smith, and Matthew D Bradley
BMJ 2002 325: 619. [Abstract] [Full Text] [PDF]