Intended for healthcare professionals

Research

Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review

BMJ 2006; 333 doi: https://doi.org/10.1136/bmj.38973.444699.0B (Published 12 October 2006) Cite this as: BMJ 2006;333:782

Cochrane bias

Once again the British Medical Journal treats its readers to the
unedifying spectacle of Cochrane Collaboration (CC) members patting
themselves on the back in public. Using the Oxman and Guyatt (O&G) index,
which is the CC's favourite quality assessment tool, was devised by
researchers at McMaster and 'validated' by asking colleagues at McMaster
if they thought it was a good idea, some CC members 'evaluate' CC meta-
analyses and find that they do rather well. What a surprise.

Amongst important matters that the O&G index does not cover is 1. Was
double counting avoided? 2. Were values used genuine or imputed? 3. If
trials were of different design (e.g. parallel, cross-over or cluster-
randomised) were they combined using appropriate methods?

Given that the CC continue to insist on the O&G index as an
appropriate way of judging the adequacy of meta-analysis, it is not
surprising that they continue to produce analyses that fail to satisfy
these three important criteria. Readers who want to try their hand at
spotting these errors might like to look at the paper by Brockelbank et al
in this journal and ask themselves if the O&G index helps them find them.

Personnally I would like to see the Cochrane Collaboration (which is
generally a force for good) spend more time thinking about analysis and
less time boasting in public.

Reference

1. Brocklebank D, Wright J, Cates C (2001) Systematic review of
clinical effectiveness of pressurised metered dose inhalers versus other
hand held inhaler devices for delivering corticosteroids in asthma.
British Medical Journal 323:896-902

Competing interests:
The author consults for the pharmaceutical industry and is an academic whose career is furthered by publishing

Competing interests: No competing interests

16 October 2006
Stephen J Senn
Professor of Statistics
Department of Statistics, University of Glasgow, Glasgow, G12 9LL