Intended for healthcare professionals

Research

Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review

BMJ 2006; 333 doi: https://doi.org/10.1136/bmj.38973.444699.0B (Published 12 October 2006) Cite this as: BMJ 2006;333:782

Inherent biases in publication process key confounders

We read with interest the article by Jorgensen and colleagues (1) and
would like to suggest some additional points of consideration with respect
to their conclusions that industry supported reviews are less transparent,
had few reservations about trial method limitations and had more
favourable conclusions.

The following items need address:

1. All 7 industry sponsored reviews were published in peer reviewed
journals with considerable restriction on space
2. All 7 Cochrane reviews were published after the industry sponsored
reviews and in some cases 3 years after, allowing time for further
critical analyses and further data to be incorporated as well as trial
method issues to be determined in the public domain.
The authors have made little attempt of trying to deal with this complex
effect of this time delay difference between the two types of
publications.
3. All clinical trials now are required to be registered - and where
possible trial protocols made available for review so that the previous
artificial lack of transparency due to limitation of publication portals
and space available is removed (2). Such facilities were not available in
the past.
4. Industry supported reviews attract a high calibre of statistical
support sufficient to pass safely through a rigorous peer reviewed process
often with a statistical review as a separate hoop. Certainly the impact
factors of the 7 industry papers are high and would reflect a high degree
of scrutiny. Having participated in both industry led and Cochrane
reviews, we would state that the scrutiny of industry internal processes
is severe and challenging - and analyses are frequently duplicated
separately to validate the findings.
5. We were unable to determine which of the Cochrane reviews had full
access to individual patient level data. Industry-supported meta-analyses
commonly use individual patient level data and are therefore often more
robust. The authors have not informed us of these differences between
studies.

We would therefore suggest that the conclusion of these authors also
merit caution and should perhaps discuss further limitations of their
exercise..

Finally however - our recommendations are that ideally the two
processes of Cochrane reviews and industry supported analyses should merge
- using independent statisticians and researchers to undertake their own
analyses but with data provided by industry without industry funding. Such
a collaborative process now somewhat exists under the health technology
appraisals undertaken by NICE (3) and it's reviews are transparent with
well documented methods. More importantly - recommendations are open to
appraisal in the public domain before being finalised - providing further
chance to address or remove biases.

Despite this - there will always be limitations in meta-analyses
which by design attempt to compare separate studies undertaken at
different times, in different populations, under different treatment
circumstances (given that the standard control treatment changes with
time). Thus such analyses always merit caution and cannot replace well-
conducted large randomised trials (4).

1. Jorgensen AW, Hilden J, Gotzsche PC. Cochrane reviews compared
with industry supported meta-analyses and other meta-analyses of the same
drugs: systematic review. BMJ. 2006 Oct 14;333(7572):782.
2. Tonks A. Registering clinical trials. BMJ 1999;319:1565-1568
3. www.nice.org.uk
4. Flather MD, Farkouh ME, Pogue JM, Yusuf S. Strengths and limitations of
meta-analysis: larger studies may be more reliable. Control Clin Trials.
1997 Dec;18(6):568-79; discussion 661-6.

Competing interests:
All authors undertake research supported by industry and non-industry related organisations in the UK

Competing interests: No competing interests

28 October 2006
Kostas Zacharias
Cardiology Fellow
Geetha Balakrishnan, Ameet Bakhai
Barnet General Hospital, Wellhouse Lane, Barnet EN5 3DJ