Intended for healthcare professionals

Research

Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review

BMJ 2006; 333 doi: https://doi.org/10.1136/bmj.38973.444699.0B (Published 12 October 2006) Cite this as: BMJ 2006;333:782

Has Cochrane really achieved its goals?

Editors

I am somewhat amazed that the BMJ chose to publish this review, given
its small sample size yet broad conclusions. I don't believe it would be
a stretch to suggest that if the results had been in the other direction,
it would be less likely to have been published.

It is true that Cochrane reviews report specific items more
thoroughly than journal based reviews. However, much of this is due to the
insistence of addressing methodological issues which are specious at times
and the fact that Cochrane reviews are not limited by page length. For
example, the issue of reporting allocation concealment, while it makes
sense, does not mean that it was not done (1), nor does it even
consistently demonstrate that it is an important methodological issue to
report.(2)

It is disappointing that the Cochrane library has become an ivory
tower, given that many of the reviews are out of date and methodologically
weak. The Cochrane library was established to be a clinically useful
resource, but is that really true? There are many Cochrane reviews that
would not be published in a paper journal, as they contain zero or just a
few trials. There are far too many Cochrane reviews stating that, although
upwards of 10 trials were found, the reporting is poor and therefore more
research is required before a clinical recommendation can be made. Do you
really think it is useful to say that several trials do not permit an
inference on effectiveness?

In closing, I would say that attacking pharma is an obvious target
and one that amounts to little more than bullying. Pharma has an obvious
conflict in wanting to publish favorable results. Why do the Cochrane
group not go after the agencies claiming to promote health for the
goodness of all, but mismanaging money and misusing evidence, such as the
World Bank or WHO (3,4)- not targets that are so uniformly accepting of
criticism.

1) Devereaux PJ, Choi PT, El-Dika S, Bhandari M, Montori VM,
Schunemann HJ, Garg AX, Busse JW, Heels-Ansdell D, Ghali WA, Manns BJ,
Guyatt GH.
An observational study found that authors of randomized controlled trials
frequently use concealment of randomization and blinding, despite the
failure to report these methods. J Clin Epidemiol. 2004 Dec;57(12):1232-6.

2) Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C,
Lau J.
Correlation of quality measures with estimates of treatment effect in meta
-analyses of randomized controlled trials.
JAMA. 2002 Jun 12;287(22):2973-82.

3) Attaran A, Barnes KI, Bate R, Binka F, d'Alessandro U, Fanello CI,
Garrett L, Mutabingwa TK, Roberts D, Sibley CH, Talisuna A, Van
Geertruyden JP, Watkins WM. The World Bank: false financial and
statistical accounts and medical malpractice in malaria treatment.

4) Attaran A, Barnes KI, Curtis C, d'Alessandro U, Fanello CI,
Galinski MR, Kokwaro G, Looareesuwan S, Makanga M, Mutabingwa TK, Talisuna
A, Trape JF, Watkins WM.
WHO, the Global Fund, and medical malpractice in malaria treatment.
Lancet. 2004 Jan 17;363(9404):237-40
Lancet. 2006 Jul 15;368(9531):247-52.

Competing interests:
None declared

Competing interests: No competing interests

14 October 2006
Marko Tostad
Epidemiologist
Mt. Sinai 10029