Cochrane reviews and other meta-analyses, authors’ reply
Tostad, Deeks and Zacharias are concerned about the impact of space
restrictions on our findings. Firstly, we believe space restrictions
should not be an excuse for omitting important details on the methods
used, as it is the authors who decide what to report within any given
space, and as many journals allow additional material on the web.
Secondly, our research reflects what is available to the readers, and not
what could have been available, and it is therefore valid from a pragmatic
perspective. If relevant details are not reported, e.g. methods used to
ensure adequate allocation concealment and blinding, the readers may be
unable to make their own assessments and conclusions, which may be
different from those of the authors. Thirdly, we found a number of other
interesting differences between Cochrane reviews and other meta-analyses
than those related to methods.
Senn comments on the validated scale we used for assessing
methodological quality. We agree that there will always be problems with
using scales, and this is precisely why we also looked at individual items
that are known to be important for reducing bias in trials and in reviews.
Inclusion of the additional items Senn suggests would not have changed our
findings.
Deeks mentions that reservations were made in his industry supported
review. That is correct, but the reservations were made in the body of the
Discussion. There were no such reservations in the abstract or in the
conclusion, neither in the short, nor in the long, webbased version of the
review which was the one we assessed (1). We evaluated the abstract and
the conclusion for all the reviews when we judged whether the conclusions
were without reservations and believe this is most relevant thing to do,
as most people read only the abstract.
Zacharias mistakenly writes that all 7 Cochrane reviews were
published after the industry supported reviews. We included 8 pairs and
not 7, and took great care to obtain comparable pairs. Our criterion was
that the time span from the publication of a paper-based review and the
date of the most recent substantive amendment of a Cochrane review (since
these reviews are regularly updated) should not differ more than two
years. As we have reported, the matching was very succesful. The median
publication year was 2000 both for the industry supported reviews and for
the matched Cochrane reviews, and the median difference in number of
included trials was zero (2).
We agree with Tostad and Coyne that some Cochrane reviews are not of
good quality, and we gave examples of this (2). We urge readers who find
problems with Cochrane reviews to submit a comment to be published as part
of the review. This is very easy to do. Use “Add/View Feedback” in the
Index that appears to the left of each review. Such feedback is most
welcome as we constantly try to improve the quality and relevance of our
reviews.
1. Deeks JJ, Smith LA, Bradley MD. Efficacy, tolerability, and upper
gastrointestinal safety of celecoxib for treatment of osteoarthritis and
rheumatoid arthritis: systematic review of randomised controlled trials.
BMJ 2002;325:619.
2. Jørgensen AW, Hilden J, Gøtzsche PC. Cochrane reviews compared
with industry supported meta-analyses and other meta-analyses of the same
drugs: systematic review. doi:10.1136/bmj.38973.444699.0B, Published 6
October 2006.
Competing interests:
AWJ and PCG are affiliated with the Nordic Cochrane
Centre. The views expressed in this letter represent those of the authors
and are not necessarily the views or the official policy of the Cochrane Collaboration.
Competing interests:
No competing interests
05 November 2006
Anders W Jørgensen
Physician
Jørgen Hilden, Peter C Gøtzsche
Nordic Cochrane Centre, Rigshospitalet, DK-2100 Copenhagen Ø, Denmark
Rapid Response:
Cochrane reviews and other meta-analyses, authors’ reply
Tostad, Deeks and Zacharias are concerned about the impact of space
restrictions on our findings. Firstly, we believe space restrictions
should not be an excuse for omitting important details on the methods
used, as it is the authors who decide what to report within any given
space, and as many journals allow additional material on the web.
Secondly, our research reflects what is available to the readers, and not
what could have been available, and it is therefore valid from a pragmatic
perspective. If relevant details are not reported, e.g. methods used to
ensure adequate allocation concealment and blinding, the readers may be
unable to make their own assessments and conclusions, which may be
different from those of the authors. Thirdly, we found a number of other
interesting differences between Cochrane reviews and other meta-analyses
than those related to methods.
Senn comments on the validated scale we used for assessing
methodological quality. We agree that there will always be problems with
using scales, and this is precisely why we also looked at individual items
that are known to be important for reducing bias in trials and in reviews.
Inclusion of the additional items Senn suggests would not have changed our
findings.
Deeks mentions that reservations were made in his industry supported
review. That is correct, but the reservations were made in the body of the
Discussion. There were no such reservations in the abstract or in the
conclusion, neither in the short, nor in the long, webbased version of the
review which was the one we assessed (1). We evaluated the abstract and
the conclusion for all the reviews when we judged whether the conclusions
were without reservations and believe this is most relevant thing to do,
as most people read only the abstract.
Zacharias mistakenly writes that all 7 Cochrane reviews were
published after the industry supported reviews. We included 8 pairs and
not 7, and took great care to obtain comparable pairs. Our criterion was
that the time span from the publication of a paper-based review and the
date of the most recent substantive amendment of a Cochrane review (since
these reviews are regularly updated) should not differ more than two
years. As we have reported, the matching was very succesful. The median
publication year was 2000 both for the industry supported reviews and for
the matched Cochrane reviews, and the median difference in number of
included trials was zero (2).
We agree with Tostad and Coyne that some Cochrane reviews are not of
good quality, and we gave examples of this (2). We urge readers who find
problems with Cochrane reviews to submit a comment to be published as part
of the review. This is very easy to do. Use “Add/View Feedback” in the
Index that appears to the left of each review. Such feedback is most
welcome as we constantly try to improve the quality and relevance of our
reviews.
1. Deeks JJ, Smith LA, Bradley MD. Efficacy, tolerability, and upper
gastrointestinal safety of celecoxib for treatment of osteoarthritis and
rheumatoid arthritis: systematic review of randomised controlled trials.
BMJ 2002;325:619.
2. Jørgensen AW, Hilden J, Gøtzsche PC. Cochrane reviews compared
with industry supported meta-analyses and other meta-analyses of the same
drugs: systematic review. doi:10.1136/bmj.38973.444699.0B, Published 6
October 2006.
Competing interests:
AWJ and PCG are affiliated with the Nordic Cochrane
Centre. The views expressed in this letter represent those of the authors
and are not necessarily the views or the official policy of the Cochrane Collaboration.
Competing interests: No competing interests