The Cochrane Collaboration’s tool for assessing risk of bias in randomised trialsBMJ 2011; 343 doi: https://doi.org/10.1136/bmj.d5928 (Published 18 October 2011) Cite this as: BMJ 2011;343:d5928
All rapid responses
I read with interest the article on the Cochrane Collaboration's tool
for assessing risk of bias by Higgins et al. I am an academic foundation
year two doctor working with a Cochrane review group based at the
University of York. My role within the team has been to help update
Cochrane reviews, adding new studies identified since the review was last
published, and to make sure the review meets the current Cochrane
Collaboration standards. One of the main aspects of this role was to
complete the new risk of bias assessment for all the studies in the review
update comprising those included before the development of the new tool.
The following is a brief summary of my experiences of assessing risk of
bias using the Cochrane Collaboration tool.
Initially, the task of updating Cochrane reviews was a daunting one
and I had a lot to learn. I found the Cochrane Handbook1 very helpful and
was well supported by the excellent team at the editorial base. For me,
the main positive aspect of the risk of bias tool is its ease of use and
transparency. The Cochrane Handbook1 gives very clear instructions on how
to use the tool and within RevMan (Cochrane's software package) the risk
of bias tables are generated and summary figures produced automatically.
The process of completing the risk of bias tables is also straightforward,
as every decision made by the review authors must be supported by a quote
from the study text. This requirement makes the decision making process
transparent and allows the reader of the review to see how the judgement
of bias was supported. Finally, the summary diagrams give a clear overview
of, not only each individual study's risk of bias, but of the review as a
I feel I have learnt a great deal in my short time working with the
review group. I quickly learnt about the different types of bias, why they
are important and how to detect them within the included studies of a
Cochrane review. I have acquired a valuable skill, now being able to
thoroughly appraise studies and assess their potential biases. I have a
better understanding of how studies should be designed and importantly
what information authors should report in the study publication to enable
readers to detect any potential biases and be able to form a judgement on
the evidence presented. I feel these skills are essential for all doctors
in order to assess the medical literature upon which their clinical
decision making is based.
1. Higgins JPT, Green S, eds. Cochrane handbook for systematic
reviews of interventions. Wiley, 2008.
Competing interests: No competing interests
To the Editor: Randomised controlled trials (RCTs) are considered to
be the gold standard of clinical research and the building blocks of most
systematic reviews and practice guidelines. Yet, the recently published
recommended criteria for assessing risk of bias in RCTs by the Cochrane
Collaboration are almost exclusively concerned with internal
validity.<1> Although the recommended standards are both timely and
welcomed, the perennial problem with poor uptake of clinical practice
guidelines, especially in primary care settings, is an on-going neglect of
the external validity or applicability in the design and conduct of
randomized controlled trials (RCTs). Some of the frequently identified
threats to external validity include narrow inclusion criteria, use of run
-in periods, and length of follow-up or clinical relevance of the primary
end-points used. <2,3,4>
Although, as stated by the authors, applicability is less relevant
without internal validity, the opposite is equally true. It is
disappointing to see that the external validity, once gain, was placed on
the back burner. Internal and external validity are equally important, and
both ought to be maximized in order to generate the evidence that is both
valid and applicable. This is not a zero-sum game.
1. Higgins JPT, Altman DG, Peter C G?tzsche PC, et al. The Cochrane
Collaboration's tool for assessing risk of bias in randomised trials. BMJ
2. Fortin M, Dionne J, Pinho G, Gignac J, Almirall J, Lapointe L.
Randomized controlled trials: do they have external validity for patients
with multiple comorbidities? Ann Fam Med 4 104-8. 2006.
3. Rothwell PM. External validity of randomised controlled trials:
"To whom do the results of this trial apply?" Lancet 365 82-93. 2005.
4. Glasgow RE, Lichtenstein E, Marcus AC. Why don't we see more
translation of health promotion research to practice? Rethinking the
efficacy-to-effectiveness transition. Am J Public Health 93, 1261-1267.
Competing interests: No competing interests