Assessing the quality of controlled clinical trials

doi:10.1136/bmj.323.7303.42

From trainee to consultant, BMJ Group offers doctors around the world tailored information, special events, learning resources and recruitment services at every step along their career path.

... by doctors, for doctors, for patients
We are open for entries!

Online learning

BMJ Learning

High-quality CME / CPD for doctors and other healthcare professionals. BMJ Learning features hundreds of accredited, peer-reviewed learning modules in text, video, and audio formats. Find out more

Courses and Qualifications

BMJ Masterclasses

BMJ Masterclasses, led by experts, help clinicians to use the latest evidence and recent guidelines in practice and meet their CPD/CME requirements. Find out more

Exam Preparation

The leading provider of online exam preparation, helping over 167,000 healthcare professionals to pass their exams. Find out more

onExamination

Search all BMJ education articles:

From

Limit by

Access to the full text of this article requires a subscription or payment. Please log in or subscribe below.

Peter Jüni, research fellowa,
Douglas G Altman, professor of statistics in medicineb,
Matthias Egger (m.egger@bristol.ac.uk), senior lecturer in epidemiology and public health medicinec

^a Department of Social and Preventive Medicine, University of Bern, Bern, 3012 Switzerland
^b Imperial Cancer Research Fund Medical Statistics Group, Centre for Statistics in Medicine, Institute of Health Sciences, Oxford OX3 7LF
^c Medical Research Council Health Services Research Collaboration, Department of Social Medicine, University of Bristol, Bristol BS8 2PR

Correspondence to: M Egger

This is the first in a series of four articles

The quality of controlled trials is of obvious relevance to systematic reviews. If the “raw material” is flawed then the conclusions of systematic reviews cannot be trusted. Many reviewers formally assess the quality of primary trials by following the recommendations of the Cochrane Collaboration and other experts. 1 2 However, the methodology for both the assessment of quality and its incorporation into systematic reviews and meta-analysis are a matter of ongoing debate.3-5 In this article we discuss the concept of study quality and the methods used to assess quality.

Components of internal and external validity of controlled clinical trials

Internal validity—extent to which systematic error (bias) is minimised in clinical trials

Selection bias: biased allocation to comparison groups
Performance bias: unequal provision of care apart from treatment under evaluation
Detection bias: biased assessment of outcome
Attrition bias: biased occurrence and handling of deviations from protocol and loss to follow up
External validity—extent to which results of trials provide a correct basis for generalisation to other circumstances
Patients: age, sex, severity of disease and risk factors, comorbidity
Treatment regimens: dosage, timing and route of administration, type of treatment within a class of treatments, concomitant treatments
Settings: level of care (primary to tertiary) and experience and specialisation of care provider
Modalities of outcomes: type or definition of outcomes and duration of follow up

Quality is a multidimensional concept, which could relate to the design, conduct, and analysis of a trial, its clinical relevance, or quality of reporting.6 The validity of the findings generated by a study clearly is an important dimension of quality. In the 1950s the social scientist Campbell proposed a useful distinction between internal and external validity (see box below). 7 8 Internal validity implies that the differences observed between groups of patients allocated to different …