Education and debate

Systematic reviews in health care

Assessing the quality of controlled clinical trials

BMJ 2001; 323 doi: 10.1136/bmj.323.7303.42 (Published 7 July 2001)
Cite this as: BMJ 2001;323:42

Access to the full text of this article requires a subscription or payment. Please log in or subscribe below.

  1. Peter Jüni, research fellowa,
  2. Douglas G Altman, professor of statistics in medicineb,
  3. Matthias Egger (m.egger@bristol.ac.uk), senior lecturer in epidemiology and public health medicinec
  1. a Department of Social and Preventive Medicine, University of Bern, Bern, 3012 Switzerland
  2. b Imperial Cancer Research Fund Medical Statistics Group, Centre for Statistics in Medicine, Institute of Health Sciences, Oxford OX3 7LF
  3. c Medical Research Council Health Services Research Collaboration, Department of Social Medicine, University of Bristol, Bristol BS8 2PR
  1. Correspondence to: M Egger

    This is the first in a series of four articles

    The quality of controlled trials is of obvious relevance to systematic reviews. If the “raw material” is flawed then the conclusions of systematic reviews cannot be trusted. Many reviewers formally assess the quality of primary trials by following the recommendations of the Cochrane Collaboration and other experts. 1 2 However, the methodology for both the assessment of quality and its incorporation into systematic reviews and meta-analysis are a matter of ongoing debate.3-5 In this article we discuss the concept of study quality and the methods used to assess quality.

    Components of internal and external validity of controlled clinical trials

    Internal validity—extent to which systematic error (bias) is minimised in clinical trials

    • Selection bias: biased allocation to comparison groups

    • Performance bias: unequal provision of care apart from treatment under evaluation

    • Detection bias: biased assessment of outcome

    • Attrition bias: biased occurrence and handling of deviations from protocol and loss to follow up

    • External validity—extent to which results of trials provide a correct basis for generalisation to other circumstances

    • Patients: age, sex, severity of disease and risk factors, comorbidity

    • Treatment regimens: dosage, timing and route of administration, type of treatment within a class of treatments, concomitant treatments

    • Settings: level of care (primary to tertiary) and experience and specialisation of care provider

    • Modalities of outcomes: type or definition of outcomes and duration of follow up

    Quality is a multidimensional concept, which could relate to the design, conduct, and analysis of a trial, its clinical relevance, or quality of reporting.6 The validity of the findings generated by a study clearly is an important dimension of quality. In the 1950s the social scientist Campbell proposed a useful distinction between internal and external validity (see box below). 7 8 Internal validity implies that the differences observed between groups of patients allocated to different …

    Access to the full text of this article requires a subscription or payment

    Subscribe

    Article access

    Article access for 1 day

    Purchase this article for £20 $30 €32*

    The PDF version can be downloaded as your personal record

    * Prices do not include VAT

    Rapid responses

    THIS WEEK'S POLL

    BMJ most popular