Education And Debate Methods in health services research

Interpreting the evidence: choosing between randomised and non-randomised studies

BMJ 1999; 319 doi: https://doi.org/10.1136/bmj.319.7205.312 (Published 31 July 1999) Cite this as: BMJ 1999;319:312
  1. Martin McKee, professor of European public health (McKeem.mckee@lshtm.ac.uk)a,
  2. Annie Britton, research fellowa,
  3. Nick Black, professor of health services researcha,
  4. Klim McPherson, professor of public health epidemiologya,
  5. Colin Sanderson, senior lecturer in health services researcha,
  6. Chris Bain, reader in social and preventive medicineb
  1. a London School of Hygiene and Tropical Medicine, London WC1E 7HT
  2. b University of Queensland Medical School, Brisbane 4006, Australia
  1. Correspondence to: Martin

    This is the first of four articles

    Evaluations of healthcare interventions can either randomise subjects to comparison groups, or not. In both designs there are potential threats to validity, which can be external (the extent to which they are generalisable to all potential recipients) or internal (whether differences in observed effects can be attributed to differences in the intervention). Randomisation should ensure that comparison groups of sufficient size differ only in their exposure to the intervention concerned. However, some investigators have argued that randomised controlled trials (RCTs) tend to exclude, consciously or otherwise, some types of patient to whom results will subsequently be applied Furthermore, in unblinded trials the outcome of treatment may be influenced by practitioners' and patients' preferences for one or other intervention. Though non-randomised studies are less selective in terms of recruitment, they are subject to selection bias in allocation if treatment is related to initial prognosis.

    Summary points

    Treatment effects obtained from randomised and non-randomised studies may differ, but one method does not give a consistently greater effect than the other

    Treatment effects measured in each type of study best approximate when the exclusion criteria are the same and where potential prognostic factors are well understood and controlled for in the non-randomised studies

    Subjects excluded from randomised controlled trials tend to have a worse prognosis than those included, and this limits generalisability

    Subjects participating in randomised controlled trials evaluating treatment of existing conditions tend to be less affluent, educated, and healthy than those who do not; the opposite is true for trials of preventive interventions

    View this table:

    Threats to validity of evaluative research and possible solutions

    These issues have led to extensive debate, although empirical evidence is limited. This paper is a brief summary of a more detailed review1 of the impact of these potential threats.

    Nature of the evidence

    The review focused …

    View Full Text

    Sign in

    Log in through your institution

    Free trial

    Register for a free trial to thebmj.com to receive unlimited access to all content on thebmj.com for 14 days.
    Sign up for a free trial

    Subscribe