Intended for healthcare professionals


Benefits and harms of drug treatments

BMJ 2004; 329 doi: (Published 01 July 2004) Cite this as: BMJ 2004;329:2
  1. Jan P Vandenbroucke, professor of clinical epidemiology (j.p.vandenbroucke{at}
  1. Department of Clinical Epidemiology, Leiden University Medical Centre, 1-C9-P, PO Box 9600, 2300 RC Leiden, Netherlands

    Observational studies and randomised trials should learn from each other

    However international medical science has become, communicating electronically at the speed of light, some fields are still worlds apart. The movement that is subsumed under the banner of evidence based medicine, with its sister movements such as the Cochrane Collaboration or the BMJ's Clinical Evidence, aims to evaluate whether the benefits of treatments that had been hoped for actually exist. This relies almost exclusively on randomised controlled trials, in particular in the study of drug interventions. In a world apart is the field of pharmacoepidemiology, devoting itself mainly to detection and systematic studies of the adverse effects of the very same treatments. Adverse drug effects are often unanticipated and are predominantly investigated by observational studies—for example, by using large databases that link routine prescriptions with the occurrence of unexpected disease.

    The protagonists of these fields barely know each other: they publish in different journals, write and read different books, and work in different departments. They are even suspicious of each other's methods. Adepts of evidence based medicine doubt whether anything reasonable can follow from observational research; they give the impression that they believe that methods of observational research lag far behind. Pharmacoepidemiologists have hitherto made little use of systematic reviews of randomised trials in their much broader job of assessing causation of harm from a variety of pharmacological, clinical, and observational data.

    Yet intellectually, both sides can and should coexist and learn from each other.1 w1 Assessment of small effects of treatment will always need randomised trials as its yardstick. At the other end of the research spectrum of all fields of observational research, research into adverse effects of drugs offers the best chances of being as unbiased as if randomised.2 3 Each medical question should be approached by using the appropriate research tools4—this effectively precludes the idea of a single grading of levels of evidence for all types of research questions.2

    Individual randomised controlled trials often do not suffice to detect adverse effects, especially if the effects are rare and late.5 w2 Systematic reviews of randomised trials have offered little solace so far, even for early and relatively common adverse effects, as adverse events had not been systematically described in similar ways in the individual trials and therefore could not be compared directly for the purpose of a systematic review.6 In addition, most systematic reviews shun observational research. Although there are exceptions,w3-w5 even established adverse effects are often not assessed in systematic reviews, presumably because no randomised evidence exists. However, a balance of benefits and harms is the only reasonable way to evaluate interventions.

    Randomised trials are good at sorting out what works and what doesn't. Even if they show a benefit, they often do not give sufficient insight into harms. A properly balanced review should include a systematic evaluation of adverse effects by the best methods of observational pharmacoepidemiology. For their part, pharmacoepidemiologists should abandon their exclusive preoccupation with one side of the question and one type of methods: they should explore collaboration with people who conduct systematic reviews or randomised trials. Hidden nuggets might be found by scraping the randomised barrel, as happened in a review that looked at untoward events in smaller trials of hormone replacement therapy.7 Also, long term follow up of randomised trials might be a powerful tool for late, but rather common, adverse effects, as with the effects of diethylstilbestrol.8 But, most importantly, better reporting of harms in randomised trials might make systematic reviews of such trials better able to quantify adverse effects. The cry for more and better routine reporting of potential adverse events in randomised trials is an old one: already in 1977 Skegg and Doll proposed to record all medical events in randomised trials.9

    In the field of medical history, the two worlds have already met. The James Lind Library ( traces the history of ideas on the fair evaluation of treatments; it has a place for the history of “casting of lots” as well as for the history of investigating adverse drug reactions. Still, actual practice at the coalface remains different.

    The solution might be in educating people. Just imagine that a select group of people who are properly trained in systematically reviewing randomised trials on drug treatment were to receive training from academic pharmacoepidemiologists, so as to learn the tricks of how to wrestle answers from large observational datasets. They would shed their fears of case-control and retrospective cohort studies and value these studies for what they can do in elucidating the frequency and the causality of adverse effects.10 w6 They would use case reports and systematic reviews of case series.11 At the same time, suppose that some pharmacoepidemiologists started training in Cochrane style systematic reviews, to learn how to mine pooled data from randomised trials on adverse effects. That would be beneficial for their own trade, and in turn might have a salutary influence on the reporting of harms in future randomised trials. To carry this dream to its ultimate conclusion: imagine a world in which they wrote reviews together. That world would marry the best evidence on benefits with the best evidence on harm in a single balanced review to assist doctors and benefit patients.


    • Embedded Image Additional references w1-w6 are on

    • Competing interests None declared.


    1. 1.
    2. 2.
    3. 3.
    4. 4.
    5. 5.
    6. 6.
    7. 7.
    8. 8.
    9. 9.
    10. 10.
    11. 11.
    View Abstract