BMJ  2004;329:2-3 (3 July), doi:10.1136/bmj.329.7456.2

Editorial

Benefits and harms of drug treatments

Observational studies and randomised trials should learn from each other

However international medical science has become, communicating electronically at the speed of light, some fields are still worlds apart. The movement that is subsumed under the banner of evidence based medicine, with its sister movements such as the Cochrane Collaboration or the BMJ's Clinical Evidence, aims to evaluate whether the benefits of treatments that had been hoped for actually exist. This relies almost exclusively on randomised controlled trials, in particular in the study of drug interventions. In a world apart is the field of pharmacoepidemiology, devoting itself mainly to detection and systematic studies of the adverse effects of the very same treatments. Adverse drug effects are often unanticipated and are predominantly investigated by observational studies—for example, by using large databases that link routine prescriptions with the occurrence of unexpected disease.

The protagonists of these fields barely know each other: they publish in different journals, write and read different books, and work in different departments. They are even suspicious of each other's methods. Adepts of evidence based medicine doubt whether anything reasonable can follow from observational research; they give the impression that they believe that methods of observational research lag far behind. Pharmacoepidemiologists have hitherto made little use of systematic reviews of randomised trials in their much broader job of assessing causation of harm from a variety of pharmacological, clinical, and observational data.

Yet intellectually, both sides can and should coexist and learn from each other.^{1 w1} Assessment of small effects of treatment will always need randomised trials as its yardstick. At the other end of the research spectrum of all fields of observational research, research into adverse effects of drugs offers the best chances of being as unbiased as if randomised.^{2 3} Each medical question should be approached by using the appropriate research tools⁴—this effectively precludes the idea of a single grading of levels of evidence for all types of research questions.²

Individual randomised controlled trials often do not suffice to detect adverse effects, especially if the effects are rare and late.^{5 w2} Systematic reviews of randomised trials have offered little solace so far, even for early and relatively common adverse effects, as adverse events had not been systematically described in similar ways in the individual trials and therefore could not be compared directly for the purpose of a systematic review.⁶ In addition, most systematic reviews shun observational research. Although there are exceptions,^w3-w5 even established adverse effects are often not assessed in systematic reviews, presumably because no randomised evidence exists. However, a balance of benefits and harms is the only reasonable way to evaluate interventions.

Randomised trials are good at sorting out what works and what doesn't. Even if they show a benefit, they often do not give sufficient insight into harms. A properly balanced review should include a systematic evaluation of adverse effects by the best methods of observational pharmacoepidemiology. For their part, pharmacoepidemiologists should abandon their exclusive preoccupation with one side of the question and one type of methods: they should explore collaboration with people who conduct systematic reviews or randomised trials. Hidden nuggets might be found by scraping the randomised barrel, as happened in a review that looked at untoward events in smaller trials of hormone replacement therapy.⁷ Also, long term follow up of randomised trials might be a powerful tool for late, but rather common, adverse effects, as with the effects of diethylstilbestrol.⁸ But, most importantly, better reporting of harms in randomised trials might make systematic reviews of such trials better able to quantify adverse effects. The cry for more and better routine reporting of potential adverse events in randomised trials is an old one: already in 1977 Skegg and Doll proposed to record all medical events in randomised trials.⁹

In the field of medical history, the two worlds have already met. The James Lind Library (www.jameslindlibrary.org/) traces the history of ideas on the fair evaluation of treatments; it has a place for the history of "casting of lots" as well as for the history of investigating adverse drug reactions. Still, actual practice at the coalface remains different.

The solution might be in educating people. Just imagine that a select group of people who are properly trained in systematically reviewing randomised trials on drug treatment were to receive training from academic pharmacoepidemiologists, so as to learn the tricks of how to wrestle answers from large observational datasets. They would shed their fears of case-control and retrospective cohort studies and value these studies for what they can do in elucidating the frequency and the causality of adverse effects.^{10 w6} They would use case reports and systematic reviews of case series.¹¹ At the same time, suppose that some pharmacoepidemiologists started training in Cochrane style systematic reviews, to learn how to mine pooled data from randomised trials on adverse effects. That would be beneficial for their own trade, and in turn might have a salutary influence on the reporting of harms in future randomised trials. To carry this dream to its ultimate conclusion: imagine a world in which they wrote reviews together. That world would marry the best evidence on benefits with the best evidence on harm in a single balanced review to assist doctors and benefit patients.

Department of Clinical Epidemiology, Leiden University Medical Centre, 1-C9-P, PO Box 9600, 2300 RC Leiden, Netherlands (j.p.vandenbroucke{at}lumc.nl)

---

Additional references w1-w6 are on bmj.com

Competing interests: None declared.

References

1. Cuervo LG, Clarke M. Balancing benefits and harms in health care. BMJ 2003;327: 65-6.[Free Full Text]
2. Glasziou P, Vandenbroucke J, Chalmers I. Assessing the quality of research. BMJ 2004;328: 39-41.[Free Full Text]
3. Vandenbroucke JP. When are observational studies as credible as randomised trials? Lancet 2004;363: 1728-31.[CrossRef][ISI][Medline]
4. Sackett DL, Wennberg JE. Choosing the best research design for each question. BMJ 1997;315: 1636.[Free Full Text]
5. Jick H. The discovery of drug-induced illness. N Engl J Med 1977;296: 481-5.[Abstract]
6. Loke YK, Derry S. Reporting of adverse drug reactions in randomised controlled trials - a systematic survey. BMC Clin Pharmacol 2001;1: 3.[CrossRef][Medline]
7. McPherson K, Hemminki E. Synthesising licensing data to assess drug safety. BMJ 2004;328: 518-20.[Free Full Text]
8. Grant A, Chalmers I. Some research strategies for investigating aetiology and assessing the effects of clinical practice. In: Macdonald RR, ed. Scientific basis of obstetrics and gynaecology. 3rd ed. London: Churchill Livingstone, 1985: 49-84.
9. Skegg DCG, Doll R. The case for recording events in clinical trials. BMJ 1977;ii: 1523-4.
10. Vandenbroucke JP. Observational research and evidence-based medicine: What should we teach young physicians? J Clin Epidemiol 1998;51: 467-72.[CrossRef][ISI][Medline]
11. Loke YK, Derry S, Aronson JK. A comparison of three different sources of data in assessing the frequencies of adverse reactions to amiodarone. Br J Clin Pharmacol 2004;57: 616-21.[CrossRef][ISI][Medline]

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Risk of myocardial infarction in patients taking cyclo-oxygenase-2 inhibitors or conventional non-steroidal anti-inflammatory drugs: population based nested case-control analysis

Julia Hippisley-Cox and Carol Coupland
BMJ 2005 330: 1366. [Abstract] [Full Text] [PDF]

Assessing the benefit-harm balance at the bedside

Yoon Kong Loke
BMJ 2004 329: 7-8. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• Grypdonck, M. H. F. (2006). Qualitative health research in the era of evidence-based practice.. Qual Health Res 16: 1371-1385 [Abstract]  
• Atthobari, J., Brantsma, A. H., Gansevoort, R. T., Visser, S. T., Asselbergs, F. W., van Gilst, W. H., de Jong, P. E., de Jong-van den Berg, L. T. W., on behalf of PREVEND study group, (2006). The effect of statins on urinary albumin excretion and glomerular filtration rate: results from both a randomized clinical trial and an observational cohort study. Nephrol Dial Transplant 21: 3106-3114 [Abstract] [Full text]  
• Papanikolaou, P. N., Christidi, G. D., Ioannidis, J. P.A. (2006). Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies. CMAJ 174: 635-641 [Abstract] [Full text]  
• Chou, R., Helfand, M. (2005). Challenges in Systematic Reviews That Assess Treatment Harms. ANN INTERN MED 142: 1090-1099 [Abstract] [Full text]  
• Hippisley-Cox, J., Coupland, C. (2005). Risk of myocardial infarction in patients taking cyclo-oxygenase-2 inhibitors or conventional non-steroidal anti-inflammatory drugs: population based nested case-control analysis. BMJ 330: 1366- [Abstract] [Full text]  
• Loke, Y. K. (2004). Assessing the benefit-harm balance at the bedside. BMJ 329: 7-8 [Full text]  

Rapid Responses:

Read all Rapid Responses

Postmarketing surveillance is present in the UK, but under-used

John S Watts
bmj.com, 2 Jul 2004 [Full text]

Epidemiologists’ long-term underestimation of harm from hormones

Ellen C G Grant
bmj.com, 3 Jul 2004 [Full text]

"Bangs for backfires" - a useful measure of benefits and harms? E.g. Rosuvastatin.

David R Purdy
bmj.com, 5 Jul 2004 [Full text]

Randomized controlled trials: the hijacking of basic sciences by mathematical logic

Vinod K Gupta, M.D.
bmj.com, 6 Jul 2004 [Full text]

Research and practice - two sides of same coin

Arun S Nanivadekar
bmj.com, 7 Jul 2004 [Full text]

Re: Randomized controlled trials: the hijacking of basic sciences by mathematical logic

Ellen C G Grant
bmj.com, 8 Jul 2004 [Full text]

Re: Re: Randomized controlled trials: the mythical hypomagnesaemia -- intrinsic noradrenergic activation -- cortical spreading depression nexus in migraine

Vinod K Gupta, MD
bmj.com, 8 Jul 2004 [Full text]

Re: Re: Re: Randomized controlled trials: the mythical hypomagnesaemia -- intrinsic noradrenergic activation -- cortical spreading depression nexus in migraine

Ellen C G Grant
bmj.com, 9 Jul 2004 [Full text]