The best RCT still trumps the best observational study

A common question in clinical consultations is: “For this person, what are the likely effects of one treatment compared with another?” The central tenet of evidence based medicine is that this task is achieved by using the best evidence combined with consideration of that person's individual needs.1 A further question then arises: “What is the best evidence?” Two recent studies in the New England Journal of Medicine have caused uproar in the research community by finding no difference in estimates of treatment effects between randomised controlled trials and non-randomised trials.

The randomised controlled trial and, especially, systematic reviews of several of these trials are traditionally the gold standards for judging the benefits of treatments, mainly because it is conceptually easier to attribute any observed effect to the treatments being compared. The role of non-randomised (observational) studies in evaluating treatments is contentious: deliberate choice of the treatment for each person implies that observed outcomes may be caused by differences among people being given the two treatments, rather than the treatments alone. Unrecognised confounding factors can always interfere with attempts to correct for identified differences between groups.

These considerations have supported a hierarchy of evidence, with randomised controlled trials and derivatives at the top, controlled …