It’s time to rebuild the evidence base

• Download figure
• Open in new tab
• Download powerpoint

For many of us, the move towards an evidence based approach to medicine has largely been a welcome one. We have learnt to evaluate therapies rigorously and be highly sceptical of expert enthusiasm for them. Perhaps most importantly, we now try to turn routinely to summaries of the evidence rather than rely on single studies. For what we assumed were good reasons, systematic reviews and meta-analyses have become gold standards, whether we are a politician, a physician, or simply a citizen. But is it fool’s gold? In our collective zeal to summarise, we have too often ignored the fact that a vast and growing proportion of those original studies are industry sponsored, which means that they tend to exaggerate benefits and play down harms. Summarising that bias doesn’t make it go away. Medicine’s prized evidence base has become debased.

An international team of researchers from across Europe and North America recently examined 29 meta-analyses published in leading medical journals.1 Those meta-analyses summarised results of more than 500 trials of top selling drugs for conditions such as cancer and heart disease. Almost 70% of the original trials that disclosed a funding source were company sponsored. Similarly, almost 70% of the original trials that disclosed the financial relationships of authors reported ties to drug companies. Yet only two of the 29 meta-analyses reported who funded the original trials, and none reported on the financial ties of authors. These alarming findings were published in JAMA earlier this year, along with the conclusion that key information about potential bias is being left out of influential reviews that guide what doctors do.

Anyone who is in any doubt that study sponsorship is associated with more favourable outcomes needn’t be. As the authors of the JAMA paper make clear, extensive research has shown that “conflicts of interest can influence the results and conclusions” of randomised controlled trials and meta-analyses. A growing body of reliable data indicates that commercial sponsors tend to get the outcomes they desire, raising serious questions about how we are supposed to trust the evidence base.

In the early 1990s researchers examined more than 50 clinical trials that compared popular anti-arthritis drugs and showed that not one of those trials found its sponsor’s drug to be inferior to a comparator.2 By 2003 there were many similar pieces of research, and summaries of that research were pointing to a “systematic bias” across the medical literature, with company funded trials far more likely than more independently funded trials to find favourable results for their products.3 In 2005 the long time editor of the BMJ, Richard Smith, wrote that “the evidence is strong that companies are getting the results they want,” and he wondered whether journals should stop publishing trials that were more about marketing than medical science.4

In 2007 a review of almost 200 trials comparing cholesterol lowering drugs found that company funding greatly influenced a study’s outcome.5 If a study found a positive result for a drug, the study was 20 times more likely to have been funded by that drug’s maker than by the manufacturer of the comparison drug. In 2008 research confirmed that the world was being fundamentally misled about the benefits of widely prescribed antidepressants, showing that many studies with unfavourable results had simply not been published.6 In 2009 researchers demonstrated that sponsored trials of new antidiabetes drugs, and trials run by authors with financial ties to companies, tended to find more favourable results for the sponsor’s product: an extremely worrying finding, given the harms associated with rosiglitazone.7 And now, in 2011, the JAMA paper suggests that the vast majority of meta-analyses fail to report such basic facts as the funding source for the original studies.

“It’s hard to find an area of the healthcare literature that is not contaminated,” says Brett Thombs, assistant professor at McGill University in Canada and one of the authors of the JAMA paper. Whether it is drugs, behavioural therapies, or patient support tools, “when anyone tests something they developed themselves,” says Thombs, “there is a great risk of bias.” He and colleagues recommend several reforms to the way reviews are reported, pushing more disclosure of funding sources and of authors’ financial ties.

But simply disclosing bias doesn’t make it go away either. The aim of any rational healthcare system is surely to generate an evidence base we can trust. The argument that trials without industry funding are not feasible is disingenuous and dangerous. Clearly, not all company sponsored studies are corrupted, and not all publicly funded studies are reliable, but a mountain of data suggests that much current evidence is deeply contaminated by commercial conflicts of interest. Of particular concern are the many sponsored trials that recruit large numbers of people and find very small benefits—where a hidden bias could make a useless and potentially harmful pill seem to be an effective treatment.

Just as the tobacco companies polluted the literature on smoking, and polluting industries try to distort the science of climate change, the pharmaceutical giants and the doctors on their payrolls are poisoning too much of the medical science with overly positive findings. The inevitable rebuilding of that evidence base may not only produce more trustworthy and less debased information but also offer societies the chance to take back the agenda in healthcare research and practice, which has been hijacked so effectively, and with such panache, by the drug industry.