Editorials

The risk of bias from omitted research

Evidence must be independently sought and free of economic interests

The rise of evidence based health care has highlighted the use of ineffective interventions, the risks of uncoordinated research, and the consequences of relying on studies published in prestigious journals while ignoring unpublished ones that have negative findings.^1-5 Systematic reviews of the best evidence are now recognised as fundamental tools in overcoming these problems because they highlight questions that need urgent answers.⁶ But is evidence based health care achieving its goals? Aren't systematic reviews which are based on existing research at risk of amplifying the irrelevant? Should we be more concerned about "bias caused by omitted research" than the well recognised pitfall of publication bias?

The increasing awareness of this danger is leading to efforts to correct this imbalance. One such attempt is the Cochrane Collaboration (an international organisation named after Archie Cochrane, the British epidemiologist), which is committed to preparing, maintaining, and disseminating systematic reviews to map the value of healthcare interventions.⁷ The public and the media are attracted to alternative medicine, while doctors, who often criticise the use of these unproved treatments, use products such as tonics, food supplements, antioxidants, memory enhancing drugs, and vasodilators for which there is no evidence that they are clinically useful. A far reaching cultural campaign is needed in Europe.

Therefore, evidence that is not only of good methodological quality but also generalisable is needed. Too often, evidence from randomised clinical trials is obtained only from selected groups of patients, and children, elderly people, and women are often excluded. ^{7 8} The use of treatments should not be extended by analogy from one group of patients to another without appropriate randomised controlled trials.

Randomised controlled trials are needed to produce generalisable evidence, and general practitioners must be involved, not just researchers. Hospital doctors can play an active part in drafting trial protocols and analysing results. The progress in cardiology achieved by the GISSI trial in Italy as well as by the international studies on infarct survival (ISIS) for the treatment of myocardial infarction should make these trials models in terms of the example they provide of involving doctors and hospitals that are representative of different levels of the healthcare systems. ^{9 10}

To produce evidence we must work independently, free from prejudice and unfettered by the economic interests at play in medicine. It is unfortunate that the industrialised countries, especially in Europe, have delegated the control of drug trials to pharmaceutical companies. We are not suggesting that the industry is wicked, and we acknowledge its role in providing essential drugs, such as antibiotics, antiulcer agents, antipsychotic drugs, and fibrinolytic agents, to name just a few. Nevertheless, delegating this responsibility places clear limitations on research, and these seem to be growing.

For economic reasons, researchers are drawn to areas likely to give the best possible financial return. This leads to a gap between public health needs and the areas on which research actually concentrates. It favours "bias by omitted research." Examples are many. Millions of people suffer from tropical diseases (malaria, leprosy, schistosomiasis), and yet no one designs realistic strategies to tackle them. Who is working to develop drugs for these diseases? The World Health Organization scrapes together a few million dollars for drug development, but the money is insufficient to develop even a single drug.

Many older drugs are marketed on the basis of what is considered poor evidence by today's standards. There are no economic incentives to improve this evidence by conducting new trials. How many antihypertensive drugs on the market have been reliably shown to reduce cardiovascular mortality in addition to lowering blood pressure? Women who have reached the menopause are given hormone replacement therapyoestrogen or progestogen, or bothsimply because indirect evidence suggests that it reduces cardiovascular mortality and fractures. And questionable generalisations are made from this evidence.¹¹ Only long term randomised controlled trials could clarify whether this is so, but about 50 000 women would have to be recruited. Who will organise such trials?

The same economic disincentives hold for comparative trials, which are rarely carried out. For many medical problems there are several classes of pharmacological drugs on the market with different mechanisms of action. How do they compare on efficacy and safety? We shall probably never know because there are clear economic reasons why no one is interested in investigating whether drug A is better than B, C, or D. Clinical equivalence is often used as an excuse not to look for differences. Investment by drug companies in research is welcome but we must urgently set up an independent European fund along the lines of the US National Institutes of Health to finance independent, randomised controlled trials in areas that are relevant to public health.

The European Union fifth framework programme defines strategic priorities for such areas as research and technological development for the period 1998-2002. The initiative is made up of four themed programmes but is meant to finance research to increase knowledge, and for pragmatic reasons it gives priority to research that interests drug companies.¹² The amount of money involved is not enough to cover the costs of independent trials. Investing resources more clearly in public health will not give an immediate return but will increase the amount of evidence on which the practice of medicine should be based.

Silvio Garattini, director. 

Istituto di Ricerche Farmacologiche "Mario Negri," Via Eritrea 62, 20157 Milan, Italy (GARATTINI{at}marionegri.it)

Alessandro Liberati, associate professor. 

Centro Cochrane Italiano, Istituto di Ricerche Farmacologiche "Mario Negri," Via Eritrea 62, 20157, Milan, Italy and Università di Modena e Reggio Emilia, Via Campa 287, Modena, Italy