Editorials

Unbiased, relevant, and reliable assessments in health care

Important progress during the past century, but plenty of scope for doing better 

C ausal inferences about the effects of treatments must always depend on best judgments. Because the lives and wellbeing of patients will be influenced for better or worse by the validity of these judgments, however, it is important to be explicit about the logic as well as the empirical evidence on which the judgments are based. This issue of the BMJ is about one important aspect of that logicthe attempt to control bias through randomisation.

There is a growing acceptance that it is logical to try to control biases of various kinds when assessing the effects of treatments. Efforts by clinicians to control biases stretch back for at least three centuries,¹ but only during the past 100 years have these become widespread. In particular, as we approach the end of the 20th century, there are now hundreds of thousands of reports of studies in which efforts have been made to control selection biases, the aim here being to distinguish differences attributable to treatments from differences that reflect the characteristics (known and unknown) of the people who have received treatment.

These studies are known as randomised trials because eligible patients are allocated at random to one of two or more alternative forms of care. This is their sole defining characteristic.² Other measures sometimes used to control biasesfor example, the use of placebos to minimise observer biasesare neither specific to nor necessary features of randomised trials.

Consensus is growing that the results of randomised trials provide the most secure basis for valid causal inferences about the effects of treatments.³ Not everyone subscribes to this view,⁴ however, and there are certainly aspects of the design and interpretation of randomised trials which continue to present real challenges. ^{5 6} The results of randomised trials usually differ from those of studies in which the comparison groups have been assembled in other ways.⁷ Although the most likely explanation for these differences would seem to be uncontrolled biases, other explanations cannot be ruled out.⁸

Two studies stand out in the history of efforts to control selection biases in clinical research. In 1898 a Danish physician, Johannes Fibiger, allocated patients with diphtheria to comparison groups on the basis of day they were admitted to hospital. He gave anti-diphtheria serum to patients admitted on alternate days and compared their progress with that of those admitted on other days. Fibiger's report is remarkable not only because it shows that he was conscious of the need to control selection biases but also because he described his methods and analyses so clearly. ^{9 10}

Whether the basis for allocating patients in an unselected series to comparison groups is alternation or random numbers, failure to adhere strictly to the allocation schedule may result in bias. ^{11 12} Fifty years ago yesterday, the BMJ carried the report of another landmark study in the history of efforts to control selection biasesthe UK Medical Research Council's randomised trial of streptomycin for pulmonary tuberculosis.^13-15 The report is especially important because it describes in detail the precautions taken by the researchers to conceal the allocation schedule from those entering patients into the trial.¹³

Randomised trials conducted over the past half century have helped to bring about a situation in which health care has been credited with three of the seven years of increased life expectancy over that time and an average of five additional years of partial or complete relief from the poor quality of life associated with chronic disease.¹⁶ But we should not be complacent. Systematic reviews of some of the hundreds of thousands of reports of trials published since 1948 are beginning to make painfully clear that, in most of these studies, inadequate steps were taken to control biases, many questions and outcomes of interest to patients were ignored,¹⁷ and insufficient numbers of participants were studied to yield reliable estimates of treatment effects.¹⁸ In brief, a massive amount of research effort, the goodwill of hundreds of thousands of patients, and millions of pounds have been wasted.

Several developments could help to ensure that efforts over the next 50 years will be more effective in yielding unbiased, relevant, and reliable assessments of the effects of health care. Information derived from systematic reviews of past research¹⁹ and from registers of continuing trials²⁰ will help to show where new trials are needed and how best to maximise the quality and relevance of the new information sought. Some of this information is likely to be in the form of qualitative data, and this implies the need for greater cooperation among clinical and social scientists in designing and running trials.²¹

Electronic publication will offer opportunities for improving the quality of research and of research reports²² through open peer review of protocols and reduction of publication bias, and by providing a mechanism through which the results of new studies can be set properly within the context of other relevant studies.²³ Improvements in the infrastructure needed to support trials²⁴ should mean that clinicians and patients faced with uncertainties about the relative merits of treatment options will more often be able to participate in the research needed to resolve these uncertainties.

The greatest potential for improving research may lie in greater public involvement. Partly because of perverse incentives to pursue particular research projects ^{25 26} researchers often seem to design trials to address questions that are of no interest to patients. Greater public involvement could help to reduce this mismatch and ensure that trials are designed to address questions that patients see as relevant. More generally, it will be important to assess whether the public understands and endorses the efforts being made to control biases in assessing the effects of health care. ^{27 28} So far, the research community has made very little effort to involve the public in discussions about this. All in all, there is plenty of scope for building on the undoubted progress made during the past century.

Iain Chalmers, Director. 

UK Cochrane Centre, Oxford OX2 7LG

Acknowledgments

I thank Doug Altman, Mike Bracken, Ray Garry, Peter Gøtzsche, Andrew Herxheimer, Tony Hope, Muir Gray, and Ann Oakley for helpful comments on an earlier draft of this article.

1. Royal College of Physicians of Edinburgh/UK Cochrane Centre. Controlled trials from history. http://www.rcpe.ac.uk/cochrane/
2. Kleijnen J, Gøtzsche P, Kunz RH, Oxman AD, Chalmers I. So what's so special about randomisation? In: Maynard A, Chalmers I, eds. Non-random reflections on health services research. London: BMJ Books, 1997:93-106.
3. Clarke MJ. Ovarian ablation in breast cancer, 1896 to 1998: milestones along hierarchy of evidence from case report to Cochrane review. BMJ 1998; 317: 1246-1248[Free Full Text].
4. Cranberg L. Evaluating new treatments. BMJ 1998; 317: 1261[Free Full Text].
5. Russell I. Evaluating new surgical procedures. BMJ 1995; 311: 1243-1244[Free Full Text].
6. McPherson K, Chalmers I. Incorporating patient preferences into clinical trials. BMJ 1998; 317: 78[Free Full Text].
7. Kunz R, Oxman A. The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ 1998; 317: 1185-1190[Abstract/Free Full Text].
8. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ 1996; 312: 1215-1218[Free Full Text].
9. Fibiger J. Om Serumbehandling af Difteri. Hospitalstidende 1898; 6: 309-325, 337-50.
10. Hróbjartsson A, Gøtzsche PC, Gluud C. The controlled clinical trial turns 100 years: Fibiger's trial of serum treatment of diphtheria. BMJ 1998; 317: 1243-1245[Free Full Text].
11. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273: 408-412[Abstract].
12. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998; 352: 609-613[Medline].
13. Medical Research Council. Streptomycin treatment of pulmonary tuberculosis: a Medical Research Council investigation. BMJ 1948; 2: 769-782.
14. Doll R. Controlled trials: the 1948 watershed. BMJ 1998; 317: 1217-1220[Free Full Text].
15. Yoshioka A. Use of randomisation in the Medical Research Council's clinical trial of streptomycin in pulmonary tuberculosis in the 1940s. BMJ 1998; 317: 1220-1223[Free Full Text].
16. Bunker JP, Frazier HS, Mosteller F. Improving health: measuring effects of medical care. Milbank Quarterly 1994; 72: 225-258.
17. Thornley B, Adams C. Content and quality of 2000 controlled trials in schizophrenia over 50 years. BMJ 1998; 317: 1181-1184[Abstract/Free Full Text].
18. Peto R, Baigent C. Trials: the next 50 years. BMJ 1998; 317: 1170-1171[Free Full Text].
19. Medical Research Council. MRC guidelines for good clinical practice in clinical trials. http://www.mrc.ac.uk/ctg.html
20. Current Science. Current Controlled Trials. http://www.controlled-trials.com.
21. Oakley A. Experimentation and social interventions: a forgotten but important history. BMJ 1998; 317: 1239-1242[Free Full Text].
22. Chalmers I, Altman DG. How can medical journals help prevent poor medical research? Some opportunities presented by electronic publishing. Lancet (in press).
23. Clarke M, Chalmers I. Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? JAMA 1998; 280: 280-282[Abstract/Free Full Text].
24. Farrell B. Efficient management of randomised controlled trials: nature or nurture. BMJ 1998; 317: 1236-1239[Free Full Text].
25. Chalmers I. The perinatal research agenda: whose priorities? Birth 1991; 18: 137-145[Medline].
26. Pieters T. Marketing medicines through randomised controlled trials: the case of interferon. BMJ 1998; 317: 1231-1233[Free Full Text].
27. Edwards SJL, Lilford RJ, Hewison J. The ethics of randomised controlled trials from the perspective of patients, the public, and healthcare professionals. BMJ 1998; 317: 1209-1212[Free Full Text].
28. Featherstone K, Donovan JL. Random allocation or allocation at random? Patients' perspectives of participation in a randomised controlled trial. BMJ 1998; 317: 1177-1180[Abstract/Free Full Text].