BMJ 1998;317:1170-1171 ( 31 October )

Editorials

Trials: the next 50 years

Large scale randomised evidence of moderate benefits 

Over the past half century there has been a vast proliferation first of randomised trials and now of meta-analyses, both of which (if appropriately analysed) can avoid bias. But to get medically reliable answers to previously unanswered questions about life or death treatment decisions it isn't enough just to avoid bias. We must also ensure that we are not seriously misled by the play of chance, and often the only way to do this reliably is to get appropriate analyses of really large scale randomised evidence.¹

At present, many wrong, or at least unreliable, therapeutic answers are being generated by non-randomised "outcomes research," by small randomised studies, by small meta-analyses, and by statistically inappropriate analyses. Moreover, even when large scale randomised evidence is available, wrong conclusions can be drawn from unduly selective emphasis on particular trials or subgroupsand such "selection biases" can cause even greater errors when there is only a limited amount of evidence to review.

Over the past 50 years randomisation has already delivered reliable answers to some important questions and it offers the promise of reliable answers to many more. For that promise to be properly realised over the next 50 years, however, medical research needs to find practicable ways of greatly increasing the size of randomised studies; otherwise moderate but worthwhile benefits will continue to be missed. One important step towards larger size is the recent emphasis on meta-analyses: ^{2 3} when many different trials have all addressed similar therapeutic questions a synthesis of all of their results not only avoids selective biases but also helps avoid random error.

But it often happens that there are no really large trials and that even a meta-analysis of all the trials in the world isn't big enough to give statistically reliable answers about major outcomes. The key question then is how, in practice, is it possible to randomise a really large number of patients? For if one is trying to decide how millions of future patients should be treated it may often be appropriate to randomise at least many thousandsas is now becoming possible in breast and intestinal canceror even tens of thousands, as has occasionally been possible in stroke and heart disease.

Generally the only practicable way to achieve this is to design trials that are extremely simple and flexible: simplify the entry criteria by use of the "uncertainty principle" (see box), simplify the treatments, and simplify enormously the data requirements. Using the uncertainty principle should allow the process of providing information and gaining consent to become much closer to what is appropriate in normal medical practice. Collecting less information may mean bigger numbers and hence better science: many trials still collect ten or a hundred times too much information per patient, often at the behest of study sponsors or their committees. Requirements for large amounts of defensive documentation imposed on trials by well intentioned guidelines on good clinical practice (or good research practice) or excessive audits may, paradoxically, substantially reduce the reliability with which therapeutic questions are answered, if their indirect effect is to make randomised trials smaller or even to prevent them starting.

The uncertainty principle A patient can be entered if, and only if, the responsible clinician is substantially uncertain which of the trial treatments would be most appropriate for that particular patient. A patient should not be entered if the responsible clinician or the patient are for any medical or non-medical reasons reasonably certain that one of the treatments that might be allocated would be inappropriate for this particular individual (in comparison with either no treatment or some other treatment that could be offered to the patient in or outside the trial).

To argue the need for some large, simple randomised trials is not, of course, to argue that all other trials are useless: indeed, many small (or complex) trials will continue to be needed for certain purposes, as will many other types of clinical research. But for many important questions about practicable therapeutic improvements in controlling the common causes of death or serious disability there is no reliable alternative to large scale randomised evidence.

The reason for this is simple: when it comes to major outcomes it is generally unrealistic to hope for large therapeutic effects. Moreover, if a particular treatment did produce a really large effect on survival then we might well be able to recognise this reliably without any randomised trials. The efficacy of penicillin, for example, was so great that it was recognised before the introduction of randomisation. Likewise, the main hazards of tobacco are so great that they were recognised without randomisation. Hence, if substantial uncertainty remains about the effects of some particular treatment on survival then these effects are likely to be small or only moderate. For example, it might be reasonable to hope that a new treatment for acute stroke or acute myocardial infarction could reduce recurrent stroke or death in hospital from 10% to 9% or 8% (as aspirin does, ^{4 5} preventing 10 000 or 20 000 deaths per million treated), but not to hope that it could halve in-hospital mortality. Many lives could, however, be saved by moderate reductions in the common causes of deathand if, eventually, several moderate benefits are reliably demonstrated their combined effects may be substantial.⁵

Thus, those who sponsor, perform, and regulate therapeutic research need to find ways of making trials much simpler and much larger. Otherwise the next 50 years of randomised evidence will not fulfil the promise of 50 years ago, when a properly⁶ randomised clinical trial was first published, ^{6 7} transforming medical research by its method of generating unbiased answers to many therapeutic questions.

Richard Peto, Professor of medical statistics and epidemiology. 
Colin Baigent, Medical Research Council scientist. 

Clinical Trial Service Unit and Epidemiological Studies Unit (CTSU), Radcliffe Infirmary, Oxford OX2 6HE

---

1. Collins R, Peto R, Gray R, Parish S. Large-scale randomised evidence: trials and overviews. In: Maynard A, Chalmers I, eds. Non-random reflections on health services research. London: BMJ Books, 1998:197-230.
2. The Cochrane Library. Cochrane Collaboration; Issue 3. Oxford: Update Software , 1998.
3. Clarke M. Ovarian ablation in breast cancer, 1896 to 1998: milestones along hierarchy of evidence from case report to Cochrane Review. BMJ 1998; 317: 1246-1248[Free Full Text].
4. CAST (Chinese Acute Stroke Trial) Collaborative Group. CAST: randomised placebo-controlled trial of early aspirin use in 20,000 patients with acute ischaemic stroke. Lancet 1997; 349: 1641-1649[Medline].
5. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction. Lancet 1988; ii: 349-360.
6. Doll R. Controlled trials: the 1948 watershed. BMJ 1998; 317: 1217-1220[Free Full Text].
7. MRC Streptomycin in Tuberculosis Trials Committee. Streptomycin treatment for pulmonary tuberculosis. BMJ 1948; ii: 769-782.

---

© BMJ 1998
CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Related Articles

Are experimental treatments for cancer in children superior to established treatments? Observational study of randomised controlled trials by the Children's Oncology Group

Ambuj Kumar, Heloisa Soares, Robert Wells, Mike Clarke, Iztok Hozo, Archie Bleyer, Gregory Reaman, Iain Chalmers, and Benjamin Djulbegovic
BMJ 2005 331: 1295. [Abstract] [Full Text] [PDF]

Clinical trials

David Barer, José Antonio Sacristán, and Inès Galende
BMJ 1999 318: 1138. [Extract] [Full Text]

Patients who are eligible but not randomised should be included as additional comparative arm in study

L Peter Fielding, Roger Grace, and Rosemary Hittinger
BMJ 1999 318: 874. [Extract] [Full Text]

This article has been cited by other articles:

• Djulbegovic, B., Kumar, A., Soares, H. P., Hozo, I., Bepler, G., Clarke, M., Bennett, C. L. (2008). Treatment Success in Cancer: New Cancer Treatment Successes Identified in Phase 3 Randomized Controlled Trials Conducted by the National Cancer Institute-Sponsored Cooperative Oncology Groups, 1955 to 2006. Arch Intern Med 168: 632-642 [Abstract] [Full text]  
• Phillips, S. J., Dai, D., Mitnitski, A., Gubitz, G. J., Johnston, K. C., Koroshetz, W. J., Furie, K. L., Black, S., Heiselman, D. E., on behalf of the GAIN Americas Investigators, (2007). Clinical Diagnosis of Lacunar Stroke in the First 6 Hours After Symptom Onset: Analysis of Data From the Glycine Antagonist In Neuroprotection (GAIN) Americas Trial. Stroke 38: 2706-2711 [Abstract] [Full text]  
• Fueglistaler, P., Adamina, M., Guller, U. (2007). Non-Inferiority Trials in Surgical Oncology. Ann. Surg. Oncol. 14: 1532-1539 [Abstract] [Full text]  
• Williams, G. W (2006). The other side of clinical trial monitoring; assuring data quality and procedural adherence. Clin Trials 3: 530-537 [Abstract]  
• Gluud, L. L. (2006). Bias in Clinical Intervention Research. Am J Epidemiol 163: 493-501 [Abstract] [Full text]  
• Kumar, A., Soares, H., Wells, R., Clarke, M., Hozo, I., Bleyer, A., Reaman, G., Chalmers, I., Djulbegovic, B. (2005). Are experimental treatments for cancer in children superior to established treatments? Observational study of randomised controlled trials by the Children's Oncology Group. BMJ 331: 1295- [Abstract] [Full text]  
• March, J. S., Silva, S. G., Compton, S., Shapiro, M., Califf, R., Krishnan, R. (2005). The Case for Practical Clinical Trials in Psychiatry. Am. J. Psychiatry 162: 836-846 [Abstract] [Full text]  
• Soares, H. P., Kumar, A., Daniels, S., Swann, S., Cantor, A., Hozo, I., Clark, M., Serdarevic, F., Gwede, C., Trotti, A., Djulbegovic, B. (2005). Evaluation of New Treatments in Radiation Oncology: Are They Better Than Standard Treatments?. JAMA 293: 970-978 [Abstract] [Full text]  
• Tinetti, M. E., Bogardus, S. T. Jr., Agostini, J. V. (2004). Potential Pitfalls of Disease-Specific Guidelines for Patients with Multiple Conditions. NEJM 351: 2870-2874 [Full text]  
• Global IDEA Scientific Advisory Committee, (2004). Health and economic benefits of an accelerated program of research to combat global infectious diseases. CMAJ 171: 1203-1208 [Full text]  
• Joffe, S., Harrington, D. P, George, S. L, Emanuel, E. J, Budzinski, L. A, Weeks, J. C (2004). Satisfaction of the uncertainty principle in cancer clinical trials: retrospective cohort analysis. BMJ 328: 1463- [Abstract] [Full text]  
• Ades, A. E., Lu, G., Claxton, K. (2004). Expected Value of Sample Information Calculations in Medical Decision Modeling. Med Decis Making 24: 207-227 [Abstract]  
• Cole, C. H., Wright, K. W., Tarnow-Mordi, W., Phelps, D. L. (2003). Resolving Our Uncertainty About Oxygen Therapy. Pediatrics 112: 1415-1419 [Full text]  
• Gates, G. A. (2003). Dilating the Clinical Research Stricture. Arch Otolaryngol Head Neck Surg 129: 155-156 [Full text]  
• Landray, M. J, Whitlock, G. (2002). Evaluating treatment effects reliably. BMJ 325: 1372-1373 [Full text]  
• Senn, S (2002). Ethical considerations concerning treatment allocation in drug development trials. Stat Methods Med Res 11: 403-411 [Abstract]  
• Rollhauser, C, Hawkey, C J (2002). Upper gastrointestinal bleeding and surrogate end points. Gut 51: 140-140 [Full text]  
• Biondi-Zoccai, G. G.L., Abbate, A., Agostoni, P., Brown, B. G., Chait, A., Fisher, L. (2002). Antioxidant Vitamins and Coronary Disease. NEJM 346: 1092-1093 [Full text]  
• Bhandari, M., Richards, R. R., Sprague, S., Schemitsch, E. H. (2002). The Quality of Reporting of Randomized Trials in The Journal of Bone and Joint Surgery from 1988 through 2000. JBJS 84: 388-396 [Abstract] [Full text]  
• Frost, H, Lamb, S E, Robertson, S (2002). A randomized controlled trial of exercise to improve mobility and function after elective knee arthroplasty. Feasibility, results and methodological difficulties. Clin Rehabil 16: 200-209 [Abstract]  
• Djulbegovic, B., Clarke, M., Schulz, K. F., Moher, D., Altman, D. G. (2001). Ethical Issues in the Reporting of Clinical Trials. JAMA 286: 1577-1578 [Full text]  
• Carding, P., Hillman, R. (2001). More randomised controlled studies in speech and language therapy. BMJ 323: 645-646 [Full text]  
• Halpern, S. D., Karlawish, J. H. T., Weijer, C., Djulbegovic, B., Clarke, M. (2001). Informing Patients of Uncertainty in Clinical Trials. JAMA 285: 2713-2714 [Full text]  
• GEDDES, J., GOODWIN, G. (2001). Bipolar disorder: clinical uncertainty, evidence-based medicine and large-scale randomised trials. Br. J. Psychiatry 178 : s191-s194 [Abstract] [Full text]  
• Djulbegovic, B., Clarke, M. (2001). Scientific and Ethical Issues in Equivalence Trials. JAMA 285: 1206-1208 [Full text]  
• Lilford, R J, Djulbegovic, B., Schuklenk, U., Rothman, K. J, Michels, K. B (2001). Declaration of Helsinki should be strengthened. BMJ 322: 299a-299 [Full text]  
• Weijer, C. (2000). The Ethical Analysis of Risk. J Law Med Ethics 28: 344-361  
• Shapiro, S. H., Glass, K. C. (2000). Why Sackett's analysis of randomized controlled trials fails, but needn't. CMAJ 163: 834-835 [Full text]  
• Weijer, C., Shapiro, S. H, Glass, K. C., Enkin, M. W (2000). For and against: Clinical equipoise and not the uncertainty principle is the moral underpinning of the randomised controlled trial FOR AGAINST. BMJ 321: 756-758 [Full text]  
• Ashcroft, R. (2000). Giving medicine a fair trial. BMJ 320: 1686-1686 [Full text]  
• Sackett, D. L. (2000). Why randomized controlled trials fail but needn't: 1. Failure to gain "coal-face" commitment and to use the uncertainty principle. CMAJ 162: 1311-1314 [Abstract] [Full text]  
• Alderson, P., Roberts, I. (2000). Should journals publish systematic reviews that find no evidence to guide practice? Examples from injury research. BMJ 320: 376-377 [Full text]  
• Murray, G. D (2000). Promoting good research practice. Stat Methods Med Res 9: 17-24 [Abstract]  
• Alderson, P., Roberts, I. (1999). Design of CRASH trial. BMJ 319: 1068a-1068 [Full text]  
• Spiegelhalter, D. J, Myles, J. P, Jones, D. R, Abrams, K. R (1999). Methods in health service research: An introduction to bayesian methods in health technology assessment. BMJ 319: 508-512 [Full text]  
• Barer, D., Sacristán, J. A., Galende, I. (1999). Clinical trials. BMJ 318: 1138-1138 [Full text]  
• Fielding, L P., Grace, R., Hittinger, R. (1999). Patients who are eligible but not randomised should be included as additional comparative arm in study. BMJ 318: 874a-874 [Full text]  
• Chalmers, I. (1998). Unbiased, relevant, and reliable assessments in health care. BMJ 317: 1167-1168 [Full text]  

Rapid Responses:

Read all Rapid Responses

Large scale trials exacerbate risk of spurious conclusion if bias is not eliminated

D B Double
bmj.com, 5 Nov 1998 [Full text]

Drug Development is Different

Stephen Senn
bmj.com, 6 Nov 1998 [Full text]

Clinical Trials, The Next 50 Years

David Barer
bmj.com, 4 Dec 1998 [Full text]