Rapid Responses to:

EDITORIALS: Richard Peto and Colin Baigent Trials: the next 50 years BMJ 1998; 317: 1170-1171 [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Large scale trials exacerbate risk of spurious conclusion if bias is not eliminated

D B Double   (5 November 1998)

Drug Development is Different

Stephen Senn   (6 November 1998)

Clinical Trials, The Next 50 Years

David Barer   (4 December 1998)

Large scale trials exacerbate risk of spurious conclusion if bias is not eliminated 5 November 1998
D B Double, Consultant Psychiatrist Norfolk Mental Health Care NHS Trust, Norwich Send response to journal: Re: Large scale trials exacerbate risk of spurious conclusion if bias is not eliminated	EDITOR - Peto and Baigent state that randomised trials can avoid bias.1 The problem with advocating large scale trials is that the chance of a spurious conclusion is increased if bias is not eliminated. In particular, small differences in mortality, as found for example in the ISIS-2 trial,2 could be due to bias. Larger differences in morbidity can be created by bias due to unblinding, particularly if psychological factors are important. For example, much of the efficacy of psychotropic medication in clinical trials could be a modification of the placebo effect produced through unblinding.3 Of course, psychological factors are not likely to have much impact on mortality and there is "no need in the search for precision to throw common sense out of the window".4 But, low-quality trials, compared with high-quality trials, are associated with an increased estimate of benefit.5 Even high quality reporting of trials does not mean that bias has been totally eliminated. Admittedly, it is difficult to see how bias could have influenced the results of the ISIS-2 trial, as allocation seems to have been adequately concealed and an "intention-to-treat" analysis deals with exclusions after randomisation. A thoroughgoing scepticism, however, should be suspicious that small differences in survival may have merely been created by large trials exacerbating bias. Large treatment effects demonstrable without randomised trials are believable as long as they are not placebo effects. The next 50 years of randomised evidence needs to acknowledge the failure to eliminate bias in the literature despite the introduction of properly randomised trials 50 years ago. 1. Peto R, Baigent C. Trials: the next 50 years. BMJ 1998;317:1170- 1. 2. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction. Lancet 1988;ii:349-60. 3. Fisher S, Greenberg RP. (eds) From Placebo to panacea. Putting psychiatric drugs to the test. Chichester: John Wiley, 1997 4. Hill AB. Medical ethics and controlled trials. BMJ 1963;i:1043 5. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998;352:609-13
Drug Development is Different 6 November 1998
Stephen Senn, Professor of Pharmaceutical and Health Statistics University College London, London WC1E 6BT Send response to journal: Re: Drug Development is Different	EDITOR- The interesting editorial by Professor Peto and Dr Baigent was written from the perspective of public health 1. The points they make are valuable but not valid for all trials. There is a world of difference between carrying out a controlled trial of a freely available treatment and investigating the properties of a yet unregistered pharmaceutical 2. It is, however in connection with the latter that the guidelines to which they implicitly refer have been developed 3. As an example of such differences, consider the uncertainty principle. Any physician who cannot satisfy the uncertainty principle in a drug trial because she feels that a yet unregistered product is highly likely to be beneficial, condemns all patients in her care to receive the inferior treatment. An alternative view of such trials is that trialists continue to experiment until either they are convinced, despite initial beliefs, that their treatment is ineffective or Society is convinced that it is effective. However, an effective treatment is much more than a molecule: it is also a formulation, a dose and a dose schedule. Many different, complex, and often of necessity small trials, have to be carried out to establish a final form of a treatment 2,4. Obtaining many measurements in such trials and paying careful attention to matters of quality may, indeed, be an appropriate thing to do. Furthermore, there are a number of practices in the conduct and analysis of pharmaceutical industry trials that it would be highly desirable for others to copy. For example if authors were to behave to editors as sponsors are required to by regulators, they would have to present an analysis that was pre-specified as well as all the original data. 1. Peto R, Baigent C. Trials: the next 50 years. BMJ 1998;317:1170- 1. 2. Senn, SJ. Statistical Issues in Drug Development, Chichester: Wiley, 1997. 3. International Conference on Harmonisation, ICH Harmonised Tripartite Guideline for General Considerations for Clinical Trials, Richmond: Brookwood, 1997. 4. Sheiner, LB, The intellectual health of clinical drug evaluation, Clin Pharmacol Ther 1991; 50: 4-9.
Clinical Trials, The Next 50 Years 4 December 1998
David Barer, Prof of Stroke Medicine Queen Elizabeth Hosp. Gateshead Send response to journal: Re: Clinical Trials, The Next 50 Years	While no-one should argue with Richard Peto and Colin Baigent [1] about the dangers of basing clinical practice on flawed "outcomes research" or isolated, undersized randomised trials, there are several reasons why the ultra-simple megatrial may not be the ideal model for the next 50 years. Although such trials have helped to banish the illusion of clinical certainty, teaching us not to expect a predictable physiological response from our treatments but merely an improvement in the odds of successful outcome, the homogenising effect of large numbers may still disguise important variations in response. Attempts to explore these variations through subgroup analysis are widely condemned because of the perceived association with post-hoc data dredging, so that the theoretical knowledge of clinicians and the individuality of patients are seen as less and less relevant to treatment decisions. The large simple trials that Peto advocates usually focus on hard endpoints such as deaths, which may be rare in some patient groups and of limited relevance in others. The ageing of the human race may be the greatest global challenge to be faced in the next 50 years, and we will surely become less concerned with the length of survival in old age than with its quality. Thus many treatments will be evaluated in terms of their effects on "healthy active life expectancy" or on the duration and severity of "terminal dependency" (research is needed to determine the optimal balance between these two in particular situations), rather than on mere survival. Relatively simple measures of dependency are available for this purpose, but trial follow up will involve more than just a body count. Older people are so physiologically diverse that we will need to look for qualitative as well as quantitative differences in treatment response in different subgroups. Trials will therefore need to estimate not just the average treatment effect but the benefits and risks in all important subgroups, so despite high event rates, even large trials will be unlikely to answer all the relevant questions about a particular treatment. Thus the next 50 years should see the monolithic megatrial replaced by preplanned collaborations between smaller studies, addressing different aspects of the same broad research question and using an agreed system for classifying patient subgroups, interventions and outcomes [2,3]. Such an approach would overcome many of the limitations of retrospective meta- analysis, and would involve clinicians to a greater extent in defining the main and subsidiary research questions, and agreeing on appropriate classifications. The risks and benefits of treatment would soon become clear in some subgroups and the focus of uncertainty would gradually change over time. Clinicians could then abandon the naive notion that treatment policy for all patients with a certain condition should be dictated by the results of a single trial or meta-analysis, and use their clinical skills, as well as the latest accumulated research evidence, to select the best treatment for individual patients. Yours faithfully David Barer 1. Peto R, Baigent C. Trials: the next 50 years. Large scale randomised evidence of moderate benefits. BMJ 1998;317:1170-1171 2. Barer D, Ellul J. From meta-analysis to epi-analysis: the European Stroke Database Project. In: Fracchia G, Haavisto K, eds. European Medicines Research, Perspectives in Clinical Trials. Cambridge: European Conference Publications, 1996: 157 -64 3. Gladman J, Barer D, Langhorne P. Specialist rehabilitation after stroke. BMJ 1996; 312:1623-24