Intended for healthcare professionals

Rapid response to:

Education And Debate

Interpreting treatment effects in randomised trials

BMJ 1998; 316 doi: https://doi.org/10.1136/bmj.316.7132.690 (Published 28 February 1998) Cite this as: BMJ 1998;316:690

Rapid Response:

Random results from randomised trials.

Editor – The proposal for analysing randomised trials by Guyatt et al1 is misguided, flies in the face of elementary statistical theory and is to be resisted. There are three obvious sources of variability in clinical trials. First, we have pure differences between patients: some are more seriously ill than others. Second, we have variability within patients: even given the same treatment regime they, or their measurements, may vary from period to period. Third, we have treatment by patient interaction: some patients may react more favourably to a given treatment than will other patients. The parallel group trial does not and cannot distinguish between the three types of variability unless we can find meaningful ways of classifying sub-groups2,3. The standard cross-over trial will distinguish between the first type of variability and the other two but not easily between the second and third4 and certainly not in the form of analysis suggested by Guyatt et al.

Guyatt et al have implicitly assumed in this paper1 that whether a patient is better treated on one treatment than another can be determined by comparing one period of treatment on each. This is at complete variance to advice that Guyatt and co-authors have give elsewhere5. There they have suggested, that if efficacy for individual patients is to be established, they should be randomised to repeated periods of treatment and control: the so-called 'n-of-1’ methodology.

Nothing from the two clinical trials as presented by Guyatt et al is inconsistent with the theory that all patients benefitted equally. If we wish to establish what proportion of patients benefit from treatments, rather than merely being satisfied with average effects, then we need random effect models and sequences of n-of-1 trials3,6. Since the methodology which they propose does not correctly partition the sources of random variability, it will simply produce random results.

1Guyatt GH, Juniper EF, Walter SD, Griffith LE, Goldstein RS. Interpreting treatment effects in randomised trials. BMJ 1998; 316:690-3
2Senn SJ. Letter to the editor: testing for individual and population equivalence based on the proportion of similar responses, Stat Med 1997; 15: 1303-5.
3 Senn SJ. Statistical Issues in Drug Development. Chichester: Wiley, 1997.
4 Senn SJ. Cross-over Trials in Clinical Research. Chichester: Wiley, 1993.
5Guyatt GH, Heyting A, Jaeschke R, Keller J, Adachi JD, Roberts RS. N of 1 randomized trials for investigating new drugs. Controlled Clin Trials 1990; 11:88-100.
6 Senn SJ. Suspended judgment: n-of-1 trials. Controlled Clin Trials, 1993; 14:1-5.

Stephen Senn Professor of pharmaceutical and health statistics
Department of Epidemiology and Public Health, Department of Statistical Science,
University College London, London WC1E 6BT
stephens@public-health.ucl.ac.uk

Competing interests: No competing interests

06 July 1998
Stephen Senn
Professor of Pharmaceutical and Health Statistics
University College London