Endgames Statistical Question

Clinical trials: outcome measures

BMJ 2015; 350 doi: https://doi.org/10.1136/bmj.h121 (Published 12 January 2015) Cite this as: BMJ 2015;350:h121
  1. Philip Sedgwick, reader in medical statistics and medical education1
  1. 1Institute for Medical and Biomedical Education, St George’s, University of London, London, UK
  1. p.sedgwick{at}sgul.ac.uk

Researchers investigated the effects of household based treatment of drinking water on the prevalence of diarrhoea in areas with a turbid water source in rural western Kenya.1 A cluster randomised controlled trial was performed. The intervention was water treatment with flocculant-disinfectant or sodium hypochlorite for 20 weeks. The control treatment was standard practice. In total, 605 family compounds, each with at least one child aged under 2 years and containing several houses, were recruited. Compounds were randomised to treatment using cluster allocation, stratified by the site of the main source of household water (pond water or river water). In total, 201 compounds (2124 people) were allocated to water treatment with flocculant-disinfectant, 203 compounds (2249) were allocated to sodium hypochlorite, and 201 compounds (2277) were allocated to control.

The primary outcome was the prevalence of diarrhoea. Secondary outcomes included all cause mortality, bacteriological (Escherichia coli) concentration, free residual chlorine concentration, and turbidity. The researchers reported that for all age groups combined, when compared with control, the prevalence of diarrhoea was significantly lower in the flocculant-disinfectant and sodium hypochlorite treatment groups. The number of deaths in each of the flocculant-disinfectant and sodium hypochlorite treatment groups did not differ significantly from the control group. Drinking water from intervention households was more likely than water from control households to meet World Health Organization guidelines for bacteriological quality. Furthermore, drinking water from flocculant-disinfectant households had much lower turbidity than samples from control or sodium hypochlorite households. There was no indication of improvement in water quality as measured by free residual chlorine concentration. It was concluded that flocculant-disinfectant is well suited as a household based treatment of drinking water in areas with turbid water.

Which of the following statements, if any, are true?

  • a) The primary outcome represented the measure of greatest therapeutic benefit

  • b) The primary and secondary outcomes will have been declared before the trial started

  • c) The measurements of E coli concentration, free residual chlorine concentration, and turbidity are referred to as surrogate outcomes

  • d) All cause mortality is referred to as a composite outcome

Answers

Statements a, b, c, and d are all true.

The effectiveness and safety of treatments in clinical trials are measured by outcomes, sometimes referred to as endpoints. These endpoints are ideally clinical outcomes that are relevant to patients—for example, survival or health related quality of life. However, as described below, the outcomes are sometimes laboratory measurements of biomarkers. The primary outcome is typically the clinical measure that represents the greatest benefit of treatment (a is true). In the above trial the primary outcome was prevalence of diarrhoea. If an intervention significantly reduced diarrhoea when compared with control, it would be regarded as effective and might influence future treatment decision making. Trials can sometimes have more than one primary outcome if several measures are considered to be of equal therapeutic importance. However, this approach is generally not recommended because it can cause problems when interpreting the effectiveness of treatment. Trials will typically have several secondary outcomes that provide information on therapeutic effects of secondary importance, side effects, or tolerability. In the above trial, the secondary outcomes included all cause mortality, E coli concentration, free residual chlorine concentration, and turbidity. The purpose of the secondary outcomes was to provide further credibility to the interpretation of the results for the primary outcome.

The primary and secondary outcomes for a trial must be declared a priori—that is, when the trial is being planned (b is true). Such an approach prevents researchers from selecting those measures that show significant differences between treatment groups when the trial results are analysed, thereby claiming effectiveness for an intervention on the basis of measures that may have limited clinical importance. Furthermore, the necessary sample size for a trial is calculated on the basis of the smallest effect of clinical interest in the primary outcome that needs to be seen in order for an intervention to be regarded as effective.2 If a trial has more than one primary endpoint, then a sample size calculation should be performed for each one, and the largest resulting sample size selected.

In the above trial, the secondary outcomes included bacteriological (E coli) concentration, free residual chlorine concentration, and turbidity of drinking water. The researchers discussed how these factors can reduce the effect of initial water treatment and also increase the risk of recontamination of water during storage. In particular, the members of the family compounds were instructed not to store treated water. These endpoints are called surrogate outcomes (c is true). They were markers of the effectiveness of water treatment and used in addition to the final patient relevant clinical outcome of diarrhoea. As a surrogate outcome, they were proxy measures for the effect of household based treatment of drinking water on the prevalence of diarrhoea. Obviously it was thought that they would be closely associated with the primary outcome, and therefore be able to be used as potential markers for the effectiveness of household based treatment of drinking water in the future.

A surrogate outcome may be used as a substitute for a clinical outcome—either primary or secondary. They are often laboratory measurements of a biomarker that denotes how a patient feels, functions, or survives. Examples include cholesterol concentrations for major cardiovascular events. A surrogate outcome may be used if the patient relevant outcome is rare or will take a long time to occur during follow-up. The surrogate outcome will occur faster or be easier to measure, thereby reducing the duration, size, and cost of a trial. Surrogate outcomes should be chosen and interpreted with care. If the surrogate outcome does not predict the clinical outcome very well, it may limit the interpretation, application, or results of a trial.

Sometimes outcomes are composite, consisting of two or more component outcomes. For example, a recent placebo controlled trial investigated the effects of B vitamins and omega 3 fatty acids on cardiovascular diseases.3 The primary outcome measure was major cardiovascular events, defined as a composite of non-fatal myocardial infarction, stroke, or death from cardiovascular disease. Patients who experienced any one of these three component outcomes were considered to have experienced the composite primary outcome. Composite outcomes tend to be used when each component outcome is rare and a large sample would be needed to show a significant difference between treatment groups for each outcome. Because the use of a composite outcome would increase the event rate, a smaller sample size would be needed, reducing the duration, size, and cost of the trial. However, the results for a trial with a composite outcome can be misleading and should be interpreted with care. For example, for the primary composite outcome of major cardiovascular events, the incidence of one component outcome may have increased and the others decreased in the intervention group, resulting in no significant difference between the intervention and placebo groups in the frequency of major cardiovascular events. It is also possible that a significant reduction in major cardiovascular events may be seen without a reduction in all three component outcomes. Therefore, each component outcome should be presented separately to ensure sensible interpretation of the results. However, subsequent statistical comparisons between treatment groups for each component outcome should be avoided because they may lack statistical power and result in problems of interpretation associated with multiple hypothesis testing.4

In the above trial of household based treatment of drinking water, the secondary outcomes included all cause mortality, which is a composite outcome (d is true) because it combines death from any cause. All cause mortality was included because it was a measure of the total effect of treatment on general health. However, some deaths may not have been related to the source of drinking water and this might have produced misleading results. Although all cause mortality may be relevant to the trial participants, knowing whether specific causes of death were affected by the treatment would have also been important. However, establishing the exact cause of death in the trial setting would probably have been difficult for the researchers.

Notes

Cite this as: BMJ 2015;350:h121

Footnotes

  • Competing interests: None declared.

References

View Abstract

Sign in

Log in through your institution

Subscribe