Randomised controlled trials: internal versus external validity

Answers

Statements a, b, and c are true, whereas d is false.

Melatonin had previously been prescribed to children with neurodevelopmental delay because of its sleep phase shifting and hypnotic properties. However, trials had conflicting results. The aim of this trial was to establish the efficacy of melatonin in treating severe sleep problems in children with neurodevelopmental disorders who had not responded to standardised sleep behaviour advice. A placebo controlled study design was used.

It was essential that the internal and external validity of the clinical trial were considered before the results could be generalised to other children with impaired sleep and neurodevelopmental disorders. Internal validity is the extent to which the observed treatment effects can be ascribed to differences in treatment and not confounding, thereby allowing the inference of causality to be ascribed to the treatment. External validity is the extent to which the study results can be generalised to the population that the sample was meant to represent.

The participants were recruited from 19 hospitals across England and Wales. It is not clear whether the sample was representative of the population. Therefore, it is difficult to assess the extent of external validity. However, the use of a multicentre trial meant that a larger number of children from different locations could be recruited than if the children had been sampled from, for example, a single hospital through convenience sampling. It is possible that the efficacy of melatonin could have varied between children from different geographical locations and demographic backgrounds. Therefore, the use of a multicentre trial resulted in the sample of children being more representative of the population and it promoted the external validity of the trial (a is true).

Participants were randomised to melatonin or placebo. Randomisation eliminated allocation bias and therefore minimised confounding. Allocation bias is the systematic difference between participants in how they are allocated to treatment. Confounding in clinical trials has been described in a previous question.2 It is the difference between treatment groups at baseline in those factors that affect treatment and outcome measures. In general, if the sample size for a trial is large enough, the randomisation of participants will result in groups of patients that are similar in baseline characteristics. Such factors include demographics, prognostic factors, and characteristics that influence participants to take part in or withdraw from a trial. If confounding is minimised then differences between treatment groups in outcome measures will be the result of differences in treatment received, not differences in characteristics at baseline. Therefore, a causal association can be inferred between treatment and outcome. Hence, the random allocation of participants to treatment promoted internal validity (b is true).

It was essential that a control group was included. Control treatments in trials are typically the standard treatment, a placebo, or no intervention. The researchers wished to establish the therapeutic benefit of melatonin. For that reason a placebo was used as the control. The placebo capsules and contents were identical in internal and external appearance to melatonin. By using a placebo and randomising children to treatment, participants and researchers were not aware of treatment allocation. This ensured that the trial was double blind. The double blind nature of the trial meant that response and assessor bias, collectively known as ascertainment bias, were minimised. Ascertainment bias, described in a previous question,3 is the systematic distortion of the assessment of outcome measures by the investigators or trial participants because they are aware of treatment allocation. Therefore, the use of a placebo to blind patients and researchers to treatment allocation meant that internal validity was promoted (c is true).

If the children and their parents had a preference for treatment it might have threatened the internal validity of the trial. Those who had a preference for melatonin might be better motivated and show greater adherence to treatment if allocated the intervention. In contrast, participants not receiving their preferred treatment might exhibit resentful demoralisation, whereby they comply poorly and possibly withdraw from the trial. Therefore, the randomised double blind placebo controlled design meant that resentful demoralisation was minimised and internal validity was promoted (c is true).

Internal validity depended on the random allocation of participants and use of placebo as described. It did not depend on external validity—that is, the extent to which the results can be generalised to the population that the sample was meant to represent (d is false). However, if the trial lacked internal validity, making it difficult to infer a causal association between treatment and outcome, then external validity might have been limited.

An intention to treat analysis, described in a previous question,4 was used to compare treatment groups in outcome. The approach compares treatment groups as originally allocated, irrespective of whether patients received or adhered to their treatment protocol. Not all of the participants in the trial above completed their follow-up. Intention to treat analysis promoted internal validity because it ensured that the treatment groups remained similar in baseline characteristics and therefore minimised confounding. Intention to treat analysis also promoted external validity because it is a pragmatic approach that aimed to evaluate the effectiveness of the intervention in routine practice.