- a Medical Statistics Unit, Fylde College, Lancaster University, Lancaster LA1 4YF
- b AstraZeneca, Mereside, Alderley Park, Macclesfield, Cheshire SK10 4TG
- Correspondence to: S Hollis
- Accepted 2 June 1999
Objectives: To assess the methodological quality of intention to treat analysis as reported in randomised controlled trials in four large medical journals.
Design: Survey of all reports of randomised controlled trials published in 1997 in the BMJ, Lancet, JAMA, and New England Journal of Medicine.
Main outcome measures: Methods of dealing with deviations from random allocation and missing data.
Results: 119 (48%) of the reports mentioned intention to treat analysis. Of these, 12 excluded any patients who did not start the allocated intervention and three did not analyse all randomised subjects as allocated. Five reports explicitly stated that there were no deviations from random allocation. The remaining 99 reports seemed to analyse according to random allocation, but only 34 of these explicitly stated this. 89 (75%) trials had some missing data on the primary outcome variable. The methods used to deal with this were generally inadequate, potentially leading to a biased treatment effect. 29 (24%) trials had more than 10% of responses missing for the primary outcome, the methodsof handling the missing responses were similar in this subset.
Conclusions: The intention to treat approach is often inadequately described and inadequately applied. Authors should explicitly describe the handling of deviations from randomised allocation and missing responses and discuss the potential effect of any missing response. Readers should critically assess the validity of reported intention to treat analyses.
Intention to treat gives a pragmatic estimate of the benefit of a change in treatment policy rather than of potential benefit in patients who receive treatment exactly as planned
Full application of intention to treat is possible only when complete outcome data are available for all randomised subjects
About half of all published reports of randomised controlled trials stated that intention to treat was used, but handling of deviations from randomised allocation varied widely
Many trials had some missing data on the primary outcome variable, and methods used to deal with this were generally inadequate, potentially leading to bias
Intention to treat analyses are often inadequately described and inadequately applied
“Intention to treat” is a strategy for the analysis of randomised controlled trials that compares patients in the groups to which they were originally randomly assigned. This is generally interpreted as including all patients, regardless of whether they actually satisfied the entry criteria, the treatment actually received, and subsequent withdrawal or deviation from the protocol. However there is a debate about the validity of excluding specific cases within each of these categories from an intention to treat analysis.1 Clinical effectiveness may be overestimated if an intention to treat analysis is not done.2
The intention to treat approach has two main purposes. Firstly, the approach maintains treatment groups that are similar apart from random variation. This is the reason for randomisation, and the feature may be lost if analysis is not performed on the groups produced by the randomisation process. For example, in a trial comparing medical and surgical treatment for stable angina pectoris, some patients allocated to surgical intervention died before being operated on.3 If these deaths are not attributed to surgical intervention using an intention to treat analysis, surgery seems to have a falsely low mortality (table 1). Secondly, intention to treat analysis allows for non-compliance and deviations from policy by clinicians. There are, of course, exceptions. Some types of deviations from randomised allocation may occur only within the trial setting and would not be expected in routine practice. For example, in a trial comparing active and placebo vaccination there is the potential for placebo vaccine to be incorrectly administered in place of active, but this could not occur outside the trial and so need not be accounted for in estimates of potential efficacy. However, most types of deviations from protocol would continue to occur in routine practice and so should be included in the estimated benefit of a change in treatment policy. Intention to treat analysis is therefore most suitable for pragmatic trials4 of effectiveness rather than for explanatory investigations of efficacy.
Deviations from randomised allocation often result in missing outcome data. A full application of the intention to treat approach is possible only when complete outcome data are available for all randomised subjects Care must always be taken to minimise missing responses and to follow up those who withdraw from treatment, but this is particularly important for the implementation of an intention to treat analysis.5 No consensus exists about how missing responses should be handled in intention to treat analyses, and different approaches may be appropriate in different situations. Practice also varies over handling of false inclusions (subjects found after randomisation not to satisfy the entry criteria). Thus, there is no single definition of an intention to treat analysis, and the phrase seems to have different meanings for different authors.6 We carried out a survey of recently published reports to examine current application of intention to treat analysis.
We identified all reports of randomised controlled trials published in 1997 in four major medical journals: BMJ, Lancet, JAMA, and New England Journal of Medicine. All except the New England Journal of Medicine have adopted the CONSORT statement,7 which requires that authors indicate whether analyses were performed on an intention to treat basis. The total number of randomised controlled trials was obtained by Medline searches for publication type “randomized controlled trials” within each journal and cross checked against the Cochrane controlled trials register.8 The journals were then hand searched to identify trials which reported an intention to treat analysis. For articles in the BMJ and Lancet, we also carried out a full text search for “intention to treat” or “intent to treat” on the internet (www.bmj.com, www.thelancet.com).
All trials that reported an intention to treat analysis were then independently assessed by both authors. We considered deviations from random allocation, false inclusions, and missing response. For each trial we recorded whether each of these occurred, and if so, the method of analysis and whether this method was explicitly stated. The assessment of missing response was limited to the primary outcomes if any were specified. Any uncertainties or disagreements between the two assessments were resolved by consensus.
About half of all the randomised controlled trials reported an analysis explicitly described as intention to treat (table 2), with similar proportions in each journal. A total of 119 reports of randomised controlled trials including an intention to treat analysis were assessed. Table 3 summarises their characteristics.
Most reports stated in the methods section that intention to treat analysis was used but did not specify how any deviations from randomised allocation, false inclusions, or missing outcomes were handled. Of the 15 reports that did not analyse according to randomised allocation, 12 specifically excluded from the analysis any patients who did not start the allocated intervention (table 4). Three papers described intention to treat analyses that do not comply with the basic principle of analysing all randomised subjects as allocated. In a report of a trial comparing conventional anterior surgery and laparoscopic surgery for repairing inguinal hernia, various patients were excluded, including those not receiving the allocated intervention:
Data on all patients who were randomly assigned … were analysed on an intention to treat basis. In this analysis we did not include patients without hernias, those who withdrew their consent before undergoing surgery, those who at the time of surgery were found to be poor candidates for general anaesthesia, and those who did not undergo the assigned operation because of a misunderstanding resulting in an unplanned open or laparoscopic repair.21
This resulted in the exclusion of 57 (5%) enrolled patients.
In a trial of endometrial resection or hysterectomy for menorrhagia, the authors excluded from the intention to treat analysis 26 (13%) women who withdrew after randomisation but before surgery.22 The researchers contacted 10 of these women and found that, “of six who had been assigned endometrial resection, four had hysterectomy and two had resection, whereas three of four assigned hysterectomy chose endometrial resection and one chose hysterectomy.” In a trial of folic acid supplementation, 17 (14%) women were excluded because of non-compliance.23 The aim of this trial was to predict the likely effect of food fortification, which would not provide the same opportunity for non-compliance as supplementation using tablets. Thus, exclusion of women who did not comply was appropriate, but it should not have been described as an intention to treat analysis.
Five reports explicitly stated that there were no deviations from random allocation. The remaining 99 reports seemed to analyse according to random allocation, but only 34 of these explicitly stated this. Of the 25 reports which stated that false inclusions had occurred, only a quarter included these cases in the reported intention to treat analysis (table 3).
Eighty nine trials had some missing data on the primary outcome variable. The most common method of handling missing data was complete case analysis (44, 49%), in which all patients with a missing response are excluded from the analysis. Twenty nine (33%) papers used all available information on each patient (28 censored at end of follow up and one used all available outcome measurements over five assessments). Fifteen (17%) imputed values for the missing response. The imputation methods used were carry forward of last observed response (seven), explicit allocation of poor outcome (four), implicit assumption of good or poor outcome by including patients with missing response in the denominator but not the numerator when calculating rates (three), and use of the group average (one). Only one paper examined the effect of using a range of methods to handle the missing responses.24 Twenty nine (24%) trials had more than 10% of responses missing for the primary outcome, the methods of handling the missing responses were similar in this subset.
Almost half the reports of randomised controlled trials included an analysis described as intention to treat. This compares with 12% of trials found in a survey of reports published in obstetric and gynaecological journals in 1990-1.25 Evidence based health care encourages appraisal of research methods, and critical appraisal guides for trials usually include a question on whether follow up was complete and whether subjects have been analysed in the groups to which they were randomised.26 This increased general awareness of intention to treat analysis may have contributed to its incomplete use in the analysis of randomised controlled trials. The trials may have not been planned with a complete strategy for the reduction and handling of deviations from the allocated intervention.
Failure to start intervention
The exclusion of patients who did not start the allocated intervention from the intention to treat analysis was fairly common (10%). In some situations this seems sensible and is unlikely to lead to bias—when the intended effect of an intervention depends on the occurrence of a subsequent event that cannot be influenced by the randomised allocation. For example, prophylaxis for prevention of transplant rejection can be effective only if a transplant is received; it seems unlikely that allocation to active treatment or placebo could affect this. Ideally, these situations should be avoided by randomisation after the necessary event, but this is not always possible in practice. Perhaps more could be achieved towards appropriate timing of randomisation, as illustrated by the surgeon who ensured randomisation after diagnosis by tossing a sterilised coin in the operating theatre once the patient's abdomen was open.27 Unless the possibility of bias can be confidently rejected, patients who did not start the allocated intervention should be included in the intention to treat analysis where possible.
If deviations from randomised allocation are due to non-compliance of the patient, the effect of the intervention if compliance had been complete may be relevant However, naive comparisons based on compliance may be misleading. For example, the coronary drug project28 found a substantially lower five year mortality in patients who complied well with clofibrate than in those who complied poorly, which seemed to indicate clofibrate was beneficial when taken as instructed. However, when compliance was examined in the placebo group, death rates in patients with both good and poor compliance were similar to those in the clofibrate group. The authors concluded that there are serious difficulties in evaluating treatment efficacy in subgroups defined by patient responses after randomisation Considerable work has been carried out on valid statistical analysis of the effect of compliance in clinical trials,29–32 but this is a complex area and should be approached with care.
False inclusions should also generally not be excluded from an intention to treat analysis.33 Their exclusion can be justified only if the reascertainment of the entry criteria is applied identically in each group. From a pragmatic viewpoint, if false inclusions occur in the controlled environment of a trial, it seems inevitable that misclassification will also occur in routine clinical practice.
The main problem in the application of intention to treat seen in this survey was the handling of missing response. Inappropriate handling of missing response can produce misleading conclusions. Table 5 shows the effect of various approaches. Complete case analysis, which was the approach used in most trials, violates the principle of intention to treat and leads to bias unless data are missing at random—that is, absence of an observation is independent of the outcome.35 36 Partial information, such as outcome at some time points, or time to drop out, may be used to produce a more efficient analysis, but this is still potentially biased.37
Various imputation methods may be used to estimate the missing responses However, clinical trials usually do not collect sufficient data to allow good estimation, and the only commonly feasible options are using the last observed response (carry forward) or assuming that all missing responses were constant. Extreme case analysis (for example, all patients lost to the group that fared better are assigned a poor outcome; all lost to the group that fared worse are assigned a good outcome) has also been recommended,38 but this is unlikely to yield a conclusive answer in practice (Meyer K, Windeler J, 19th International Society for Clinical Biostatistics meeting, Dundee 1998). More sophisticated techniques for handling missing data are available39 but depend on assumptions about the missing data mechanism which cannot be completely verified in most clinical trials. In general, imputation is used to produce a conservative estimate of treatment effect. However, no imputation method can give an unbiased estimate of the treatment effect unless the assumptions made about the missing data are valid. To fully appreciate the potential influence of missing responses, some form of sensitivity analysis is recommended, examining the effect of different strategies on the conclusions.
Recommendations for intention to treat analysis “ITT is better regarded as a complete trial strategy for design, conduct and analysis rather than as an approach to analysis alone”5
Decide whether the aim is pragmatic or explanatory. For pragmatic trials, intention to treat is essential
Justify in advance any inclusion criteria which when violated would merit exclusion from intention to treat analysis
Minimise missing response on the primary outcome
Follow up subjects who withdraw from treatment
Include all randomised subjects in the groups to which they were allocated
Investigate the potential effect of missing response
Specify that intention to treat analysis has been carried out, explicitly describing the handling of deviations from randomised allocation and missing response
Report deviations from randomised allocation and missing response
Discuss the potential effect of missing response
Base conclusions on the results of intention to treat analysis
Full reporting of any deviations from random allocation and missing response is essential in the assessment of the necessity and appropriateness of an intention to treat approach, as emphasised in the CONSORT guidelines on the reporting of randomised controlled trials.7 However, the CONSORT guidelines do not address intention to treat analysis in any detail and so we have provided recommendations for its implementation (box).
Our survey revealed that the intention to treat approach is often inadequately described and inadequately applied. We hope that future researchers will take note of our recommendations, but we advise readers to assess critically the validity of reported intention to treat analyses.
Contributors: SH initiated the research and is guarantor. SH and FC searched for eligible reports, assessed the reports, and participated in data analysis and writing of the paper.
Competing interests None declared.