Jump to: Page Content, Site Navigation, Site Search,
You are seeing this message because your web browser does not support basic web standards. Find out more about why this message is appearing and what you can do to make your experience on this site better.
Sally Hollis a Medical Statistics
Unit, Fylde College, Lancaster University, Lancaster LA1 4YF, b AstraZeneca,
Mereside, Alderley Park, Macclesfield, Cheshire SK10 4TG
Correspondence
to: S Hollis s.hollis{at}lancs.ac.uk
| |
Abstract |
|---|
|
|
|---|
Objectives:
To assess the methodological quality of
intention to treat analysis as reported in randomised controlled trials in four large medical journals.
Design:
Survey of all reports of randomised controlled trials published in 1997 in the BMJ, Lancet, JAMA,
and New England Journal of Medicine.
Main outcome measures:
Methods of dealing with
deviations from random allocation and missing data.
Results:
119 (48%) of the reports mentioned intention to treat analysis. Of these, 12 excluded any patients who did not start
the allocated intervention and three did not analyse all randomised
subjects as allocated. Five reports explicitly stated that there were
no deviations from random allocation. The remaining 99 reports seemed
to analyse according to random allocation, but only 34 of these
explicitly stated this. 89 (75%) trials had some missing data on the
primary outcome variable. The methods used to deal with this were
generally inadequate, potentially leading to a biased treatment effect.
29 (24%) trials had more than 10% of responses missing for the
primary outcome, the methods of handling the missing responses were
similar in this subset.
Conclusions:
The intention to treat approach is often
inadequately described and inadequately applied. Authors should
explicitly describe the handling of deviations from randomised
allocation and missing responses and discuss the potential effect of
any missing response. Readers should critically assess the validity of
reported intention to treat analyses.
|
Key messages
|
| |
Introduction |
|---|
|
|
|---|
"Intention to treat" is a strategy for the analysis of randomised controlled trials that compares patients in the groups to which they were originally randomly assigned. This is generally interpreted as including all patients, regardless of whether they actually satisfied the entry criteria, the treatment actually received, and subsequent withdrawal or deviation from the protocol. However there is a debate about the validity of excluding specific cases within each of these categories from an intention to treat analysis.1 Clinical effectiveness may be overestimated if an intention to treat analysis is not done.2
The intention to treat approach has two main purposes. Firstly, the approach maintains treatment groups that are similar apart from random variation. This is the reason for randomisation, and the feature may be lost if analysis is not performed on the groups produced by the randomisation process. For example, in a trial comparing medical and surgical treatment for stable angina pectoris, some patients allocated to surgical intervention died before being operated on.3 If these deaths are not attributed to surgical intervention using an intention to treat analysis, surgery seems to have a falsely low mortality (table 1). Secondly, intention to treat analysis allows for non-compliance and deviations from policy by clinicians. There are, of course, exceptions. Some types of deviations from randomised allocation may occur only within the trial setting and would not be expected in routine practice. For example, in a trial comparing active and placebo vaccination there is the potential for placebo vaccine to be incorrectly administered in place of active, but this could not occur outside the trial and so need not be accounted for in estimates of potential efficacy. However, most types of deviations from protocol would continue to occur in routine practice and so should be included in the estimated benefit of a change in treatment policy. Intention to treat analysis is therefore most suitable for pragmatic trials4 of effectiveness rather than for explanatory investigations of efficacy.
|
Deviations from randomised allocation often result in missing
outcome data. A full application of the intention to treat approach is
possible only when complete outcome data are available for all
randomised subjects. Care must always be taken to minimise missing
responses and to follow up those who withdraw from treatment, but this
is particularly important for the implementation of an intention to
treat analysis.5 No consensus exists about how missing
responses should be handled in intention to treat analyses, and
different approaches may be appropriate in different situations. Practice also varies over handling of false inclusions (subjects found
after randomisation not to satisfy the entry criteria). Thus, there is
no single definition of an intention to treat analysis, and the phrase
seems to have different meanings for different authors.6
We carried out a survey of recently published reports to examine
current application of intention to treat analysis.
| |
Methods |
|---|
|
|
|---|
We identified all reports of randomised controlled trials published in 1997 in four major medical journals: BMJ, Lancet, JAMA, and New England Journal of Medicine. All except the New England Journal of Medicine have adopted the CONSORT statement,7 which requires that authors indicate whether analyses were performed on an intention to treat basis. The total number of randomised controlled trials was obtained by Medline searches for publication type "randomized controlled trials" within each journal and cross checked against the Cochrane controlled trials register.8 The journals were then hand searched to identify trials which reported an intention to treat analysis. For articles in the BMJ and Lancet, we also carried out a full text search for "intention to treat" or "intent to treat" on the internet (www.bmj.com, www.thelancet.com).
All trials that reported an intention to treat analysis were then
independently assessed by both authors. We considered deviations from
random allocation, false inclusions, and missing response. For each
trial we recorded whether each of these occurred, and if so, the method
of analysis and whether this method was explicitly stated. The
assessment of missing response was limited to the primary outcomes if
any were specified. Any uncertainties or disagreements between the two
assessments were resolved by consensus.
| |
Results |
|---|
|
|
|---|
About half of all the randomised controlled trials reported an analysis explicitly described as intention to treat (table 2), with similar proportions in each journal. A total of 119 reports of randomised controlled trials including an intention to treat analysis were assessed. Table 3 summarises their characteristics.
|
|
Most reports stated in the methods section that intention to treat analysis was used but did not specify how any deviations from randomised allocation, false inclusions, or missing outcomes were handled. Of the 15 reports that did not analyse according to randomised allocation, 12 specifically excluded from the analysis any patients who did not start the allocated intervention (table 4). Three papers described intention to treat analyses that do not comply with the basic principle of analysing all randomised subjects as allocated. In a report of a trial comparing conventional anterior surgery and laparoscopic surgery for repairing inguinal hernia, various patients were excluded, including those not receiving the allocated intervention:
Data on all patients who were randomly assigned ... were analysed on an intention to treat basis. In this analysis we did not include patients without hernias, those who withdrew their consent before undergoing surgery, those who at the time of surgery were found to be poor candidates for general anaesthesia, and those who did not undergo the assigned operation because of a misunderstanding resulting in an unplanned open or laparoscopic repair.21
This resulted in the exclusion of 57 (5%) enrolled patients.
|
In a trial of endometrial resection or hysterectomy for menorrhagia, the authors excluded from the intention to treat analysis 26 (13%) women who withdrew after randomisation but before surgery.22 The researchers contacted 10 of these women and found that, "of six who had been assigned endometrial resection, four had hysterectomy and two had resection, whereas three of four assigned hysterectomy chose endometrial resection and one chose hysterectomy." In a trial of folic acid supplementation, 17 (14%) women were excluded because of non-compliance.23 The aim of this trial was to predict the likely effect of food fortification, which would not provide the same opportunity for non-compliance as supplementation using tablets. Thus, exclusion of women who did not comply was appropriate, but it should not have been described as an intention to treat analysis.
Five reports explicitly stated that there were no deviations from random allocation. The remaining 99 reports seemed to analyse according to random allocation, but only 34 of these explicitly stated this. Of the 25 reports which stated that false inclusions had occurred, only a quarter included these cases in the reported intention to treat analysis (table 3).
Eighty nine trials had some missing data on the primary outcome
variable. The most common method of handling missing data was complete
case analysis (44, 49%), in which all patients with a missing response
are excluded from the analysis. Twenty nine (33%) papers used all
available information on each patient (28 censored at end of follow up
and one used all available outcome measurements over five assessments).
Fifteen (17%) imputed values for the missing response. The imputation
methods used were carry forward of last observed response (seven),
explicit allocation of poor outcome (four), implicit assumption of good
or poor outcome by including patients with missing response in the
denominator but not the numerator when calculating rates (three), and
use of the group average (one). Only one paper examined the effect of
using a range of methods to handle the missing
responses.24 Twenty nine (24%) trials had more than 10%
of responses missing for the primary outcome, the methods of handling
the missing responses were similar in this subset.
| |
Discussion |
|---|
|
|
|---|
Almost half the reports of randomised controlled trials included an analysis described as intention to treat. This compares with 12% of trials found in a survey of reports published in obstetric and gynaecological journals in 1990-1.25 Evidence based health care encourages appraisal of research methods, and critical appraisal guides for trials usually include a question on whether follow up was complete and whether subjects have been analysed in the groups to which they were randomised.26 This increased general awareness of intention to treat analysis may have contributed to its incomplete use in the analysis of randomised controlled trials. The trials may have not been planned with a complete strategy for the reduction and handling of deviations from the allocated intervention.
Failure to start intervention
The exclusion of patients who did not start the allocated
intervention from the intention to treat analysis was fairly common
(10%). In some situations this seems sensible and is unlikely to lead
to bias
when the intended effect of an intervention depends on the
occurrence of a subsequent event that cannot be influenced by the
randomised allocation. For example, prophylaxis for prevention of
transplant rejection can be effective only if a transplant is received;
it seems unlikely that allocation to active treatment or placebo could
affect this. Ideally, these situations should be avoided by
randomisation after the necessary event, but this is not always
possible in practice. Perhaps more could be achieved towards
appropriate timing of randomisation, as illustrated by the surgeon who
ensured randomisation after diagnosis by tossing a sterilised coin in
the operating theatre once the patient's abdomen was
open.27 Unless the possibility of bias can be confidently
rejected, patients who did not start the allocated intervention should
be included in the intention to treat analysis where possible.
Non-compliance
If deviations from randomised allocation are due to non-compliance
of the patient, the effect of the intervention if compliance had been
complete may be relevant. However, naive comparisons based on
compliance may be misleading. For example, the coronary drug
project28 found a substantially lower five year mortality
in patients who complied well with clofibrate than in those who
complied poorly, which seemed to indicate clofibrate was beneficial
when taken as instructed. However, when compliance was examined in the
placebo group, death rates in patients with both good and poor
compliance were similar to those in the clofibrate group. The authors
concluded that there are serious difficulties in evaluating treatment
efficacy in subgroups defined by patient responses after randomisation.
Considerable work has been carried out on valid statistical analysis of
the effect of compliance in clinical
trials,29-32 but this is a complex area and
should be approached with care.
False inclusions
False inclusions should also generally not be excluded from an
intention to treat analysis.33 Their exclusion can be
justified only if the reascertainment of the entry criteria is applied
identically in each group. From a pragmatic viewpoint, if false
inclusions occur in the controlled environment of a trial, it seems
inevitable that misclassification will also occur in routine clinical practice.
Missing response
The main problem in the application of intention to treat
seen in this survey was the handling of missing response. Inappropriate
handling of missing response can produce misleading conclusions. Table
5 shows the effect of various approaches. Complete case analysis, which
was the approach used in most trials, violates the principle of
intention to treat and leads to bias unless data are missing at
random
that is, absence of an observation is independent of the
outcome.
35 36
Partial information, such as outcome at
some time points, or time to drop out, may be used to produce a more
efficient analysis, but this is still potentially biased.37
|
|
Recommendations for intention to treat analysis "ITT is
better regarded as a complete trial strategy for design, conduct and
analysis rather than as an approach to analysis
alone"5
Design
Conduct
Analysis
Reporting
|
Implications
Full reporting of any deviations from random allocation and
missing response is essential in the assessment of the necessity and
appropriateness of an intention to treat approach, as emphasised in the
CONSORT guidelines on the reporting of randomised controlled
trials.7 However, the CONSORT guidelines do not address
intention to treat analysis in any detail and so we have provided
recommendations for its implementation (box).
| |
Acknowledgments |
|---|
Contributors: SH initiated the research and is guarantor. SH and FC searched for eligible reports, assessed the reports, and participated in data analysis and writing of the paper.
| |
Footnotes |
|---|
Funding: None.
Competing interests: None declared.
| |
References |
|---|
|
|
|---|
| 1. | Fisher LD, Dixon DO, Herson J, Frankowski RK, Hearon MS, Pearce KE. Intention to treat in clinical trials. In: Pearce KE, ed. Statistical issues in drug research and development. New York: Marcel Dekker, 1990:331-350. |
| 2. |
Bollini P, Pampallona S, Tibaldi G, Kupelnick B, Munizza C.
Effectiveness of antidepressants. Meta-analysis of dose-effect relationships in randomised clinical trials.
Br J Psychiatry
1999;
174:
297-300 |
| 3. | European Coronary Surgery Study Group. Coronary-artery bypass surgery in stable angina pectoris: Survival at two years. Lancet 1979; i: 889-893. |
| 4. |
Roland M, Torgerson DJ.
Understanding controlled trials. What are pragmatic trials?.
BMJ
1998;
316:
285 |
| 5. |
Lewis JA, Machin D.
Intention to treat who should use ITT?
Br J Cancer
1993;
68:
647-650[Medline].
|
| 6. | Issues in trial reporting. Bandolier 1996; 3: 6-7. |
| 7. | Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996; 276: 637-639[Medline]. |
| 8. | Cochrane Controlled Trials Register. Cochrane library. Cochrane Collaboration. Oxford: Update Software , 1997. |
| 9. | Spruance SL, Rea TL, Yhoming C, Tucker R, Saltzman R, Boon R. Penciclovir cream for the treatment of herpes simplex labialis. A randomized, multicenter, double-blind, placebo-controlled trial. Topical Penciclovir Collaborative Study Group. JAMA 1997; 277: 1374-1379[Abstract]. |
| 10. | Fazekas F, Deisenhammer F, Strasser-Fuchs S, Nahler G, Mamoli B. Randomised placebo controlled trial of monthly intravenous immunoglobulin therapy in relapsing-remitting multiple sclerosis. Austrian Immunoglobulin in Multiple Sclerosis Study Group. Lancet 1997; 349: 589-593[Medline]. |
| 11. | CAESAR Coordinating Committee. Randomised trail of addition of lamivudine or lamivudine plus loviride to zidovudine-containing regimens for patients with HIV-1 infection: the CAESAR trial. Lancet 1997; 349: 1413-1421[Medline]. |
| 12. | Landoni F, Maneo A, Colombo A, Placa F, Milani R, Perego P, et al. Randomised study of radical surgery versus radiotherapy for stage Ib-IIa cervical cancer. Lancet 1997; 350: 535-540[Medline]. |
| 13. | Rutgeerts P, Rauws E, Wara P, Swain P, Hoos A, Solleder E, et al. Randomised trial of single and repeated fibrin glue compared with injection of polidocanol treatment of bleeding peptic ulcer. Lancet 1997; 350: 692-696[Medline]. |
| 14. | Nashan B, Moore R, Amlot P, Schmidt AG, Abeywickrama K, Soulillou JP. Randomised trial of basiliximab versus placebo for control of acute cellular rejection in renal allograft recipients. CHIB 201 International Study Group. Lancet 1997; 350: 1193-1198[Medline]. |
| 15. |
Jacobson JM, Greenspan JS, Spritzler J, Ketter N, Fahey JL, Jackson JB, et al.
Thalidomide for the treatment of oral aphthous ulcers in patients with human immunodeficiency virus infection. National Institute of Allergy and Infectious Diseases AIDS Clinical Trials Group.
N Engl J Med
1997;
336:
1487-1493 |
| 16. |
Kaplan LD, Straus DJ, Testa MA, Von Roenn J, Dezube BJ, Cooley TP, et al.
Low dose compared with standard dose m-BACOD chemotherapy for non-Hodgkin's lymphoma associated with human immunodeficiency virus infection. National Institute of Allergy and Infectious Diseases AIDS Clinical Trials Group.
N Engl J Med
1997;
336:
1641-1648 |
| 17. |
Englund JA, Baker CJ, Raskino C, McKinney RE, Petrie B, Fowler MG, et al.
Zidovudine, didanosine, or both as the initial treatment for symptomatic HIV-infected children. AIDS Clinical Trials Group Study 152 Team.
N Engl J Med
1997;
336:
1704-1712 |
| 18. |
Tardif JC, Cote G, Lesperance J, Bourassa M, Lambert J, Doucet S, et al.
Probucol and multivitamins in the prevention of restenosis after coronary angioplasty. Multivitamins and Probuco Study Group.
N Engl J Med
1997;
337:
365-372 |
| 19. |
Guilhot F, Chastang C, Michallot M, Guerci A, Harousseau JL, Maloisel F, et al.
Interferon alfa-2b combined with cytarabine versus interferon alone in chronic myelogenous leukemia. French Chronic Myeloid Leukemia Study Group.
N Engl J Med
1997;
337:
223-229 |
| 20. |
Daoud EG, Strickberger SA, Man KC, Goyal R, Deeb GM, Bolling SF, et al.
Preoperative amiodarone as prophylaxis against atrial fibrillation after heart surgery.
N Engl J Med
1997;
337:
1785-1791 |
| 21. |
Liem MS, van der Graaf Y, van Steensel CJ, Boelhouwer RU, Clevers GJ, Meijer W, et al.
Comparison of conventional anterior surgery and laparoscopic surgery for inguinal-hernia repair.
N Engl J Med
1997;
336:
1541-1547 |
| 22. | O'Connor H, Broadbent JA, Magos AL, McPherson K. Medical Research Council randomised trial of endometrial resection versus hysterectomy in management of menorrhagia. Lancet 1997; 349: 879-901. |
| 23. | Daly S, Mills JL, Molloy AM, Conley IM, Lee YJ, Kirke PN, et al. Minimum effective dose of folic acid for food fortification to prevent neural-tube defects. Lancet 1997; 350: 1662-1665[Medline]. |
| 24. |
Post Coronary Artery Bypass Graft Trial Investigators.
The effect of aggressive lowering of low-density lipoprotein cholesterol levels and low-dose anticoagulation on obstructive changes in saphenous-vein coronary-artery bypass grafts.
N Engl J Med
1997;
336:
153-162 |
| 25. |
Schulz KF, Grimes DA, Altman DG, Hayes RJ.
Blinding and exclusions after allocation in randomised controlled trials: survey of published parallel group trials in obstetrics and gynaecology.
BMJ
1996;
312:
742-744 |
| 26. |
Guyatt GH, Sackett DL, Cook DJ.
Users' guides to the medical literature.2. How to use an article about therapy or prevention. A. Are the results of the study valid?
JAMA
1993;
270:
2598-2601 |
| 27. |
Newell DJ.
Intention-to-treat analysis: implications for quantitative and qualitative research.
Int J Epidemiol
1992;
21:
837-841 |
| 28. | Coronary Drug Project Research Group. Influence of adherence to treatment and response of cholesterol on mortality in the coronary drug project. N Engl J Med 1980; 303: 1038-1041[Abstract]. |
| 29. | Cuzick J, Edwards R, Segnan N. Adjusting for non-compliance and contamination in randomized clinical trials. Stat Med 1997; 16: 1017-1029[Medline]. |
| 30. | Efron B, Feldman D. Compliance as an explanatory variable in clinical trials. J Am Stat Assoc 1991; 86: 9-17. |
| 31. | Goetghebeur EJ, Shapiro SH. Analysing non-compliance in clinical trials: ethical imperative or mission impossible? Stat Med 1996; 15: 2813-2826[Medline]. |
| 32. | Sommer A, Zeger SL. On estimating efficacy from clinical trials. Stat Med 1991; 10: 45-52[Medline]. |
| 33. | Senn SJ. Statistical issues in drug development Chichester: Wiley , 1997. |
| 34. |
Savage MP, Douglas Jr JS, Fischman DL, Pepine CJ, King III SB, Werner JA, et al.
Stent placement compared with balloon angioplasty for obstructed coronary bypass grafts.
N Engl J Med
1997;
337:
740-747 |
| 35. | Little RA, Rubin DB. Statistical analysis with missing data. New York: Wiley , 1987. |
| 36. | Choi SC, Lu IL. Effect of non-random missing data mechanisms in clinical trials. Stat Med 1995; 14: 2675-2684[Medline]. |
| 37. | Lagakos SW, Lim LL, Robins JM. Adjusting for early treatment termination in comparative clinical trials. Stat Med 1990; 9: 1417-1424[Medline]. |
| 38. | Sackett DL, Richardson WS, Rosenberg WS, Haynes RB. Evidence-based medicine. New York: Churchill Livingstone , 1997. |
| 39. | Little R, Yau L. Intent-to-treat analysis for longitudinal studies with drop-outs. Biometrics 1996; 52: 1324-1333[Medline]. |
(Accepted 2 June 1999)
Read all Rapid Responses
What can you learn from this BMJ paper? Read Leanne Tite's Paper+