Cluster randomised controlled trial of expert system based on the transtheoretical (“stages of change”) model for smoking prevention and cessation in schoolsBMJ 1999; 319 doi: https://doi.org/10.1136/bmj.319.7215.948 (Published 09 October 1999) Cite this as: BMJ 1999;319:948
- Paul Aveyard, lecturer in public health medicine (, )
- K K Cheng, professor of epidemiology,
- Joanne Almond, research associate,
- Emma Sherratt, research fellow,
- Robert Lancashire, computing officer,
- Terry Lawrence, senior health development adviser,
- Carl Griffin, research associate,
- Olga Evans, research fellow
- Correspondence to: P Aveyard
- Accepted 24 September 1999
Objectives: To examine whether a year long programme based on the transtheoretical model of behaviour change, incorporating three sessions using an expert system computer program and three class lessons, could reduce the prevalence of teenage smoking.
Design: Cluster randomised trial comparing the intervention to a control group exposed only to health education as part of the English national curriculum.
Setting: 52 schools in the West Midlands region.
Participants: 8352 students in year 9 (age 13-14 years) at those schools.
Main outcome measures: Prevalence of teenage smoking 12 months after the start of the intervention.
Results: Of the 8352 students recruited, 7444 (89.1%) were followed up at 12 months. The intention to treat odds ratio for smoking in the intervention group relative to control was 1.08 (95% confidence interval 0.89 to 1.33). Sensitivity analysis for loss to follow up and adjustment for potential confounders did not alter these findings.
Conclusions: The smoking prevention and cessation intervention based on the transtheoretical model, as delivered in this trial, is ineffective in schoolchildren aged 13-14.
The transtheoretical model proposes that individuals move through a series of stages in behaviour change
A computer programme gave 13 and 14 year old school students tailored information about what stage they were in and what to do to move to the next stage
Students given this information were no more likely to move stage, refrain from smoking, or stop smoking than those exposed to ordinary classroom health education
There is no evidence that the computerised expert system based on the transtheoretical model is effective in smoking prevention and cessation
Editorial by Reid
Between 1993 and 1996 the percentage of regular smokers among 15 year olds in England increased from 19% to 28% in boys and from 26% to 33% in girls.1 The British government is committed to reducing this.2 School programmes are attractive vehicles for this because most schools teach health education as part of personal health and social education. The results of school interventions to prevent smoking have been disappointing, however.3–5 Short term reductions in smoking prevalence that were found in some studies disappeared after three years. 4 5
The transtheoretical model proposes that people change behaviour by moving through a sequence of stages—“stages of change.” 6 7 The model describes both how people become smokers and how they stop. Ten psychological processes move people through the stages; some processes are important for movement from one particular stage and not others. The other elements of the transtheoretical model comprise decisional balance (the balance of the pros and cons of smoking), self efficacy (the degree of confidence in oneself to accomplish the change to non-smoking or to remain a non-smoker), and temptations (to smoke). This influential model is incorporated in many health promotion programmes.8 The most exciting aspect of the theory is that it leads directly to interventions. Validated questionnaires measure the key elements of the transtheoretical model.9–11 An individual can be characterised as being in one particular stage of change Feedback, together with helpful strategies for increasing confidence, resisting temptation, and thinking about their smoking in the correct way, should help that individual progress to the next stage of change.12 This process of diagnosis, feedback, and a stock of helpful strategies for how to move stage have been incorporated into a computer program—an expert system 7 13 14 An expert system for adults has been tested and was more effective in smoking cessation than stage based manuals alone.15 The only published study that used the adolescent system to help school age smokers stop was a feasibility study and was too small to test the efficacy of the intervention.16 Here we report a large school based intervention study incorporating the expert system for smoking prevention and cessation in adolescents based on the transtheoretical model.
We chose school year 9, with students aged 13-14 years, to participate in the trial. We calculated the intraclass correlation coefficient (0.008) for smoking prevalence for this age group in schools from the West Midlands young people's lifestyle survey.17 Using this, the predicted prevalence of smoking in year 10 and the mean size of the year 9 groups, we calculated that a sample of 8500 was necessary to achieve 90% power to detect a 4% difference in the prevalence of smoking with a 5% type 1 error Most school based programmes have found effect sizes larger than this at one year of follow up.4 We aimed to test the intervention in a random sample of children in year 9 attending state schools in the West Midlands health region. We sampled schools with probability proportional to the size of their year 9 population. We approached 89 schools and 53 agreed to participate. Once schools had been randomised (see below) we visited them with baseline questionnaires. The research team administered questionnaires to whole classes as part of personal health and social education lessons. Individuals were able to opt out, though none chose to do so. The questionnaires were marked confidential and this was emphasised in the standard instructions read out before the questionnaire. We left questionnaires for non-attendees to complete later under teacher supervision according to a protocol so that young people had confidence their teachers would not see the data Participation in the cohort depended on filling in the baseline questionnaire, and over 90% of potential participants were recruited (see figure on website).
Once schools had agreed to participate we randomly allocated schools, not individuals, to receive the intervention or be controls We ensured that the arms were balanced by ordering schools into five groups based on numbers of students in year 9. We allocated each school a number between 1 and n (the maximum number in the group). A computer program generated n/2 random numbers between 1 and n, and these schools were allocated to intervention. One school allocated to the intervention dropped out after randomisation and before baseline questionnaires were administered.
The intervention group received six sessions of two types: one computer session and one class lesson for each of the three terms of year 9 (autumn 1997 to summer 1998). For the computer session, the research team set up a classroom with about 30 computers and removed these at the end of the day. Whole classes came in turns and each student used a computer with headphones. The computer program was based on that developed by Prochaska and colleagues, containing questionnaires measuring the key concepts of the transtheoretical model.13 After each questionnaire students received feedback both through the headphones and on screen of how their temptations, for example, compared to stage based data collected by Pallonen et al18 (normative feedback) and in second and third sessions, what change had occurred since last time (ipsative feedback). The questionnaires were interspersed with video clips of young people talking about their thoughts about smoking that were relevant to the stage of change of the student concerned. The other transtheoretical model intervention was a one hour lesson delivered by ordinary class teachers. The teachers attended a two day training course organised by Public Management Associates, who had developed licensed training and lesson plans in consultation with Prochaska and colleagues. The three lessons developed the young people's understanding of the stages of change and how the pros and cons of smoking would vary in different stages, and the lessons got young people to use these concepts. More details of how we delivered the intervention are available.19
Our aim for students in the control group was that they would be exposed to no intervention other than the normal health education on tobacco, which is part of the English national curriculum. However, as a reward for participation, teachers in control group schools were given three lesson plans and handouts on smoking.20 These lessons consisted of quizzes on facts about tobacco and one lesson on different ways of persuading someone to stop smoking. The content of the lessons was all taken from generally available teaching support material.21 The lesson plans and materials were provided to all control group schools, but teachers in these schools received no training in smoking issues or delivery of the lessons and it was up to the individual schools whether or not they used the materials.
We administered a questionnaire to all students at baseline and approximately one year after the start of the intervention (about five months after the last intervention) to assess the outcome. The primary outcome was regular smoking (one or more cigarettes per week). We used information from a number of questions and an algorithm to code smoking status. We created a variable to show where there was contradiction between the questions. We examined the test-retest reliability of smoking status derived from the algorithm (regular smoker or not) in a separate study of 122 year 9 students, with tests two weeks apart. The κ statistic was 0.87 (95% confidence interval 0.70 to 1.00), indicating excellent reliability. Of the 8352 students, we followed up 7444 (89.1%) and could allocate smoking status to 7413 (99.6% of those followed up); 7147 (96.0% of those followed up) gave consistent answers. Over 98% of students in both groups had at least two interventions.
All analysis was done using MLwiN (multi-level modelling for windows22) to account for cluster randomisation. We entered school as a random effect and all other variables as fixed effects in our logistic regression models. We calculated odds ratios and 95% confidence intervals. All percentages quoted in the results represent the modelled percentage for the average school from the population of all schools from which our sample of schools was obtained (that is, the random effect is zero).
For the outcome of smoking status, we analysed the data in three main ways. Firstly, we included everyone who started in the cohort, whether or not they were followed up (intention to treat analysis). We repeated the analysis making four different assumptions about those lost to follow up (see table 3). Secondly, we included only those for whom we knew the smoking status at follow up. Thirdly, we included only those students whose smoking status was known and who did not contradict themselves on any question pertaining to smoking status in the questionnaire. We produced three models for the outcome: “unadjusted” for any variable, “adjusted for baseline smoking status” as defined in table 1, and “fully adjusted” adjusted for all variables in table 1 except stage.
The distribution of baseline characteristics and other potential confounders was reasonably even, though the intervention group had slightly fewer never smokers and boys and slightly more children whose parents also smoked (table 1).
Most students received the intervention as intended (methods of process assessment are given on the website). Rates of completion were high, with over 77% receiving all three computerised interventions, though baseline smokers were less likely to attend. Most students did not speed through the computer session, though smokers were less likely to spend long enough to receive the individualised messages Students found the computer program easy to use and interesting, though slightly fewer found it useful or valuable, and these percentages were lower for smokers. Smokers' and non-smokers' ratings of interest and usefulness declined the more they used the intervention (table 2).
All teachers reported that all intervention lessons were delivered, but we have no record of which individuals received the class based intervention. However, the process of receiving the intervention required the same input from students as that for the computer intervention—that is, being present on the day that particular lesson was scheduled—and so the participation rates were probably similar. Teachers were reluctant to return their questionnaires, despite prompting. Most teachers would have taught the same lesson to several year 9 classes. Although they should have completed a questionnaire for every class they taught, many teachers returned a single questionnaire summarising all of that term's lessons Those who returned their questionnaire showed that they were happy with the lesson delivery and felt that the students had understood the lesson well (table on BMJ website). We have no data on whether the controls actually received the lessons on smoking that were distributed to teachers at control schools.
There were no statistically significant changes in smoking overall between the groups, or in the subgroups defined by initial smoking status (table 3). The odds ratio for the intention to treat analysis assuming that those lost to follow up did not change smoking status from baseline was 1.08 (0.89 to 1.33). There was little confounding by the variables in table 1 as shown by the small changes in odds ratios after adjustment.
Our pragmatic trial resulted in successful delivery of both the expert system and supporting lessons to students because our intervention was incorporated into the personal and social education curriculum Our study showed that smokers were less likely to be present and more likely not to take long enough on the expert system, and that they felt that the expert system was less valuable. Charlton and Blair have shown that regular smokers were about twice as likely to be absent from school as non-smokers,23 which explains the higher non-participation seen in our smokers. We had only a minority of all possible returns of questionnaire information about each class lesson. It is likely that more enthusiastic teachers would return their questionnaires, but the major factor accounting for non-return was probably competing demands on teachers' time. It is unlikely that this response was severely biased, but we cannot exclude this possibility Nevertheless, our data indicate that we delivered an intervention that was popular with teachers and students, even on the third occasion.
Effect of the intervention
This study shows that the intervention based on the transtheoretical model had no effect on the prevalence of regular smoking. Examination of the subgroups by initial smoking status revealed no effect. The confidence intervals and point estimates of the effect of the intervention show that it is unlikely that it reduces adolescent smoking prevalence by more than 2%, and it is more likely that it has no effect. Elders et al report that 80% of 16 year old American smokers were still smoking five or six years later.24 Taken together, this means we cannot exclude the possibility that the intervention would reduce smoking prevalence in early adulthood by 1% (a small but worthwhile public health benefit). One possibility is that we have moved participants along the stage of change but not yet influenced their behaviour. We have scheduled a two year follow up to see if this occurs, but our analysis on change in stage between the arms (data not presented) showed no benefit of the intervention for this outcome either.
Random allocation eliminated selection bias. There is no possibility of serious contamination in this intervention. As the only access to the intervention was by attending schools on the day we visited with the computers or the day of the lesson, individuals who did not attend these schools could not have received any important component of the intervention. Individuals who swapped to schools in the intervention arm would have been allocated a new identification number and completed the intervention, but they would only have been included in the analysis as dropouts from their original allocation; it is unlikely that there were more than a handful of such people Information bias is an unlikely explanation, because drop out was low and similar in both arms (10.7% for the intervention and 11.0% control). Sensitivity analysis that included the dropouts and assumed a range of possibilities about their smoking status did not alter our results.
Another cause of information bias is that some students give wrong information about their smoking status. It is unlikely that this was differential with respect to the arms of the study. Follow up was by the questionnaire alone, and standard instructions were given to each class. The follow up data were collected at least three months after the last intervention.
Non-differential misclassification may have affected these results, however, which would tend to reduce the apparent effect of any true differences between the arms. Our outcome of smoking more than one cigarette a week could be insensitive to changes between the arms. For example, a participant who smoked one cigarette of cannabis at the weekend is unlikely to be touched by the intervention and was included as a regular smoker under this definition. Similarly, some individuals in both arms may lie about their smoking status, but this is unlikely to have obscured the effectiveness of the transtheoretical model intervention for several reasons. The questionnaire did not include the participant's name, and all participants were assured that the questionnaire was confidential. The questionnaire showed high test-retest reliability. There was excellent agreement between the smoking status recorded on the questionnaire and that recorded on the computer for those in the intervention arm (κ=0.85, 0.82 to 0.87). In addition, our baseline and follow up smoking rates are similar to national data (smoking prevalence in year 9 at baseline was 13.2% (12.4% to 13.9%) compared with 10.5% (8.1% to 13.4%) in England1; in year 10 at follow up it was 19.0% (18.2% to 20.0%) compared with 18.5% (15.4% to 21.9%) in England1). Finally, data from cotinine validation studies suggest that questionnaire data on adolescents' smoking is valid.25 All this reduces the likelihood that non-differential misclassification obscured the effect of the intervention.
It remains possible that confounding that was not controlled by cluster randomisation or by measurement and adjustment explains the apparent lack of effect. We measured and controlled for some but not all the factors related to smoking,4 but we controlled for most of those that are unequivocally linked to smoking in adolescents It is unlikely that major uneven distribution of unmeasured confounders across the arms obscured the intervention effect.
Despite high rates of delivery of a programme that teachers and students found interesting, it had no effect on smoking prevalence among participants. The expert system used in this study12 is in current use in some parts of the United Kingdom, and it has been claimed to be effective.26 However, this large trial provides no justification for using it.
Public Management Associates developed the anglicised version of the computerised expert system and the lesson plans for the teachers and also trained the teachers. We had a great deal of help from Professor Jim Prochaska and his colleagues at the University of Rhode Island and we are grateful to them. We also worked closely with Birmingham City Council's health education unit. Mrs Sheila Hirst and Mrs Helen Evans administered this project and we are very grateful to them. We are also grateful to the 52 schools and their year 9 students and teachers for taking part in this study. This study would not have come about with Professor Rod Griffiths. He provided the impetus for the study, guidance on obtaining funding, and support and direction throughout the study, and we are very grateful to him for that.
Contributors: TL and KKC had the original idea for the study. All authors were members of the steering committee and contributed actively to the protocol, data analysis, and interpretation of the data. PA, JA, RL, ES and KKC prepared and analysed the data and discussed this with the others. PA wrote the first draft of the paper and KKC, JA, ES, RL, TL, CG, OE made revisions. PA will act as study guarantor.
Competing interests None declared.
Funding Health authorities of the West Midlands.
website extra Information on process assessment and a figure showing flow of participants are on the BMJ's website www.bmj.com