Cost effectiveness of computer tailored and non-tailored smoking cessation letters in general practice: randomised controlled trialBMJ 2001; 322 doi: https://doi.org/10.1136/bmj.322.7299.1396 (Published 09 June 2001) Cite this as: BMJ 2001;322:1396
- A Scott Lennox (), clinical research fellowa,
- Liesl M Osman, senior research fellowb,
- Ehud Reiter, lecturerc,
- Roma Robertson, research assistantc,
- James Friend, professor of respiratory medicineb,
- Ian McCann, research assistanta,
- Diane Skatun, research fellowd,
- Peter T Donnan, medical statisticiane
- a Department of General Practice and Primary Care, University of Aberdeen, Aberdeen AB25 2AY
- b Department of Medicine and Therapeutics, University of Aberdeen
- c Department of Computing Science, University of Aberdeen
- d Health Economics Research Unit, University of Aberdeen
- e Medicines Monitoring Unit, Department of Clinical Pharmacology, University of Dundee, Dundee DD1 9SY
- Correspondence to: A Scott Lennox
- Accepted 8 March 2001
Objectives: To develop and evaluate, in a primary care setting, a computerised system for generating tailored letters about smoking cessation.
Design: Randomised controlled trial.
Setting: Six general practices in Aberdeen, Scotland.
Participants: 2553 smokers aged 17 to 65.
Interventions: All participants received a questionnaire asking about their smoking. Participants subsequently received either a computer tailored or a non-tailored, standard letter on smoking cessation, or no letter.
Main outcome measures: Prevalence of validated abstinence at six months; change in intention to stop smoking in the next six months.
Results: The validated cessation rate at six months was 3.5% (30/857) (95% confidence interval 2.3% to 4.7%) for the tailored letter group, 4.4% (37/846) (3.0% to 5.8%) for the non-tailored letter group, and 2.6% (22/850) (1.5% to 3.7%) for the control (no letter) group. After adjustment for significant covariates, the cessation rate was 66% greater (−4% to 186%; P=0.07) in the non-tailored letter group than that in the no letter group. Among participants who smoked <20 cigarettes per day, the cessation rate in the non-tailored letter group was 87% greater (0% to 246%; P=0.05) than that in the no letter group. Among heavy smokers who did not quit, a 76% higher rate of positive shift in “stage of change” (intention to quit within a particular period of time) was seen compared with those who received no letter (11% to 180%; P=0.02). The increase in cost for each additional quitter in the non-tailored letter group compared with the no letter group was £89.
Conclusions: In a large general practice, a brief non-tailored letter effectively increased cessation rates among smokers. A tailored letter was not effective in increasing cessation rates but promoted shift in movement towards cessation (“stage of change”) in heavy smokers. As a pragmatic tool to encourage cessation of smoking, a mass mailing of non-tailored letters from general practices is more cost effective than computer tailored letters or no letters.
What is already known on this topic
What is already known on this topic Brief opportunistic advice on stopping smoking that is given face to face by health professionals increases rates of cessation by 2-3%
Intensive, expert-led interventions increase cessation rates by up to 20% or more but are expensive and reach only a small proportion of smokers
Written advice tailored to an individual's “stage of change” (intention to stop in a particular period of time) has been claimed to be as effective as intensive interventions, but previous studies of tailored written advice did not biochemically validate cessation
What this paper adds
What this paper adds A simple standard letter sent to patients of general practices that gave brief advice on stopping smoking increased the biochemically validated rate of cessation by 2%
A letter tailored to the individual's “stage of change” was not more effective than the non-tailored standard letter
Although the increase in cessation resulting from the non-tailored standard letter was small, this intervention was highly cost effective
Cigarette smoking continues to be a major preventable source of illness and premature death in Scotland. Intensive, expert-led interventions have relatively high success rates but reach only a small proportion of smokers. The real potential for reducing the national prevalence of smoking lies in the widespread implementation of brief interventions. 1 2 However, there are constraints on effective health promotion by primary healthcare professionals, particularly lack of time and skills.3 Consequently effective, low cost interventions in primary care that require only minimal input from health professionals should be sought.
Two studies in North America investigated computer generated personalised letters as a method of encouraging smoking cessation. 4 5 Such letters allow smokers to receive expert input without much demand on health professionals' time. The findings were positive but can only be regarded as preliminary. The numbers of participants were small, and in neither study were smokers' claims to have stopped smoking validated biochemically. We were encouraged by the results of these studies, but believed that a larger study, with biochemical validation, was needed on a population with a wider socioeconomic range.
We hypothesised that computer tailored letters would be more effective and cost effective than non-tailored letters in helping smokers to stop. We planned subgroup analyses—light versus heavy smokers, and intention versus no intention to quit in the next six months—because the two previous studies had indicated differential effects according to heaviness of smoking and readiness to quit.
Our randomised controlled trial compared the effect on smoking cessation of a computer tailored letter, a non-tailored letter, and no letter. Ethical approval was obtained from the Grampian joint ethical committee.
At the start of the study we sent all participants a questionnaire that asked about their current smoking behaviour, attitudes to smoking, perception of barriers to quitting, and intention to quit in the next six months or in one month. After we received their questionnaire, we sent each participant a computer tailored letter, a non-tailored letter, or just a letter thanking them for participating in the study (“no letter”).
Tailored letter—We developed a computerised system for generating tailored letters. The system made decisions on the text to be included in each participant's letter, based on the answers the participant gave in the questionnaire. The phrases and decision rules were devised by experts on smoking cessation and on patient information, in collaboration with the developers of the software. The experts were informed by their clinical experience and their knowledge of various models of behaviour change,6 in particular the “stage of change” model of smoking cessation (described below).7 Smokers were first categorised according to their intention to stop smoking and their “decisional balance” (their rating of the pros and cons of smoking). This determined the main topics to be included in the letters, which were further personalised in response to other answers in the questionnaire.
Non-tailored letter—This was essentially a default tailored letter produced by scanning a blank questionnaire. To this extent, both interventions were expert interventions, based on a considerable input of time, knowledge, and experience.
No letter—We sent control participants a letter thanking them for their participation and informing them that they would receive material at the end of the study (either a tailored or a non-tailored letter, should either have been shown to be effective).
How the letters were tailored
Overall structure of the letters
Letters were printed on four A5 sized pages (one piece of A4 paper folded in half). The front page contained introductory text and the middle pages contained most of the tailored information. The back page was selected from 16 possible versions. Information was tailored on two levels: overall content was based on category of smoker, while specific text within sections was determined by each participant's specific answers to the questionnaire.
Categories of smoker
Smokers' “stage of change” was determined using the standard questions “Are you intending to stop smoking in the next six months?” (No = a “pre-contemplator”) and “If yes, are you intending to stop smoking within the next month?” (No = a “contemplator”, yes = a “preparer”).
Pre-contemplators were further divided into three categories.
Category 1—answered “no” to the question “Would you like to stop smoking if it was easy?”
Category 2—answered “yes” or “not sure” to the question “Would you like to stop smoking if it was easy?” and had a mixed or negative “decisional balance.” (Decisional balance refers to the things the smoker likes and dislikes about smoking and the weighting put on them. It can be positive (the dislikes outweigh the likes), negative (the likes outweigh the dislikes), or mixed. We assumed that someone with a positive decisional balance is well motivated to stop.)
Category 3—answered “yes” or “not sure” to the question “Would you like to stop smoking if it was easy?” and had a positive decisional balance.
Contemplators and preparers were also divided into two further categories according to their decisional balance (see box).
Categorisation of smokers
Category 1 (pre-contemplator)—would not like to stop even if it was easy
Category 2 (pre-contemplator)—would like (or unsure whether) to stop if it was easy, and has a mixed or negative decisional balance
Category 3 (pre-contemplator)—would like (or unsure whether) to stop if it was easy, and has a positive decisional balance
Category 4 (contemplator)—mixed or negative decisional balance
Category 5 (contemplator)—positive decisional balance
Category 6 (preparer)—mixed or negative decisional balance
Category 7 (preparer)—positive decisional balance
Content of the letters
All the letters apart from those for smokers in category 1 were based on five possible sections.
An introductory paragraph focusing on the intention or desire to stop smoking and how to achieve this.
Motivation, with three possible parts:
A comment on the reasons given for liking and disliking smoking
A paragraph on each of the three most important reasons for disliking smoking
A health message for those with serious respiratory or circulatory conditions.
(3) Confidence building, with three possible parts:
A general message on confidence
Specific reasons for being confident
Advice on coping with the three main difficulties envisaged when stopping.
(4) Advice on stopping:
“Small steps”: three changes to try in the next month in preparation for stopping
Ideas of what to do when planning to stop smoking, eg pick a date, tell friends
“Go for it”: list of things that other people have found useful when they have tried to stop.
(5) Closing section, reiterating the main emphasis of the letter.
Table 1 shows which sections were selected for categories 2-7. The detailed content of each section was tailored according to the answers in individual questionnaires.
Focus of information for each category
Category 1—The letter acknowledged that the participant was not currently interested in stopping smoking. If participants were concerned about their health some information was given about how stopping smoking would improve their health. Reasons to be confident of successfully stopping were listed. If the participant was a heavy smoker, the use of nicotine replacement was recommended should they eventually try to stop.
Category 2—These participants indicated that they might like to stop smoking but did not seem to be highly motivated. The main aim of the letter was therefore to increase motivation. It also aimed to increase confidence in their ability to stop and suggested strategies for coping with any envisaged difficulties should they decide to try to stop.
Category 3—These participants were aware of the negative aspects of smoking but possibly lacked confidence in their ability to stop. The main aim of the letter was therefore to increase confidence by addressing some of the difficulties they might face should they try to stop. It also suggested three simple preparatory “small steps” to an attempt at cessation.
Categories 4 and 5—Advice was given on planning an attempt to quit, including choosing a date for stopping. The letter also gave information aimed at maintaining motivation and increasing confidence. Additional motivational text was included for participants in category 4.
Categories 6 and 7—The main focus was specific advice on how to stop smoking. The letter also included information to reinforce reasons for stopping and to boost confidence.
Second level tailoring
Within each section, inclusion of text was based on participants' answers to the questionnaire. The following are examples of tailoring at this level.
The level of importance of the confidence messages was determined by how confident the person felt about stopping smoking. If they were not at all confident, the confidence messages were given a high level of importance.
Reasons for liking and disliking smoking were itemised; these were based on the participant's answers.
The three main barriers that make it difficult for a participant to stop were inferred from answers to questions about the difficulty of stopping, reasons for returning to smoking, and what they liked about smoking. The text aimed to provide support in overcoming these barriers.
The age and sex of a participant were taken into consideration when ideas about types of exercise they might like to take were given.
We recruited participants from smokers aged 17 to 65 years registered at six general practices in Aberdeen. From the computerised records of the practices we identified 7427 patients, who were sent a covering letter, a consent form, and a questionnaire to collect information to form the basis of the tailoring. We sent two reminders at intervals of three weeks.
Assignment and mailing of the letters
The unit of randomisation was the individual participant. After the questionnaires and consent forms were returned, we randomised the participants to the groups using computer generated random numbers. We mailed materials appropriate to each group immediately after randomisation.
Follow up at six months was by postal questionnaire, with two reminders at intervals of three weeks. We attempted telephone follow up of non-respondents.
The main outcome measure was point abstinence at six months, defined as a negative response to the question “Have you smoked a cigarette, even a puff, in the last seven days?” We validated self reports of smoking cessation by salivary cotinine assay.8 Participants lost to follow up and those whose report of cessation could not be confirmed biochemically were classed as continuing smokers.
We also measured movement in “stage of change” with respect to smoking, analysed as either a positive shift or null shift (no change or negative shift). Participants' initial stage of change was obtained from the postal questionnaire and classified as pre-contemplator (no intention to quit in the next six months), contemplator (intending to quit in the next six months), or preparer (intending to quit in the next month). Initial stage of change was compared with the intention reported in the final follow up questionnaire (final stage of change). We excluded non-validated quitters because their final stage of change could not be categorised. The main economic outcome measure was cost effectiveness, expressed as the cost for each additional non-smoker at six months.
We used the point abstinence rate at six months to determine sample size. We estimated the expected rate in the control group at 8%, based on reported rates of between 7% and 11% in randomised controlled trials of smoking cessation interventions in primary care.9–11 Given the low level of input from health professionals required by the intervention, even a modest increase in the cessation rate would be clinically worthwhile. We therefore chose an increase from 8% to 13% as being worthwhile. To detect this difference with a power of 80% at the 95% significance level required 590 participants in each group.
Methods of analysis
We used χ2 tests to analyse categorical variables. Data on continuous variables were tabulated as means and standard deviations. Differences between groups were assessed by using analysis of variance. We used multiple logistic regression to assess relations between outcomes and group membership.12 Analyses were adjusted for age, sex, level of social deprivation, heaviness of smoking, time to first cigarette of the day, and initial stage of change. Multiple logistic models tested for interaction terms for heaviness of smoking by group and initial stage of change by group. Logistic models were assessed for each of the four subgroups defined by heaviness of smoking (<20 and ≥20 cigarettes per day) and initial stage of change (pre-contemplator and contemplator or preparer).
Response rates and overall cessation rates
Figure 1 shows the flow of participants through the study. The number of valid mailings after exclusion of those sent to the wrong address or to participants who were currently non-smokers or who were excluded was 6155. Of the 6155 valid mailings, 2612 responses were valid (42.4%). A total of 2553 participants did not withdraw, and the follow up rate was 78.1% (1995/2553).
A total of 154 (6.0%) participants claimed to have stopped smoking after the intervention. The research nurse tried to contact these participants to obtain a salivary sample for biochemical validation. Samples were obtained from 99 participants; one sample was lost in the laboratory. Seven participants refused to give a sample, 33 had started smoking again before a sample could be obtained, eight could not be contacted, and a suitable appointment could not be arranged for another seven. Eighty nine participants were validated as having stopped smoking, giving an overall validated cessation rate of 3.5% (89/2553).
Characteristics of respondents
There were no significant differences between the groups in age, sex, level of social deprivation, or initial stage of change. The percentage of heavy smokers was significantly higher in the tailored letter group than in the non-tailored letter group, which had the lowest proportion of heavy smokers.
Sex, age, and heaviness of smoking were not associated with cessation, but there was a significant inverse association with level of social deprivation. Participants whose initial stage of change was contemplator or preparer were more likely to have stopped than pre-contemplators, as were participants who had their first cigarette later in the day (table 2).
Validated cessation rates were 3.5% (30/857; 95% confidence interval 2.3% to 4.7%) in the tailored letter group, 4.4% (37/846; 3.0% to 5.8%) in the non-tailored letter group, and 2.6% (22/850; 1.5% to 3.7%) in the control group. After adjusting for confounding variables, we found that participants receiving a non-tailored letter were 66% more likely to have quit than those receiving no letter (−4% to 186%; P=0.07), and participants receiving a tailored letter were 39% more likely to have quit than those receiving no letter (−21% to 146%; P=0.25). After adjustment for confounding variables, participants who received either a tailored or non-tailored letter were 53% more likely to have quit than those receiving no letter (−7% to 151%; P=0.09).
Among participants who smoked <20 cigarettes a day, those who received a non-tailored letter were 87% more likely to have quit than those who received no letter (table 3). Among participants who smoked ≥20 cigarettes a day and among pre-contemplators there were no differences between either the tailored letter group or the non-tailored letter group and the no letter group. On the other hand, contemplators or preparers who received the non-tailored letter had a higher cessation rate than those who received no letter (P=0.08).
Shift in stage of change among participants who did not stop smoking
Among participants who did not stop smoking, heavy smokers who received the tailored letter were 76% more likely (11% to 180%) to have made a positive shift in stage of change compared with those who received no letter (table 4).
The follow up questionnaire asked participants if they remembered receiving the letter, had kept it, or had found it useful or relevant. Just over a fifth (146/705) of respondents who did not receive a letter on smoking cessation “remembered” receiving such a letter, compared with 64% (422/657) who received a tailored letter and 46% (290/633) who received a non-tailored letter (P<0.001); the difference between the recollection rates of the tailored and non-tailored letter groups was also significant (P<0.01). Participants were more likely to have kept the tailored letter than the non-tailored letter (29% (188/657) v 19% (118/633); P<0.01). The tailored letter and non-tailored letter groups did not differ significantly regarding recipients' perception of the usefulness or relevance of the letter.
Cost effectiveness of the non-tailored letter intervention
Thirty seven of the 846 participants who received a non-tailored letter stopped smoking, compared with 22 of the 850 participants who received no letter. Costs based on the actual number of participants recruited indicate that these 15 additional quitters were gained at a total cost of £464.
Although the analysis was based on only the 846 smokers who responded to the initial contact, cost effectiveness analysis should also consider the potential benefits of the intervention to people who did not respond to the experimental intervention, because in real life the non-tailored letter would be distributed to everyone in the target population. The worst case scenario assumes that only the 846 participants were smokers, giving a cost effectiveness ratio of £89 per additional quitter. The best case scenario assumes that all mistargeted participants (that is, those responding to the initial contact as non-smokers or those who received wrongly addressed letters) had declared themselves, leaving the remaining 1219 participants as smokers. Assuming they behave in a similar manner to the 846 who received the non-tailored letter, this gives a cost effectiveness ratio of £37 per quitter.
The value of the intervention in terms of the expected life years gained can be calculated.13 Using a discount rate of 5% gives a cost per life year gained of between £50 and £122.
Our main finding was that, compared with no letter, the non-tailored letter was effective and cost effective in helping smokers to stop smoking, whereas the computer tailored letter was not. However, the computer tailored letter encouraged heavy smokers to move forward in their stage of readiness to consider stopping smoking.
The cessation rate of 4.4% is low compared with rates of 19% and 25% in two previous studies of computer generated letters and 21% in a mass media intervention by the Health Education Board for Scotland. 4 5 14 However, our study had methodological strengths: it was carried out on a randomly chosen population who had not actively volunteered to take part in the intervention and had no special motivation to quit; it used an intention to treat analysis, with all participants lost to follow up being classed as continuing smokers; claims of participants to have stopped smoking were biochemically validated; and the tailored and non-tailored letters were created from the same text base.
In contrast, the high rates of cessation in the other studies were based on self reported cessation, and subjects who dropped out were omitted from the calculations of rates of continuing smoking. In some studies the form of the materials used for the control group was very different from that in the tailored intervention. None of these studies used biochemical validation of non-smoking. Contrary to the argument that biochemical validation is unnecessary in brief intervention studies,15 our findings indicate that not validating cessation results in an overestimate of cessation. Furthermore, one study based its success rate on a subgroup of light smokers who had intended to quit smoking.4 If we had used these methods, our rate of cessation would have been 20% or more.
Raw et al summarised evidence on the validated effect of different types of cessation intervention.16 The most effective is nicotine replacement therapy, which increases the rate of cessation by 8% at six months. Brief advice from a doctor increases abstinence at six months by 2-3%. Two validated studies by the British Thoracic Society found that up to three personalised but non-tailored letters, from doctors to outpatients in chest clinics, increased cessation by 2-3%.17 The present study has found that even one short non-tailored letter from a patient's general practice is as effective as these last two brief interventions.
Can we conclude that tailored letters are not effective?
Our hypothesis that tailored letters would be more effective than non-tailored letters was not supported by the findings. However, using a weaker concept of tailoring, we might consider that our non-tailored letter was in fact tailored—or at least personalised—to some extent. Although the non-tailored letter was not tailored to individuals, it was more personal than a general leaflet giving advice on smoking cessation: it was in a letter format, with the crest of the local university and the logo of the patient's general practice, and was ostensibly from “the practice.” This degree of personalisation may account for some of its effect, given the evidence from the British Thoracic Society's study that a non-tailored letter signed by a physician is more effective than an unsigned control letter.17
The tailored letter was effective in increasing heavy smokers' readiness to stop smoking, whereas the non-tailored letter was ineffective. Although this finding was from a subanalysis done after the main analysis, it was significant at the level of P=0.02. The finding needs to be confirmed by further investigation but is reported here because it is potentially important for this group of smokers, which is held to be little affected by brief interventions. Progression in stage of change will lead to eventual cessation for some smokers; in a large group of smokers it could bring important long term gains.
Participants were more likely to remember receiving the tailored letter than the non-tailored letter, and more likely to keep it. This suggests a potential advantage of tailored over non-tailored letters.
Previous studies have shown that tailored interventions may be effective only in certain subgroups.18 This may reflect incomplete knowledge of what information works for which kind of smokers in the context of a letter or leaflet—tailored information may work only for certain categories of smoker. In the present study we got the information right for heavy smokers at an earlier stage of change—cognitive strategies are generally more appropriate for participants at earlier stages of change7—and the success of the tailored letter may reflect the importance it placed on raising confidence in achieving goals (“self efficacy”) among these smokers. It is unclear why heavy smokers fared better than light smokers.
The greater effectiveness of the non-tailored letter among smokers intending to quit in the next six months may be due to the fact that all participants in the non-tailored letter group received specific advice on how to prepare for and cope with difficulties during an attempt to quit, whereas many smokers in the tailored letter group did not receive this behavioural information. Instead they received more cognitive input aimed at boosting motivation, confidence, and self efficacy. Although both cognitive and behavioural input is appropriate for such smokers,7 letters may be better suited to conveying behavioural than cognitive interventions.
The evidence from other studies of tailored interventions is equivocal. A recent review drew overly positive conclusions: a critical reading of the source material shows that, of eight methodologically sound studies, three showed no effect of tailoring.19 One of these three was the only trial to compare a one-off tailored letter with both a non-tailored letter and a control. Of the five successful trials, one was a trial of “iterative” tailored feedback (successive cycles of questionnaire and intervention in which the responses to the questionnaire inform the contents of the next intervention) and stage-matched manuals, two were of tailored feedback and nicotine replacement therapy, one was of a non-tailored booklet and tailored feedback, and one was of iterative feedback. Even in these five, as pointed out above, the lack of validation of self reported cessation brings into question the reliability of their results.
However, the present study's findings on the impact of tailoring on shift in stage of change in heavy smokers, and the fact that recipients of tailored letters were more likely to remember receiving and to have kept their letter, suggest that it would be premature to conclude that tailoring is ineffective.
Cost effectiveness of the non-tailored letter
The cost per quitter of the non-tailored letter is estimated at between £37 and £89, which compares very favourably with other cessation interventions. 14 20–22 The cost effectiveness ratio of the Health Education Board for Scotland's mass media intervention was between £168 and £369, corresponding to a cost per discounted life year saved of between £304 and £656 (1993 prices).14 However, as pointed out above, this intervention used self reported quitting. If the true rate of quitting were lower, the corresponding cost effectiveness would also have been lower.
The present intervention is highly cost effective. This is because of the low cost of its delivery. Using existing data held in general practices means that the delivery of the information does not require the target groups to be persuaded to receive the intervention. Neither does it depend on the input of health professionals, so staff costs are low.
The potential for implementation of the non-tailored letter intervention
Intervention by primary care professionals in the form of brief opportunistic advice increases smoking cessation by about 2-3% over control intervention, 16 23 but its implementation is limited by various constraints on health professionals.3 In contrast, the 2% increase in cessation found in the present study could be widely and easily realised by using the computerised data now usually held by general practices. Indeed, the quantity and quality of data on smoking held in general practices are set to improve as computers become more user friendly. Soon most practices will be capable of implementing this type of intervention, which would be well suited to implementation at the level of primary care groups or local healthcare cooperatives, and which could be part of a national strategy on smoking cessation. Coordinators from the smoking cessation services that have recently been set up in all health authorities and boards could play a central role.
Other potential settings include smoking helplines and workplaces. The only conditions necessary for implementation are a political will, a database of current smokers, and an administrative structure capable of producing and sending the letters.
We thank Martin Pucci and Margaret Taylor for their contribution to the expert group; Steven Porter, Duncan MacIver, and Yvonne McKay for their programming expertise; Annette Hermse, the validation nurse; and the general practitioners and practice managers of participating practices for their cooperation.
Contributors: ASL contributed to the design of the study and to the expert group, led the evaluation, and prepared the manuscript for publication. LMO contributed to the design of the study and to the expert group, carried out some of the analysis, and helped prepare the manuscript for publication. ER contributed to the design of the study, led the development of the computer tailoring system, and helped prepare the manuscript for publication. RR coordinated the development of the computer tailoring system and helped prepare the manuscript for publication. JF contributed to the design of the study and to the expert group. IM was responsible for designing the questionnaire, data collection, and data entry. DS contributed to the design of, carried out, and reported the economic evaluation. PTD advised on statistical aspects of the study design and carried out most of the analysis. ASL, LMO, and ER are the guarantors for the paper.
Funding The Chief Scientist Office, Scottish Executive Health Department, with additional funding from the Engineering and Physical Sciences Research Council. The Health Economics Research Unit is funded by the Chief Scientist Office. The views expressed in this paper are those of the authors and not those of the funding bodies.
Competing interests None declared.