Effect of incorporating a 10 minute point of care test for salivary nicotine metabolites into a general practice based smoking cessation programme: randomised controlled trial
BMJ 2005; 331 doi: https://doi.org/10.1136/bmj.38621.463900.7C (Published 27 October 2005) Cite this as: BMJ 2005;331:999
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Barnfather and colleagues describe results from a smoking cessation
trial to test the effect of giving immediate smoke intake feedback from a
nicotine metabolites saliva test. The test appears to be aimed at
providing an alternative to expired-air CO, used for many years throughout
the NHS. Such tests are important at three stages when treating smokers:
(1) Prior to stopping, to motivate a quit attempt by demonstrating the
level of harmful smoke constituents ingested, (2) In the early stages of a
quit attempt to monitor progress and, hopefully, demonstrate improvement,
(3) After quitting, to verify self-reports of abstinence. Compared to the
standard CO monitor the new test has utility in only the first of these
stages because a high proportion of smokers will use nicotine replacement
therapy (NRT) in their quit attempt and the test would fail at stages (2)
and (3) if NRT were used. Department of Health monitoring of Stop Smoking
Services shows that about 80% of smokers use NRT, following NICE
advocation of its use in March 2002. This position is unlikely to change
substantially since as well as being safe and cheap, NRT is likely to
remain the only proven effective medication that virtually all smokers can
use, and that is also readily available over the counter and on general
sale. Because the test cannot be used throughout whole treatment cycle it
is not a viable alternative to the cheaper, more immediate, and less time
-consuming CO test. The CO test also has the considerable advantage of
showing the level of a tobacco smoke constituent that is known to be
harmful in the body. Nicotine metabolites are not known to be harmful, and
any attempt at conversion between metabolite level from the test and the
more relevant CO level would only be approximate due to large individual
differences in inhalation patterns and nicotine metabolism.
Aside from the doubtful utility of the new product, there appear to
be statistical and design flaws in the current report that readers should
be aware of. Although the exact figures used in the comparison of
cessation rates in the two groups are not given, they are described in the
discussion as being 23% vs 6% (net improvement 17%, p<0.001). Such
rates only appear achievable by using different base samples in the two
groups. The intervention group cessation rate (10/44 = 22.7%) is based on
those who attended follow-up at 8 weeks, whereas the control rate (3/49 =
6.1%) is based on all who attended the initial session. This differential
selection of a base sample clearly favours the intervention, since those
who fail to return are more likely to be continuing smokers. Even this
comparison, unacceptable to most scientists and journals, yields a
significant result (p=0.034) by only 1 case. Using a standard intention-to
-treat analysis where all those randomized are included regardless of
attendance, the difference is no longer significant at the conventional 5%
level [10/50 (20%) vs 3/50 (6%); difference = 14%, p= 0.071].
It is also difficult to appreciate the thinking behind the sample
size (statistical power) calculations. The key component was an
expectation that immediately feedback produces a 20% improvement in quit
rate at 8 weeks. This is reported as indicating a requirement for exactly
100 subjects, 50 in each of intervention and control conditions, to
achieve an 0.8 chance (power) of detecting a difference should it exist.
The anticipated rates in the two groups are not given, although they are
fundamental to the calculation. Only by assuming success rates due to
delayed and immediate feedback of 2% and 22%, respectively, is a power
level of 80% achieved in such a small sample. To have expected such a low
rate among controls and a 20% improvement with such a minimal intervention
appears to unrealistic. It represents an odds ratio of 12 in favour of the
intervention, which is larger by a factor of about 5 than for any other
effective smoking cessation intervention. It is a larger difference than
achieved between NRT or bupropion and placebo, or in a brief intervention
setting such as this, between intervention with counseling plus bio-
feedback plus NRT and no-intervention. The most optimistic a-priori
expectation for such a modest intervention is more realistically an
advantage in the region of 5% (say, 5% vs 10%). As many who have
contemplated undertaking smoking cessation trials will know, to achieve
even a modest 80% power level about 1,000 subjects are required when
attempting to detect a true difference of 5% vs 10%. Based on this more
realistic assessment, one wonders how such a small sample size could have
been set in advance.
Regardless of the issues above, the best estimate of the net effect
of this intervention was still a very large and clinically, if not
statistically, significant 14% (odds ratio = 4), and the question remains
as to how such a large effect could have been achieved by such a modest
intervention. The answer might not be due to the immediate feedback test
but rather to the fact that this was essentially a non-blind trial that
relied on the motivation and expectations of the patients for effect. Lack
of blinding could have boosted the effect size in several ways. One
possibility is that after the allocation to the immediate feedback
condition was known the clinician inadvertently give additional counseling
and, perhaps more importantly, encouragement towards making a quit attempt
prior to the endpoint 8 weeks later. Conversely, those allocated to wait
for their feedback may have gained the impression that they should not
make a special effort to quit until after their result was given, by which
time it would have been too late to be recorded as a success.
Competing interests:
JAS has advised the manufacturers of smoking cessation products, for which he received renumeration. He has also advised the Department of Health, MRC, and NICE, for which he did not receive renumeration.
Competing interests: No competing interests
Re: Don’t give up the CO monitor just yet
We read with interest Dr Stapleton’s reply on the aims and outcomes
of our study (BMJ 2005; 331: 999). Unfortunately we believe Dr Stapleton
has mis-interpreted the text. We therefore respond to his comments as
factually as we are able to and do this in the order in which he has
written them.
Firstly, the test is not “aimed at providing an alternative to
expired-air CO”. We feel that the last paragraph of our introduction,
which serves as the aim of the study makes this clear. Our aims in the
accepted manuscript read: “The aims of this study were twofold:
1. To assess the impact of providing smokers with visual and personalised
feedback in a primary care setting, on their salivary nicotine metabolite
values (SNV) and upon quitting.
2. To assess patients’ opinions on the utility of such a PoC test in
helping them to quit smoking.”
The Editor changed this, but the paragraph used still makes the aims very
clear. Dr Stapleton’s interpretation is thus incorrect. Indeed, we
acknowledge that use of NRT would limit the utility of this test as a
“measure of smoking habit” and we state this clearly in our discussion,
but that is not the proposed purpose of its use in this study.
His second paragraph states “Aside from the doubtful utility of the
new product…”. This statement has no scientific foundation and is his
personal opinion. As scientists we prefer not to use such emotive
narrative as this biases readers unfairly. There are no major design
flaws or statistical flaws in our view and this was also the view of the
BMJ hanging committee and review panel. We are sure that Dr Stapleton
must be familiar with CONSORT statements and the tremendous rigour with
which the BMJ reviews submitted manuscripts. We received very detailed
comments from the statistical advisor for the BMJ, who dissected very
perceptively all data. We re-analysed all data in several different ways
to ensure its accuracy. The quit rate is clearly stated as 7% for
controls in our abstract and our results section and 23% for test
subjects. The 6% that appears in the discussion is a typographical error
and inconsistent with the rest of the paper. In fact the true figures
were 6.9% (controls) and 22.7% (test). Dr Tim Coleman, who provided a
scientific critical review of our paper (BMJ 2005;331:979-980 (29
October), doi:10.1136/bmj.331.7523.979) apparently has no difficulty
identifying the 7% quit rate for controls. Dr Stapleton is once more
incorrect in his assertion that “different base samples” were used. The
data was calculated, exactly as we state (very clearly) in figure 2 and
also in the text. One cannot use 50 as the denominator for the
calculation, because 6 test and 7 control subjects failed to attend the
recall and their quit status was thus unknown. We again discussed the
best way of representing this data with the BMJ review panel and this was
felt to be the clearest way of achieving this. Indeed we had to re-plot
figure 2 to present that data in this way at the request of the BMJ
statistical advisor. Dr Stapleton’s calculation is thus an inappropriate
one. A Chi-square test provides a statistically significant difference in
quit rates between the groups. However, we would agree with Dr
Stapleton’s comment that the results are clinically significant and that
this is more relevant than statistical significance.
Dr Stapleton is correct in that the power calculation is incorrectly
documented. One cannot predict with accuracy the likely differences in a
study, where there is no published prior data of this type. The power
calculation was therefore an estimate of the numbers needed to achieve a
power of 80% at the 0.05 level and indeed it required us to exit 37
subjects per cell to show a 2-fold difference between groups. The BMJ
statistician suggested we re-word our narrative in this section, to that
which appears in the paper. However, we accept the 20% should read “2-
fold” and we must acknowledge this typographical error and are grateful to
Dr Stapleton for pointing this out. We recruited 50 volunteers in each
cell to ensure we had some leeway with losing volunteers from the study.
A post-hoc analysis confirmed that the study was more than adequately
powered, and the BMJ statistician confirmed this by re-calculating
independently.
The 7% quit rate for controls (who were recruited with no specific
stated desire to quit smoking) is not a low quit rate as Dr Stapleton
implies, but consistent with several previous studies, which we do in fact
quote in our introduction.
In Dr Stapleton’s final paragraph, he asserts that the trial was
“essentially a non-blind trial…”. We strongly disagree with this
statement and ask Dr Stapleton to read our manuscript again. The BMJ
insists on CONSORT statements being made, upon blinding and true
randomisation and the review panel explored in almost forensic detail the
nature of the blinding of all staff involved in this study. The reason we
believe that such a “minimal intervention” had such an effect is explained
in our discussion. Feedback is now accepted as a valuable factor in
producing behavioural change. The visual and educational element of this
approach helps the smoker appreciate the personalised nature of the
intervention and helps them interact with the information provided. This
personalisation is a significantly better approach than the blanket
approach, whereby all smokers, are given the same advice to quit. Evidence
shows that many smokers ignore this advice or do not recall being given it
when questioned later. The visualisation of change induced by smoking
within their own bodies seems to be a far more powerful tool than warning
them about lung or heart disease that may or may not result from their
habit, many years into the future.
In summary, we feel our paper is as robust as any scientifically
conducted clinical trial can be and the BMJ review panel clearly felt that
too. All studies have their limitations and we try to report our data as
clearly as possible and always acknowledge any limitations, as is good
scientific practice.
Competing interests:
None declared
Competing interests: No competing interests