Rapid Responses to:
|
|
Rapid Responses: Rapid Responses published:
-
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif)
Don’t give up the CO monitor just yet
- John A Stapleton (1 November 2005)
-
Re: Don’t give up the CO monitor just yet
- Iain L C Chapple, Kristian D Banfather (9 November 2005)
|
|
|||
|
John A Stapleton, Senior Lecturer Kings College London - Institute of Psychiatry, London SE5 8AF.
Send response to journal:
|
Barnfather and colleagues describe results from a smoking cessation trial to test the effect of giving immediate smoke intake feedback from a nicotine metabolites saliva test. The test appears to be aimed at providing an alternative to expired-air CO, used for many years throughout the NHS. Such tests are important at three stages when treating smokers: (1) Prior to stopping, to motivate a quit attempt by demonstrating the level of harmful smoke constituents ingested, (2) In the early stages of a quit attempt to monitor progress and, hopefully, demonstrate improvement, (3) After quitting, to verify self-reports of abstinence. Compared to the standard CO monitor the new test has utility in only the first of these stages because a high proportion of smokers will use nicotine replacement therapy (NRT) in their quit attempt and the test would fail at stages (2) and (3) if NRT were used. Department of Health monitoring of Stop Smoking Services shows that about 80% of smokers use NRT, following NICE advocation of its use in March 2002. This position is unlikely to change substantially since as well as being safe and cheap, NRT is likely to remain the only proven effective medication that virtually all smokers can use, and that is also readily available over the counter and on general sale. Because the test cannot be used throughout whole treatment cycle it is not a viable alternative to the cheaper, more immediate, and less time -consuming CO test. The CO test also has the considerable advantage of showing the level of a tobacco smoke constituent that is known to be harmful in the body. Nicotine metabolites are not known to be harmful, and any attempt at conversion between metabolite level from the test and the more relevant CO level would only be approximate due to large individual differences in inhalation patterns and nicotine metabolism. Aside from the doubtful utility of the new product, there appear to be statistical and design flaws in the current report that readers should be aware of. Although the exact figures used in the comparison of cessation rates in the two groups are not given, they are described in the discussion as being 23% vs 6% (net improvement 17%, p<0.001). Such rates only appear achievable by using different base samples in the two groups. The intervention group cessation rate (10/44 = 22.7%) is based on those who attended follow-up at 8 weeks, whereas the control rate (3/49 = 6.1%) is based on all who attended the initial session. This differential selection of a base sample clearly favours the intervention, since those who fail to return are more likely to be continuing smokers. Even this comparison, unacceptable to most scientists and journals, yields a significant result (p=0.034) by only 1 case. Using a standard intention-to -treat analysis where all those randomized are included regardless of attendance, the difference is no longer significant at the conventional 5% level [10/50 (20%) vs 3/50 (6%); difference = 14%, p= 0.071]. It is also difficult to appreciate the thinking behind the sample size (statistical power) calculations. The key component was an expectation that immediately feedback produces a 20% improvement in quit rate at 8 weeks. This is reported as indicating a requirement for exactly 100 subjects, 50 in each of intervention and control conditions, to achieve an 0.8 chance (power) of detecting a difference should it exist. The anticipated rates in the two groups are not given, although they are fundamental to the calculation. Only by assuming success rates due to delayed and immediate feedback of 2% and 22%, respectively, is a power level of 80% achieved in such a small sample. To have expected such a low rate among controls and a 20% improvement with such a minimal intervention appears to unrealistic. It represents an odds ratio of 12 in favour of the intervention, which is larger by a factor of about 5 than for any other effective smoking cessation intervention. It is a larger difference than achieved between NRT or bupropion and placebo, or in a brief intervention setting such as this, between intervention with counseling plus bio- feedback plus NRT and no-intervention. The most optimistic a-priori expectation for such a modest intervention is more realistically an advantage in the region of 5% (say, 5% vs 10%). As many who have contemplated undertaking smoking cessation trials will know, to achieve even a modest 80% power level about 1,000 subjects are required when attempting to detect a true difference of 5% vs 10%. Based on this more realistic assessment, one wonders how such a small sample size could have been set in advance. Regardless of the issues above, the best estimate of the net effect of this intervention was still a very large and clinically, if not statistically, significant 14% (odds ratio = 4), and the question remains as to how such a large effect could have been achieved by such a modest intervention. The answer might not be due to the immediate feedback test but rather to the fact that this was essentially a non-blind trial that relied on the motivation and expectations of the patients for effect. Lack of blinding could have boosted the effect size in several ways. One possibility is that after the allocation to the immediate feedback condition was known the clinician inadvertently give additional counseling and, perhaps more importantly, encouragement towards making a quit attempt prior to the endpoint 8 weeks later. Conversely, those allocated to wait for their feedback may have gained the impression that they should not make a special effort to quit until after their result was given, by which time it would have been too late to be recorded as a success. Competing interests: JAS has advised the manufacturers of smoking cessation products, for which he received renumeration. He has also advised the Department of Health, MRC, and NICE, for which he did not receive renumeration. |
|||
|
|
|||
|
Iain L C Chapple, Professor of Periodontology Birmingham Dental School B4 6NN, Kristian D Banfather
Send response to journal:
|
We read with interest Dr Stapleton’s reply on the aims and outcomes of our study (BMJ 2005; 331: 999). Unfortunately we believe Dr Stapleton has mis-interpreted the text. We therefore respond to his comments as factually as we are able to and do this in the order in which he has written them. Firstly, the test is not “aimed at providing an alternative to expired-air CO”. We feel that the last paragraph of our introduction, which serves as the aim of the study makes this clear. Our aims in the accepted manuscript read: “The aims of this study were twofold: 1. To assess the impact of providing smokers with visual and personalised
feedback in a primary care setting, on their salivary nicotine metabolite
values (SNV) and upon quitting.
The Editor changed this, but the paragraph used still makes the aims very clear. Dr Stapleton’s interpretation is thus incorrect. Indeed, we acknowledge that use of NRT would limit the utility of this test as a “measure of smoking habit” and we state this clearly in our discussion, but that is not the proposed purpose of its use in this study. His second paragraph states “Aside from the doubtful utility of the new product…”. This statement has no scientific foundation and is his personal opinion. As scientists we prefer not to use such emotive narrative as this biases readers unfairly. There are no major design flaws or statistical flaws in our view and this was also the view of the BMJ hanging committee and review panel. We are sure that Dr Stapleton must be familiar with CONSORT statements and the tremendous rigour with which the BMJ reviews submitted manuscripts. We received very detailed comments from the statistical advisor for the BMJ, who dissected very perceptively all data. We re-analysed all data in several different ways to ensure its accuracy. The quit rate is clearly stated as 7% for controls in our abstract and our results section and 23% for test subjects. The 6% that appears in the discussion is a typographical error and inconsistent with the rest of the paper. In fact the true figures were 6.9% (controls) and 22.7% (test). Dr Tim Coleman, who provided a scientific critical review of our paper (BMJ 2005;331:979-980 (29 October), doi:10.1136/bmj.331.7523.979) apparently has no difficulty identifying the 7% quit rate for controls. Dr Stapleton is once more incorrect in his assertion that “different base samples” were used. The data was calculated, exactly as we state (very clearly) in figure 2 and also in the text. One cannot use 50 as the denominator for the calculation, because 6 test and 7 control subjects failed to attend the recall and their quit status was thus unknown. We again discussed the best way of representing this data with the BMJ review panel and this was felt to be the clearest way of achieving this. Indeed we had to re-plot figure 2 to present that data in this way at the request of the BMJ statistical advisor. Dr Stapleton’s calculation is thus an inappropriate one. A Chi-square test provides a statistically significant difference in quit rates between the groups. However, we would agree with Dr Stapleton’s comment that the results are clinically significant and that this is more relevant than statistical significance. Dr Stapleton is correct in that the power calculation is incorrectly documented. One cannot predict with accuracy the likely differences in a study, where there is no published prior data of this type. The power calculation was therefore an estimate of the numbers needed to achieve a power of 80% at the 0.05 level and indeed it required us to exit 37 subjects per cell to show a 2-fold difference between groups. The BMJ statistician suggested we re-word our narrative in this section, to that which appears in the paper. However, we accept the 20% should read “2- fold” and we must acknowledge this typographical error and are grateful to Dr Stapleton for pointing this out. We recruited 50 volunteers in each cell to ensure we had some leeway with losing volunteers from the study. A post-hoc analysis confirmed that the study was more than adequately powered, and the BMJ statistician confirmed this by re-calculating independently. The 7% quit rate for controls (who were recruited with no specific stated desire to quit smoking) is not a low quit rate as Dr Stapleton implies, but consistent with several previous studies, which we do in fact quote in our introduction. In Dr Stapleton’s final paragraph, he asserts that the trial was “essentially a non-blind trial…”. We strongly disagree with this statement and ask Dr Stapleton to read our manuscript again. The BMJ insists on CONSORT statements being made, upon blinding and true randomisation and the review panel explored in almost forensic detail the nature of the blinding of all staff involved in this study. The reason we believe that such a “minimal intervention” had such an effect is explained in our discussion. Feedback is now accepted as a valuable factor in producing behavioural change. The visual and educational element of this approach helps the smoker appreciate the personalised nature of the intervention and helps them interact with the information provided. This personalisation is a significantly better approach than the blanket approach, whereby all smokers, are given the same advice to quit. Evidence shows that many smokers ignore this advice or do not recall being given it when questioned later. The visualisation of change induced by smoking within their own bodies seems to be a far more powerful tool than warning them about lung or heart disease that may or may not result from their habit, many years into the future. In summary, we feel our paper is as robust as any scientifically conducted clinical trial can be and the BMJ review panel clearly felt that too. All studies have their limitations and we try to report our data as clearly as possible and always acknowledge any limitations, as is good scientific practice. Competing interests: None declared |
|||