Rapid Responses to:

GENERAL PRACTICE: Tom Fahey, Nigel Stocks, and Toby Thomas Quantitative systematic review of randomised controlled trials comparing antibiotic with placebo for acute cough in adults BMJ 1998; 316: 906-910 [Abstract] [Full text]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Unbalanced methods and conclusion of the review

Christopher Cates   (23 May 1998)

Unbalanced methods and conclusion of the review 23 May 1998
Christopher Cates, GP Manor View Practice, Bushey Health Centre, London Road, Bushey, Herts Send response to journal: Re: Unbalanced methods and conclusion of the review	Dear Editor, Simon Chapman has highlighted the confusion caused by the different interpretations of the IRAC report on the effects of passive smoking ; by ignoring the size and direction of the effect and focussing on the lower confidence interval the erroneous conclusion was that passive smoking does not cause lung cancer. Unfortunately Fahey, Stocks and Thomas have fallen into the same trap in reporting the results of a systematic review of antibiotic use in acute cough . They state categorically in their discussion "This systematic review shows that antibiotic treatment has no effect on the resolution of acute cough". This conclusion is not justified by the data in their review. Both of the outcomes measured on resolution of productive cough and clinical improvement show a pooled effect which favours antibiotics, but does not reach clinical significance using a random effects model at the 95% level. The authors appear to have confused the statistical significance of these findings, with the size of the effect. There is around a one in forty chance of this pooled result arising due to random variation, rather than due to a real difference between antibiotic and placebo; this is hardly grounds to claim that the review shows that antibiotics have no effect. The authors do not show an even-handed approach when they deal with the data concerning antibiotic efficacy and side effects in treatment. In the case of efficacy the authors state that " antibiotic treatment was no better than placebo…", whilst in the case of the side effects the data are reported as showing " a non-significant increase in the risk of side effects from antibiotics". They then proceed to exclude the only trial that showed more side effects in the placebo group than the antibiotic group (on the grounds that this reduces heterogeneity) and found that the pooled result was then significant. Excluding a study because of the direction of its result is not an acceptable method of generating statistical significance. Heterogeneity in this outcome was not excessive (Chi squared=7.80, df=5, 0.2>p>0.1) and the same technique could have been used to generate a statistically significant benefit in the outcomes for cough resolution, by excluding the Williamson trial!