Reanalysis of epidemiological evidence on lung cancer and passive smokingBMJ 2000; 320 doi: http://dx.doi.org/10.1136/bmj.320.7232.417 (Published 12 February 2000) Cite this as: BMJ 2000;320:417
- J B Copas, professor (, )
- J Q Shi, research fellow
- Correspondence to: J B Copas
- Accepted 1 November 1999
Objective: To assess the epidemiological evidence for an increase in the risk of lung cancer resulting from exposure to environmental tobacco smoke.
Design: Reanalysis of 37 published epidemiological studies previously included in a meta-analysis allowing for the possibility of publication bias.
Main outcome measure: Relative risk of lung cancer among female lifelong non-smokers, according to whether her partner was a current smoker or a lifelong non-smoker.
Results: If it is assumed that all studies that have ever been carried out are included, or that those selected for review are truly representative of all such studies, then the estimated excess risk of lung cancer is 24%, as previously reported (95% confidence interval 13% to 36%, P<0.001). However, a significant correlation between study outcome and study size suggests the presence of publication bias. Adjustment for such bias implies that the risk has been overestimated. For example, if only 60% of studies have been included, the estimate of excess risk falls from 24% to 15%.
Conclusion: A modest degree of publication bias leads to a substantial reduction in the relative risk and to a weaker level of significance, suggesting that the published estimate of the increased risk of lung cancer associated with environmental tobacco smoke needs to be interpreted with caution.
A systematic review of epidemiological studies on passive smoking estimated the increased risk of lung cancer as 24%
There is clear evidence of publication bias in these studies
Reanalysis of the data allowing for the possibility of publication bias substantially lowers the estimate of relative risk
Exposure to environmental tobacco smoke (passive smoking) is widely accepted to increase the risk of lung cancer, but different epidemiological studies have produced varying estimates of the size of the relative risk. Hackshaw et al reviewed the results of 37 such studies that estimated the relative risk of lung cancer among female lifelong non-smokers, comparing those whose spouses (or partners) were current smokers with those whose spouses had never smoked.1 Of the 37 studies, 31 reported an increase in risk, and the increase was significant in seven studies. The remaining six studies reported negative results, but none of these was significant. Pooling these results using a method which allows for statistical heterogeneity between studies, Hackshaw et al concluded that there is an overall excess risk of 24% (95% confidence interval 13% to 36%).1 This is strong epidemiological evidence for an association between lung cancer and passive smoking (P<0.001).
The approach used by Hackshaw et al does not allow for the possibility of publication bias—that is, the possibility that published studies, particularly smaller ones, will be biased in favour of more positive results. We reanalysed the results and looked for evidence of publication bias.
Methods and results
The figure shows the relative risks from the 37 epidemiological studies analysed by Hackshaw et al1 plotted against a measure of the uncertainty in that relative risk. This uncertainty (s) decreases as the size of the study increases so that large studies are on the left of the plot and small studies on the right. The plot shows a trend for smaller studies to give more positive results than the larger studies (correlation=0.35, P<0.05, or P=0.012 by Egger's test2). This graph is similar to the funnel plot used in the meta-analysis of clinical trials, when a trend such as this is interpreted as a sign of publication bias.3 This bias arises when a study is more likely to be written up and submitted to a journal and more likely to be accepted for publication if it reports positive results than if its results are inconclusive or negative. Since it is reasonable to assume that publication is more likely for larger (small s) than smaller (large s) studies, the problem of publication bias will be most evident among the smaller studies, as suggested by the figure. By “publication” we mean the whole process of selecting a study for review.
We reanalysed the results of the 37 epidemiological studies to allow for the trend evident in the figure. Our method describes the apparent relation between relative risk and study size by a curve. This gives a good fit to the observed points. The basic idea of the method is that there is no real relation between study outcome and study size, the relation that we observe is simply an artefact of the process of selecting these studies.
Our method has been published,4 and further details are available from us on request. The estimated average relative risk depends on a statistical parameter that can be interpreted as the probability that a paper with a certain value of s is published (publication probability. If the publication probability is 1,all papers are published and so there is no possibility of publication bias; the relative risk is then estimated as 1.24 (24% risk excess), agreeing as expected with Hackshaw et al's result.1 But smaller values of publication probability give smaller estimates of relative risk. We do not know how many unpublished studies have been carried out. Therefore there is no way of estimating the publication probability from any data: all we know is that there is a significant correlation in the funnel plot, so that some degree of publication bias is needed to explain this trend.
The table gives the estimated relative risk for values of publication probability between 0.6 and 1, together with 95% confidence intervals and P values. The P value is less than 5% only when the publication probability is more than about 0.7. The indirect estimate of 19% excess risk derived from studies on biochemical markers (table 5 of Hackshaw et al's paper1) agrees with the epidemiological analysis when the publication probability is about 0.9.
For any given value of publication probability it is possible to estimate the number of studies which have been undertaken but not published. This is shown in the final two columns of the table. If the publication probability is 0.8 then there are a total of 23 unpublished studies so that the 37 selected ones represent a sample of 37/60=62% of all such studies that have been undertaken. If this is the case, then the excess risk is likely to be closer to 15% than 24%. The dashed line in the figure shows the fit from our statistical model when the publication probability is 0.8; this curve fits the available evidence well
Although the trend in the figure seems clear, Bero et al suggest that the number of unpublished studies is unlikely to be large,5 and so the problem of publication bias may be less severe here than in systematic reviews of other aspects of medicine. However, the possibility of publication bias cannot be ruled out altogether, and at least some publication bias is needed to explain the trend we found. Our results show that the publication probability does not have to fall much below 1.0 before there is quite a substantial reduction in the estimated risk.
Contributors: The reanalysis was done jointly by JBC and JQS. JBS was responsible for the presentation of the paper and is the guarantor.
Funding Economic and Social Research Council.
Competing interest None declared.