Introduction

Top Abstract Introduction Methods and results Conclusions References

Exposure to environmental tobacco smoke (passive smoking) is widely accepted to increase the risk of lung cancer, but different epidemiological studies have produced varying estimates of the size of the relative risk. Hackshaw et al reviewed the results of 37 such studies that estimated the relative risk of lung cancer among female lifelong non-smokers, comparing those whose spouses (or partners) were current smokers with those whose spouses had never smoked.¹ Of the 37 studies, 31 reported an increase in risk, and the increase was significant in seven studies. The remaining six studies reported negative results, but none of these was significant. Pooling these results using a method which allows for statistical heterogeneity between studies, Hackshaw et al concluded that there is an overall excess risk of 24% (95% confidence interval 13% to 36%).¹ This is strong epidemiological evidence for an association between lung cancer and passive smoking (P<0.001).

The approach used by Hackshaw et al does not allow for the possibility of publication biasthat is, the possibility that published studies, particularly smaller ones, will be biased in favour of more positive results. We reanalysed the results and looked for evidence of publication bias.

	Methods and results

Top Abstract Introduction Methods and results Conclusions References

The figure shows the relative risks from the 37 epidemiological studies analysed by Hackshaw et al¹ plotted against a measure of the uncertainty in that relative risk. This uncertainty (s) decreases as the size of the study increases so that large studies are on the left of the plot and small studies on the right. The plot shows a trend for smaller studies to give more positive results than the larger studies (correlation=0.35, P<0.05, or P=0.012 by Egger's test²). This graph is similar to the funnel plot used in the meta-analysis of clinical trials, when a trend such as this is interpreted as a sign of publication bias.³ This bias arises when a study is more likely to be written up and submitted to a journal and more likely to be accepted for publication if it reports positive results than if its results are inconclusive or negative. Since it is reasonable to assume that publication is more likely for larger (small s) than smaller (large s) studies, the problem of publication bias will be most evident among the smaller studies, as suggested by the figure. By "publication" we mean the whole process of selecting a study for review.

View larger version (10K): [in this window] [in a new window]   Plot of relative risk of lung cancer versus s (standard deviation of log relative risk). Overall weighted average of relative risk is shown as dotted line and fitted value (publication probability=0.8) as dashed line

We reanalysed the results of the 37 epidemiological studies to allow for the trend evident in the figure. Our method describes the apparent relation between relative risk and study size by a curve. This gives a good fit to the observed points. The basic idea of the method is that there is no real relation between study outcome and study size, the relation that we observe is simply an artefact of the process of selecting these studies.

Our method has been published,⁴ and further details are available from us on request. The estimated average relative risk depends on a statistical parameter that can be interpreted as the probability that a paper with a certain value of s is published (publication probability. If the publication probability is 1, all papers are published and so there is no possibility of publication bias; the relative risk is then estimated as 1.24 (24% risk excess), agreeing as expected with Hackshaw et al's result.¹ But smaller values of publication probability give smaller estimates of relative risk. We do not know how many unpublished studies have been carried out. Therefore there is no way of estimating the publication probability from any data: all we know is that there is a significant correlation in the funnel plot, so that some degree of publication bias is needed to explain this trend.

The table gives the estimated relative risk for values of publication probability between 0.6 and 1, together with 95% confidence intervals and P values. The P value is less than 5% only when the publication probability is more than about 0.7. The indirect estimate of 19% excess risk derived from studies on biochemical markers (table 5 of Hackshaw et al's paper¹) agrees with the epidemiological analysis when the publication probability is about 0.9. 

For any given value of publication probability it is possible to estimate the number of studies which have been undertaken but not published. This is shown in the final two columns of the table. If the publication probability is 0.8 then there are a total of 23 unpublished studies so that the 37 selected ones represent a sample of 37/60=62% of all such studies that have been undertaken. If this is the case, then the excess risk is likely to be closer to 15% than 24%. The dashed line in the figure shows the fit from our statistical model when the publication probability is 0.8; this curve fits the available evidence well.

	Conclusions

Top Abstract Introduction Methods and results Conclusions References

Although the trend in the figure seems clear, Bero et al suggest that the number of unpublished studies is unlikely to be large,⁵ and so the problem of publication bias may be less severe here than in systematic reviews of other aspects of medicine. However, the possibility of publication bias cannot be ruled out altogether, and at least some publication bias is needed to explain the trend we found. Our results show that the publication probability does not have to fall much below 1.0 before there is quite a substantial reduction in the estimated risk.

	Acknowledgments

   Contributors: The reanalysis was done jointly by JBC and JQS. JBS was responsible for the presentation of the paper and is the guarantor.

	Footnotes

Funding: Economic and Social Research Council.

Competing interest: None declared.

	References

Top Abstract Introduction Methods and results Conclusions References

1.	Hackshaw AK, Law MR, Wald NJ. The accumulated evidence on lung cancer and environmental tobacco smoke. BMJ 1997; 315: 980-988[Abstract/Free Full Text].
2.	Egger M, Smith GD, Schneider M, Minder C. Bias in meta-analysis detected by a simple graphical test. BMJ 1997; 315: 629-634[Abstract/Free Full Text].
3.	Egger M, Smith GD. Misleading meta-analysis. BMJ 1995; 310: 752-754[Free Full Text].
4.	Copas JB. What works; selectivity models and meta analysis. J R Stat Soc Am 1999; 162: 95-109.
5.	Bero LA, Glantz SA, Rennie D. Publication bias and public health policy on environmental tobacco smoke. JAMA 1994; 272: 133-136[Abstract].

(Accepted 1 November 1999)