Re: Indoor tanning and non-melanoma skin cancer: systematic review and meta-analysis
This article  and another article published in BMJ  were relied upon in a recently published meta-analysis of the international prevalence of indoor tanning . Based on the summary relative risks of non-melanoma skin cancer and melanoma associated with indoor tanning, estimated from meta-analyses of 12 studies  and 27 studies , respectively, and using summary prevalence estimates of ever exposure to indoor tanning among adults in the United States, Northern and Western Europe, and Australia from a total of 18 studies, Wehner et al.  estimated that more than 450 000 non-melanoma skin cancer cases and more than 10 000 melanoma cases in these regions are attributable to indoor tanning each year.
We are writing to express concerns with the methods used to derive these estimates of attributable skin cancer cases. One problem lies with the use of meta-analytic methods to combine heterogeneous observational data into a single summary estimate. Appreciable heterogeneity was observed across studies included in these meta-analyses, as indicated by I2 values of 36.8%, 47.1%, and 56% for the associations between indoor tanning and basal cell carcinoma, squamous cell carcinoma, and melanoma, respectively [1, 2], and 96.5%, 99.9%, 99.9%*, and 99.9% for the prevalence of ever indoor tanning among adults in the United States, Northern and Western Europe, Australia, and all areas combined, respectively . For example, the estimated prevalence of ever indoor tanning ranged from 19% to 74% in the United States, and from 11% to 64% in Northern and Western Europe. Further, crude categorization of ever vs. never exposure results in conflation of different levels of exposure with, presumably, different degrees of risk.
In the presence of such substantial heterogeneity, a single summary estimate may not be scientifically meaningful [4, 5]. A random-effects summary estimate provides an average of results across studies, but that estimate may not reflect the actual result in any study population and may not be applicable to any real population. Using a random-effects model does not overcome study heterogeneity or circumvent the need to explore potential sources of such heterogeneity. Moreover, using the 95% confidence interval around a random-effects point estimate, as was done with the estimates of indoor tanning prevalence to calculate a range in the number of attributable cases of skin cancer , does not take account of study heterogeneity. As stated in the Cochrane Handbook : “The confidence interval from a random-effects meta-analysis describes uncertainty in the location of the mean of systematically different effects in the different studies. It does not describe the degree of heterogeneity among studies as may be commonly believed.”
Given the high degree of heterogeneity in prevalence estimates of indoor tanning, with I2 consistently > 90% and p-values for heterogeneity consistently < 0.001 , how should one interpret the data? At a minimum, given that the quality of a meta-analysis is only as good as that of the contributing studies, it is important to evaluate the individual studies to consider potential sources of bias. To assess the internal validity and external generalizability of the 88 underlying studies included in the Wehner et al. (2014) meta-analysis , we systematically reviewed the source populations, recruitment methods, and participation rates reported in those studies. We found that prevalence estimates from the majority of these studies were based on highly selected or non-representative populations. These source populations call into question whether the results from these studies can be generalized to the entire populations of the United States, Northern and Western Europe, or Australia. Furthermore, low participation rates and non-randomized sampling methods in many studies likely resulted in biased findings. Publication bias was also evident , with preferential publication of studies reporting a higher prevalence of indoor tanning, further undermining the validity of the meta-analysis results.
Regardless of whether meta-analysis results are scientifically informative in the presence of extreme heterogeneity, the estimates of the number of skin cancer cases attributable to indoor tanning  do not adequately account for this heterogeneity or for the uncertainty in the underlying data. The estimates were calculated from three other types of estimates: the prevalence of indoor tanning among adults in each geographic region, the relative risks of non-melanoma skin cancer and melanoma associated with indoor tanning, and the annual incidence of non-melanoma skin cancer and melanoma in each geographic region. In estimating the number of attributable skin cancer cases, Wehner et al.  took only the 95% confidence intervals around the tanning prevalence estimates into account, ignoring the confidence intervals around the relative risk estimates (1.29 to 2.17 for squamous cell carcinoma, 1.08 to 1.53 for basal cell carcinoma , and 1.09 to 1.43 for melanoma ). The annual cancer incidence estimates also have inherent uncertainty, although confidence intervals appear not to have been reported by the sources relied upon by Wehner et al. . Thus, the reported 95% confidence intervals around the estimated number of skin cancer cases attributable to indoor tanning are not true confidence intervals because they do not incorporate the uncertainty in the relative risk and cancer incidence estimates. Furthermore, as stated earlier, the meta-analysis confidence intervals describe only statistical error; they do not describe the extent of study heterogeneity. In other words, the estimates of attributable skin cancer cases are much more uncertain and unstable than reported and do not provide a valid estimate of the true prevalence (if there is a single prevalence) of indoor tanning in the general population.
The relationship between ultraviolet radiation exposure and human health is complex, as is the scientific research on this topic. We are currently undertaking a systematic effort to develop a consensus methodology for characterizing research methods on ultraviolet radiation and human health, so that stakeholders in this research—including policymakers, legislators, regulators, clinicians, and public and commercial entities—can better evaluate the utility of studies for their decision-making objectives. All interested parties are invited to participate in this effort. Our hope is that consumers of this research will be better able to understand and interpret the science, including studies such as these.
* Based on the prevalence estimates of 11% in all three included studies, the reported I2 estimate of 99.9% for adults in Australia may be incorrect.
1. Wehner MR, Shive ML, Chren MM, Han J, Qureshi AA, Linos E. Indoor tanning and non-melanoma skin cancer: systematic review and meta-analysis. BMJ 2012;345:e5909.
2. Boniol M, Autier P, Boyle P, Gandini S. Cutaneous melanoma attributable to sunbed use: systematic review and meta-analysis. BMJ 2012;345:e4757.
3. Wehner MR, Chren MM, Nameth D, Choudhry A, Gaskins M, Nead KT, Boscardin WJ, Linos E. International prevalence of indoor tanning: a systematic review and meta-analysis. JAMA Dermatol 2014;150(4):390-400.
4. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.
5. Shapiro S. Meta-analysis/Shmeta-analysis. Am J Epidemiol 1994;140(9):771-8.
Competing interests: Ms. Kuehn and Dr. Chang are employed by Exponent, Inc., a consulting company that received funding for this work from the American Suntanning Association.