Indoor tanning and non-melanoma skin cancer: systematic review and meta-analysisBMJ 2012; 345 doi: https://doi.org/10.1136/bmj.e5909 (Published 02 October 2012) Cite this as: BMJ 2012;345:e5909
All rapid responses
This article  and another article published in BMJ  were relied upon in a recently published meta-analysis of the international prevalence of indoor tanning . Based on the summary relative risks of non-melanoma skin cancer and melanoma associated with indoor tanning, estimated from meta-analyses of 12 studies  and 27 studies , respectively, and using summary prevalence estimates of ever exposure to indoor tanning among adults in the United States, Northern and Western Europe, and Australia from a total of 18 studies, Wehner et al.  estimated that more than 450 000 non-melanoma skin cancer cases and more than 10 000 melanoma cases in these regions are attributable to indoor tanning each year.
We are writing to express concerns with the methods used to derive these estimates of attributable skin cancer cases. One problem lies with the use of meta-analytic methods to combine heterogeneous observational data into a single summary estimate. Appreciable heterogeneity was observed across studies included in these meta-analyses, as indicated by I2 values of 36.8%, 47.1%, and 56% for the associations between indoor tanning and basal cell carcinoma, squamous cell carcinoma, and melanoma, respectively [1, 2], and 96.5%, 99.9%, 99.9%*, and 99.9% for the prevalence of ever indoor tanning among adults in the United States, Northern and Western Europe, Australia, and all areas combined, respectively . For example, the estimated prevalence of ever indoor tanning ranged from 19% to 74% in the United States, and from 11% to 64% in Northern and Western Europe. Further, crude categorization of ever vs. never exposure results in conflation of different levels of exposure with, presumably, different degrees of risk.
In the presence of such substantial heterogeneity, a single summary estimate may not be scientifically meaningful [4, 5]. A random-effects summary estimate provides an average of results across studies, but that estimate may not reflect the actual result in any study population and may not be applicable to any real population. Using a random-effects model does not overcome study heterogeneity or circumvent the need to explore potential sources of such heterogeneity. Moreover, using the 95% confidence interval around a random-effects point estimate, as was done with the estimates of indoor tanning prevalence to calculate a range in the number of attributable cases of skin cancer , does not take account of study heterogeneity. As stated in the Cochrane Handbook : “The confidence interval from a random-effects meta-analysis describes uncertainty in the location of the mean of systematically different effects in the different studies. It does not describe the degree of heterogeneity among studies as may be commonly believed.”
Given the high degree of heterogeneity in prevalence estimates of indoor tanning, with I2 consistently > 90% and p-values for heterogeneity consistently < 0.001 , how should one interpret the data? At a minimum, given that the quality of a meta-analysis is only as good as that of the contributing studies, it is important to evaluate the individual studies to consider potential sources of bias. To assess the internal validity and external generalizability of the 88 underlying studies included in the Wehner et al. (2014) meta-analysis , we systematically reviewed the source populations, recruitment methods, and participation rates reported in those studies. We found that prevalence estimates from the majority of these studies were based on highly selected or non-representative populations. These source populations call into question whether the results from these studies can be generalized to the entire populations of the United States, Northern and Western Europe, or Australia. Furthermore, low participation rates and non-randomized sampling methods in many studies likely resulted in biased findings. Publication bias was also evident , with preferential publication of studies reporting a higher prevalence of indoor tanning, further undermining the validity of the meta-analysis results.
Regardless of whether meta-analysis results are scientifically informative in the presence of extreme heterogeneity, the estimates of the number of skin cancer cases attributable to indoor tanning  do not adequately account for this heterogeneity or for the uncertainty in the underlying data. The estimates were calculated from three other types of estimates: the prevalence of indoor tanning among adults in each geographic region, the relative risks of non-melanoma skin cancer and melanoma associated with indoor tanning, and the annual incidence of non-melanoma skin cancer and melanoma in each geographic region. In estimating the number of attributable skin cancer cases, Wehner et al.  took only the 95% confidence intervals around the tanning prevalence estimates into account, ignoring the confidence intervals around the relative risk estimates (1.29 to 2.17 for squamous cell carcinoma, 1.08 to 1.53 for basal cell carcinoma , and 1.09 to 1.43 for melanoma ). The annual cancer incidence estimates also have inherent uncertainty, although confidence intervals appear not to have been reported by the sources relied upon by Wehner et al. . Thus, the reported 95% confidence intervals around the estimated number of skin cancer cases attributable to indoor tanning are not true confidence intervals because they do not incorporate the uncertainty in the relative risk and cancer incidence estimates. Furthermore, as stated earlier, the meta-analysis confidence intervals describe only statistical error; they do not describe the extent of study heterogeneity. In other words, the estimates of attributable skin cancer cases are much more uncertain and unstable than reported and do not provide a valid estimate of the true prevalence (if there is a single prevalence) of indoor tanning in the general population.
The relationship between ultraviolet radiation exposure and human health is complex, as is the scientific research on this topic. We are currently undertaking a systematic effort to develop a consensus methodology for characterizing research methods on ultraviolet radiation and human health, so that stakeholders in this research—including policymakers, legislators, regulators, clinicians, and public and commercial entities—can better evaluate the utility of studies for their decision-making objectives. All interested parties are invited to participate in this effort. Our hope is that consumers of this research will be better able to understand and interpret the science, including studies such as these.
* Based on the prevalence estimates of 11% in all three included studies, the reported I2 estimate of 99.9% for adults in Australia may be incorrect.
1. Wehner MR, Shive ML, Chren MM, Han J, Qureshi AA, Linos E. Indoor tanning and non-melanoma skin cancer: systematic review and meta-analysis. BMJ 2012;345:e5909.
2. Boniol M, Autier P, Boyle P, Gandini S. Cutaneous melanoma attributable to sunbed use: systematic review and meta-analysis. BMJ 2012;345:e4757.
3. Wehner MR, Chren MM, Nameth D, Choudhry A, Gaskins M, Nead KT, Boscardin WJ, Linos E. International prevalence of indoor tanning: a systematic review and meta-analysis. JAMA Dermatol 2014;150(4):390-400.
4. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.
5. Shapiro S. Meta-analysis/Shmeta-analysis. Am J Epidemiol 1994;140(9):771-8.
Competing interests: Ms. Kuehn and Dr. Chang are employed by Exponent, Inc., a consulting company that received funding for this work from the American Suntanning Association.
We thank Dr. Tunnicliff for her thoughtful comments. We agree that it was important to use a very conservative method to synthesize these observational studies that were predominantly small, retrospective, and case-control in design. We used a random-effects model to accomplish this goal; the χ2 p-values and I2 values were included for completeness but did not affect our choice of model.
Dr. Tunnicliff mentioned that some original studies were "less than optimum" and certainly conclusions of meta-analyses are always limited by the quality of included studies. We focused on what we judged to be the most significant sources of bias when combining effect estimates: lack of consistency in the exposure measurement and differences in adjustment for confounders. We combined studies based on a consistent exposure measure: 'ever exposure to indoor tanning.' As we noted, the primary analysis excluded the 2 of 12 studies that did not report an ever-exposure estimate. Also, the conclusions did not change in sensitivity analyses that excluded studies that did not fully adjust for confounders.
As Dr. Tunnicuff mentioned, observational data from different studies should be combined with caution, and we believe our conservative approach represents the most accurate synthesis of results currently available.
Competing interests: No competing interests
The authors report the X² test for heterogeneity was non-significant for both BCC (p=0.14) and SCC (p=0.09). This would infer that a significance level of p<0.05 has been selected, rather than p<0.1 or p<0.2, usually recommended for the test for heterogeneity because it is conservative. The reported I² statistics suggest that there is no evidence of heterogeneity, however the width of the confidence intervals indicates an imprecise estimate and therefore uncertainty about these results. Despite reporting no evidence of heterogeneity, the authors adopted a random effects model (used to limit heterogeneity) to calculate the summary statistics. The reasons for selecting this model over a fixed-effects model are not given. This made us wonder about the original studies.
A brief review showed that the original studies are less than optimum. For example: Two of the studies only included nurses as the study participants (Zhang et al. 2012; Han et al. 2006); within studies there was a lack of clarity, and across studies there was a lack of consistency about how exposure to indoor tanning was measured; the potential extent of recall bias due to various attempts to measure lifetime exposure to indoor tanning and other variables. These factors are not highlighted in the report.
This study addresses an important public health question. However, while we agree with the conclusions, some caution should be taken in assuming population effects from such a diverse and variable set of studies.
Competing interests: No competing interests