Re: Vitamin D and multiple health outcomes: umbrella review of systematic reviews and meta-analyses of observational studies and randomised trials
Umbrella review of vitamin D and multiple health outcomes: Which way up was the umbrella…did this review collect knowledge or shield us from it?
Author John Ioannidis described umbrella reviews  as “subjective and suboptimal”, “limited by the data in the primary studies” and the process of “patching together pre-existing reviews”, thus open to subjective interpretations based on the authors opinions.
This umbrella review of Vitamin D by four authors including John Ioannidis  found a null result without exploring the reasons for their findings. The reason Vitamin D meta-analyses and reviews fail to produce useful results is thought to be biological flaws in primary studies [3, 4]. These flaws lead to null results as the intervention does not change the Vitamin D status; however these flaws may be overlooked when evaluating the research for Vitamin D and other nutrients [5, 6]. Furthermore, the methodology for differentiating flawed and non-flawed studies has been described by Heaney .
The review authors  describe flaws as “difficulties in relation to RCTs” such as “low dose vitamin D supplementation”, “large differences in baseline plasma concentrations of 25-hydroxyvitamin D” and “contamination with private use of vitamin D” in the randomised controlled trials that might be “inadequate to raise the body’s vitamin D concentrations enough to show a difference between the arms of a trial”. The review authors ascribe the concepts to Autier (2014)  although previously described by Heaney and others as biological flaws [5, 6]. Whilst this umbrella review  recognised these “difficulties” or biological flaws in their primary studies, the authors choose not to compare meta-analyses of flawed and unflawed primary studies.
The review authors  identify 107 systematic reviews and 74 meta-analyses with null and conflicting results, yet they have simply repeated the same process. Repeating the same thing over and over again expecting a different result has been described as insanity (a quote ascribed to Albert Einstein).
A previous review of Vitamin D  focused on studies without flaws concluded there was evidence to support the use of vitamin D supplementation in many non-skeletal diseases. Furthermore, there is a systematic review and meta-analysis on vitamin D supplementation and depression  that shows a positive result for primary studies without flaws, a negative result for studies with flaws and a null effect when studies are combined. These findings need validating by further research for depression and in other disorders.
Thus, there is a theoretical basis, a published methodology, and a meta-analysis to support the contention that Vitamin D reviews find and will continue to find null results unless primary studies with these ‘difficulties’ or biological flaws are excluded from the process.
The question remains why did these authors not present a review of flawed and unflawed studies in a review claimed to be all-encompassing?
1. Ioannidis JP. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. CMAJ2009;181:488-93.
2. Theodoratou E, Tzoulaki I, Zgaga L, Ioannidis JP. Vitamin D and multiple health outcomes: umbrella review of systematic reviews and meta-analyses of observational studies and randomised trials. BMJ2014 Apr 1;348:g2035. doi: 10.1136/bmj.g2035.
3. Scragg RDo we need to take calcium with vitamin D supplements to prevent falls, fractures, and death? Curr Opin Clin Nutr Metab Care. 2012 Nov;15(6):614-24. doi: 10.1097/MCO.0b013e328359ef21.
4. Lappe, J.M.; Heaney, R.P. Why randomized controlled trials of calcium and Vitamin D sometimes fail. Dermatoendocrinol2012, 4, 95-100.
5. Heaney RP. Vitamin D: basal status and effective dose. N Engl J Med2012; 367:77–78.
6. Biesalski, H.K.; Aggett, P.J.; Anton, R.; Bernstein, P.S.; Blumberg, J.; Heaney, R.P. Scientific substantiation of health claims: Evidence-based nutrition. 26th Hohenheim Consensus Conference; Nutr2011, 27, S1–S20.
7. Heaney RP. Guidelines for optimizing design and analysis of clinical studies of nutrient effects. Nutr Rev2014; 72: 48054.
8. Autier P, Boniol M, Pizot C, Mullie P. Vitamin D status and ill health: a systematic review. Lancet Diabetes Endocrinol2014; 2: 76-89.
9. Spedding S, Vanlint S, Morris H, Scragg R. Does Vitamin D Sufficiency Equate to a Single Serum 25-Hydroxyvitamin D Level or Are Different Levels Required for Non-Skeletal Diseases? Nutrients2013; 5(12):5127-5139.
109. Spedding S. Vitamin D and depression: A systematic review and meta-analysis comparing studies with and without biological flaws. Nutrients2014 in press.
Competing interests: No competing interests