Vitamin D supplementation to prevent acute respiratory tract infections: systematic review and meta-analysis of individual participant data
BMJ 2017; 356 doi: https://doi.org/10.1136/bmj.i6583 (Published 15 February 2017) Cite this as: BMJ 2017;356:i6583
All rapid responses
I thank Professor Colquhoun for taking the time to comment on our work. I am happy to be able to reassure him that there was no breakdown in the editorial process, and that – unlike the accompanying editorial – our article was subjected to independent peer review.
As Professor Colquhoun will be aware, absolute risk reduction is commonly represented by presenting the ‘Number needed to treat’ (NNT) (the inverse of the absolute risk reduction), which in the case of our study represents the average number of people who need to take a vitamin D supplement in order to prevent one individual from having at least one acute respiratory infection (ARI). I draw his attention to the panel on the bottom left of page 1 of the article, where NNT data are presented. The 2% absolute risk reduction alluded to by Avenell and Bolland is a raw value. More accurate estimates of absolute effect size for the study population as a whole are generated from adjusted odds ratios (aOR) as presented in the summary of findings table (supplementary Table S3), which shows a reduction in risk of having at least one ARI from 42.2% to 39.1% using the adjusted odds ratio (aOR) of 0.88 generated by the one-step IPD meta-analysis. Using the aOR from the two-step IPD meta-analysis (0.80, 95% CI 0.69 to 0.93), the absolute risk reduction is from 42.2% to 36.9%. The corresponding NNTs are 33 for the estimate from one-step IPDMA and 19 for the estimate from two-step IPDMA. To put these numbers into context, Cochrane reviews report the NNT to prevent one influenza-like illness with parenteral inactivated influenza vaccine to be 40 in adults (1) and 28 in children (2). Influenza vaccination programmes are motivated by the principle that when a disease is common, even minor reductions in incidence can have significant public health benefits; vitamin D fortification programmes might well be motivated by the same principle, particularly given that ARI is a major cause of industrial absenteeism, health service use and antibiotic prescribing (3). Approximately 30% of the UK population is estimated to be vitamin D deficient in winter and spring (4) – in this group the NNT with daily/weekly vitamin D supplementation to prevent one ARI falls to four. This represents a large protective effect by any standards.
I am fully in agreement with Professor Colquhoun that the press are responsible for the publicity this article has received. I can only presume that this is because the statistics above suggest that our findings are of potential importance for public health.
1. Demicheli V, Rivetti D, Deeks JJ, Jefferson TO. Vaccines for preventing influenza in healthy adults. Cochrane Database Syst Rev. 2004(3):CD001269.
2. Jefferson T, Rivetti A, Di Pietrantonj C, Demicheli V, Ferroni E. Vaccines for preventing influenza in healthy children. Cochrane Database Syst Rev. 2012(8):CD004879. Epub 2012/08/17.
3. Fendrick AM, Monto AS, Nightengale B, Sarnes M. The economic burden of non-influenza-related viral respiratory tract infection in the United States. Arch Intern Med. 2003;163(4):487-94. Epub 2003/02/18.
4. Cashman KD, Dowling KG, Skrabakova Z, Gonzalez-Gross M, Valtuena J, De Henauw S, et al. Vitamin D deficiency in Europe: pandemic? Am J Clin Nutr. 2016;103(4):1033-44. Epub 2016/02/13.
Competing interests: I am the corresponding author of the paper on which Professor Colquhoun has commented.
Martineau et al conducted a systematic review and meta-analysis of individual participant data from randomized controlled trials on the efficacy of vitamin D supplementation to prevent acute respiratory tract infections (1). A meta-analysis of almost 11,000 individuals across 25 randomised clinical trials provided a summary estimate, using a random effects meta-analysis model, of an adjusted odds ratio of 0.88 (95% confidence interval [95%CI] 0.81 to 0.96) providing evidence that supplementation with vitamin D may indeed lead to a reduction in risk of acute respiratory tract infections. The authors estimate that the number needed to treat to prevent a respiratory tract infection is 31 (1).
The article by Martineau et al (1) has received widespread media attention, indeed the authors themselves state that on the basis of their analyses, “These findings support the introduction of public health measures such as food fortification to improve vitamin D status, particularly in settings where profound vitamin D deficiency is common”.
The central findings are reported in the forest plot shown in Figure 2 of the main manuscript (1). Of note, the trials with the larger number of cases of acute respiratory tract infections tended to have the smallest magnitude of effects (i.e. closest to the null), which can point towards presence of small study bias (i.e. publication bias, a major problem in the scientific literature (2)). To investigate this, the authors conduct a funnel plot (shown in Figure S5 in the supplement), and report this as showing a “degree of asymmetry”. Interestingly, the presence of small study bias isn’t tested formally, using methods such as Egger’s test (3).
The meta-analysis performed by Martineau et al (1) was conducted using random effects modelling, which can give inflated summary estimates in the presence of small study bias (4). Indeed, when using the trial-specific adjusted odds ratios and corresponding 95%CI reported by the authors(1) in Figure 2, I obtain the same overall summary effect estimate as the authors when using random-effects modelling (adjusted OR 0.80; 95%CI: 0.69, 0.93; https://twitter.com/mvholmes/status/832359268938547202). When using fixed-effect modelling, the OR is closer to the null: adjusted OR 0.88 (95%CI: 0.81, 0.96).
An Egger test(3) for small study bias provided evidence against the null hypothesis (of no presence of small study bias; P=0.01), and on grouping trials by the number of cases, from >0 to <100, >=100 to <500 and >=500, there was a striking diminution in the summary effect estimates from small, medium to large trials (https://twitter.com/mvholmes/status/832361337401516032). Eleven small trials (<100 cases reported in each trial) with a total 577 acute respiratory tract infections had a summary OR for the effect of vitamin D supplementation on acute respiratory tract infections of 0.59 (95%CI: 0.47, 0.74) for fixed-effect and OR 0.57 (95%CI: 0.41, 0.79) for random-effects modelling. Ten medium sized trials (>=100 to <500 cases per trial) with a total 1920 acute respiratory tract infections had a corresponding summary OR of 0.87 (95%CI: 0.75, 1.00) for fixed-effect and 0.87 (95%CI: 0.71, 1.06) for random effects modelling. The three largest trials (>=500 cases per trial) with a total 1980 cases of acute respiratory tract infection had a summary OR of 1.00 (95%CI: 0.88, 1.13) from both fixed and random-effects modelling, with no heterogeneity identified between the three large trials (I2=0%).
When limiting the analysis to the largest trials, there is no evidence that supplementation with vitamin D leads to a reduction in the incidence of acute respiratory tract infections. The data therefore do not support recommendations for population-wide supplementation with vitamin D to prevent acute respiratory tract infections.
References:
1: Martineau AR, Jolliffe DA, Hooper RL et al. Vitamin D supplementation to prevent acute respiratory tract infections: systematic review and meta-analysis of individual participant data. BMJ 2017;356:i6583
2: Ioannidis JPA. Why Most Published Research Findings Are False. PLoS Medicine 2005 2: e124.
3: Egger M, Davey Smith G, Schneider M and Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997;315:629
4: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.handbook.cochrane.org.
Competing interests: No competing interests
This result was reported in many media this morning, despite the fact that the paper shows very little. The paper appears to disobey BMJ guidelines by failing to give the absolute risks in the abstract. There seems to have been a breakdown in the editorial process,
I suspected the press might be responsible for all the publicity for a paper that contributes very little. The press release does cite the excellent editorial (by Boland and Avenell) that accompanied the paper. but it seems that journalists didn't bother to follow the link. The reporting might have been better if the press release had quoted the editorial:
"the primary result is a reduction from 42% to 40% in the proportion of participants experiencing at least one acute respiratory tract infection. It seems unlikely that the general population would consider a 2% absolute risk reduction sufficient justification to take supplements. "
Of course had that been done, the paper might not have been reported in the media at all.
Competing interests: No competing interests
Media reporting of the vitamin D story
As a former national newspaper science writer, I'm familiar with journalists being blamed for "hyping" academic research findings. In their defence, however, I would point out that the paper by Martineau et al reports findings of widespread interest on a public health issue and appears in a leading peer-reviewed journal. Furthermore, journalists are expected to seek comment from experts independent of the authors, and in this specific case there was no shortage of experts willing to comment positively (see eg http://bit.ly/2kQApZJ).
Journalists who still decline to report the findings on the debatable grounds that the overall NNT is somewhat on the high side can expect a career-limiting "frank exchange" with their news editor when it turns out every other news outlet has reported it.
Those journalists who went through due diligence and did cover the story, in contrast, have brought this public health issue to the attention of the public at large, which is surely what journalists should be doing.
Competing interests: No competing interests