Implausible results in human nutrition research

BMJ 2013; 347 doi: https://doi.org/10.1136/bmj.f6698 (Published 14 November 2013) Cite this as: BMJ 2013;347:f6698

Re: Implausible results in human nutrition research

Dear Editors,

We read with great interest the article by Dr Ioannidis. He describes accurately many of the difficulties and disappointments of research results when trying to causally connect nutrition to risk of chronic diseases. Despite that, we believe that some of his conclusions may be overstated and premature. He may be correct that strong effects are improbable for many individual nutrients, a focus of much previous work. However, important effects for some foods/food groups are still entirely possible. On theoretical grounds, important effects on risk may be present and yet be very difficult to consistently detect with commonly used analytic approaches. Measurement errors are often large, and when statistical models have many covariates measured with error, residual confounding may often seriously bias effect estimates, usually toward to null.(1,2) Another issue is that care is needed with specific interpretation of effects from observational studies given that any exposure may be confounded by unmeasured variables that are tightly linked to it. Insufficient attention to these issues may largely explain the inconsistent results that Ioannidis highlights. We agree with him that large studies address only precision and not bias.

Nevertheless, observational studies have produced many results with sufficient consistency to allow consensus labels of “probable” referring to causal associations. For instance, there is in our opinion quite strong evidence from observational studies that red meats (especially processed meats) increase the risk of colorectal cancer, that nut consumption reduces risk of cardiovascular disease, and that some classes of fatty acids also affect risk of cardiovascular disease. The few intervention trials of nuts or Mediterranean diets may have produced some over-optimistic results, as stated by Ioannidis, yet these observational and trial results should not be dismissed so quickly. Given the largely consistent results from observational studies, despite the impediments mentioned above, and the supportive results from trials in selected populations, tentative conclusions that important effects do exist seem more prudent than a dismissal.

As another example of interest, our observational findings showing that American Adventist vegetarians have lower risk than their non-vegetarian counterparts for total mortality, cardiovascular disease, diabetes, and certain cancers are now consistent across two large American cohorts.(3-6) These results seem un-confounded by other non-dietary factors that we can identify. Although studies of dietary patterns do not identify specific dietary components that influence risk, they imply that such factors probably exist. It is a strong assumption that effects of dietary patterns result only from the sum of many very small and individually unidentifiable effects. In our view it is more likely that among these effects there will be some that stand out and be of greater interest. To conclude prematurely that diet has only small effects on risks of chronic disease would be a serious public health failure given the low cost of such preventive measures and the very low risk of side effects.

Although there are acknowledged problems with observational studies, in our view they are still our best hope to advance our understanding. Further efforts to minimize measurement errors and their effects (e.g. repeated dietary assessments perhaps using hand-held electronic technology, use of biomarkers in regression calibration etc) and the use of newer methods that improve control of confounding are ways forward. The mega-trials that Ioannidis proposes promise to be exceedingly expensive if used to investigate dietary effects on risk of cancer, for instance, and very likely infeasible. The intervention would need to last for a minimum of 10 years. We do not know how to effectively intervene on such a complex behavior for such a long period. Attempts to succeed would likely involve such highly selected subjects, perhaps with respect to socioeconomic status and baseline health, that broader application of results may be difficult. Even if it were possible to effectively intervene for at least a decade, the intervention would always be “unnatural” and the natural motivations for the diet and behavioral and sociologic associations with the diet, would be missing. Natural populations will often have subscribed to an approximately consistent dietary pattern for many decades, if not the whole of life. Another serious limitation of intervention studies is that only one (or perhaps two) interventions can be investigated, whereas observational studies can evaluate hypotheses about many potential causes.

In summary, it seems to us that observational studies continue to have great value. Although they will not individually produce “conclusive” results, mainly consistent results across such studies will certainly suggest causal effects. These should be followed by basic science investigations to see whether mechanisms can or cannot be found to support a causal hypothesis. Intervention trials may or may not be possible depending on the complexity of the necessary dietary intervention and its required duration. Trials would be more valuable if they are large enough to incorporate a naturally diverse population of subjects while retaining adequate power. At present this does not seem realistic.

1. Day NE, Wong MY, Bingham S, Khaw KT, Luben R, Michels KB et al. Correlated measurement error—Implications for nutritional epidemiology. Int J Epidemiol2004; 33:1373-81.
2. Schatzkin A, Kipnis V, Carroll RJ, Midthune D, Subar AF, Bingham S, Schoeller DA, Troiano RP, Freedman LS. A comparison of a food frequency questionnaire with a 24-hour recall for use in an epidemiological cohort study: results from the biomarker-based Observing Protein and Energy Nutrition (OPEN) study. Int J Epidemiol. 2003 Dec;32(6):1054-62.
3. Fraser GE. Diet, life expectancy and chronic disease. Studies of the health of vegetarians. Oxford University Press, New York, 2003.
4. Tonstad S, Stewart K, Oda K, Batech M, Herring RP, Fraser GE. Vegetarian diets and incidence of diabetes in the Adventist Health Study-2. Nutr Metab Cardiovasc Dis [Internet]. 2013 Apr;23(4):292–9. Available from: http://dx.doi.org/10.1016/j.numecd.2011.07.004
5. Tantamango-Bartley Y, Jaceldo-Siegl K, Fan J, Fraser GE. Vegetarian diets and the incidence of cancer in a low-risk population. Cancer Epidemiology Biomarkers & Prevention. 2013 Feb;22(2):286–94.
6. Orlich MJ, Singh PN, Sabaté J, Jaceldo-Siegl K, Fan J, Knutsen S, Beeson WL, Fraser GE. Vegetarian dietary patterns and mortality in Adventist Health Study 2. JAMA Intern Med. 2013 Jul 8;173(13):1230-8. doi: 10.1001/jamainternmed.2013.6473. PMID:23836264 [PubMed - indexed for MEDLINE]

Competing interests: No competing interests

16 December 2013
Gary E Fraser
nutritional epidemiologist/cardiologist
Michael Orlich, Synnove Knutsen, Pramil Singh, Joan Sabate
Loma Linda University
Nichol Hall, Loma Linda University, Loma Linda, CA 92350 USA