Editorials

Implausible results in human nutrition research

BMJ 2013; 347 doi: https://doi.org/10.1136/bmj.f6698 (Published 14 November 2013) Cite this as: BMJ 2013;347:f6698

Re: Implausible results in human nutrition research

INTERPRETING NUTRITION RESEARCH – MORE COMPLEX THAN APPARENT PLAUSIBILITY

RESPONSE TO IOANNIDIS

Martin J Wiseman FRCP FRCPath FAfN (Corresponding author)
Visiting Professor of Human Nutrition

Alan A Jackson CBE MD FRCP FRCPath FRCPH FAfN
Professor of Human Nutrition

NIHR Biomedical Research Centre in Nutrition
Southampton General Hospital (MP 113)
Tremona Road
Southampton SO16 6YD, UK

There is much to applaud in Ioannidis’s plea for a more rigorous interpretation of nutrition research (1). However there are four areas that he fails to address adequately. As in all medicine it is important to be able to take decisions in the face of limited evidence. Furthermore, while biomedical evidence is necessary, is unlikely to be sufficient to address the challenges posed by complex problems rooted in sociocultural experience and behaviour – such as nutrition.

First, it is easy to finds among the extensive literature single studies that report almost any finding. The question is whether those findings represent the body of evidence. The problem has been addressed by World Cancer Research Fund (WCRF)/ American Institute for Cancer Research 2007 Expert Report (2) and continues to be a matter for specific consideration within the Continuous Update Project (3). This is a thorough and exhaustive review of the world literature as it relates to diet, nutrition, physical activity and cancer, and is founded on the need to view all results in the context of a synthesis of all the evidence available. Even with such a comprehensive approach, publication bias can distort the available information.

Second, the nature of the epidemiology is to throw up several associations, only some of which are selected for reporting. Ioannidis draws attention to the findings for specific nutrients, and though he alludes to the issue, does not emphasise sufficiently that these reported findings can only be interpreted as markers of a much more complex set of exposures (the “exposome”) which include, but cannot be limited to diet (4). Equally the nature of nutrition is far more complex than the linear pseudo-pharmacological model implied by Ioannidis. The naïve interpretation of such findings as representing a simple causal pathway rather than a component of and marker for an aspect of a complex integrated system has led to randomised controlled trials (RCTs) testing nutrient supplements, out of context. Unsurprisingly these trials have mostly not proved positive (and some have shown adverse effects) (5); while positive results from RCTs can demonstrate efficacy (with causality needing to be inferred) it is much more difficult to interpret negative results. A RCT may be negative for a variety of reasons other than the overall lack of efficacy of the intervention. It may have been tested on an atypical population (high risk) using atypical exposures (nutrient supplements at high dose) at the wrong time (late in disease aetiology) or for an insufficient period (weeks or months rather than years or decades).

This difficulty with RCTs highlights the third area, where Ioannidis implies that hypotheses generated from observational studies can be refuted by negative RCTs. For the reasons stated above this is fallacious. The hypotheses generated by observational studies are more complex, and difficult to address, than is possible in RCTs. Therefore RCTs tend to address hypotheses that are simplifications of those posed by the epidemiology and selected as being amenable to an RCT. All types of evidence have advantages and disadvantages, and the results always need to be incorporated into an overall synthesis which recognises the broader issues.

Finally it is important to understand the mechanistic underpinning of any observed associations. It is easy to find a mechanistic study that reports results that can be used to describe a plausible biological explanation, but even more than for epidemiology, the preclinical literature presents challenges of interpretation and is prone to publication bias. Interpretation of the mechanistic literature is particularly challenging because the experimental conditions for studies using animal models or in vitro systems are rarely identical to those already reported, so that the reproducibility of particular findings is often speculative. Further, the validity of specific animal models or in vitro systems to human metabolism is also often unclear. These issues (6) have been the reason for WCRF International to commission work to enable a systematic review of the mechanistic evidence linking nutritional exposures to cancer.

A major driver, or purpose, of nutrition research should be to improve health or management of disease. Ioannidis goes some way to helping that but only offers an incomplete interpretation. In clinical practice, we always have uncertainties in how we manage our patients but despite this, are not paralysed but use the art of professional practice to act in their best interests. This approach is equally valid in the field of public health. Bradford Hill in 1965 (7) set out his framework for inferring causality from observational studies. This was not a dry academic exercise, as it is sometimes portrayed, but had at its heart the need – when presented with a problem where RCTs were not plausible to conduct – to decide what action to take, on the best evidence available, however flawed. It may be that the current literature appears to overstate the impact of single nutrients, but the ecological variations in geographic distribution of disease and their plasticity with time suggest that the whole ”exposome” of nutrition and other environmental determinants is in fact not over- but underestimated by a naïve interpretation of the published literature.

Competing interests:
All authors have completed the Unified Competing Interest form, and have no interests to declare.

MJW is Medical and Scientific Adviser to WCRF International and was Project Director for the WCRF/AICR 2007 Expert Report.

AAJ was a member of the independent Expert Panel for the 2007 Expert Report and is Chair of the independent Expert Panel for the Continuous Update Project

References

1. Ioannidis J. Implausible results in human nutrition research. BMJ 2013;347:f6698

2. WCRF/AICR. Food, Nutrition, Physical Activity, and the Prevention of Cancer: a Global Perspective. Washington DC: AICR, 2007

3. WCRF. 3ww.wcrf.org/cancer_research/cup/ (accessed 3 December 2013)

4. Wild C P. The exposome: from concept to utility. Int J Epi 2012;41:24–32

5. Virtamo J, Pietinen P, Huttunen J K et al. Incidence of cancer and mortality following alpha-tocopherol and beta-carotene supplementation: a postintervention follow-up. JAMA 2003;290:476-85

6. Begley C G and Ellis L M. Raise standards for preclinical cancer research. Nature 2012;483:531-2

7. Hill A B. The environment and disease: association or causation? Proc R Soc Med 1965;58:295-300

Competing interests: No competing interests

04 December 2013
Martin J Wiseman
Physician
Alan A Jackson
University of Southampton
SGH, MP113, Tremona Road, Southampton SO16 6YD