Intended for healthcare professionals

Editorials

Developing reliable dietary guidelines

BMJ 2017; 359 doi: https://doi.org/10.1136/bmj.j4845 (Published 03 November 2017) Cite this as: BMJ 2017;359:j4845
  1. Lisa Bero, professor
  1. Charles Perkins Centre, University of Sydney, New South Wales, Australia
  1. lisa.bero{at}sydney.edu.au

Robust measures to reduce bias and improve methods are required

The National Academy of Sciences (NAS) recently published two reports critiquing the process used to develop the US dietary guidelines.12 The reports identify important deficiencies in the process, including a lack of transparency in forming guideline committees, problems with the methods of systematic reviews underpinning guidelines, and a lack of clarity about how evidence is finally translated into public health recommendations. These deficiencies have received considerable attention already. Our recent analysis of the methods used to synthesise evidence and grade recommendations in 32 national food based dietary guidelines found similar problems, suggesting that improvements are required globally.3

The NAS reports’ headline recommendation is a complete redesign of the guideline development process into three steps conducted by separate groups. Federal staff and consultants would start by gathering data—conducting new systematic reviews and identifying existing reviews, meta-analyses, and descriptive datasets. A dietary guidelines scientific advisory committee would then evaluate and integrate all this evidence and develop recommendations for a scientific report. Finally, government employees would use this report to draft policy recommendations in the Dietary Guidelines for Americans.

Although a redesign may help resolve some of the problems, it could introduce others. Systematic reviews in nutrition are complicated, and excluding systematic reviewers from guideline development groups could hinder the group’s understanding of the heterogeneity and other key characteristics of the data. Developers of dietary guidelines would benefit from an expert’s deeper understanding of systematic review methods, including meta-analysis of animal and mechanistic data, risk of bias assessment, “living systematic reviews,” and network meta-analysis.

Tackling bias

Furthermore, the proposed redesign does not resolve well documented biases in primary nutritional studies, such as those related to funding source and author conflicts of interest.45 Dietary guidelines, like other public health guidelines, are influenced by political, economic, and social factors. But if the primary evidence is biased, the entire foundation for systematic reviews, guidelines, food regulation, clinical advice, and consumer information crumbles. Bias in dietary guidelines cannot be eliminated without tackling bias in the evidence used to support them.

Bias in the research agenda, possibly steered by the financial interests of funders, can also influence dietary guidelines. An analysis of 470 Cochrane reviews found that most evaluated supplements and micronutrients, reflecting primary trials that focus more on specific nutrients, such as a single vitamin or fat subtype, than on whole food groups or diets.67 Cochrane Nutrition is one effort aimed at improving the evidence base for dietary guidelines by supporting global coordination of systematic reviews in nutrition and strengthening the methods of both reviews and primary studies.8

The NAS reports make no specific recommendations about the final critical step in developing guidelines—translating evidence into recommendations. Guideline developers should be aware of the many available options, ranging from consensus approaches, through structured consensus methods such as GRADE,9 to quantitative (including bayesian) methods for rating the certainty of a recommendation.

Public health guidelines differ from clinical practice guidelines because they include more diverse types of evidence. Most of the focus to date has been on reducing bias in clinical practice guidelines,10 although the same standards should clearly apply to both. In Australia, this gap is being tackled by the National Health and Medical Research Council through an advisory group on the synthesis and translation of research evidence (STORE). The group was established to improve the development and implementation of guidelines and systematic reviews as well as decision making.11 STORE’s remit includes improving guidelines in environmental and public health.12

Those developing dietary guidelines could learn from other public health areas facing similar challenges with diverse evidence to consider.1314 For example, the NAS committee convened to review the Environmental Protection Agency process for environmental risk assessment recommended that the agency develop protocols for all systematic reviews, use empirically based tools for assessing risks of bias in different types of studies, incorporate of risk of bias assessments into the evaluation of evidence within each systematic review, use GRADE-like structured processes to formulate conclusions from evidence integration, and explore quantitative methods for evidence integration. US Congress acted on these recommendations, mandating that the EPA must now use systematic reviews in all environmental risk assessments.15

Nutrition researchers and guideline developers should continue to champion efforts to reduce bias such as open data access, study registries, better reporting, and more transparency regarding funding sources and conflicts of interest. It is unclear how the redesign recommended by the NAS will reduce bias in dietary guidelines, but it will substantially increase their cost. The 2015 US guidelines cost about $905 000 (£690 000; €770 000) plus an estimated 22.2 fulltime equivalents of staff over two years.2 These resources would be better spent on initiatives to reduce bias in nutrition research and to develop new and improved methods for evaluating and synthesising diverse types of evidence.

Footnotes

  • Competing interests: I have read and understood BMJ policy on declaration of interests and declare that I am a member of the NHMRC advisory group on the synthesis and translation of research evidence. I was a member of the National Research Council committee that published Review of EPA’s Integrated Risk Information System (IRIS) Process. I was cochair of the Cochrane governing board from 2013 to 2017, for which the University of Sydney received remuneration.

  • Provenance and peer review: Commissioned; not externally peer reviewed.

References

View Abstract