Outcome of Post-publication review of article by Nina Teicholz
In response to concerns raised about this article[1] by the Center for Science in the Public Interest (CSPI),[2] [3] we asked Cochrane to recommend experts in systematic reviews of evidence for guidelines who might undertake a formal post-publication review. Two people were recommended to us, and through initial correspondence with them we identified two others. Two of these four agreed to review the article: Professor Lisa Bero, Chair of Medicines Use and Health Outcomes at the University of Sydney, and Professor Mark Helfand, Professor of Medical Informatics and Clinical Epidemiology at the Oregon Health & Science University. We asked each of them to pay special attention to the points raised in the CSPI’s letter, to give us their views on whether we should retract Teicholz’s article, and to allow their signed reviews to be published. Our letter to them and their reviews are available on thebmj.com [see attachments below].
Both reviewers agreed with the authors of the CSPI letter in finding that the report of the Dietary Guidelines Advisory Committee (DGAC) described methods (such as a search strategy and predefined inclusion criteria) for the selection of evidence for its report. But they also noted problems with the committee’s methods and rejected the letter’s contention that Teicholz’s article should be retracted. The problems noted by the reviewers included the committee’s methods being out of date and lacking sufficient detail, which could have introduced bias.
Bero describes how some of the reviews by the DGAC “circumvent the systematic search process,” and how, “regarding the search strategies, inclusion of studies and quality assessment,” there are aspects of the methods of the DGAC report that “sometimes lack sufficient detail and may introduce bias.” She concludes, “Teicholz’s criticisms of the methods used by DGAC are within the realm of scientific debate.” Helfand says: “I found nothing to contradict Teicholz’ central concern that the DGAC’ processes to protect against bias are inadequate. It is clear that further investigation of the composition of the committee, as well as its conflict of interest policies and work group structure, are warranted. The NEL and DGAC do not appear to have incorporated key developments in methodology and governance of evidence-based guideline development since 2010. The DGAC’s role in grading the evidence and the lack of an evidence to decision framework are examples of practices that may fall below the current international standard for conducting systematic reviews.”
On the basis of these reviews and our own internal assessment of the issues raised, we find no grounds for retraction of the article. We did find that some of the points raised by the CSPI merited either correction or clarification, and we have published a notice to that effect: http://www.bmj.com/content/355/bmj.i6061. Nina Teicholz has provided a response to the post publication review of her article, which can be found here, http://www.bmj.com/content/351/bmj.h4962/rr-49.
We stand by Teicholz’s article and its critique of this highly influential advisory committee’s processes for reviewing the evidence, and we echo her conclusion: “Given the ever increasing toll of obesity, diabetes, and heart disease, and the failure of existing strategies to make inroads in fighting these diseases, there is an urgent need to provide nutritional advice based on sound science.”
Neither Teicholz nor The BMJ are new to criticism. Healthcare is rife with controversy, and the field of nutrition more so than many, characterised as it is by much weak science, polarised opinion, and powerful commercial interests.[4] But nutrition is perhaps one of the most important and neglected of all health disciplines, traditionally relegated to non-medical nutritionists rather than being, as we believe it deserves to be, a central part of medical training and practice. The current state of nutrition research should be a matter of grave concern to those attempting to develop evidence based health and economic policies that truly serves the public interests. The BMJ plans to continue to provide a forum for debate on the science and politics of food; and is collaborating with researchers at the University of Cambridge and Tufts University in Massachusetts on a series of articles examining the science and politics of food, which is due to be published next year.
Rapid Response:
Outcome of Post-publication review of article by Nina Teicholz
In response to concerns raised about this article[1] by the Center for Science in the Public Interest (CSPI),[2] [3] we asked Cochrane to recommend experts in systematic reviews of evidence for guidelines who might undertake a formal post-publication review. Two people were recommended to us, and through initial correspondence with them we identified two others. Two of these four agreed to review the article: Professor Lisa Bero, Chair of Medicines Use and Health Outcomes at the University of Sydney, and Professor Mark Helfand, Professor of Medical Informatics and Clinical Epidemiology at the Oregon Health & Science University. We asked each of them to pay special attention to the points raised in the CSPI’s letter, to give us their views on whether we should retract Teicholz’s article, and to allow their signed reviews to be published. Our letter to them and their reviews are available on thebmj.com [see attachments below].
Both reviewers agreed with the authors of the CSPI letter in finding that the report of the Dietary Guidelines Advisory Committee (DGAC) described methods (such as a search strategy and predefined inclusion criteria) for the selection of evidence for its report. But they also noted problems with the committee’s methods and rejected the letter’s contention that Teicholz’s article should be retracted. The problems noted by the reviewers included the committee’s methods being out of date and lacking sufficient detail, which could have introduced bias.
Bero describes how some of the reviews by the DGAC “circumvent the systematic search process,” and how, “regarding the search strategies, inclusion of studies and quality assessment,” there are aspects of the methods of the DGAC report that “sometimes lack sufficient detail and may introduce bias.” She concludes, “Teicholz’s criticisms of the methods used by DGAC are within the realm of scientific debate.” Helfand says: “I found nothing to contradict Teicholz’ central concern that the DGAC’ processes to protect against bias are inadequate. It is clear that further investigation of the composition of the committee, as well as its conflict of interest policies and work group structure, are warranted. The NEL and DGAC do not appear to have incorporated key developments in methodology and governance of evidence-based guideline development since 2010. The DGAC’s role in grading the evidence and the lack of an evidence to decision framework are examples of practices that may fall below the current international standard for conducting systematic reviews.”
On the basis of these reviews and our own internal assessment of the issues raised, we find no grounds for retraction of the article. We did find that some of the points raised by the CSPI merited either correction or clarification, and we have published a notice to that effect: http://www.bmj.com/content/355/bmj.i6061. Nina Teicholz has provided a response to the post publication review of her article, which can be found here, http://www.bmj.com/content/351/bmj.h4962/rr-49.
We stand by Teicholz’s article and its critique of this highly influential advisory committee’s processes for reviewing the evidence, and we echo her conclusion: “Given the ever increasing toll of obesity, diabetes, and heart disease, and the failure of existing strategies to make inroads in fighting these diseases, there is an urgent need to provide nutritional advice based on sound science.”
Neither Teicholz nor The BMJ are new to criticism. Healthcare is rife with controversy, and the field of nutrition more so than many, characterised as it is by much weak science, polarised opinion, and powerful commercial interests.[4] But nutrition is perhaps one of the most important and neglected of all health disciplines, traditionally relegated to non-medical nutritionists rather than being, as we believe it deserves to be, a central part of medical training and practice. The current state of nutrition research should be a matter of grave concern to those attempting to develop evidence based health and economic policies that truly serves the public interests. The BMJ plans to continue to provide a forum for debate on the science and politics of food; and is collaborating with researchers at the University of Cambridge and Tufts University in Massachusetts on a series of articles examining the science and politics of food, which is due to be published next year.
References
1. http://www.bmj.com/content/351/bmj.h4962
2. http://cspinet.org/bmj-retraction-letter.html
3. http://www.bmj.com/content/351/bmj.h4962/rr-36
4. http://www.bmj.com/content/347/bmj.f6698
Competing interests: I am the editor of The BMJ and responsible for all it contains