I have reposted it to this article by Heneghan and colleagues (discussed in the box "What the research found" in Cohen's feature) because it is more relevant to the full online text.
A shorter version of this rapid response was published as a letter in the print issue of 18 August 2012 http://www.bmj.com/content/345/bmj.e5429 using the online article by Heneghan and colleagues as reference 1 rather than the feature by Cohen.
Sharon Davies
Letters editor
8 October 2012
------
Dear Editor
Scientists in all fields accept that differing views and the resulting (often vociferous) discussions are central to improving understanding. On this basis it is important that concepts are challenged, especially when a particular view or opinion has taken hold in the scientific community and even the general population. However, it was with an overriding sense of disappointment that I read the features “Forty years of sports performance research and little insight gained” and “The truth about sports drinks”.
The features focus on the “poor quality” and “bias” of a substantial body of literature, yet the critique offered includes subjective judgements, errors in fact and selective reporting that leave the authors open to exactly the criticisms that they level at the literature they have reviewed. Unfortunately, far from igniting a healthy debate, this approach can only serve to polarise opinion when, as with most things in life and certainly in science, the “truth” (an interesting choice of wording given that this suggests verified and indisputable fact) is not a simple message because of complexities of context. The lack of balance in these features falls short of providing a useful commentary as to what can and cannot be accepted as likely (since science deals with probability and does not claim to determine absolute truth) benefits of specific supplements.
The limitations of the article by Heneghan and colleagues are numerous, but the accusation of bias is based on certain specific criteria: lack of randomisation, lack of allocation concealment, lack of blinding, lack of intention to treat and a surrogate outcome measure used. Of course these are important considerations in the design of any study, but they are not relevant to all study designs and to all primary outcome measures. To suggest that these have not been considered in the design of the studies cited is naïve, and to suggest they have been consistently ignored in the interest of “positive” findings is insulting. As a simple example, I would ask the authors of this article how it is possible to conceal to a participant or double-blind a study where one trial involves drinking and one does not. I would also ask the authors what relevance intention to treat has in a laboratory based study with a small number of visits to the laboratory (unless poor tolerance within one of the trials results in withdrawal, which would normally be reported were it to happen). However, the most fundamental problem in this article is that the authors have taken an approach that artificially introduces an outcome measure for all studies and ignores the objectives of many (these are often mechanistic questions where performance within that study is not of primary interest). The authors then claim that the studies have used surrogate outcome measure, as opposed to acknowledging that they are posing a surrogate question. By taking this approach, the authors of this study have introduced the very bias they then report. On this basis, the relevance of the approach taken by the authors must be questioned, and, while independent, non-subject-specific expert review of the quality of the science is a laudable aim, the lack of understanding of the field that the authors display has clearly impacted on their ability to judge the quality of the literature.
I understand the importance of scientists offering a clear message (an important distinction from a simple message). However, this is not as simple an issue as is being portrayed and when considering the efficacy of “sports drinks”, issues around the individuals involved interact with the type, duration and intensity of exercise, which in turn interact with the amount, type and timing of specific nutrition. There are many examples in the literature of reviews by experts in the field that tackle these issues in a way that is both more balanced and more effective than the features presented.
Competing interests:
Some of my published research has been funded by GlaxoSmithKline
Rapid Response:
Re: Forty years of sports performance research and little insight gained
Editorial note
This rapid response http://www.bmj.com/content/345/bmj.e4737/rr/596007 was originally posted on 30 July 2012 in response to the feature by Cohen entitled "The truth about sports drinks" http://www.bmj.com/content/345/bmj.e4737
I have reposted it to this article by Heneghan and colleagues (discussed in the box "What the research found" in Cohen's feature) because it is more relevant to the full online text.
A shorter version of this rapid response was published as a letter in the print issue of 18 August 2012 http://www.bmj.com/content/345/bmj.e5429 using the online article by Heneghan and colleagues as reference 1 rather than the feature by Cohen.
Sharon Davies
Letters editor
8 October 2012
------
Dear Editor
Scientists in all fields accept that differing views and the resulting (often vociferous) discussions are central to improving understanding. On this basis it is important that concepts are challenged, especially when a particular view or opinion has taken hold in the scientific community and even the general population. However, it was with an overriding sense of disappointment that I read the features “Forty years of sports performance research and little insight gained” and “The truth about sports drinks”.
The features focus on the “poor quality” and “bias” of a substantial body of literature, yet the critique offered includes subjective judgements, errors in fact and selective reporting that leave the authors open to exactly the criticisms that they level at the literature they have reviewed. Unfortunately, far from igniting a healthy debate, this approach can only serve to polarise opinion when, as with most things in life and certainly in science, the “truth” (an interesting choice of wording given that this suggests verified and indisputable fact) is not a simple message because of complexities of context. The lack of balance in these features falls short of providing a useful commentary as to what can and cannot be accepted as likely (since science deals with probability and does not claim to determine absolute truth) benefits of specific supplements.
The limitations of the article by Heneghan and colleagues are numerous, but the accusation of bias is based on certain specific criteria: lack of randomisation, lack of allocation concealment, lack of blinding, lack of intention to treat and a surrogate outcome measure used. Of course these are important considerations in the design of any study, but they are not relevant to all study designs and to all primary outcome measures. To suggest that these have not been considered in the design of the studies cited is naïve, and to suggest they have been consistently ignored in the interest of “positive” findings is insulting. As a simple example, I would ask the authors of this article how it is possible to conceal to a participant or double-blind a study where one trial involves drinking and one does not. I would also ask the authors what relevance intention to treat has in a laboratory based study with a small number of visits to the laboratory (unless poor tolerance within one of the trials results in withdrawal, which would normally be reported were it to happen). However, the most fundamental problem in this article is that the authors have taken an approach that artificially introduces an outcome measure for all studies and ignores the objectives of many (these are often mechanistic questions where performance within that study is not of primary interest). The authors then claim that the studies have used surrogate outcome measure, as opposed to acknowledging that they are posing a surrogate question. By taking this approach, the authors of this study have introduced the very bias they then report. On this basis, the relevance of the approach taken by the authors must be questioned, and, while independent, non-subject-specific expert review of the quality of the science is a laudable aim, the lack of understanding of the field that the authors display has clearly impacted on their ability to judge the quality of the literature.
I understand the importance of scientists offering a clear message (an important distinction from a simple message). However, this is not as simple an issue as is being portrayed and when considering the efficacy of “sports drinks”, issues around the individuals involved interact with the type, duration and intensity of exercise, which in turn interact with the amount, type and timing of specific nutrition. There are many examples in the literature of reviews by experts in the field that tackle these issues in a way that is both more balanced and more effective than the features presented.
Competing interests: Some of my published research has been funded by GlaxoSmithKline