Rapid responses are electronic comments to the editor. They enable our users
to debate issues raised in articles published on bmj.com. A rapid response
is first posted online. If you need the URL (web address) of an individual
response, simply click on the response headline and copy the URL from the
browser window. A proportion of responses will, after editing, be published
online and in the print journal as letters, which are indexed in PubMed.
Rapid responses are not indexed in PubMed and they are not journal articles.
The BMJ reserves the right to remove responses which are being
wilfully misrepresented as published articles or when it is brought to our
attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not
including references and author details. We will no longer post responses
that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Editor – The update by Doshi addressing the corrections to the 2012 CHEST study clearly points out several inconsistencies (1). In this context, the change to the Supplementary Appendix deserves special attention. On its first page it reads: “PDF updated on March 11, 2016”. Identifying what has been updated is made difficult because the original Supplementary Appendix is no longer electronically accessible. Only if one had saved the original version and is thereby now able to compare it with the ‘updated’ one, will one realize that ‘updated’ here means addition of entirely new text.
On page 5 of the ‘update’, the newly added text describes in detail the “Reporting and analysis of Adverse Events”. Part of this reads:
“Safety analyses were conducted on the “safety” set rather than the “intention-to-treat” set.
There is no definition of “safety set”. Why should mortality, renal replacement therapy, use of ventilator and organ failure be studied on an “intention-to-treat set”, but adverse events otherwise?
It continues to read:
“For overall safety and tolerability assessment, patients are categorised according to the treatment received that is not always the same as the treatment to which they were randomly assigned.”
In the publications explaining the planned study protocol (2) and statistical analysis (3) (both preceding the publication of the CHEST study (4)), the authors had stated without restriction: “All analyses will be conducted on an intention-to-treat basis”.
Another part of the added text reads:
“In order to conduct a safety analysis, that included an assessment using a standardised questionnaire to determine the incidence of pruritus, a recognised adverse effect associated with the administration of HES, we conducted the safety analysis by categorising patients as “saline only” vs. HES ± Saline”, that included patients assigned to saline who received HES pre-randomization.”
First, there is no description of the “standardised questionnaire”, on how the questioning was performed in sedated and ventilated patients, and on whether the assessors were blinded to the treatment.
Second, in those two publications (3, 4) preceding the publication of the CHEST study, saline and hydroxyethyl starch (HES) were both defined as study drugs. In the trial registration (5) saline was explicitly declared an ‘active comparator’. The categorisation now given in the ‘updated’ Supplementary Appendix means comparison of study drug I vs. study drug I ± II. That does not make any sense. Such categorization might have possibly been justified if HES had been compared to an inactive placebo or standard care.
Most importantly, however, the ‘update’ implies that the methodology as now described is identical to that used at the time of the study. This inevitably raises many questions. (i) Why had this methodology not been described in any of the aforementioned publications (2, 3, 5)? (ii) Why is this ‘update’ published almost 4 years after the publication of the original article? (iii) Why is there no explanation on why and when it was decided to publish the ‘update’? After all, the editor-in-chief of the NEJM had been informed about considerable inconsistencies in data analysis and reporting back in 2013 in a letter from Fresenius company, and at the end of December 2015 in a letter from me. In both cases, he considered the published data as being accurately reported and declared the matter closed.
The recommendations of the International Committee of Medical Journal Editors (ICMJE) on how to handle corrections are unambiguous (6). Part of those read:
“The journal should archive all prior versions of the article. This archive can be either directly accessible to readers or can be made available to the reader on request.(…) Previous electronic versions should prominently note that there are more recent versions of the article.”
The Code of Conduct by the Committee on Publication Ethics (COPE) states (7): “… editors should always be willing to publish corrections, clarifications, retractions and apologies when needed.”
The COPE retraction guideline states (8): “Journal editors should consider retracting a publication if: they have clear evidence that the findings are unreliable, either as a result of misconduct (e.g. data fabrication) or honest error (e.g. miscalculation or experimental error).”
A recent position statement from the ICMJE reads (9): “The ICMJE believes that there is an ethical obligation to responsibly share data generated by interventional clinical trials because participants have put themselves at risk.” This was also published by the NEJM.
Bauchner et al. wrote in a recent editorial in JAMA (10): ”Because it is unlikely that the original researchers can or will conduct all useful analyses, the data must be available to other researchers to continue to gain insights or to replicate the findings. The absence of data sharing therefore constitutes a failure of the obligation of researchers to the study participants, and therefore a failure of the ethical underpinnings of conducting clinical trials.”
In this spirit, it is time that the editor-in-chief of the NEJM has, at a minimum, the data on adverse events be retracted and re-analyzed by an independent third party.
References
1. Doshi P. Update: New England Journal of Medicine publishes correction to 2012 CHEST trial of hydroxyethyl starch versus colloids. BMJ 2016;352:i1571
2. The Crystalloid versus Hydroxyethyl Starch Trial (CHEST) Management Committee. The Crystalloid versus Hydroxyethyl Starch Trial: protocol for a multi-centre randomised controlled trial of fluid resuscitation with 6% hydroxyethyl starch (130/0.4) compared to 0.9% sodium chloride (saline) in intensive care patients on mortality. Intensive Care Med 2011;37:816-23
3. Myburgh J, Li Q, Heritier S, Dan A, Glass P. Crystalloid Versus Hydroxyethyl Starch Trial (CHEST) Management Committee. Statistical analysis plan for the Crystalloid Versus Hydroxyethyl Starch Trial (CHEST). Crit Care Resusc 2012;14:44-52.
4. Myburgh JA, Finfer S, Bellomo R, et al. CHEST Investigators Australian and New Zealand Intensive Care Society Clinical Trials Group. Hydroxyethyl starch or saline for fluid resuscitation in intensive care. N Engl J Med 2012;367:1901-11.
5. https://clinicaltrials.gov/ct2/show/NCT00935168
6. http://www.icmje.org/recommendations/browse/publishing-and-editorial-iss...
7. http://publicationethics.org/files/Code%20of%20Conduct_2.pdf
8. http://publicationethics.org/files/retraction%20guidelines_0.pdf
9. http://www.icmje.org/news-and-editorials/M15-2928-PAP.pdf
10. Bauchner H, Golub RM, Fontanarosa PB. Data sharing: an ethical and scientific imperative. JAMA 2016;315:1238-40
Competing interests:
I had received two lecture honoraria form Fresenius Kabi
Re: Update: New England Journal of Medicine publishes correction to 2012 CHEST trial of hydroxyethyl starch versus colloids
Editor – The update by Doshi addressing the corrections to the 2012 CHEST study clearly points out several inconsistencies (1). In this context, the change to the Supplementary Appendix deserves special attention. On its first page it reads: “PDF updated on March 11, 2016”. Identifying what has been updated is made difficult because the original Supplementary Appendix is no longer electronically accessible. Only if one had saved the original version and is thereby now able to compare it with the ‘updated’ one, will one realize that ‘updated’ here means addition of entirely new text.
On page 5 of the ‘update’, the newly added text describes in detail the “Reporting and analysis of Adverse Events”. Part of this reads:
“Safety analyses were conducted on the “safety” set rather than the “intention-to-treat” set.
There is no definition of “safety set”. Why should mortality, renal replacement therapy, use of ventilator and organ failure be studied on an “intention-to-treat set”, but adverse events otherwise?
It continues to read:
“For overall safety and tolerability assessment, patients are categorised according to the treatment received that is not always the same as the treatment to which they were randomly assigned.”
In the publications explaining the planned study protocol (2) and statistical analysis (3) (both preceding the publication of the CHEST study (4)), the authors had stated without restriction: “All analyses will be conducted on an intention-to-treat basis”.
Another part of the added text reads:
“In order to conduct a safety analysis, that included an assessment using a standardised questionnaire to determine the incidence of pruritus, a recognised adverse effect associated with the administration of HES, we conducted the safety analysis by categorising patients as “saline only” vs. HES ± Saline”, that included patients assigned to saline who received HES pre-randomization.”
First, there is no description of the “standardised questionnaire”, on how the questioning was performed in sedated and ventilated patients, and on whether the assessors were blinded to the treatment.
Second, in those two publications (3, 4) preceding the publication of the CHEST study, saline and hydroxyethyl starch (HES) were both defined as study drugs. In the trial registration (5) saline was explicitly declared an ‘active comparator’. The categorisation now given in the ‘updated’ Supplementary Appendix means comparison of study drug I vs. study drug I ± II. That does not make any sense. Such categorization might have possibly been justified if HES had been compared to an inactive placebo or standard care.
Most importantly, however, the ‘update’ implies that the methodology as now described is identical to that used at the time of the study. This inevitably raises many questions. (i) Why had this methodology not been described in any of the aforementioned publications (2, 3, 5)? (ii) Why is this ‘update’ published almost 4 years after the publication of the original article? (iii) Why is there no explanation on why and when it was decided to publish the ‘update’? After all, the editor-in-chief of the NEJM had been informed about considerable inconsistencies in data analysis and reporting back in 2013 in a letter from Fresenius company, and at the end of December 2015 in a letter from me. In both cases, he considered the published data as being accurately reported and declared the matter closed.
The recommendations of the International Committee of Medical Journal Editors (ICMJE) on how to handle corrections are unambiguous (6). Part of those read:
“The journal should archive all prior versions of the article. This archive can be either directly accessible to readers or can be made available to the reader on request.(…) Previous electronic versions should prominently note that there are more recent versions of the article.”
The Code of Conduct by the Committee on Publication Ethics (COPE) states (7): “… editors should always be willing to publish corrections, clarifications, retractions and apologies when needed.”
The COPE retraction guideline states (8): “Journal editors should consider retracting a publication if: they have clear evidence that the findings are unreliable, either as a result of misconduct (e.g. data fabrication) or honest error (e.g. miscalculation or experimental error).”
A recent position statement from the ICMJE reads (9): “The ICMJE believes that there is an ethical obligation to responsibly share data generated by interventional clinical trials because participants have put themselves at risk.” This was also published by the NEJM.
Bauchner et al. wrote in a recent editorial in JAMA (10): ”Because it is unlikely that the original researchers can or will conduct all useful analyses, the data must be available to other researchers to continue to gain insights or to replicate the findings. The absence of data sharing therefore constitutes a failure of the obligation of researchers to the study participants, and therefore a failure of the ethical underpinnings of conducting clinical trials.”
In this spirit, it is time that the editor-in-chief of the NEJM has, at a minimum, the data on adverse events be retracted and re-analyzed by an independent third party.
References
1. Doshi P. Update: New England Journal of Medicine publishes correction to 2012 CHEST trial of hydroxyethyl starch versus colloids. BMJ 2016;352:i1571
2. The Crystalloid versus Hydroxyethyl Starch Trial (CHEST) Management Committee. The Crystalloid versus Hydroxyethyl Starch Trial: protocol for a multi-centre randomised controlled trial of fluid resuscitation with 6% hydroxyethyl starch (130/0.4) compared to 0.9% sodium chloride (saline) in intensive care patients on mortality. Intensive Care Med 2011;37:816-23
3. Myburgh J, Li Q, Heritier S, Dan A, Glass P. Crystalloid Versus Hydroxyethyl Starch Trial (CHEST) Management Committee. Statistical analysis plan for the Crystalloid Versus Hydroxyethyl Starch Trial (CHEST). Crit Care Resusc 2012;14:44-52.
4. Myburgh JA, Finfer S, Bellomo R, et al. CHEST Investigators Australian and New Zealand Intensive Care Society Clinical Trials Group. Hydroxyethyl starch or saline for fluid resuscitation in intensive care. N Engl J Med 2012;367:1901-11.
5. https://clinicaltrials.gov/ct2/show/NCT00935168
6. http://www.icmje.org/recommendations/browse/publishing-and-editorial-iss...
7. http://publicationethics.org/files/Code%20of%20Conduct_2.pdf
8. http://publicationethics.org/files/retraction%20guidelines_0.pdf
9. http://www.icmje.org/news-and-editorials/M15-2928-PAP.pdf
10. Bauchner H, Golub RM, Fontanarosa PB. Data sharing: an ethical and scientific imperative. JAMA 2016;315:1238-40
Competing interests: I had received two lecture honoraria form Fresenius Kabi