Rapid Responses to:

NEWS:
Jeanne Lenzer
Journal retracts article about age of transfused blood three years after publication
BMJ 2009; 338: b2057 [Full text]
*Rapid Responses: Submit a response to this article

Rapid Responses published:

[Read Rapid Response] Editorial Responsibility
Steven L Shafer   (16 September 2009)
[Read Rapid Response] Muddy Waters: "Fraud" Vs. "Lost Data" And An Editor's Conflict Of Interest
Shailendra Joshi   (25 November 2009)

Editorial Responsibility 16 September 2009
Next Rapid Response Top
Steven L Shafer,
Editor-in-Chief, Anesthesia & Analgesia, Professor of Anesthesiology, Columbia University
Columbia University Medical Center 622 W. 168th St, PH 5-505 New York, NY 10032-3725

Send response to journal:
Re: Editorial Responsibility

 To the Editor:

A recent BMJ News article "Journal retracts article about age of transfused blood three years after publication"1 describes how Anesthesia & Analgesia handled a manuscript by Basran and colleagues2 when alert readers noted an error in the tables3 and the authors were unable to find their original spreadsheets. As Editor-in-Chief of Anesthesia & Analgesia, I analyzed the published tables and concluded that the error was likely in the calculation of the confidence intervals. The authors disclosed the loss of their data,4 and my accompanying letter noted that the "findings stand, albeit weakly".5 The authors' correspondence and my response were judiciously handled by Dr. Lawrence Saidman, Correspondence Editor for Anesthesia & Analgesia and former Editor-in-Chief of Anesthesiology. A query from the BMJ in February 2009 was the first correspondence of any kind received by the Editorial Board regarding the decision.

How should a journal respond when data are lost? The era of written laboratory notebooks has passed. This problem will likely recur because investigators archive data on spreadsheets hosted on their laptop computers. As Harvey Marcovitch, former chair of the Committee on Publication Ethics, observed in the BMJ report,1 "the data having gone missing is not satisfactory." True, but what does one do when spreadsheets can't be found? Again quoting Dr. Marcovitch, "the best approach is always one of transparency."1 I agree - there was no question that the authors had to disclose the loss of the data.

Should the article be retracted? Retraction sends a strong message about authorship responsibility for archiving data. Had the manuscript touted a new drug, device, or procedure, then we would have retracted it. However, retracting a manuscript precludes access to the report. The manuscript raised an important safety concern, subsequently verified,6 that mortality might increase following transfusion of old blood. We allowed the paper to stand, "albeit weakly", in the interest of patient safety, and to encourage repetition of the study.

Journals have neither the authority nor the resources to investigate questions that arise regarding the conduct of research. That responsibility lies with the academic institution. In January 2009 we were informed that there was an inconsistency between the conclusions of the manuscript and a prior abstract by the authors. Without original data these differences could not be explained, so the manuscript was retracted.7,8

These are serious issues. Learning from this experience, Anesthesia & Analgesia now requires identification of an "archival author" responsible for safekeeping data following publication.9,10 This requires investigators to designate an individual to accept this responsibility. The archival author will be identified on every published research report.

Our decision in this case should not be viewed as a precedent for how journals should handle loss of research data. Rather, the precedent we would hope to set is the process of carefully examining the available data, considering all options, and making a responsible editorial decision that weighs the implications of all possible courses of action.

References
  1. Lenzer J. Journal retracts article about age of transfused blood three years after Publication BMJ 2009;338:b2057
  2. Basran S, Frumento RJ, Cohen A, Lee S, Du Y, Nishanian E, Kaplan HS, Stafford-Smith M, Bennett-Guerrero E. The association between duration of storage of transfused red blood cells and morbidity and mortality after reoperative cardiac surgery. Anesth Analg. 2006;103:15-20
  3. Rothmann M, Braun MM, Ng TH. On the hazard ratios and corresponding confidence intervals that appear in Basran et al. (2006). Anesth Analg. 2007;104:1597
  4. Basran S, Frumento RJ, Cohen A, Lee S, Du Y, Nishanian E, Kaplan HS, Stafford-Smith M, Bennett-Guerrero E. Authors reply. Anesth Analg. 2007;104:1597
  5. Shafer SL. Occam's razor. Anesth Analg. 2007;104:1597-8
  6. Koch CG, Li L, Sessler DI, Figueroa P, Hoeltge GA, Mihaljevic T, Blackstone EH. Duration of red-cell storage and complications after cardiac surgery. N Engl J Med 2008;358:1229–39
  7. Basran S, Frumento R, Cohen A, Lee S, Du Y, Nishanian E, Kaplan HS, Stafford-Smith M, Bennett-Guerrero E. Request for retraction. Anesth Analg. 2009;108:1991
  8. Shafer SL. Notice of retraction. Anesth Analg. 2009;108:1953
  9. 2008–2009 Editorial Board, Anesthesia & Analgesia. 2009 Anesthesia & Analgesia Guide for Authors. Anesth. Analg. 2009 109: 217-231.
  10. http://www.aaeditor.org/Authors/home.html (last accessed September 4, 2009)

    11. Competing interests: Editor in Chief, Anesthesia & Analgesia
Muddy Waters: "Fraud" Vs. "Lost Data" And An Editor's Conflict Of Interest 25 November 2009
Previous Rapid Response Top
Shailendra Joshi,
Assistant Professor
Columbia University, NY10032

Send response to journal:
Re: Muddy Waters: "Fraud" Vs. "Lost Data" And An Editor's Conflict Of Interest

To The Editor, The BMJ,

Far from clarifying the situation, Dr. Shafer’s response to Ms. Lenzer’s article (1,2) raises more scientific and editorial questions about the editorial handling of the Basran paper particularly when there seems to be an apparent conflict of interest.(3)

• An important question it raises is regarding the significance of the loss of data while defending a scientific paper? The Basran paper was questioned by Rothmann et al. soon after publication, well within the Office of Research Integrity (ORI) guidelines that recommend data and techniques to be preserved for at least three years, or longer if they are considered significant.(4,5) When the Anesthesia and Analgesia (A&A) decided not to retract the paper Dr. Shafer wrote, “How embarrassing!”(6) More recently in his comments to the Anesthesiology News he considered retraction of the Ruben papers to be a case of fraud but the Basran paper he said was different, a case of lost data that ‘“did not involve fraud”.(7)

Dr. Shafer is correct that loss of data by itself does not amount to fraud or scientific misconduct. The current definition of scientific misconduct (42 CFR 93.103) does not specifically include loss of data even when there is a challenge immediately after publication, while "falsification" and "fabrication" are clearly defined (http://law.justia.com/us/cfr/title 42/42- 1.0.1.8.71.1.29). If we accept the loss of data as a valid excuse for not defending the results, then it will be virtually impossible to subsequently investigate any allegations of falsification or fabrication. The lack of penalty for not archiving data for any length of time seems to provide a loophole in the definition of scientific misconduct that needs to be addressed by the ORI.

• Dr. Shafer says, “the era of written laboratory notebooks has passed. This problem will likely recur because investigators archive data on spreadsheets hosted on their laptop computers.” Dr. Shafer’s excuse that the data were lost because they were on missing laptops does not apply to the Basran paper. The Basran paper was a retrospective analysis of clinical data that are still available through hospital, blood bank and US social security death index databases, page 16.(3) The original data is not experimental data that was archived on a notebook or a laptop. With some effort it should possible to replicate the data set from the very same databases that were used by the authors. Furthermore, it is impossible to accept that all key authors, such as the research coordinator, statistician, first author and the corresponding author, all those who should have had copies of this large data set, had the files on missing laptops. The Bennett-Guerrero/Frumento team published several papers, many of them in the A&A, and should have been familiar with the importance of preserving data.

Dr. Shafer’s explanation for the lost data to the BMJ, is slightly different from that in his editorial the Occam’s razor.(6) In the editorial he had asserted that the data were lost as the authors moved. Although at the time when Rothmann et al. questioned the study, five of the nine authors, including the first author, were at their respective institutions.

• Dr. Shafer says that “Retraction sends a strong message about authorship responsibility for archiving data” but he makes an exception in this case because the publication did not describe a new drug, device or a procedure so he let it stand "albeit" weakly. Not true! In 2007, the range of penalties that the A&A could impose on an author for "academic misconduct" could be, "Sanctions against authors range from requesting a Letter to the Editor acknowledging the error and voluntarily withdrawing a manuscript, to a lifetime ban on publication in Anesthesia & Analgesia." (14) When it came to the Basran paper the journal applied the least possible penalty for not archiving data, by merely requesting a “letter acknowledging the error”. In the letter the authors did not even have to disclose how the data were lost within days of publication. The penalty imposed for the loss of data then was more was in line with the “How embarrassing!” comment than it was to send any strong message. The message “Occam’s razor” sent was contrary to the commitment to data archiving that Dr. Shafer now apparently supports.(6)

In addition, Dr. Shafer seems to be arguing for a two-tier system for peer- review based on the contents of the publication (devices, drugs and procedures vs. others, such as basic science papers) that is fundamentally unfair and potentially dangerous in the long run. Be as it may, the publication by Basran et al. was not an insignificant one for the A&A. The paper’s findings could have huge impact on the operations of (procedures at) the blood banks and it could also increase liability of medical professionals. The New York Times (15), Science Now (16), the Red Cross (17), and the FDA (18) quoted this paper! How many papers in the A&A achieve such recognition? If this paper did not merit scrutiny, will any other paper ever will?

• Dr. Shafer says the findings of this study have been “verified” by the Koch’s paper(19). The significance of the Basran paper is not just in pointing to the hazards of blood transfusion but when they are likely to occur. The Basran paper draws that line at >30 days while the Koch paper draws it at >14 days.(3,19) Not the same results. The difference in the results could have a huge impact on the operations of the blood banks.(20,21) In May 2008, a joint statement by the American Association of Blood Banks (AABB), American Red Cross (ARC) and America’s Blood Centers (ABC), pointed to the problems with the Koch’s study and recommended no changes in blood transfusion practices based on these publications.(17)

• Dr. Shafer says that, “Journals have neither the authority nor the resources to investigate questions that arise regarding the conduct of research. That responsibility lies with the academic institution." Certainly true, however, the readership of the journal and the larger society expects the editors to do their job, to ask pertinent questions and to demonstrate some curiosity as 'how" and "why" did something happen not just report the "what". If the criteria for accepting a publication is the belief that the finding can be trusted (6), sooner or later A&A will permit fraudulent research to get by its review process.(22)

Note that contrary to what the authors repeatedly emphasize that there is “an error in our paper”, suggesting a single error, (13,23) there are many errors in the paper beyond those that Rothmann et al. were concerned with.(24) There are unusually strong P-values such as "P = 0.000" (page 18) or "P < 0.000" (Page 17), the data in text and tables are internally inconsistent in several places, whether this is poor editing, sloppy rounding, or statistical errors is hard to determine.(3) Although disturbing, these are not the major problems with the paper.

• The major problems with the Basran study are that data presented in the final manuscript are incompatible with the preliminary data presented as earlier in abstracts.(3,9,10) While the underlying data are missing, we do have access to preliminary results published as abstracts from the same cohort. (9,10) In their retraction letter (13) the authors have acknowledged, the paper was retracted because the number of patients with acute renal dysfunction (ARD, 58 cases) was the same for the 392 patients reported in the abstract (9) as it was for the 321 patients that were reported in the final analysis.(3) There was not a single case of ARD in the 71 patients that were excluded from the final analysis. Exclusions in the final analysis were mostly due to the patients receiving "irradiated blood".(3) With 18% overall incidence of ARD (3) the chances of this happening is exceedingly remote, less than 1:1000,000. Alternately, if we accept the data in the abstract and the paper, then we will come to the stunning and improbable conclusion that “irradiated blood” provides absolute protection against renal failure during repeat cardiac surgery!

Furthermore, there were a total of 434 patients with cardiac re- operations in the abstract and the paper. According to the abstracts, 42 patients were excluded because they were not transfused.(9,10) Of the remaining 392 that were transfused, "Ninety-two patients were excluded because they received ≥1 U of irradiated RBCs." (3) Therefore, there could be no more than 300 eligible patients in this cohort, even if we ignore the other exclusion criterions. Yet, the study describes results from 321 patients! Where did these 21 additional patients come from?

The apparent incompatibility between the data presented in the abstract and the final manuscript is not some simple error as the authors (13,23), and Dr. Shafer (6), refer to but they raise the possibility of a compromised data set. Either the inclusion and exclusion criteria were not properly applied, or there were serious data entry errors, or there was "cherry picking" of the data particularly with regards to patients with renal failure. The absence of any effort on the part of the authors' to replicate their results - when they could apparently have done so – under these circumstances, also challenges one's imagination.

• What has really made the Basran paper unique is the editorial handling of the issues. One thing Dr. Shafer does not categorically state is whether he has/had any conflict of interest in the defending the Basran paper although he has commented on the paper several times by now.(1,6,7,25) An unambiguous statement in the matter will help us understand what he and the journal consider to be a conflict of interest?(26) A recent article and accompanying editorial in the Proceedings of the Mayo Clinic points to complex bidirectional issues regarding the conflicts of interest pertaining to medical journal editors.(27,28) Under the usual circumstances, if Dr. Shafer was seeking a job or transitioning into one, at the department of origin of the paper (2) he should have at least declared his conflicts of interest and ideally should have excused himself from intervening in the process.(29)

In his closing comments Dr. Shafer certainly wants to put all this behind. However, Dr. Shafer’s comments and actions are deeply concerning whether he is providing changing reasons for data loss,(1,6) applying a rhetorical argument “Occum’s razor” to correct statistical errors,(6) or "apparently" ignoring his conflicts of interest.(2) Instead of proposing new rules, the A&A under Dr. Shafer should be more alert and less gullible, and it should follow the existing rules. A good starting point for Dr. Shafer will be to follow the lead of the Basran paper and retract his editorial “Occam’s razor” that in my opinion undermines the significance of data loss. (6) For the rest of us, and for the regulatory authorities, we should address the deficiencies in the definition of research misconduct and find methods to enforce at least some over-sight of medical editors from the ground up and top down.

Sincerely,

Shailendra Joshi, MD

References: 1. Shafer SL. Editorial Responsibilities. The BMJ 2009;Rapid Response:b 2057.

2. Lenzer J. Journal retracts article about age of transfused blood three years after publication. The BMJ 2009;338:b 2057.

3. Basran S, Frumento RJ, Cohen A et al. The association between duration of storage of transfused red blood cells and morbidity and mortality after reoperative cardiac surgery. Anesth Analg 2006;103:15-20.

4. Steneck NH. ORI Introduction to the Responsible Conduct of Research: US Government Printing Office, 2007.

5. Coulehan MB, Wells JF. Guidelines for Responsible data management in Scientific Research. Clinical Tools:http://ori.dhhs.gov/education/products/clinicaltools/data.pdf.

6. Shafer SL. Occam's razor. Anesth Analg 2007;104:1597-8.

7. Editorial. Burned by Fraud, Anesthesia Journal Grids Author Rules. Anesthesiology News 2009; 35 :1 and 70.

8. Lenzer J, Brownlee S. Government Orders Columbia to Tell Patients 'True Nature" of Drug Study. Huffington Post 2009:http://www.huffingtonpost.com/2009/10/07/ government-orders columbi_n_312536.html.

9. Basran S, Frumento R, Cohen A et al. Association between Length of Storage of Erythrocytes and Postoperative Acute Renal Dysfunction in Patients Undergoing Reoperative Cardiac Surgery. Anesthesiology 2004;Proceedings of the Annual Meeting of the American Society of Anesthesiologists 2004:A205 http://www.asaabstracts.com/strands/asaabstracts/search.

10. Frumento R, Basran S, Cohen A et al. Association between the Length of Storage of Transfused Red Cells and Length of Stay in Patients Undergoing Reoperative Cardiac Surgery. Anesthesiology 2004;Proceeding of the Annual Meeting of the American Society of Anesthesiologists 2004:A-179 http://www.asaabstracts.com/strands/asaabstracts/search.

11. Girshin M, Frumento RJ. Pediatric Mortality Related to Anesthesia outside of the Operating Room. ASA abstract (A-1408) 2007;American Society of Anesthesiologists Annual Meeting Abstracts (abstract index) (index):http://www.asaabstracts.com/strands/asaabstracts/abstractList.htm;j sessionid=2C938BC55238FF8149087F60BCB77BDD?year=2007&index=16.

12. Jindal M, Frumento R. Can ASA Grade Predict QA Respiratory Events in Bariatric Surgery? An Analysis of 1,625 Patients (A-925). Annual meeting of the American Society of Anesthesiologists (abstract index) 2007:http://www.asaabstracts.com/strands/asaabstracts/abstractList.htm;js essionid=F5C6B1F8B6AFE9AFE0271CAF9AC52A68?year=2007&index=15.

13. Basran S, Frumento R, Cohen A et al. Request for Retraction. Anesth Analg 2009;108:1991.

14. Editorial. Guide of Authors 2006-07. Anesth Analg 2007;105:187-99.

15. Balakar N. Age of Trasfused Blood May Play Part in Recovery New York Times. New York, 2006:http://www.nytimes.com/2006/06/27/health/27blood.html.

16. Gray B. Blood Gone Bad? Science 2006;Science Now:http://sciencenow.sciencemag.org/cgi/content/full/2006/622/2.

17. Triulzi D. Clinical Significance of Red Cell Age in Transfusions. Statement Before the Advisory Committee on Blood Safety and Availability 2008;May 30, 2008 :http://www.aabb.org/Content/News_and_Media/Statements/jointstatement0 53008.html

18. He P. FDA's Criteria for Evaluation of Red Blood Cell Products. Proceedings of the Blood Products Advisory Committee (91st) Meeting, Rockville MD 2008:http://www.fda.gov/ohrms/dockets/AC/08/slides/2008- 4355S1-12_files/frame.htm.

19. Koch CG, Li L, Sessler DI et al. Duration of red-cell storage and complications after cardiac surgery. N Engl J Med 2008;358:1229-39.

20. Pereira A. Blood inventory management in the type and screen era. Vox Sang 2005;89:245-50.

21. Owens W, Tokessy M, Rock G. Age of blood in inventory at a large tertiary care hospital. Vox Sang 2001;81:21-3.

22. Shafer SL. Tattered threads. Anesth Analg 2009;108:1361-3.

23. Basran S, Frumento R, Cohen A et al. Author reply. Anesth Analg 2007;104:1597.

24. Rothmann M, Braun MM, Ng TH. On the hazard ratios and corresponding confidence intervals that appear in Basran et al. (2006). Anesth Analg 2007;104:1597; author reply.

25. Shafer SL. Notice of Retraction. Anesth Analg 2009;108:1953.

26. Shafer SL. Full disclosure matters! Anesth Analg 2008;106:1017.

27. Lanier WL. Bidirectional conflicts of interest involving industry and medical journals: who will champion integrity? Mayo Clin Proc 2009;84:771- 5.

28. Hirsch LJ. Conflicts of interest, authorship, and disclosures in industry- related scientific publications: the tort bar and editorial oversight of medical journals. Mayo Clin Proc 2009;84:811-21.

29. Editorial. International Committee for Medical Journal Editors: Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publication. 2008 :http://www.icmje.org/icmje.

Competing interests: none