To
The Editor,
The BMJ,
Far from clarifying the situation, Dr. Shafer’s response to Ms.
Lenzer’s article
(1,2) raises more scientific and editorial questions about the editorial
handling of the Basran paper particularly when there seems to be an
apparent
conflict of interest.(3)
• An important question it raises is regarding the significance of
the loss of
data while defending a scientific paper? The Basran paper was questioned
by
Rothmann et al. soon after publication, well within the Office of Research
Integrity (ORI) guidelines that recommend data and techniques to be
preserved for at least three years, or longer if they are considered
significant.(4,5) When the Anesthesia and Analgesia (A&A) decided not to
retract the paper Dr. Shafer wrote, “How embarrassing!”(6) More recently
in
his comments to the Anesthesiology News he considered retraction of the
Ruben papers to be a case of fraud but the Basran paper he said was
different, a case of lost data that ‘“did not involve fraud”.(7)
Dr. Shafer is correct that loss of data by itself does not amount to
fraud or
scientific misconduct. The current definition of scientific misconduct
(42 CFR
93.103) does not specifically include loss of data even when there is a
challenge immediately after publication, while "falsification" and
"fabrication"
are clearly defined (http://law.justia.com/us/cfr/title 42/42-
1.0.1.8.71.1.29). If we accept the loss of data as a valid excuse for not
defending the results, then it will be virtually impossible to
subsequently
investigate any allegations of falsification or fabrication. The lack of
penalty
for not archiving data for any length of time seems to provide a loophole
in
the definition of scientific misconduct that needs to be addressed by the
ORI.
• Dr. Shafer says, “the era of written laboratory notebooks has
passed. This
problem will likely recur because investigators archive data on
spreadsheets
hosted on their laptop computers.” Dr. Shafer’s excuse that the data were
lost because they were on missing laptops does not apply to the Basran
paper. The Basran paper was a retrospective analysis of clinical data
that are
still available through hospital, blood bank and US social security death
index
databases, page 16.(3) The original data is not experimental data that was
archived on a notebook or a laptop. With some effort it should possible
to
replicate the data set from the very same databases that were used by the
authors. Furthermore, it is impossible to accept that all key authors,
such as
the research coordinator, statistician, first author and the corresponding
author, all those who should have had copies of this large data set, had
the
files on missing laptops. The Bennett-Guerrero/Frumento team published
several papers, many of them in the A&A, and should have been familiar
with
the importance of preserving data.
Dr. Shafer’s explanation for the lost data to the BMJ, is slightly
different from
that in his editorial the Occam’s razor.(6) In the editorial he had
asserted that
the data were lost as the authors moved. Although at the time when
Rothmann et al. questioned the study, five of the nine authors, including
the
first author, were at their respective institutions.
• Dr. Shafer says that “Retraction sends a strong message about
authorship
responsibility for archiving data” but he makes an exception in this case
because the publication did not describe a new drug, device or a procedure
so he let it stand "albeit" weakly. Not true! In 2007, the range of
penalties
that the A&A could impose on an author for "academic misconduct" could be,
"Sanctions against authors range from requesting a Letter to the Editor
acknowledging the error and voluntarily withdrawing a manuscript, to a
lifetime
ban on publication in Anesthesia & Analgesia." (14) When it came to
the
Basran paper the journal applied the least possible penalty for not
archiving
data, by merely requesting a “letter acknowledging the error”. In the
letter
the authors did not even have to disclose how the data were lost within
days
of publication. The penalty imposed for the loss of data then was more was
in
line with the “How embarrassing!” comment than it was to send any strong
message. The message “Occam’s razor” sent was contrary to the
commitment to data archiving that Dr. Shafer now apparently supports.(6)
In addition, Dr. Shafer seems to be arguing for a two-tier system for
peer-
review based on the contents of the publication (devices, drugs and
procedures vs. others, such as basic science papers) that is fundamentally
unfair and potentially dangerous in the long run. Be as it may, the
publication by Basran et al. was not an insignificant one for the A&A.
The
paper’s findings could have huge impact on the operations of (procedures
at)
the blood banks and it could also increase liability of medical
professionals.
The New York Times (15), Science Now (16), the Red Cross (17), and the FDA
(18) quoted this paper! How many papers in the A&A achieve such
recognition? If this paper did not merit scrutiny, will any other paper
ever
will?
• Dr. Shafer says the findings of this study have been “verified” by
the Koch’s
paper(19). The significance of the Basran paper is not just in pointing to
the
hazards of blood transfusion but when they are likely to occur. The
Basran
paper draws that line at >30 days while the Koch paper draws it at
>14
days.(3,19) Not the same results. The difference in the results could
have a
huge impact on the operations of the blood banks.(20,21) In May 2008, a
joint statement by the American Association of Blood Banks (AABB),
American
Red Cross (ARC) and America’s Blood Centers (ABC), pointed to the problems
with the Koch’s study and recommended no changes in blood transfusion
practices based on these publications.(17)
• Dr. Shafer says that, “Journals have neither the authority nor the
resources
to investigate questions that arise regarding the conduct of research.
That
responsibility lies with the academic institution." Certainly true,
however, the
readership of the journal and the larger society expects the editors to do
their job, to ask pertinent questions and to demonstrate some curiosity as
'how" and "why" did something happen not just report the "what". If the
criteria for accepting a publication is the belief that the finding can be
trusted (6), sooner or later A&A will permit fraudulent research to get by
its
review process.(22)
Note that contrary to what the authors repeatedly emphasize that
there is “an
error in our paper”, suggesting a single error, (13,23) there are many
errors
in the paper beyond those that Rothmann et al. were concerned with.(24)
There are unusually strong P-values such as "P = 0.000" (page 18) or "P
<
0.000" (Page 17), the data in text and tables are internally inconsistent
in
several places, whether this is poor editing, sloppy rounding, or
statistical
errors is hard to determine.(3) Although disturbing, these are not the
major
problems with the paper.
• The major problems with the Basran study are that data presented in
the
final manuscript are incompatible with the preliminary data presented as
earlier in abstracts.(3,9,10) While the underlying data are missing, we do
have
access to preliminary results published as abstracts from the same cohort.
(9,10) In their retraction letter (13) the authors have acknowledged, the
paper
was retracted because the number of patients with acute renal dysfunction
(ARD, 58 cases) was the same for the 392 patients reported in the abstract
(9) as it was for the 321 patients that were reported in the final
analysis.(3)
There was not a single case of ARD in the 71 patients that were excluded
from the final analysis. Exclusions in the final analysis were mostly due
to the
patients receiving "irradiated blood".(3) With 18% overall incidence of
ARD (3)
the chances of this happening is exceedingly remote, less than 1:1000,000.
Alternately, if we accept the data in the abstract and the paper, then we
will
come to the stunning and improbable conclusion that “irradiated blood”
provides absolute protection against renal failure during repeat cardiac
surgery!
Furthermore, there were a total of 434 patients with cardiac re-
operations in
the abstract and the paper. According to the abstracts, 42 patients were
excluded because they were not transfused.(9,10) Of the remaining 392 that
were transfused, "Ninety-two patients were excluded because they received
≥1 U of irradiated RBCs." (3) Therefore, there could be no more than
300
eligible patients in this cohort, even if we ignore the other exclusion
criterions. Yet, the study describes results from 321 patients! Where did
these 21 additional patients come from?
The apparent incompatibility between the data presented in the
abstract and
the final manuscript is not some simple error as the authors (13,23), and
Dr.
Shafer (6), refer to but they raise the possibility of a compromised data
set.
Either the inclusion and exclusion criteria were not properly applied, or
there
were serious data entry errors, or there was "cherry picking" of the data
particularly with regards to patients with renal failure. The absence of
any
effort on the part of the authors' to replicate their results - when they
could
apparently have done so – under these circumstances, also challenges one's
imagination.
• What has really made the Basran paper unique is the editorial
handling of
the issues. One thing Dr. Shafer does not categorically state is whether
he
has/had any conflict of interest in the defending the Basran paper
although
he has commented on the paper several times by now.(1,6,7,25) An
unambiguous statement in the matter will help us understand what he and
the journal consider to be a conflict of interest?(26) A recent article
and
accompanying editorial in the Proceedings of the Mayo Clinic points to
complex bidirectional issues regarding the conflicts of interest
pertaining to
medical journal editors.(27,28) Under the usual circumstances, if Dr.
Shafer
was seeking a job or transitioning into one, at the department of origin
of
the paper (2) he should have at least declared his conflicts of interest
and
ideally should have excused himself from intervening in the process.(29)
In his closing comments Dr. Shafer certainly wants to put all this
behind.
However, Dr. Shafer’s comments and actions are deeply concerning whether
he is providing changing reasons for data loss,(1,6) applying a rhetorical
argument “Occum’s razor” to correct statistical errors,(6) or "apparently"
ignoring his conflicts of interest.(2) Instead of proposing new rules, the
A&A
under Dr. Shafer should be more alert and less gullible, and it should
follow
the existing rules. A good starting point for Dr. Shafer will be to
follow the
lead of the Basran paper and retract his editorial “Occam’s razor” that in
my
opinion undermines the significance of data loss. (6) For the rest of us,
and
for the regulatory authorities, we should address the deficiencies in the
definition of research misconduct and find methods to enforce at least
some
over-sight of medical editors from the ground up and top down.
Sincerely,
Shailendra Joshi, MD
References:
1. Shafer SL. Editorial Responsibilities. The BMJ 2009;Rapid Response:b
2057.
2. Lenzer J. Journal retracts article about age of transfused blood three
years after publication. The BMJ 2009;338:b 2057.
3. Basran S, Frumento RJ, Cohen A et al. The association between duration
of storage of transfused red blood cells and morbidity and mortality after
reoperative cardiac surgery. Anesth Analg 2006;103:15-20.
4. Steneck NH. ORI Introduction to the Responsible Conduct of Research: US
Government Printing Office, 2007.
5. Coulehan MB, Wells JF. Guidelines for Responsible data management in
Scientific Research. Clinical
Tools:http://ori.dhhs.gov/education/products/clinicaltools/data.pdf.
6. Shafer SL. Occam's razor. Anesth Analg 2007;104:1597-8.
7. Editorial. Burned by Fraud, Anesthesia Journal Grids Author Rules.
Anesthesiology News 2009; 35 :1 and 70.
8. Lenzer J, Brownlee S. Government Orders Columbia to Tell Patients 'True
Nature" of Drug Study. Huffington Post
2009:http://www.huffingtonpost.com/2009/10/07/ government-orders
columbi_n_312536.html.
9. Basran S, Frumento R, Cohen A et al. Association between Length of
Storage of Erythrocytes and Postoperative Acute Renal Dysfunction in
Patients
Undergoing Reoperative Cardiac Surgery. Anesthesiology 2004;Proceedings of
the Annual Meeting of the American Society of Anesthesiologists 2004:A205
http://www.asaabstracts.com/strands/asaabstracts/search.
10. Frumento R, Basran S, Cohen A et al. Association between the Length of
Storage of Transfused Red Cells and Length of Stay in Patients Undergoing
Reoperative Cardiac Surgery. Anesthesiology 2004;Proceeding of the Annual
Meeting of the American Society of Anesthesiologists 2004:A-179
http://www.asaabstracts.com/strands/asaabstracts/search.
11. Girshin M, Frumento RJ. Pediatric Mortality Related to Anesthesia
outside
of the Operating Room. ASA abstract (A-1408) 2007;American Society of
Anesthesiologists Annual Meeting Abstracts (abstract index)
(index):http://www.asaabstracts.com/strands/asaabstracts/abstractList.htm;j
sessionid=2C938BC55238FF8149087F60BCB77BDD?year=2007&index=16.
12. Jindal M, Frumento R. Can ASA Grade Predict QA Respiratory Events in
Bariatric Surgery? An Analysis of 1,625 Patients (A-925). Annual meeting
of
the American Society of Anesthesiologists (abstract index)
2007:http://www.asaabstracts.com/strands/asaabstracts/abstractList.htm;js
essionid=F5C6B1F8B6AFE9AFE0271CAF9AC52A68?year=2007&index=15.
13. Basran S, Frumento R, Cohen A et al. Request for Retraction. Anesth
Analg 2009;108:1991.
14. Editorial. Guide of Authors 2006-07. Anesth Analg 2007;105:187-99.
15. Balakar N. Age of Trasfused Blood May Play Part in Recovery New York
Times. New York,
2006:http://www.nytimes.com/2006/06/27/health/27blood.html.
16. Gray B. Blood Gone Bad? Science 2006;Science
Now:http://sciencenow.sciencemag.org/cgi/content/full/2006/622/2.
17. Triulzi D. Clinical Significance of Red Cell Age in Transfusions.
Statement
Before the Advisory Committee on Blood Safety and Availability 2008;May
30,
2008
:http://www.aabb.org/Content/News_and_Media/Statements/jointstatement0
53008.html
18. He P. FDA's Criteria for Evaluation of Red Blood Cell Products.
Proceedings of the Blood Products Advisory Committee (91st) Meeting,
Rockville MD 2008:http://www.fda.gov/ohrms/dockets/AC/08/slides/2008-
4355S1-12_files/frame.htm.
19. Koch CG, Li L, Sessler DI et al. Duration of red-cell storage and
complications after cardiac surgery. N Engl J Med 2008;358:1229-39.
20. Pereira A. Blood inventory management in the type and screen era. Vox
Sang 2005;89:245-50.
21. Owens W, Tokessy M, Rock G. Age of blood in inventory at a large
tertiary
care hospital. Vox Sang 2001;81:21-3.
22. Shafer SL. Tattered threads. Anesth Analg 2009;108:1361-3.
23. Basran S, Frumento R, Cohen A et al. Author reply. Anesth Analg
2007;104:1597.
24. Rothmann M, Braun MM, Ng TH. On the hazard ratios and corresponding
confidence intervals that appear in Basran et al. (2006). Anesth Analg
2007;104:1597; author reply.
25. Shafer SL. Notice of Retraction. Anesth Analg 2009;108:1953.
26. Shafer SL. Full disclosure matters! Anesth Analg 2008;106:1017.
27. Lanier WL. Bidirectional conflicts of interest involving industry and
medical journals: who will champion integrity? Mayo Clin Proc 2009;84:771-
5.
28. Hirsch LJ. Conflicts of interest, authorship, and disclosures in
industry-
related scientific publications: the tort bar and editorial oversight of
medical
journals. Mayo Clin Proc 2009;84:811-21.
29. Editorial. International Committee for Medical Journal Editors:
Uniform
Requirements for Manuscripts Submitted to Biomedical Journals: Writing and
Editing for Biomedical Publication. 2008 :http://www.icmje.org/icmje.
Competing interests:
none