Inappropriate referencing in researchBMJ 2009; 339 doi: https://doi.org/10.1136/bmj.b2049 (Published 21 July 2009) Cite this as: BMJ 2009;339:b2049
All rapid responses
Interesting commentary to go along with an interesting article.
However, it seems a little naive to be suggesting a more extensive
critical review of the literature in funding applications. The trend these
days is for applications to be shorter, not longer. Hence, the onus is
going to be on the reviewers of these applications to be more vigilant.
Sadly, reviewers of grant applications have their own biases, and more
often than not the applicant will cater to those biases in the body of the
application. To do otherwise will result in an unfunded application.
Competing interests: No competing interests
Dean Ferguson notes that most systematic reviews focus on clinical
evidence of treatment effectiveness but suggests that systematic reviews
on basic research would also be useful. The Cochrane Collaboration has
been influential in promoting systematic reviews of treatment and in
developing robust techniques for doing such reviews. Maybe we need a
similar collaboration for basic biomedical research?
Since it is now accepted that funding proposals for clinical trials
should include a systematic review, perhaps this principle should also be
extended to preclinical research. In which case, perhaps the major
biomedical funders might consider organizing such a collaboration -- in
the long-term it might increase the efficiency of research spending and
actually save money.
Competing interests: No competing interests
We read with great interest the article of Dr Fergusson, entitled
"Inappropriate referencing in research" recently published in the BMJ.
We are in agreement with the author as "Improper citation is not a benign
practice; adequate and accurate citation is a necessity of scientifically
and methodologically sound research" 1.
The scientists have been entangled with the obsession of producing
science all around the world for several years. Investigators have been
endeavoring to produce scientific papers as articles every day
increasingly,and consequently submit the papers to scientific communities and
especially academic journals as their research and investigation output.
There are thousands of scholarly papers registered in literature databases
throughout the world of science and the number increases every day; for
instance Medline or PubMed (U.S. National Library of Medicine) has
announced inclusion of hundreds of papers per diem. Despite the
overwhelming efforts in preparation of such papers, authorities believe
that considerable volume of the records are worthless.
The foregoing all occur while the obsession of producing scholarly
papers triggers new scientific journals in various disciplines. The
increasing process has expanded so abundantly that one can witness
establishment of subspecialty or even sub-subspecialty journals in diverse
disciplines, focusing sometimes on infrequent diseases or conditions. The
diversification and abundance of such academic journals made the academic
communities to attend and assess the quality of the journals using a
systematic approach. In fact, assessment and placement of the scientific
journals have been in the spotlight recently and each territory has
created its criteria for assessment and ranking each discipline's
journals and to rank thousands of scholarly and professional journals
within their discipline or sub-discipline. These criteria seem to be
essential for educated decisions about selecting and deselecting journals
in college libraries, and gauging the prestige and influence of journals.
There are multifold values to the categorizations and evaluations (like as
impact factor [IF], devised by Eugene Garfield, the founder of the
Institute for Scientific Information) announced by different measures 2,
They are important indicators of how frequently current investigators
and scientists are using individual journals. By tabulating and
aggregating citation and article counts, they offer a unique perspective
for scientific journal evaluation and comparison. 2
This has made the authors consider factors such as IF more than
before; on the other hand it could be foreseen that this would set off
competition between editors of every single subspecialty journal. This
has manifested in the advertisements: Many of the journals specify their
rank and IF among the other journals in their home pages and/or first
pages of publications in bold.
Every competition has the risk of ignoring the principles as a
consequence. Publication ethics have been addressed numerously during the
recent years; hence, the editors and managers are expected to comply with
publication ethics in the scientific journalism competitions as they are
among highly educated and professionals of the discipline. Are the
principles really complied with in the scientific journalism competitions?
We did not intend to judge this issue personally, but some observations
and occurrences have drawn our attention to some unhealthy competitions
during the recent months. In late 2007, we prepared a paper pertinent to a
specific subspecialty and reviewed the last (2005) IFs of ten journals
included in ISI in the discipline related to the manuscript's topic. The
approximate journals' IFs range from 0.76-4.68.
Considering the quality of the paper, we determined to submit it to a
journal with IF of 1.1, ranking 9 of 10. Fortunately, the paper was
accepted and published by the journal. A few months ago, we prepared another research paper; we again reviewed the IFs
of the journals of the same subspecialty and there were the same ten
journals still included in ISI, but we were astonished by dramatic changes
in the 2007 IFs. For example the journal which had been ranked seventh
among the ten journals in 2005 IFs was ranked fourth and its impact factor
was improved by 1.43 (almost 80% increase), while the mean change in other
journals was significantly lower (+0.43) and the ranking of the remaining
journals changed at most by 1 rank (except one journal which its ranking
decreased by 5). We were really astonished and asked ourselves: "what
could the editors of this journal have done that the journal's IF such a
significant improvement had?"
The question remained unanswered and we
submitted the paper to this interesting journal without paying attention
to the response. Several months later we were notified of the opinion of
the editorial board and the journal's decision in regards to our paper
via email. The result would read: "It was stated that the paper should be
accepted for publication if certain revisions could be made. This involves
revising the manuscript in accordance with the reviewer comments listed
below". Such a decision making might not look very weird at the very
first sight, and researchers and scientists who write numerous papers
consider that as a reasonable and positive decision made by an editorial
board. But we are almost sure that you will change your mind as well in
case you know the editorial board's request. The email addressed only one
point to be revised as: "There should be a total of at least 8
references. The following (1-3) can be added." The three reference nominated by the editor to cite were all among
the papers published in the same journal during the previous Gregorian
year (some irrelevant to our paper to some extent). Probably the esteemed editorial board
intended to improve the journal's IF. The aforementioned email made the
basis for improving the journal's IF and ranking between 2005 and 2007
clear for us to some extent.
The pessimism was soon settled through the vain consolation that such a
thing might be seen only in this journal or this particular sub-specialty.
Several weeks later we were notified on the decision made by the editorial
board of another (X) journal in regards to one of
our papers (in another subspecialty field) via email. One of the explicit
requests of the editorial board was: ¡§Please consider to cite at least one
paper published in 2007 or 2008 in (X) Journal, related to the subject of
your manuscript.¡¨ Also recently, we received an email from a different journal in another specialty requesting proofreading of our recently accepted
paper and interestingly, the chief editor, even after notification of the
final acceptance (following our preliminary revision based on the
reviewers' comments) requested that: "Your article entitled ¡§¡K¡¨ will
be published in the May-June issue of the ¡§Y¡¨ Journal. The Editors would
appreciate if you could cite articles recently published in the ¡§Y¡¨
Journal (2007-2008). Possible articles include the following papers: 1)
¡K, and 2) ¡K.. "
Now, we believe that the new strategy of editors in not limited to a
specific discipline or sub-specialty, but at least a remarkable portion of
the scientific medical journals are affected. In our opinion, an epidemic
trick among scientific journals is spreading as a consequence of "IF
Thus, it seems that one of the new tricks of the editors of some
journals in medicine to promote the IF is to force the authors and
scientists to cite previously published papers in the same journal. It
should be noted that it is not important in IF measurement whether the
reference is cited in the same journal or another and in fact "Self
citations are not corrected for" 2. Although "Thomson Reuters is aware
of the fact that some journals use extensive reference to their prior
content to influence their citation metrics and the contribution of so-
called journal self-citation has been included in Journal Citation Reports
since it first appeared in 1975" 4, this unethical approach of the
journals' editors makes their journals' IF promote falsely among the
other journals. In our opinion, this unethical approach, as well as some
other problems associated with the use of journal Ifs 2) will devaluate
criteria such as IF in near future and authors and scientists will lose
their faith in such criteria, unfortunately. It has been stated that
"some journals received no listing in Journal Citation Reports previous
years because of exceptionally high self-citation counts and their titles
are listed in the Notices file on the journal's website"4, but in our
opinion the problem is spreading and not completely solved by such a
strategy . These concerns have been emphasized by others, too 5.
The foregoing has been neglected in medical journals. It
should be attended by scientific communities, principally by specialized
committees of publication ethics to be uprooted and prevent the
disease from spreading throughout the fragile but gigantic body of
scientific journalism. We believe similar instances, besides the three
aforementioned cases, certainly exist and those who endeavor in research
and develops papers continuously have definitely encountered such
Exclusion of self-citation from IF criteria can be proposed as an
approach; moreover, the journals' editors should be taught about fair
play and they should also be formally and ethically asked not to prejudice
authors' rights and freedom as to promote the journals' ranking.
Notwithstanding the fact that the proposal to cite some new references (in
case stipulated without prejudice) is invaluable and helps the authors
promote the quality of manuscripts (we, as reviewers, propose such in our
reviews scarcely), it is unethical and unseemly to make the authors
encounter such dilemma of citation or rejection. To our point of view
editors and the journals' managers, in line with their endeavors in
scientific journalism competitions, should seek more reasonable and
substantive approached to promote the journals' quality ranking and avoid
such unethical methods.
1. Fergusson D. Inappropriate referencing in research. BMJ. 2009; 339:
2. Seglen PO. Why the impact factor of journals should not be used for
evaluating research. BMJ. 1997; 314: 497.
3. ISI web of Knowledge:
4. Testa J. Playing the system puts self-citation's impact under review.
Nature. 455: 729.
5. Sala SD, Brooks J. Multi-authors' self-citation: a further impact
factor bias? Cortex. 2008; 44(9): 1139-45.
Competing interests: No competing interests
Citation distortion undoubtedly contributes to creating unfounded authority even by our most reputable establishments. In 2007 I pointed out a citation error in the NICE guideline for caesarean section, and my letter was published in the BMJ. The guideline document remains unchanged since then. I quote the relevant section on page 64 of the guideline verbatim.
Suggested benefits of delayed cord clamping include decreased neonatal anaemia; better
systemic and pulmonary perfusion; and better breastfeeding outcomes. Possible harms are
polycythaemia, hyperviscosity, hyperbilirubinaemia, transient tachypnoea of the newborn and
risk of maternal fetal transfusion in rhesus negative women. reference 402 “
The evidence that the timing of cord clamping affects the health of the baby is not overwhelming however there is the principle that we should never interfere with the functions of the human body unless we have evidence that it is of benefit or at least does no harm.
Let us look at the wording of this document. It starts with “suggested benefits” with the implication that these benefits have been proposed or put forwards by someone or some people. The wording implies that there is no evidence to support the “suggested benefits”.
Moving now to “possible harms”. This has the implication that these harms are likely, probable or potential. This is followed by the reference 402 which is for
Mercer JS. Current best evidence: a review of the literature on umbilical cord clamping. J Midwifery Womens Health
In this review paper Mercer references numerous research papers to support benefit for delayed cord clamping. She specifically states that there is no evidence to suggest that delayed cord clamping causes polycythemia or hyperbilrubinaemia.
References are usually given immediately after the statement that they are supporting. Clearly this reference cannot be used to support the possible harms. Even if it was intended to be used to support the suggested benefits, the reference gives much more than just suggested benefits since the results of numerous randomised controlled trials are quoted in this review. Whether or not the assessment of the evidence in this review is a fair one or not is irrelevant. It was used to support one or both of two statements for which it is clearly not supportive.
Those readers who check my BMJ letter will see that some of my references do not support the statements made! I apologise for this, and assure readers that that these were numbering errors which I did not check carefully enough. I am therefore including the text of my letter with the correct referencing numbers. Perhaps similar errors crept into the NICE guideline. If that is the case can the web document not be corrected?
My BMJ letter
NICE is encouraging artificial intervention
As well as seeming to discourage the detection of postpartum depression, (1) the National Institute for Health and Clinical Excellence (NICE) is also encouraging the artificial intervention of immediate cord clamping. (2)
The NICE guideline for caesarean section includes a section on cord clamping. It states a number of "suggested" benefits and a number of "possible" harms. The reference is a review paper in a midwifery journal. (3) This review clearly provides evidence for the benefits, not simply a "suggestion," and clearly disputes any evidence of harm from polycythemia, hyperviscosity, or hyperbilirubinaemia.
The NICE draft guideline on intrapartum care never mentions cord clamping (www.nice.org.uk/page.aspx?o=334322). The authors recognise that the major rapid physiological changes that take place enable the baby to adapt to life outside the womb. How rapidly should we expect these changes to take place? Is it reasonable to expect these physiological changes to occur within a few seconds and at the whim of a bystander? Active management of the third stage is considered to be an established part of good intrapartum care, and early cord clamping an integral part of it. (5)
How precise does the practice need to be? The timing of the oxytocic agent varies among the studies and in different parts of the world. There is no real logic for incorporating early cord clamping in a strategy to reduce post-partum haemorrhage. Removing the clamp and draining the residual placental blood seems to shorten the third stage.(4) This is recommended practice in rhesus negative women, in an effort to reduce the risk of fetomaternal haemorrhage. It is therefore totally illogical to recommend immediate cord clamping and cutting, followed by drainage of the residual placental blood. This is blood that is physiogically required by the newborn baby. It would be better to "drain the placenta" into the newborn baby or at least provide the baby with the amount of blood that it requires. Any residual blood at that stage can be allowed to drain away.
David J R Hutchon, consultant obstetrician
Darlington Memorial Hospital, Darlington DL3 6HX
Competing interests: None declared.
1. Coyne JC, Mitchell AJ. NICE may be discouraging detection of postpartum depression. BMJ 2007 doi: 10.1136/bmj.39150.424896.BE[CrossRef][Medline]
2. Kent A What's new in the other journals? BJOG 2007 114:379-80.
3. Mercer JS. Current best evidence: a review of the literature on umbilical cord clamping. J Midwifery Women's Health 2001;46:402–14.[CrossRef][Web of Science][Medline]
4. Soltani H, Dickinson F, Symonds I. Placental cord drainage after spontaneous vaginal delivery as part of the management of the third stage of labour. Cochrane Database Syst Rev 2005 (4): CD004665.
5. Prendiville W, Elbourne D. Care during the third stage of labour. In: Chalmers I, Enkin M, Keirse MJNC, eds. Effective care in pregnancy and childbirth. Oxford: Oxford University Press, 1989:1145-69.
Belief in the risk of delayed cord clamping is widespread, through distorted description of the physiological transition at birth, the distorted citation which has cascaded over the last 50 years to standard practice in obstetric and midwifery textbooks. It is taught with such authority it takes considerable effort by the student to realise that there may just be something totally illogical or unphysiological in this intervention at birth.
Competing interests: No competing interests