Intended for healthcare professionals

Rapid response to:

Feature

How impact factors changed medical publishing—and science

BMJ 2007; 334 doi: https://doi.org/10.1136/bmj.39142.454086.AD (Published 15 March 2007) Cite this as: BMJ 2007;334:561

Rapid Response:

Impact of the impact factor in Spain

Spanish researchers have observed with interest and no little irony
the debate on the virtues and vices of the impact factor and its potential
use in the UK Research Assessment Exercises beginning in 2008. Perhaps
Spain is not so different from other European countries, since the same
debate took place in our country more than 15 years ago.

In Spain publication of research reports in journals with a high
impact factor has, since 1989, been an official part of the national
system for evaluating researchers’ productivity. As stated in the Spanish
parliamentary record (1), a bonus is awarded only for “those articles of
scientific worth in journals of recognized prestige, which shall be
accepted to mean those that occupy relevant positions in the lists for
science fields in the Subject Category Listings of the Journal Citation
Reports of the Science Citation Index (Institute for Scientific
Information (ISI), Philadelphia, PA, USA)”. This criterion, which is the
determining factor in the government’s evaluation of a researcher’s
output, has become the key which opens the door to a professional career,
pubic funding, and favourable judgements of scientific productivity (2).

The aim of this reward system was to improve the quality and the
international visibility of Spanish science, but what effects has this
system, based almost entirely on a bibliometric criterion, actually had in
Spain?

The most immediate effect was mass emigration of the best research
articles to foreign journals. The increase in the number of Spanish source
items in the Thomson Scientific databases since this policy came into
effect has been spectacular (3). The mean increase in Spanish source items
in the ISI database was 592 items per year until 1990, but jumped to 1512
items per year between 1991 and 2004, a 255% increase. Nevertheless, this
huge increase beginning in the 1990s coincided with a freeze in public
investment in research and development (R&D), which decreased in real
terms after adjustment for inflation, to the point that the unit cost per
published article declined steadily in Spain for several years.

Other effects of the change in policy were the internationalization
of Spanish science, and its incorporation into mainstream research, a
process which has enhanced the rigour and quality of work done in this
country, and has raised the bar in terms of the range and scope of the
goals the research aims to achieve. Articles authored by Spanish
researchers have increased not only in quantity but also in expected and
observed impact.

However, progress toward international-quality research has been
accompanied by increasing neglect of Spanish journals, to which
researchers rarely submit their best work (4). Another negative effect has
been the destruction of Spanish as a language of science. These negative
consequences obviously cannot be extrapolated to the UK, a country which
speaks English and whose scientists were not cut off from the rest of the
international scientific community during most of the 20th century.
However, other, more worrisome and unpredictable consequences have begun
to appear which may eventually infect UK research activity and assessment:

1. Many research groups have altered their research agendas. In
Spain, research with potential practical applications, and research on
topics that are local, regional and national in scope, has been replaced
by basic research in topics more likely to be better received by the
international research community, and therefore more likely to have a
greater impact in terms of citation counts.

2. Impactitis (5), a new disease, has become epidemic in Spain. Its
symptoms are altered publication and citation behaviour in response to an
obsessive compulsion to use the impact factor as the single,
incontrovertible quality criterion for scientific articles. This epidemic
has been propagated by all types of academic institutions and researchers,
invading all tissues of science and infecting all those who take part in
scientific communication. Numerous cases are diagnosed daily in authors
and editors. Among the former, the signs of illness are choosing the
target journal on the basis of the impact factor alone without considering
which audience is most appropriate for their work, hypertrophic self-
citing, joining invisible citation colleges intended to increase the
impact of their publications, and deliberately omitting to cite their
scientific rivals or enemies. Infected editors, determined to see their
journal indexed in Thomson Scientific’s databases, sink to manipulating
editorial policies in order to increase the journal’s repercussion both
among scholars and in the mass media.

Publishing in journals that occupy the leading positions in the
Thomson Scientific impact factor rankings has become the fuel which fires
Spanish science. Although the reasons why the impact factor should be
used with caution have been spelled out (6), and even though Garfield
himself has warned against this type of abuse of the ranking (7), the
impact factor has become the number that is devouring Spanish science (8).

The consequences, both positive and negative, of using bibliometric
criteria to evaluate research in Spain may be predictive of some of the
effects that such a policy could have for science in the UK in the next
few years if a similar method of research evaluation is adopted.

(1) Boletín Oficial del Estado, Pub. L. No. 28563, (September 9,
1989). Boletín Oficial del Estado, Pub. L. No. 3566, (February 6, 1990).
Boletín Oficial del Estado, Pub. L. No. 37030, (December 3, 1994). Boletín
Oficial del Estado, Pub. L. No. 35028, (November 20, 1996).
(2) Jiménez Contreras E, Moya Anegón F, Delgado López-Cózar E. The
evolution of research activity in Spain. The impact of the National
Commission for the Evaluation of Research Activity (CNEAI). Research
Policy 2003; 32(1): 123-42.
(3) Jiménez Contreras E, Delgado López-Cózar E, Ruiz Pérez R, Fernández
VM. Impact-factor rewards affect Spanish research. Nature 2002; 417(6892):
898
(4) Díaz M, Asensio B, Llorente GA, Moreno E, Montori A, Palomares F, et
al. El futuro de las revistas científicas españolas: un esfuerzo
científico, social e institucional. Rev Esp Doc Cient. 2001; 24(3): 306-
14.
5. Camí J. Impactolatría: diagnóstico y tratamiento. Med Clín (Barc) 1997;
109: 515-24.
6. Seglen PO. Why the impact factor of journals should not be used for
evaluating research. BMJ 1997; 314: 498-502.
7. Garfield E. The Agony and the Ecstasy. The History and Meaning of the
Journal Impact Factor. International Congress on Peer Review and
Biomedical Publication. Chicago, Sep 2005 16. [cited 2007 Ap 19].
Available from:
http://garfield.library.upenn.edu/papers/jifchicago2005.pdf
8. Monastersky R. The Number That’s Devouring Science. The Chronicle of
Higher Education [serial on the Internet]. 2005 [cited 2007 Ap 19];52(8,)
A12. Available from: http://chronicle.com/free/v52/i08/08a01201.htm

Translator: K. Shashok

Competing interests:
None declared

Competing interests: No competing interests

20 April 2007
Emilio Delgado López-Cózar
Associate Professor
Rafael Ruiz-Pérez, Evaristo Jiménez
Departamento de Biblioteconomía y Documentación, Universidad de Granada, Granada 18071, Spain