Evidence based policy: proceed with careCommentary: research must be taken seriouslyBMJ 2001; 323 doi: https://doi.org/10.1136/bmj.323.7307.275 (Published 04 August 2001) Cite this as: BMJ 2001;323:275
All rapid responses
SIR: Recent contributions in the debate on the role of scientific
evidence in decision-making are timely, and have aired some powerful
points. A basic issue that needs further exploration, however, is the
nature of the dynamics within health care systems, and in particular how
vested interests operate to diminish the place of research evidence.
The commentators [1-2] take for granted that health policy should be
based on science, and furthermore seem content that clinicians, at least,
are willing to embrace evidence-based medicine. Says Professor Black, "it
seems difficult to argue with the idea that scientific research should
In actual fact, quite a few clinicians see little difficulty at
all. Suspicion exists among practitioners that bureaucrats are
altogether too willing to listen to researchers, and insufficiently
responsive to their own needs. The appeal that "whatever the research
says, I know my patients benefit", follows (among other reasons) from
clinicians' sense that they bear responsibility for patients, while
administrators give preference to ivory-tower evidence and thereby deprive
them of a full range of treatment options.
Such misgivings are supported when, on another side, many funders see
assessment research as providing an economic management tool, tailor-made
for reigning in overstretched budgets. Evidence that points to inefficacy
is 'powerful' - evidence that implies expenditure is 'idealistic'.
According to Professor Black, the hold-up in progress is the tendency
of researchers not to get the big picture. Their political naivety leads
them to have unrealistic expectations, which would be better directed in
contributing to central rather than local policy; and in the 'ethereal'
dimension, rather than the nitty-gritty.
The trouble is that none of the health system stakeholders (to use
the terminology most inclusive and least stratifying) are disposed to get
the big picture. This would cut against the pursuit of individual
interests, which are always present, usually divergent, and oftentimes
contradictory. Such interest-based thinking may not seem especially
desirable, but it does reflect realities of the health policy picture. Dr
Donald  refers to the miasma of wider factors that operate to sideline
sound research. While all sides may be content to have some science in
the mix, there is no interest group un-motivated by non-scientific
Without this understanding, it is not especially helpful to suggest
that the principles and lessons are transferable from clinical practice to
health policy. Unfortunately, many of the problems regarding research
uptake are equally transferable.
Self-interest goals are not necessarily 'bad'. In fact, it is
essential to recognize that interest groups within systems cannot be
expected to have the same goals; or even that these should coincide with
good 'patient health'. The question then becomes, how to use evidence
appropriately to find the path through these competing interests towards
the end-point of patient benefit.
Administrators, of course, suffer no shortage of 'information' but
generally not of the quality that offers support in 'mapping the decision-
making terrain'. In fact, administrators can easily become lost in a fog
of competing 'evidence' which accompanies demands, whereupon their
understandable inclination will be to listen for the most resonant klaxon
sounded by clinicians, and follow a path of costs towards it.
Professor Black is, however, right to point to the sub-structure of
mutual understanding that must be in place before evidence-based decision
making can become fully established. The professor himself identifies the
critical concept: Context. This should be understood to incorporate the
contexts both of what is to be decided, and how the decisions are to be
The absolute key is to ensure research findings are delivered to
decision-makers in a form that they can use. Contrary to one suggestion
in the article, many policy-makers are keen to be involved in the
direction of research, by targetting funding, or assisting in the
formulation of research questions. But research activity is currently
inclined to focus on systematic reviews of efficacy trials. While
fundamentally important, such work is still only one part of the picture.
Researchers need to be aware that their efforts can and should
accommodate a comprehensive examination of population health issues. If
the scope of research is not fully developed, administrators will be
correspondingly limited in the use they can make of its findings.
Evidence-based assessments should properly be conducted along
comprehensive contextual dimensions: not only efficacy and safety, but
also legal, ethical, and social implications including equality of access,
and economic impact. The tools exist for such evidence synthesis.
Research conclusions can be then be provided to decision-makers in a form
which meets the diversity of their needs.
The ultimate aim of evidence-based decision making is simple enough:
it is to provide the best health care for the greatest number of people.
While such an approach should not be seen as offering a straight path to
the shining city of collective (and objective) agreement, provided it is
maximally inclusive of relevant dimensions of assessment, it offers the
best means of setting many differing interests within an evaluative
context. In short, it can provide a directional compass in the fog of
competing claims and expensive choices. 
 Black N. Evidence based policy: proceed with care. BMJ
 Walsh K. Evidence based policy: don't be timid. [Electronic
letter] BMJ 2001;Aug 3. Available from URL:
 Kernick D. Evidence based policy: more fundamental concerns.
[Electronic letter] BMJ 2001;Aug 13. Available from URL:
 Donald A. Commentary: research must be taken seriously. BMJ
 Kazanjian A, Green CJ, Bassett K, Brunger F. Bone mineral
density testing in social context. International Journal of Technology
Assessment in Health Care 1999;15:4:679-685.
 Kazanjian A. A Compass in the Fog: Using HTA for better health
policy. [Workshop presentation] International Society for Technology
Assessment in Health Care 17th Annual Meeting, Philadelphia PA, June 3-6
2001 (in press)
Competing interests: No competing interests
Evidence based policy in Italy
EDITOR-We read with interest the paper by Black and the commentary by
Donald on Evidence based policy recently published on BMJ (1-2).
We would like to discuss their statements on the relationship between
scientific research and political decision making on the basis of our
“Italian case”, i.e. the experience of the Italian government in the last
In fact the present and the past Ministries of Health were both famous
scientists (professor Veronesi is an outstanding oncologist and professor
Sirchia a very well known expert in transplantation immunology), but an
overall glance at their positive work as the most important responsible
persons of our National Health System indicates that both behaved mostly
following the rules of politics rather than those of their personal
Having in mind this example could we say that scientific knowledge is not
important in political decisions? Certainly being a famous doctor could
have a positive psychological impact on public opinion; even if general
public has no more the idea that bus drivers and pilots may be clever
ministers of transportation and generals good ministers of defence, the
psychological impact of being a scientist in politics have an excess of
value. However, in the real every day decision making, the importance of
“money, power and procedure” prevails, together with the need to mediate
between ideological and practical differences often very hard. Political
decisions have the aim to harmonise the needs of the health sector with
those of other areas (education, social services, transportation, etc.)
evaluating their sustainability. Exempla could be for the “practice
policies” to decide if the effects of anti-dementia drugs have to be
considered as necessary in the present context; for the “service policies”
if the anticoagulation centres have the same social relevance as of the
anti-diabetic centres; and for the “governance policies” to decide the
timing to implement the DRG system in different areas of a large and
Good research (but only if it is really good) may offer a scenario upon
which politicians have the duty to decide, independently assuming the
responsibilities which may not be merely ascribed to scientific data.
The strong debate on the side effects on cerivastatine in these days in
Italy and in Europe could be an example: even in a such technical field
the decision stems on various factors, the epidemiology and clinical data
being only a part of the question, while social, psychological and
economical considerations are the most relevant determinants.
Our Italian experience indicates that the idea to give political and
decisional power to doctors or scientist is rhetoric and not enforceable
in the everyday practice. The needs of a complex society are broader and
more indented of those that may be described through predefined categories
such the ones operating in a neat “laboratory” world.
Renzo Rozzini, MD
Marco Trabucchi, MD
Geriatric Research Group and
Medical Unit for the Acute Care of the Elderly
Poliambulanza Hospital, Via Bissolati 57, 25124 Brescia, Italy
1. Black N. Evidence based policy: proceed with care BMJ 2001; 323: 275-
2. Donald A. Commentary: research must be taken seriously BMJ 2001; 323:
Competing interests: No competing interests
It is with great interest that we have read Nick Black's article on
evidence based policy(1). Having, as one of its priorities, the
identification and proposal of governmental interventions to reduce health
inequities, PAHO has subsidized several studies on health equity, most of
which have had limited impact on policy making.
We therefore agree that changes are needed in the research process,
including the creation of “policy communities”, as was suggested by Lomas.
The concept of a more inclusive work process, involving multiple
stakeholders, has also been described in the area of industrial
In their study on the social distribution of knowledge, Gibbons and
collaborators point out the need for a higher level of interaction between
researchers, decision-makers, users, and funders in the identification and
formulation of problems. Research agendas are said to be defined by an
application context, i.e. an ad hoc issue-oriented coalition or long-
standing alliance between different players interacting to solve problems
While we agree that more interaction between health researchers and
policy-makers is needed, we firmly believe that the end-users, the members
of civil society, are also important stakeholders of health research and
should therefore be regarded as key players within the proposed policy
communities. This idea forms the basis of PAHO’s technical cooperation
strategy, called DECIDES (the Spanish acronym for “Democratizing Knowledge
and Information for the Right to Health”).
Within the framework of DECIDES, PAHO’s Research Coordination Program
and Mexico’s FUNSALUD convened a meeting this summer on the “Utilization
of Health Research for Decision-Making in Health Equity” (3)in Cuernavaca,
Mexico. From three days of presentations and discussions involving a total
of 70 researchers, decision-makers, representatives of NGOs and
journalists, there emerged a general consensus to use the new technologies
of information and communication to build multisectoral alliances in the
Americas for research in population health, particularly on health equity.
1. Black, N. “Evidence based policy: proceed with care”, BMJ 2001;
2. Gibbons M, Limoges C, Nowotny H, Schwartzman S, Scott P, Trow M. The
New Production of Knowledge: The dynamics of science and research in
contemporary societies. Thousand Oaks, CA: SAGE Publications, 1999: 46-69.
3. Supported in part by the Rockefeller Foundation
Competing interests: No competing interests
In his paper on evidence base policy, Black(1) confirms what primary
care practitioners and managers have known for years - that the resources
invested into research activity are not commensurate with their impact on
policy making at grass roots level.
Despite a burgeoning literature, General Practitioners still don’t
follow guidelines(2) but make do with pragmatic, adaptive characteristics
implementing small changes that they consider as evolving and improving on
current systems(3). And despite an emphasis on a primary care led NHS,
even the most pragmatic of health economic techniques have little impact
on decision making at grass roots level(4). The environment is one of “
organisations under siege, barely coping” with managers good at presenting
“glossy corporate images that belie the problems of working in
organisations as complex as the NHS”(5) .
An important reason for this dissonance between research rhetoric and
health service reality is that research activity remains largely within
University academic departments. Here researchers work within a fixed
hierarchical structure that can inhibit flexibility and innovation.
Funding spirals, assessment exercises and internal politics often divorce
researchers from their service commitments and there remains a dissonance
from the view of the world of those who guide decision making in health
care and those who commission and provide it(6).
University academia creates theory which practitioners and managers
are expected to apply in practice. But knowledge is institutionalised and
entrapped within a particular world view that invariably bears no
relationship to the requirement of end users who need pragmatic coping
strategies that allow them to process citizens through the health care
Although research general practices and networks have evolved they
remain within a University framework. It is time for university
researchers to get out of their ivory towers and into the trenches. To
develop decision-making frameworks that facilitate the complex and
contingent nature of decisions that are made by policy makers in the real
Until they do so, evidence based policy should not proceed with care
as Black suggests. Its considerable opportunity cost should be recognised
and the scarce resources that it consumes be devoted to areas of
healthcare where there is a more direct and pressing need.
1. Black N. Evidence based policy: proceed with care. BMJ
2. Haynes B, Haines A. Barriers and bridges to evidence based
clinical practice. BMJ 1998;317:273-6.
3. Salisbury C, Bosanquet N, Wilkinson E, Bosanquet A, Hasler J.
The implementation of evidence based medicine in general practice
prescribing. British Journal of General Practice 1998;48:1849-1851.
4. McIver S, Baines D, Ham C et Al. Setting priorities and managing
demand in the NHS. Health Services Management Centre. University of
5. Marshall M. Improving quality in general practice: qualitative
case study of barriers faced by health authorities. BMJ 1999;319:164-7.
6. Kernick D, Stead J, Dixon M. Moving the research agenda to where
it matters. BMJ 1999;329:206-7.
Competing interests: No competing interests
Nick Black's discussion of the problems of evidence-based policy
seems to conclude that its all very difficult . Without wanting to
appear too gung-ho about the prospects for getting health managers and
policy makers to use research more productively, I would suggest that such
timidity is not the best way forward. Researchers should surely be
asserting the value - to policy makers and society - of the evidence they
produce. They are already engaging with policy makers and other
stakeholders to build the kind of "policy community" which Black concludes
First, few people would argue with the principle that health policy
should be evidence-based, that is that it should make full and proper use
of research findings and research methods in policy development,
implementation and evaluation. But is misleading to impute, as Black
seems to, that this would mean that every policy decision should be based
on research evidence and that other values or factors would be ignored.
It simply means that research evidence plays its part alongside other
influences in the policy process, when it has often been almost entirely
absent in the past.
Second, Black sets up a mechanistic, linear model of research
dissemination and implementation which no researcher or policymaker I know
would support, and then proceeds to knock it down rather easily.
Researchers and policy makers are not as naive and uninformed about each
others' worlds as his paper suggests, and one of the most encouraging
trends of recent years has been the increasing dialogue and interplay
between the two groups. Donald's commentary  captures nicely the sense
of a change in the zeitgeist, towards a more evaluative, interactive and
symbiotic relationship between research and health policy and practice.
Third, those who are interested in seeing the greater use of evidence
in health policy and management decision making can learn a great deal
from the development of evidence-based clinical practice . Indeed,
Black's paper is curiously reminiscent of some of the early scepticism
about evidence-based medicine, and the concerns expressed about its
potential impact. Though the form of evidence-based health policy and
management may be very different from evidence-based clinical practice,
many of the underlying principles and lessons are eminently transferable.
1. Black N. Evidence based policy: proceed with care. BMJ 2001;
2. Donald A. Commentary: research must be taken seriously. BMJ
3. Walshe K, Rundall T. Evidence based management: from theory to
practice in healthcare. Milbank Quarterly 2001 (Sept): in press.
Dr Kieran Walshe
Senior Research Fellow,
Health Services Management Centre,
University of Birmingham, Park House, 40 Edgbaston Park Road,
Birmingham, B15 2RT.
Competing interests: No competing interests