Public health and humanitarian interventions: developing the evidence baseBMJ 2000; 321 doi: https://doi.org/10.1136/bmj.321.7253.101 (Published 08 July 2000) Cite this as: BMJ 2000;321:101
- a Medical Emergency Relief International (MERLIN), London W1M 1HW,
- b Health Policy Unit, London School of Hygiene and Tropical Medicine, London WC1E 7HT
- Correspondence to: N Banatvala, Department of Public Health, Suffolk Health, PO Box 55, Ipswich IP3 8NN
This is the first of four articles
Worldwide, millions of people are annually affected by conflict and over $2bn was spent on non-food emergency aid each year between 1991 and 1997.1 Recently, 30 million people were estimated to be internally displaced and 23 million to be refugees (seeking refuge across international borders), the vast majority of whom were fleeing conflict zones.2 More agencies than ever are working in relief activities; over 200 humanitarian agencies responded to the Rwandan genocide and population displacement.3
Populations affected by armed conflict experience severe public health consequences as a result of food insecurity, population displacement, the effects of weapons, and the collapse of basic health services. 4 5 Though most conflicts after the second world war took place in Africa, the Middle East, Asia, and Latin America, since the end of the Cold War and break up of the Soviet Union we have also witnessed conflicts in Europe and the former Soviet Union, notably in Tajikistan, Chechnya, former Yugoslavia, and Nagorno-Karabakh.6 Increasingly, with relatively few exceptions, conflicts are internal rather than waged between states.
This article argues that the evidence base for humanitarian health interventions should be actively developed and explores mechanisms for its promotion.
Humanitarian interventions are increasingly complex and are difficult and costly to resource
Research to identify effective and efficient approaches to the delivery of aid warrants additional investment
Data on the public health effects of war and on delivery of public health in settings affected by conflict are increasingly being assembled, but the effectiveness of many humanitarian initiatives has not been adequately evaluated
Evaluation of the effectiveness of intervention in conflict settings needs to make explicit the humanitarian principles on which interventions are based
Generating knowledge and promoting an evidence based culture will require collaborative initiatives between implementing agencies, academics, and donors
Incentives to reward lesson learning and derivation of good practice should be explicitly identified
Why is evidence on the agenda?
Current debates regarding evidence based medicine 7 8 and evidence based policy9 have permeated all spheres of health care, including those associated with humanitarian health. Basing policies and practice on the best available evidence is essential to maximising the value of available resources. Key questions regarding the nature of evidence remain: in addition to evidence of effectiveness and efficiency, evidence related to other dimensions of health interventions, such as their humanity, equity, local ownership, and political and financial feasibility, is important. How these relate to humanitarian principles of independence, impartiality, and neutrality warrants further analysis and debate.
Magnitude of the problem
A wealth of evidence has accumulated over the past 25 years on the massive effect of war on public health. 3 10 Refugees and internally displaced people typically experience high mortality immediately after being displaced10; the most common causes of death are diarrhoeal diseases (including cholera and dysentery), measles, acute respiratory infections, and malaria, often exacerbated by malnutrition. 10 11 Morbidity from communicable diseases and psychological distress is common, 12 13 and injuries from firearms, antipersonnel landmines, interpersonal violence, and other causes have not been adequately explored and documented. Disabilities related to injury are likely to require long term health care, and providing such care may be costly. 10 14 Food insecurity, crowding, poor access to water and sanitation, and stress increase susceptibility to illness. The damage and breakdown of infrastructures increases exposure to disease and diminishes opportunities for health. 5 15 Recent data on the negative consequences of sanctions and embargoes further illustrate these points.16 Excess mortality occurs especially in children,11 and unaccompanied and orphaned children and pregnant women are especially vulnerable to a variety of diseases. 17 18
Although there is a wealth of technical knowledge on which to base effective programmes, there are many constraints to implementing timely, efficient, and effective relief programmes. Conditions that are common in the area affected by disaster are often exacerbated, and displaced people may introduce novel infections into a host community or may become susceptible to conditions present within the area to which they have fled. 19 20 Lack of resistance to infection, immaturity of the immune system in very young children, and immunosuppression associated with malnutrition make children especially vulnerable. Despite dramatic improvements in emergency relief, the American Public Health Association concluded that “a large body of information documents the inability of the international community to prevent high rates of suffering and death in virtually all refugee situations … major failings in logistics, administration and an inability to establish sustainable programs are serious barriers to providing effective emergency relief.”21 These problems can be compounded by reactive and often ineffective practices sometimes carried out by inexperienced field teams. 22 23 In one study, experienced logisticians from a variety of non-governmental organisations, given a hypothetical crisis, were in little agreement about how best to provide essential emergency provisions, such as blankets, water, and fuel: “such a lack of consensus among experienced crises operators is both surprising and of concern.”24 These and other reports suggest that relief programmes tend to be ad hoc and would be more effective if they were based on the most up to date and valid knowledge bases, drew on a cadre of more rigorously trained professionals, and assured earlier and more effective programme planning and coordination.25
There is increasing recognition that relief efforts must be accountable both to the affected populations (potential “beneficiaries”) and to their donors. However, there are often few data regarding how potential recipients value or prioritise the aid response. Clearer conceptualisation of what affected populations seek from the international humanitarian response to their needs would be valuable. The UK Department for International Development has indicated a commitment to promoting good practice in humanitarian relief 26 through support to efforts such as the Sphere Project,27 which aims to establish minimum standards for good practice in the humanitarian field, and the Ombudsman Project, 28 29 which seeks to develop an accountability structure to ensure that the views of beneficiaries can be heard and that humanitarian agencies are more accountable to the populations they seek to serve. Governments, in turn, must expect to be challenged on their initiatives to reduce or manage conflict, as should donors in relation to their humanitarian and development assistance policies and practices.
Value for money
The large sums of money disbursed in response to complex emergencies, and the high costs of providing health care in these settings, has led to donors' concerns with “value for money.” Research to identify more effective and efficient approaches to the delivery of aid should therefore be promoted. Indeed, there has been little study to examine the effect of aid on the duration, magnitude, or outcome of war.
A detailed critique of the international response to the 1994 Rwanda crisis concluded that in the face of massive resources from governments and the general public (in the order of $1.4bn between April and December 1994),3 several factors, notably an enhanced level of policy coherence (see box), would have increased effectiveness and value for money.3 There is a dearth of relevant literature on the cost effectiveness of humanitarian interventions, with few exceptions,30 highlighting a major gap in the existing evidence base.
Findings of the joint evaluation of emergency assistance to Rwanda3
Lack of policy coherence
Lack of prior investment in disaster preparedness measures
Lack of humanitarian early warning and contingency planning
Lack of coordination between UN and humanitarian agencies as well as government teams and military contingents
Poor quality healthcare delivery from many non-governmental organisations
Inadequate accountability of agencies and inability of agencies to assess their impact
Poor security in camps
Inadequate food distribution practices and poorly coordinated registration of refugees
Importance of the media in developing the international response
International community slow to provide compensation to communities negatively affected by the displaced population, and as a consequence the host community resentful of the presence of refugees
Inappropriate or poor quality health care has serious negative effects: increased morbidity, mortality, and disability; further spread of communicable diseases; emergence of resistant organisms; community dissatisfaction and distress. Individuals may experience considerable personal risk and costs to reach health services; if quality is poor this is a net disbenefit to all involved.
Increasingly, humanitarian aid workers may be targets of violence,31 and the risks of undertaking operational, evaluation, or research activities merits careful deliberation. While it is difficult to justify operating a programme where the impact is impossible to determine and the project cannot be evaluated, a slavish reliance on quantitative indicators of impact may obscure intended and unintended effects.
Principles of research and evaluation in conflict: generating the evidence
The logistic, safety, and practical difficulties of undertaking research during wars and political violence are considerable.32 Current data are often lacking and historical data have often been destroyed. There are often additional political and resource constraints on undertaking research. In the relief setting, research on health services and systems, programme operation, and health problems is more feasible and usually more appropriate than intervention and aetiological research. Nevertheless, both observational and intervention research is possible, and careful evaluation of ongoing practice and the lessons learned is valuable; such evaluation has been promoted actively by some organisations.
Priorities for health services and systems research are numerous and include understanding how best to upgrade health services for the host population alongside those available to refugees and how to most humanely and efficiently provide good quality services, and identifying key determinants for interagency and intersectoral cooperation and coordination (see box).
Research questions in complex emergencies
Food security—evaluate methods for determining food security and needs during different stages of an emergency
Caring capacity—determine the effect of humanitarian relief on the caring capacity of households and communities
Micronutrition—develop practical approaches to preventing micronutrient deficiencies
Feeding programmes—identify the reasons for low coverage of, or ineffective, feeding programmes
Reproductive health and women's health
Violence—determine strategies to prevent and respond to gender based violence
Basic care—identify core aspects of essential obstetrics care
Sexually transmitted infections—develop cost effective strategies to decrease sexually transmitted infections, including HIV, in displaced populations
Reproductive health packages—assess the value and limitations of the package of reproductive health services in emergencies promoted by UNHCR and partners
Water—determine affordable and efficacious distribution strategies; better determine quality and quantity standards
Cholera—assess research opportunities and feasibility for cholera vaccine
Malaria—undertake research on rapid diagnostic tests and the use of mosquito nets impregnated with insecticide in populations on the move
Acute respiratory infections—improve strategies for case finding and case management
Gender—consider gendered impact of conflict on communicable diseases and on ability to access appropriate services
Health service management
Assessment and resource mobilisation—identify mechanisms to improve the decision making process and the involvement of affected communities
Organisation—develop methods to ensure a rapidly established optimal and coordinated health service
Evaluation and impact—set up approaches for managers to establish or adapt evaluation methods and measures of impact
Data collection—improve tools for population estimation, such as mapping and satellite photographs
Data analysis and use—develop standardised systems for rapid assessment and surveillance; develop manuals, user friendly software, and practice oriented guidelines
Data interpretation—refine evidence-based standards and guidelines
New technologies—define the role of new technologies in collecting, making available, and reporting information
Assessment—develop methods for rapid assessment of health needs and resources required
Service delivery—develop affordable, effective, acceptable, and culturally valid interventions at community level
Violence—determine how to provide appropriate and effective population based care
Ethics and evidence
People caught up in complex emergencies are often highly vulnerable and may have been severely abused. Though conventions and guidelines have been developed to promote the rights of refugee and internally displaced populations, little specific protection is offered to these populations in relation to participating in research. This is especially important given their lack of power and control over their environment. The key elements of an ethical approach are maximising benefit and minimising harm, obtaining informed consent, ensuring confidentiality, and treating individuals with appropriate clinical care and dignity. Humanitarian agencies and their sponsors will need to adopt explicit principles in relation to research and evaluation activity. The promotion of equitable access to services and ensuring that communities have opportunities to benefit from available interventions should also underlie research undertaken in these complex settings. Recent initiatives to establish ethical guidelines for research related to populations affected by conflict, and to ensure that well established ethical guidelines are applied in these settings (J Leaning, personal communication), deserve support. Mechanisms for ensuring the protection of affected populations in relation to research initiatives, some of which may be sponsored by groups with other agendas (for example, development of new drugs or technologies), warrant attention.
Developing evidence: opportunities for partnership
Improving the evidence base requires partnerships between non-governmental organisations, academic units, United Nations and government agencies, donors, and affected communities. New initiatives in evaluation, and in operational and policy research, require an interdisciplinary, transparent, and process oriented approach. The generation of knowledge and the methods used to undertake research must be relevant and appropriate if research findings and recommendations are to be implemented and a cycle of continuous development of good practice and improved standards is to be sustained.
Improved collaboration between individuals in the field and those in the academic environment can help promote an appropriate blend of operational expertise with the collection, analysis, critical interpretation, and dissemination of data. A 1995 Lancet editorial argued that “academics may well be the best people to survey and audit the efforts of humanitarian agencies, mediate their interactions, and help them achieve their common purpose.”35 Whether one agrees or not, it is clear that research institutions could do more by establishing effective partnerships with field-based organisations, and contributing actively to policy formulation, intervention evaluation, good practice dissemination, and training.
Examples of collaborative projects include the Sphere Project, a programme involving a range of non-governmental agencies across the globe in developing and promoting standards of good practice,27 and the Steering Committee for Humanitarian Response (an alliance of several international humanitarian agencies). Both projects aim to develop minimum standards for the delivery of health care in emergencies. However, setting of standards will not necessarily improve the quality of humanitarian response and the accountability of humanitarian agencies to beneficiaries. Ensuring that donor funding to agencies depends in part on their application of the standards of good practice and includes a publicly stated commitment to critical review of their performance, to making available data and assessments of their activities, and to instituting measures to improve practice may be helpful. However, good practice standards have not yet been adequately tested and validated.
Encouraging evidence based practice
Promoting the uptake of good practice is difficult in the emergency aid sector, which is characterised by rapid staff turnover, the perception that there is little time to learn lessons given that there is always another emergency, and the scarcity of resources available for encouraging evidence based practice.
Traditional methods of continuing professional development through printed media, conferences and workshops, and training courses are of value but have inherent limitations.36 Electronic technologies, including the new wireless applications, offer unique mechanisms for keeping practitioners informed of developments and debates and may help to ensure that current practice is increasingly based on evidence. The aid community generates valuable evaluations and critiques of current practice, but more can be done to develop effective and efficient means of disseminating and generalising from field experience. Improving opportunities and funding to facilitate linkages of academic institutions and non-governmental organisations and to establish mechanisms for disseminating and debating key findings with relevant stakeholders— donors, host governments, service providers, and, wherever possible, representatives of affected communities—will increase the likelihood of benefits being derived from earlier investments in research and evaluation.
More formal methods of audit and review of relief programmes may help in developing improved standards of care and in documenting successes and failures, in considering the equity implications of interventions, and in deriving good practice. Project management tools such as the logical framework and other related approaches, such as use of agreed measures of effectiveness, could become helpful disaster management tools in complex emergencies. 27 37
Reporting programme activities and outputs is a basic requirement of donors. Robust evaluation methods can facilitate objective assessment of practice through monitoring indicators of achievement. Donors can encourage good practice not only by determining what has been achieved but by rewarding organisations willing to declare their failures and institute robust corrective measures. Though promoting agency membership of the Code of Conduct for the International Red Cross and Red Crescent Movement and Non-Governmental Organisations38 is valuable, this alone will not assure good practice. The British government has affirmed its will to seek the best possible assessment of needs and a clear framework of standards and accountability from those delivering aid.26 Agencies that continually underperform can expect to receive less support from institutional donors; pressure to demonstrate effectiveness and efficiency is increasingly present.
A tension exists between saving lives by instituting short term, resource intensive humanitarian interventions and promoting longer term health and systems development. In some circumstances, short term aid may impede the identification of political solutions or may fuel ongoing conflict. There are also legitimate concerns regarding the extent to which humanitarian assistance bypasses other health service structures, thus undermining them and reducing their longer term sustainability. Finally, evidence is required on how best to combine the essential professionalism required to manage the public health of populations in such complex settings while maintaining the humanitarian ethos.39 Two of the greatest challenges to humanitarian organisations are to institutionalise a sensitive and inclusive culture informed by evidence and to build sustainable mechanisms of crystallising policy advice from the vast and valuable foundation of field experience.40
Series editor: Anthony Zwi ()
Competing interests Both authors have been involved in actively promoting an innovative linkage between the London School of Hygiene and Tropical Medicine and MERLIN.