Introduction

Top Abstract Introduction Methods Results Discussion References

Policymakers have long been concerned with improving the performance of health systems. ^{1 2} Reforms have targeted financing (for example, social health insurance and user charges), provision (for example, managed care, autonomous hospitals), stewardship (for example, regulation of the private sector, health legislation), and resource development (for example, retraining of staff). ^{1 3-5} The impact of these reforms is increasingly being studied, ^{6 7} but for the results to be useful to policymakers across different settings, studies need a consistent framework for assessing performance and a measurable indicator.⁸

The World Health Report 2000 defined three intrinsic goals of health systemsimproving health, increasing responsiveness to the legitimate demands of the population, and ensuring that financial burdens are distributed fairly.⁹ For health and responsiveness, systems should improve levels and reduce inequalities. The report published first attempts to measure the attainment of these goals by 191 countries and considered how well countries were performing given their available resources.⁹ This paper describes the methods used for measuring and monitoring performance of health systems. Since improving health is the defining goal of the health system, we report performance in terms of that goal. Data sources have been given elsewhere.¹⁰

	Methods

Top Abstract Introduction Methods Results Discussion References

Theory
Efficiency is defined as the ratio of the observed level of attainment of a goal to the maximum that could have been achieved with the observed resources. Normally, outputs are zero when inputs are zero. In health, however, health levels would not be zero if there were no health expendituresthat is, no health systems. So to measure the contribution of the health system we have to determine what it achieves in excess of what would be achieved in its absence (the minimum). Accordingly, we define performance as the current level of population health, in excess of the estimated minimum, compared with the maximum achievable level of health given the inputs. Because of the similarity between performance and efficiency, we use the terms interchangeably.

Data
We estimated the efficiency of 191 countries from data for 1993-7. Population health was measured as healthy life expectancy (box). The health system input was health expenditure per capita measured in 1997 US dollars (adjusted for the cost of a generic basket of goods in different settings).

	Results

Top Abstract Introduction Methods Results Discussion References

Table 1 gives the coefficient estimates used in the regression equation to determine efficiency. We investigated numerous specifications of the regression equation, but they gave stable estimates of efficiency and rank.

Table 2 shows the efficiency and ranks for the highest and lowest 10 performers and the United Kingdom. Estimated efficiency varies from 0.08 to nearly 1, implying that although some countries may be close to their potential, others are not reaching anywhere near maximum levels of health. Figure 1 depicts the efficiency for all countries; the full results are available on the BMJ 's website.⁹

Figure 2 shows that efficiency is positively related to health expenditure per capita, especially at low expenditure. Performance sharply increases with expenditure up to about $80 (£53) per capita a year.

	Discussion

Top Abstract Introduction Methods Results Discussion References

Perceptions about the relative performance of health systems in different countries have been based on anecdote or case studies. For example, Sri Lanka and China are believed to have been efficient in producing health, ^{17 18} but our results show that both perform less well than other countries at similar levels of development. On the other hand, Oman performs extremely wellperhaps because it has reduced child mortality from 310 to 18 per 1000 live births over the past 40 years.¹⁹

View larger version (30K): [in this window] [in a new window]   Fig 1.   Global distribution of health system efficiency (performance) in maximising population health

View larger version (24K): [in this window] [in a new window]   Fig 2.   Efficiency (performance) in maximising population health versus health expenditure per capita, with 80% confidence intervals

Our efficiency scores compare current population health levels with the maximum possible for observed levels of health expenditure and education in a country. This does not mean that 100% efficiency can be reached immediately. There will be time lags between some actions and their outcomes, and efficiency in many low performing countries is hampered by civil unrest or a high prevalence of HIV and AIDS (fig 1). Healthy life expectancy is reduced by up to 15 years in African countries with the highest prevalence of HIV, clearly restricting the ability of these systems to reach full efficiency in the short term.

Validity of findings
Although other non-health variables affect health (housing quality, environmental conditions, etc), relevant indicators are difficult to find or estimate for many countries. In addition, many are highly correlated with educational attainment, which we used because it functions as a broad measure of non-health inputs.

Reasons for inefficiency
We found that efficiency is positively related to health expenditure per capita. Performance increased greatly with expenditure up to about $80 per capita a year, suggesting it is difficult for systems to be efficient at low expenditure. There seems to be a minimum level of health expenditure below which the system simply cannot work well. We estimate it would cost just over $6bn a year (<0.3% of global annual health expenditure) to increase health spending to this threshold in the 41 countries with lowest expenditures.

Future research
Our conclusions are, of course, tentative. The quality of data across countries varies greatly, and only some of this is accounted for in our uncertainty analysis. Our main objective was to show that the attainment and efficiency of health systems can be measured and compared across countries and over time. Much can be done to improve the data and methods, and WHO is currently working on this with member countries and academic experts. We believe this is critical work for health policymakers considering reforms. Without the ability to measure the inputs and outputs of health systems, they cannot know if the reforms achieve their objectives.

	Acknowledgments

The views expressed are solely those of the authors and do not necessarily represent those of WHO.

	Footnotes

Contributors: DBE conceived the idea of applying the frontier production function approach to measuring the performance of health systems, supervised the performance research team, and wrote and revised the manuscript. AT, CJLM, JAL, and DBE developed, performed, and interpreted the econometric analysis. AT and CJLM developed the methods for uncertainty analysis, and AT and DBE put together the data required for the educational attainment variable. CJLM coordinated the World Health Report 2000 research teams, conceptualised the framework for analysing and measuring attainment and performance, and contributed to the development of the health system assessment framework. JAL estimated historical levels of health system attainment. All four authors revised the manuscript and approved the final version. Raymond Hutubessy, Yukiko Asada, and JAL researched historical income and education levels. Alan Lopez, Colin Mathers, Ritu Sadana, Josh Salomon, Omar Ahmad, and Doris Mafat estimated life expectancy and healthy life expectancy. Jean-Pierre Pouillier, Patricia Hernandez, and Chandika Indikadahena estimated health expenditure. Julio Frenk had a major input to the health system assessment framework. DBE is guarantor.

Funding: None.

Competing interests: None declared.

Further details of the methods and full results are available on the BMJ's website

	References

Top Abstract Introduction Methods Results Discussion References

1.	Maynard A, Bloor K. Health care reform: informing difficult choices. Int J Health Plann Manage 1995; 10: 247-264[Medline].
2.	Collins C, Green A, Hunter D. Health sector reform and the interpretation of policy context. Health Policy 1999; 47: 69-83[CrossRef][Medline].
3.	Hussein AK, Mujinja PG. Impact of user charges on government health facilities in Tanzania. East Afr Med J 1997; 74: 751-757[Medline].
4.	Feldman R. The ability of managed care to control health care costs: how much is enough? J Health Care Finance 2000; 26: 15-25[Medline].
5.	Moore M. Public sector reform; downsizing, restructuring, improving performance. Geneva: World Health Organization, 1996 (Forum on health sector reform discussion paper 7).
6.	Durham G, Kill B. Public health funding mechanisms in New Zealand. Aust Health Rev 1999; 22: 100-112[Medline].
7.	Ron A. NGOs in community health insurance schemes: examples from Guatemala and the Philippines. Soc Sci Med 1999; 48: 939-950.
8.	DeRosario JM. Healthcare system performance indicators: a new beginning for a reformed Canadian healthcare system. J Health Qual 1999; 21: 37-41.
9.	World Health Organization. World Health Report 2000. Geneva: WHO, 2000.
10.	Evans DB, Bendib L, Tandon A, Lauer JA, Ebener S, Hutubessy R, et al. Estimates of income per capita, literacy, educational attainment, absolute poverty, and income Gini coefficients for the World Health Report 2000. Geneva: World Health Organization, 2000. (Global programme on evidence for health policy discussion paper No 7.)
11.	Tengs TO, Adams ME, Pliskin JS, Safran DG, Siegel JE, Weinstein MC, et al. Five-hundred life-saving interventions and their cost-effectiveness. Risk Anal 1995; 15: 369-390[CrossRef][Medline].
12.	Murray CJL, Kreuser J, Whang W. Cost-effectiveness analysis and policy choices: investing in health systems. Bull World Health Org 1994; 74: 663-674.
13.	Murray CJL, Evans DB, Acharya A, Baltussen RM. Development of WHO guidelines on generalized cost-effectiveness analysis. Health Econ 2000; 9: 235-251[CrossRef][Medline].
14.	Caldwell JC. Education as a factor in mortality decline: an examination of Nigerian data. Pop Stud 1979; 33: 395-413[CrossRef].
15.	Caldwell JC, Caldwell P. Education and literacy as factors in health. In: Halstead SB, Walsh JL, Warren KS, eds. Good health at low cost. New York: Rockefeller Foundation, 1985:181-185.
16.	Evans D, Tandon A, Murray CJL, Lauer J. The comparative efficiency of national health systems in producing health: an analysis of 191 countries. Geneva: World Health Organization, 2000. (Global programme on evidence for health policy discussion paper No 29.)
17.	Halstead SB, Walsh JA, Warren KS, eds. Good health at low cost. New York: Rockefeller Foundation, 1985.
18.	Hsiao W. What should macroeconomists know about health care policy? A primer. Washington, DC: International Monetary Fund, 2000. (IMF working paper.)
19.	Ahmad OB, Lopez AD, Inoue M. The decline in child mortality: a reappraisal. Bull World Health Org 2000; 78: 1175-1179[Medline].
20.	Tengs TO. Dying too soon: how cost-effectiveness analysis can save lives. Dallas, TX: National Center for Policy Analysis, 1997. (NCPA policy report No 204.)
21.	Tandon A, Murray CJL, Lauer J, Evans D. Measuring overall health system performance for 191 countries. Geneva: World Health Organization, 2000 (Global programme on evidence for health policy discussion paper No 30.)

(Accepted 17 April 2001)