Intended for healthcare professionals

  1. Research
  2. Impact of Feed the...
  3. Impact of Feed the Future initiative on nutrition in children aged less than 5 years in sub-Saharan Africa: difference-in-differences analysis
CCBYNC Open access
Research

Impact of Feed the Future initiative on nutrition in children aged less than 5 years in sub-Saharan Africa: difference-in-differences analysis

BMJ 2019; 367 doi: https://doi.org/10.1136/bmj.l6540 (Published 11 December 2019) Cite this as: BMJ 2019;367:l6540
  1. Theresa Ryckman, PhD candidate1,
  2. Margaret Robinson, medical student2,
  3. Courtney Pedersen, medical student2,
  4. Jay Bhattacharya, professor of medicine1 3,
  5. Eran Bendavid, associate professor of medicine1 4
  1. 1Center for Health Policy and the Center for Primary Care and Outcomes Research, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA
  2. 2Stanford University School of Medicine, Stanford, CA 94305, USA
  3. 3National Bureau of Economic Research, Cambridge, MA, USA
  4. 4Division of Primary Care and Population Health, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA
  1. Correspondence to: T Ryckman ryckmant{at}stanford.edu (or @rycktessman on Twitter)
  • Accepted 7 November 2019

Abstract

Objective To evaluate the impact of the US government’s Feed the Future initiative on nutrition outcomes in children younger than 5 years in sub-Saharan Africa.

Design Difference-in-differences quasi-experimental approach.

Setting Households in 33 low and lower middle income countries in sub-Saharan Africa.

Population 883 309 children aged less than 5 years with weight, height, and age recorded in 118 surveys conducted in 33 countries between 2000 and 2017: 388 052 children were from Feed the Future countries and 495 257 were from non-Feed the Future countries.

Main outcome measures A difference-in-differences approach was used to compare outcomes among children in intervention countries after implementation of the initiative with children before its introduction and children in non-intervention countries, controlling for relevant covariates, time invariant national differences, and time trends. The primary outcome was stunting (height for age >2 standard deviations below a reference median), a key indicator of undernutrition in children. Secondary outcomes were wasting (low weight for height) and underweight (low weight for age).

Results Across all years and countries, 38.3% of children in the study sample were stunted, 8.9% showed wasting, and 21.3% were underweight. In the first six years of Feed the Future’s implementation, children in 12 countries with the initiative exhibited a 3.9 percentage point (95% confidence interval 2.4 to 5.5) greater decline in stunting, a 1.1 percentage point (0.1 to 2.1) greater decline in wasting, and a 2.8 percentage point (1.6 to 4.0) greater decline in underweight levels compared with children in 21 countries without the initiative and compared with trends in undernutrition before Feed the Future was launched. These decreases translate to around two million fewer stunted and underweight children aged less than 5 years and around a half million fewer children with wasting. For context, about 22 million children were stunted, 11 million children were underweight, and four million children were wasted in the Feed the Future countries at baseline.

Conclusions Feed the Future’s activities were closely linked to notable improvements in stunting and underweight levels and moderate improvements in wasting in children younger than 5 years. These findings highlight the effectiveness of this large, country tailored initiative focused on agriculture and food security and have important implications for the future of this and other nutrition interventions worldwide.

Introduction

Undernutrition is the single greatest risk factor for child morbidity and mortality globally,1 underlying 56% of all deaths in children younger than 5 years and accounting for more than 8000 deaths each day in 2017.1 In 2016, 32% of children younger than 5 years living in sub-Saharan Africa were chronically undernourished, or stunted (low height for age), and 8% were acutely undernourished, or wasted (low weight for height).2 Undernutrition in childhood is strongly linked with diminished physical and cognitive development and lower lifetime earning potential.345

Although the direct causes of undernutrition include insufficient dietary intake, illness, and malabsorption of energy and micronutrients, previous work has identified links between nutritional status and food security, socioeconomic status, and women’s decision making power (so called nutrition sensitive factors).678910 Over the past several decades, health interventions targeting undernutrition have moved away from nutrition sensitive sectors, particularly agriculture, toward nutrition specific programs that focus on energy intake, breast feeding, and micronutrients.11 Nutrition specific interventions have been shown to lead to improvements in growth in children12 but in isolation might be insufficient to reduce stunting.13 A 2013 review concluded that nutrition specific approaches, even at 90% coverage, need to be paired with nutrition sensitive programs to reduce current burdens by more than 20%.12

In response to the 2008 global food price crisis, the US Agency for International Development (USAID), with support from 10 other US government agencies, announced the launch of Feed the Future in 2010. This initiative aims to reduce poverty and stunting among children younger than 5 years by 20% in its target geographies14 using both nutrition sensitive and nutrition specific approaches. Most of the initiative’s investments target food security and agriculture,15 including the promotion of high quality agricultural inputs (eg, weather resistant seeds, fertilizer), agricultural and post-harvest infrastructure (eg, food storage technologies and irrigation systems), financial services for farmers (eg, agricultural lines of credit), and private sector engagement (eg, partnerships with in-country businesses). Feed the Future also supports nutrition specific activities, including breastfeeding promotion and micronutrient supplementation, and provides support for women farmers.1617 The initiative was rolled out in 19 focus countries from 2011 to 2012, including 12 in sub-Saharan Africa, which are the focus of this study (Ethiopia, Ghana, Kenya, Liberia, Malawi, Mali, Mozambique, Rwanda, Senegal, Tanzania, Uganda, and Zambia). Feed the Future concentrates its activities in subnational regions, known as zones of influence (see appendix eTable 1 for additional details on Feed the Future’s activities, strategic priorities, and zones of influence). In general, the US government selected focus countries with high burdens of undernutrition and poverty and the potential for effective partnership (based on country commitment, stability, institutions, and opportunities for growth).16 Funding for focus countries in sub-Saharan Africa totaled around $370m (£288m; €336m) annually through 2015.18

As of 2019, Feed the Future is almost a decade old, but to our knowledge comparative evaluations of the initiative’s impact on nutritional outcomes are lacking. Moreover, few evaluations of large multicountry nutrition programs exist in the literature, and evidence on effectiveness is scarce.19 Such assessments are crucial to improving program design, resource allocation, and scalability for Feed the Future and similar initiatives. We present an impact analysis of one of the world’s largest nutrition and agriculture initiatives on child nutrition outcomes.

Methods

Using a difference-in-differences design, we analyzed the impact of Feed the Future by comparing trends in undernutrition among children aged less than 5 years before and after implementation of the initiative in focus countries compared with countries where the initiative had not been implemented. We compared exposure to Feed the Future at a national level while controlling for individual, household, and country level characteristics. Additional analyses tested underlying assumptions, hypotheses about the effects of Feed the Future over time, and the robustness of our findings to alternate assumptions and model specifications.

Data sources

Our primary outcome of interest is a binary indicator of stunting (height for age >2 standard deviations below the median of a WHO reference population—that is, height for age z score less than −2).20 Reducing stunting is one of the high level objectives of Feed the Future,21 and stunting is both reflective of chronic undernutrition22 and closely associated with socioeconomic conditions and health over a child’s lifetime.23 We additionally analyzed two secondary outcomes for undernutrition: wasting (weight for height z score less than −2) and underweight (weight for age z score less than −2), which are also included in Feed the Future’s goals.21

We drew on data from Demographic and Health Surveys and Multiple Indicator Cluster Surveys, both of which collect anthropometric data (including weight and height measured by survey enumerators according to international standards) along with individual, maternal, and household characteristics from nationally representative samples of households using standardized questionnaires.24 Additional details on both surveys, including data processing and quality assurance, are available elsewhere.2526 Our analysis was restricted to surveys conducted between 2000 and 2017 in low and lower middle income countries in sub-Saharan Africa (chosen because of Feed the Future’s geographic focus, the high burden of undernutrition, and abundant survey data in the region). We calculated nutrition outcomes for each child younger than 5 years based on anthropometric observations in the survey data using the WHO Anthro tool, a Stata macro that calculates z scores, flags implausible observations that fall outside of WHO defined ranges (extreme values, >5 standard deviations or >6 standard deviations away from the mean, depending on the outcome), and determines each child’s stunting, wasting, and underweight status.20

Statistical analysis

We used a difference-in-differences approach to compare trends among children living in Feed the Future countries with those living in non-Feed the Future countries. This method isolates changes in outcomes that are related to Feed the Future exposure while controlling for pre-existing differences, time invariant country level differences, and secular time trends common to all countries.27 Inferences from difference-in-differences models depend on two main assumptions: that outcomes would continue to follow preintervention trends in the absence of the initiative, and that these preintervention trends are similar between treated and control populations.

Difference-in-differences analyses compare treated and untreated populations before and after the initiation of treatment. We thus classified surveys as treated or control and by whether they preceded or followed Feed the Future’s implementation. Surveys conducted in the 12 Feed the Future sub-Saharan Africa focus countries were classified as treated and surveys from other countries were classified as control. Although exact implementation dates vary by country, we assumed Feed the Future could begin to have an effect starting in 2012, the year when commencement of the initiative’s activities was announced. The appendix includes alternative survey classifications.

We used unadjusted and adjusted linear probability models. In our unadjusted models, we isolate the change in nutritional outcomes unique to treated countries after implementation of Feed the Future, controlling only for trends in outcomes before implementation of the initiative across both treated and control countries, and post-Feed the Future trends in control countries. The main predictor variable in this model was an interaction term between binary indicators of whether the child resided in a Feed the Future country (FTF) and whether the observation was made after the initiative’s implementation (post) (equation 1):

Equation 1: Yitc=β0+β1(FTFc)+β2(postt)+β3(FTFc×postt)+εitc

The coefficient on this interaction term (β3) reflects the unadjusted average difference in the proportion of children living in Feed the Future countries after the initiative’s implementation who are stunted, wasted, or underweight, compared with children living in control countries and before implementation of Feed the Future. In equation 1, the subscripts i, t, and c refer to an individual, period (eg, year), and country, respectively. We used linear models in these analyses because of their simple interpretability and lower computational requirements, but we also tested logistic regression models in sensitivity analysis.

In our adjusted models, we control for several additional factors that could affect or explain trends in nutritional outcomes, improving our ability to isolate the impact of Feed the Future and controlling for sources of potential misattribution. In our fully adjusted analysis (equation 2), we included individual level covariates (Xitc) from the Demographic and Health Surveys and Multiple Indicator Cluster Surveys for the child’s sex, age (in years), maternal education, mother’s age at child’s birth, number of siblings younger than 5 years, household setting (urban or rural), household size, whether the survey was administered during the rainy season, and access to an improved drinking water source; country level covariates (Ztc) from other sources for gross national income per capita,28 life expectancy at birth,28 a governance score calculated from the World Bank’s World governance indicators,29 diphtheria-tetanus-pertussis (DTP3) vaccination coverage,30 and non-US nutrition and agriculture official development assistance31; country indicator variables (αc); and time indicator variables (αt). Appendix eTable 2 provides additional details on these covariates. Country and time indicator variables control for time invariant national differences and secular time trends common to all countries, respectively. We selected these covariates based on expert opinion and use in similar studies and to control for the stated selection criteria of Feed the Future.16323334 Importantly, we believe these covariates could affect undernutrition levels but would not be affected by the initiative itself. Alternate selections of covariates are tested in sensitivity analyses, including the omission of some covariates that could plausibly be affected by Feed the Future (eg, life expectancy), alternate governance and official development assistance terms, and varying time indicator variables.

Equation 2: Yitc=β0+β1(FTFc×postt)+βn1Xitc+βn2Ztc+αc+αt+εitc

The coefficient of interest in the fully adjusted model is β1 in equation 2. The non-interacted FTF and post terms from equation 1 are omitted from equation 2 because of collinearity with country and time indicator variables, as is standard in difference-in-differences analyses.35 We also implemented combinations of the unadjusted and fully adjusted models, with and without covariates, country indicator variables, and time indicator variables. In all models, we applied sample weights, which are necessary for survey data to be nationally representative and to avoid undue influence of small countries on results (see methods in appendix), and estimated standard errors clustered at the level of stratums, a combination of subnational region and urban or rural setting. Clustering allows for unexplained variation in nutrition outcomes to be correlated within a subnational rural or urban area. It is standard practice in program evaluations to cluster standard errors at the level at which a program was implemented.36 Stratums were chosen because Feed the Future operates in specific subnational regions (the zones of influence) and because activities differ appreciably between rural and urban areas. However, in a small selection of sensitivity analyses we varied the level at which errors are clustered.

We hypothesized that a large program such as Feed the Future might have a more pronounced impact over time, after activities have been fully rolled out and have reached the households and children they are designed to help. Many of Feed the Future’s interventions are aimed at strengthening markets or even policies and regulations, and the effects of these interventions could take time to trickle down to households. Thus we also tested whether Feed the Future can be linked to improvements in nutrition outcomes that increase over time. In one model, we added an interaction between the treatment effect term (FTF×post) and the number of years since Feed the Future was first implemented (one year for 2012, up to six years for 2017). This model tests for a linear annual decrease in undernutrition associated with Feed the Future. In another model, we added multiple interaction terms between FTF×post and six indicator variables for each year after implementation of Feed the Future (for example, FTF×post×2014 would equal 1 for observations from Feed the Future countries in 2014 only and 0 for other years and non-Feed the Future countries). This model tests for a varying effect of the initiative on nutritional outcomes each year, with no structure imposed on how this effect changes over time.

We estimated the reductions in total numbers of children stunted, wasted, and underweight that can be attributed to Feed the Future by combining our results with data on the population of children aged less than 5 years in each country and national undernutrition prevalence estimates from secondary sources.237 To estimate the decrease in prevalence associated with Feed the Future we applied the percentage point reductions derived from our fully adjusted analysis. The product of the difference between before and after prevalence estimates and population provides an approximate number of cases of undernutrition averted in children. Some children experience multiple undernutrition conditions; we did not treat these children differently in our estimates.

The causal strength of a difference-in-differences analysis depends on the assumption that outcomes would be similar between treatment and control groups in the absence of the intervention. This assumption was probed by examining whether treated countries and control countries were on similar undernutrition trajectories before the initiative’s implementation. To statistically assess pre-Feed the Future trends in outcomes, we ran regressions of each outcome on survey year, a binary indicator of whether the observation was from a Feed the Future country or not (same as the FTF variable in equations 1 and 2) and an interaction between the two variables (FTF×year). If the coefficient on the interaction term is not statistically significant, this is a sign that pre-trends were relatively similar between treated and control populations. As is common in difference-in-differences analyses we also assessed the parallel trends assumption graphically.

We tested the robustness of our findings through a range of additional sensitivity analyses, including falsification tests in which we measure the impact of Feed the Future on outcomes that it should not plausibly affect and in years before the initiative was implemented, when future selection as a Feed the Future focus country should not be associated with improvements in nutritional outcomes. We hypothesized that Feed the Future would have a greater impact in countries where its zones of influence represent a greater share of the population, and thus we tested models excluding Feed the Future countries where fewer than half of the stunted children lived in zones of influence. Since most stunting takes place in the first 1000 days after conception,38 we tested a model that focused on the association between Feed the Future and declines in stunting among only those children who were aged less than 2 years when the initiative was rolled out. Finally, since many Feed the Future interventions are focused in rural areas, we conducted an analysis that limited our sample to rural households. The appendix provides detailed descriptions of these and other sensitivity analyses. All analyses were conducted in Stata 15.39 Our analysis and reporting conform to STROBE guidelines (see appendix eTable 3) for observational studies.40

Patient and public involvement

This study draws on secondary data collected as part of two large surveys and made available for academic research. Patients and the public were not directly involved. Dissemination to study participants and patient organizations is not applicable.

Results

Data were assembled from a total of 118 Demographic and Health Surveys and Multiple Indicator Cluster Surveys: 49 surveys from all 12 Feed the Future focus countries in sub-Saharan Africa and 69 surveys from 21 control countries (table 1). The cleaned dataset included 883 309 children aged less than 5 years (table 1, appendix eTables 4 and 5). Across the 18 years and 33 countries, 38.3% of children in the study sample were stunted, 8.9% showed wasting, and 21.3% were underweight. Table 2 shows Feed the Future group level statistics before 2012 for the countries in the analysis. Baseline absolute differences in wasting prevalence, average age, maternal education, average family size, urban or rural residence, season when households were surveyed, governance, life expectancy at birth, and development assistance for nutrition and agriculture were statistically significant (whereas stunting and underweight prevalence were similar). These characteristics were controlled for in the analysis. Based on both graphical and regression assessments of data from pre-Feed the Future years, trends in stunting, wasting, and underweight before the implementation of the initiative were generally parallel between treated and control countries (appendix eTable 6 and eFigure 1). Before Feed the Future’s rollout, prevalence of stunting and underweight had been decreasing by almost 1 percentage point per year, while prevalence of wasting was decreasing by less than 0.5 percentage points annually.

Table 1

Survey years, total number of children surveyed across all years for each country, and 2015 total population of children aged less than 5 years in each country

View this table:
Table 2

Baseline comparison of Feed the Future focus countries with non-Feed the Future countries from surveys conducted between 2000 and 2011

View this table:

Table 3 shows the unadjusted and adjusted regression results for both primary and secondary outcomes. A statistically significant decrease occurred in all three measures of undernutrition. When covariates, time periods, and countries (model 1, table 3) were not adjusted for, being a Feed the Future country after implementation of the initiative was associated with relative declines of 5.5 percentage points in stunting prevalence and 4.0 percentage points in underweight prevalence (fig 1) and was not associated with a statistically significant decrease in wasting. The coefficients of interest for stunting, wasting, and underweight are significant under all other specifications, including with and without covariates, country indicator variables, and time indicator variables (models 2-8 in table 3). Reporting and interpretation are focused on the fully adjusted regression results, corresponding to equation 2 in the Methods section (model 8, table 3). After Feed the Future was implemented, controlling for all covariates, the average child in a Feed the Future focus country was 3.9 percentage points (95% confidence interval 2.4 to 5.5) less likely to be stunted, 1.1 percentage points (0.1 to 2.1) less likely to show wasting, and 2.8 percentage points (1.6 to 4.0) less likely to be underweight compared with children in non-Feed the Future countries and children surveyed before implementation of the initiative. Unadjusted and adjusted trends by country are also available in the appendix (appendix eFigure 2).

Table 3

Estimates of association between Feed the Future initiative and undernutrition from unadjusted and adjusted regression models

View this table:
Fig 1
Fig 1

Average stunting, wasting, and underweight levels by group before and after implementation of Feed the Future (FTF) based on the data and unadjusted model (model 1 in table 3). Absolute group level differences and difference in these differences (corresponding to coefficient from model 1 in table 3) are overlaid on the graphs

Assuming these improvements are attributable to Feed the Future, the number of children who were prevented from being stunted, wasted, and underweight as a result of the initiative’s activities can be estimated. Feed the Future has resulted in 2.2 million (95% confidence interval 1.3 to 3.1 million) fewer children who are stunted, 614 000 (59 000 to 1.2 million) fewer children with wasting, and 1.6 million (0.9 to 2.2 million) fewer children who are underweight. For context, about 22 million children were stunted, 11 million were underweight, and four million were wasted in the Feed the Future countries at baseline.

This study hypothesized that the impact of Feed the Future would be greater in later years, when surveyed children had been more exposed to the initiative and more policy interventions had been put in place. Results from an additional analysis (with the addition of an interaction term between whether a country is a Feed the Future country and the number of years after the initiative’s implementation) provided evidence that the impact of the initiative on stunting levels has increased over time (table 4, fig 2, appendix eTable 7). Feed the Future was associated with an average annual decrease in stunting prevalence of 1.2 percentage points (0.8 to 1.6) (model 2, table 4), and with larger and more statistically significant overall decreases in stunting in later years (model 3, table 4).

Table 4

Estimates of association between Feed the Future initiative and undernutrition over time from various adjusted regression models

View this table:
Fig 2
Fig 2

Predicted values from fully adjusted model (model 8 in table 3), collapsed to show average trends in pre and post periods by group (Feed the Future (FTF) and non-Feed the Future). Unadjusted data from individual countries are overlaid with dashed lines. See appendix eFigure 2 for results for individual countries

A wide range of sensitivity analyses tested for possible biases in these results. We focus here on stunting, the primary outcome of this study. The findings for stunting remain stable under all these analyses, including models that use logistic regression instead of linear regression; include, omit, or vary country level covariates; exclude or include additional surveys and countries; weight observations differently; and cluster errors at varying levels (appendix eTables 8-10). The estimated effect on underweight remains stable under most sensitivity analyses, whereas the estimated effect on wasting becomes smaller and statistically insignificant in several analyses.

When we restricted our sample to countries where Feed the Future coverage was highest (based on geographic area), the effect size on stunting was more pronounced (4.5 percentage point decrease in stunting prevalence, 95% confidence interval 2.4 to 6.7). The results held when we analyzed only children who could have been affected by the initiative in their first 1000 days, and larger effect sizes were found when analyzing rural populations. When the analysis was limited to only children aged less than 2 years when Feed the Future started, in rural households, in countries where Feed the Future had greater geographic coverage, effect sizes increased substantially; Feed the Future was associated with a 4.6 percentage point decline in stunting (2.1 to 7.2) and a 4.6 percentage point decline in underweight (2.4 to 6.8) (appendix eTable 11).

The results for stunting pass two sets of falsification tests, one in which pre-Feed the Future years are treated as treatment years and the other in which the impact of the initiative is tested on variables Feed the Future should not plausibly affect, such as mother’s education, height, and age, and birthweight for children born before Feed the Future’s rollout but surveyed after (appendix eTable 12). The results also remain stable when single countries are dropped from the analysis, an indication that the findings for stunting are not driven by country outliers (appendix eTable 13, eFigure 3). A detailed summary and discussion of all sensitivity analyses are included in the appendix (see appendix methods, eFigure 4).

Discussion

This study found that implementation of Feed the Future, a multi-billion dollar initiative to reduce undernutrition and poverty by focusing on nutrition, household income, food security, and agriculture, is linked to a 4 percentage point relative reduction in stunting and a 3 percentage point relative reduction in underweight prevalence among children younger than 5 years in its 12 focus countries in sub-Saharan Africa. These absolute reductions translate into 1.2 and 0.8 annual percentage point declines in stunting and underweight, respectively. The magnitude and statistical significance of these findings, especially for stunting, our primary outcome, are robust to all alternate model specifications and sensitivity analyses. We also found that Feed the Future is associated with a 1 percentage point relative reduction in wasting. Our estimated effect sizes correspond to approximately 2.2 million fewer stunted children and 1.6 million fewer underweight children. We find compelling evidence that a multisectoral initiative focused predominantly on agriculture is linked to significant improvements in child nutrition indicators within six years after implementation.

Comparison with other studies

In context, a 4 percentage point decline in stunting over a six year period is substantial although arguably not exceptional. Stunting prevalence has remained persistently high over the past decade, even while other measures of child health have improved markedly. A 2016 systematic review found only 14 programs over 25 years across all low and middle income countries that had demonstrable success at reducing stunting; only four of these were in sub-Saharan Africa, and they generally targeted a narrow population or geographic area.13 A 2012 systematic review on the impact of nutrition sensitive agricultural interventions found little evidence to indicate that they have a substantial effect on undernutrition, particularly stunting.41 Large reductions in stunting have been documented in Latin America and South Asia: 10 percentage points in Peru from 2005 to 2011, 30 percentage points in Brazil from 1974 to 2007, 18 percentage points in Bangladesh from 1997 to 2011, and 15 percentage points in Vietnam from 1993 to 1998.42434445 Many of these success stories translate to reductions in the range of 1-2 percentage points annually, which is consistent with our findings (see table 4). We also find evidence that Feed the Future’s overall impact on stunting and underweight may be increasing over time (models 2 and 3, table 4), which could eventually put its overall effects in line with these other successful initiatives.

Implications of findings

The specific pathways of Feed the Future’s estimated impacts are challenging to isolate because activities vary by country, and the pathways connecting agriculture and nutrition are complex.46 We speculate that three features of the initiative’s design contributed to its effectiveness: its country tailored approach, its focus on underlying drivers of nutrition (particularly smallholder farming), and its large scale and volume of funding. Previous research posits that international efforts for nutrition often fail because of inadequate funding and a lack of focus on national priorities19 and emphasizes the importance of context specific solutions.13 Feed the Future works with in-country experts to identify opportunities across multiple sectors and address them financially at a large scale: annual funding for Feed the Future is comparable with total official development assistance for all nutrition specific initiatives.4748

Evaluations by the US government suggest that Feed the Future succeeded in designing country tailored strategies and focusing on agricultural activities that generate income.21 The program concentrates agricultural interventions on specific foods (known as value chains) in each focus country. Value chains include both staples and nutrient dense crops, and they were chosen based on nutrition, potential for growth and generation of income, local priorities, and cultural applicability (including for women). Most value chains have met production targets and are classified as nutrition sensitive because of their linkages with increased income and food consumption from households’ own production. Future work could further explore these pathways by testing the impact of Feed the Future on agricultural production, poverty, dietary diversity and quality, and women’s position in the household. Other future work could include an analysis of the initiative’s cost effectiveness. Though we estimate that funding for Feed the Future in sub-Saharan African focus countries has totaled $370m annually, a robust cost effectiveness analysis would include detailed cost data and incorporate the impact of the initiative not just on child undernutrition but also on poverty, nutritional status of women and older children, and other important outcomes.

Country selection and identification strategy

Because our study is observational, we carefully consider the possibility that unobserved time varying factors could confound our observed effects. The assumption that pre-Feed the Future trends in nutrition would continue in the absence of Feed the Future is not directly testable. The US government presumably selected countries based on need and capacity, but we do not observe differential trends in undernutrition in the years leading up to its implementation. Still, we account for time invariant and pre-existing differences among countries, and we control for the US government’s stated selection criteria (eg, governance) and other potential time varying confounders as much as possible. We also identify accentuated effects in populations more exposed to Feed the Future by restricting analyses to rural areas, high coverage countries, and children younger than 2 years, increasing the likelihood that we are measuring the effects of the initiative, not unobserved confounders.

Our analytical approach would be threatened only if changes starting in 2012 in primarily Feed the Future countries led to improvements in nutrition but were uncorrelated with regression covariates. We identified a major funding increase to several Feed the Future countries from the Canadian government around 2012 but found that the impact in these countries was not driving our results (appendix eTable 12 and methods). We identified no other initiatives of the same scope, time, and locations, or other drivers (eg, weather events, conflicts) unique to Feed the Future countries starting around 2012, other than the initiative itself. However, we remain cautious in attributing all of the observed nutritional improvements to Feed the Future; as with any observational analysis we cannot completely rule out the possibility that other factors might have also played a role.

Strengths and limitations of this study

Feed the Future is generally administered to a set of subnational regions (zones of influence) in each country. Our analysis was conducted at the national level because precise locations of zones of influence are not always available or have changed since the initiative’s start, and many surveys do not contain detailed data on location. Our regressions thus average effects across treated and untreated areas within focus countries, implying we may underestimate the true effect of Feed the Future on the specific areas where it operates. The estimated reduction in stunting associated with Feed the Future becomes more pronounced when we focus the analysis on Feed the Future countries with greater subnational coverage. This estimate may more closely approximate the initiative’s treatment effect on the treated.

Our analysis is weighted such that large countries have more impact on results, therefore nutrition levels might not be improving in all Feed the Future countries. For instance, we observed stable or flatline trends in Ghana and Uganda (appendix eFigure 2). Though these descriptive analyses are not adjusted for relevant covariates, they do point to the potential for heterogeneous effects across Feed the Future countries and programs. Furthermore, Feed the Future intervened in countries where less than 40% of stunted children in sub-Saharan Africa reside, was active in only parts of these countries, and operated in only five of the 10 sub-Saharan African countries with the highest prevalence for stunting. It remains to be seen whether Feed the Future’s approach would be equally effective in other countries with different burdens, weaker institutions, or less governmental commitment.

Our findings are a substantial addition to the current evidence base on the impact of large scale nutrition initiatives. The US government’s 2016 evaluation of Feed the Future acknowledges that it is “difficult to understand the degree to which ZOI [zones of influence]-wide changes in impact level indicators for poverty and nutrition were driven by Feed the Future interventions or broader long-term trends.”21 We estimate that Feed the Future has prevented 2.2 million children from becoming stunted, whereas the US government evaluation estimates 3.4 million. This difference could be due to the US government evaluation’s lack of a counterfactual comparison but is also partly explained by our focus on only countries in sub-Saharan Africa. If we inflate our estimate by the number of stunted children in all Feed the Future focus countries, our estimate is closer to that of the US government, although this assumes that the impact was similar in non-sub-Saharan Africa countries.

Studies have shown the importance of addressing the indirect determinants of poor nutrition,67810 but to date there has been a lack of large scale, robust analyses evaluating multisectoral initiatives such as Feed the Future that focus on these determinants.10 Most evaluations of nutrition programs focus on interventions implemented in one or two countries at most.134950 Feed the Future’s rollout across several countries offers an ideal natural experiment to infer the effectiveness of a large multifaceted approach at scale. We analyze the impact of Feed the Future using large datasets from two publicly available sources, allowing for future replication, that cover hundreds of thousands of children in more than 30 countries. Our design controls for pre-Feed the Future trends in outcomes, time variant confounders, and Feed the Future’s stated selection criteria to isolate improvements in nutrition uniquely associated with the program. We conduct extensive sensitivity analyses that further verify the robustness of our findings.

Conclusions and policy implications

Evaluations of large donor funded initiatives such as Feed the Future are often conducted with considerable donor involvement. A recent editorial stressed the increasing importance of conducting independent evaluations of such programs.51 Prospectively designed independent evaluations could advance the value of programs and should include better measures of exposure, standardized intermediate and primary outcomes, thoughtful selection of counterfactual populations, sample size calculations that allow for measurement of average and heterogeneous effects, and engagement of beneficiaries (eg, households) to assess both quantitatively and qualitatively how programs succeeded and how they could improve.5152

Our results are consistent with the hypothesis that the average child residing in a Feed the Future focus country is less likely to be undernourished than he or she would have been in the absence of the initiative. Feed the Future is now concentrating its activities on a smaller set of eight focus countries in sub-Saharan Africa. Though our findings indicate initial and increasing progress, it will be several years before we know whether the initiative’s efforts were sustainable in either set of countries. Although a 3.9 percentage point decrease in stunting (and a 1.2 percentage point annual decrease) across several countries is noteworthy, many countries face stunting levels above 30%. The United Nation’s Sustainable Development Goals include reducing stunting by 40% and wasting to less than 5% by 2025.5354 Based on our analysis, Feed the Future may be making a noticeable difference, but its impact alone will not be enough to meet these targets. While we show the promise of an approach such as Feed the Future, addressing the complexities of undernutrition in the long term will require sustained effort across multiple sectors for years to come.

What is already known on this topic

  • Prevalence of child undernutrition in low and middle income countries has remained consistently high, even as other measures of child health have improved

  • Debate is ongoing about the relative effectiveness of programs that address underlying determinants of undernutrition versus those that focus on nutrition directly

  • Feed the Future, an agriculture and nutrition assistance initiative, has been in operation since 2011 and focuses on many of these underlying determinants; however, robust evidence on the program’s effectiveness in reducing stunting is sparse

What this study adds

  • The Feed the Future initiative was associated with meaningful improvements in nutrition outcomes in children younger than 5 years of age

  • Adjusted stunting prevalence decreased 3.9 percentage points more in Feed the Future countries than non-Feed the Future countries, translating into 2.2 million fewer children stunted

  • This analysis supports the value of a multisectoral approach, including a focus on agriculture and food security, to improve nutrition in children

Acknowledgments

This material is based on work supported by the National Science Foundation graduate research fellowship program (grant No DGE-1656518). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

We thank Kiersten Johnson, monitoring and evaluation advisor at the United States Agency for International Development’s Bureau for Food Security, who provided clarifications on the structure and early implementation of Feed the Future.

Footnotes

  • Contributors: EB and TR were involved in initial conceptualization and methodology development. TR and MR cleaned, curated, and analyzed the data. CP, MR, and TR contributed to the first draft of the manuscript. EB, JB, TR, MR, and CP contributed to study design, interpretation of results, development of visualizations, writing, and initial editing. EB and JB were involved in validation of results and conclusions and act as guarantors of the study. All authors have read and approved the final draft of the manuscript. All authors had access to all data in the study and had final responsibility for the decision to submit for publication. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding: This study received no specific funding.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf and declare: EB receives funding from the Doris Duke Charitable Foundation and the National Institute for Allergy and Infectious Diseases; JB was partially supported by the US National Institute on Aging through the Stanford Center for Demography and Economics of Health and Aging (grant No P20-AG17253); TR receives support for her PhD research as a National Science Foundation graduate research fellow and Stanford graduate fellow in science and engineering; no support from any organization specifically for the submitted work, no financial relationships with any organizations that might have an interest in the submitted work in the previous three years, and no other relationships or activities that could appear to have influenced the submitted work. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

  • Ethical approval: Not required.

  • Data sharing: All data used in the analysis are available through the Demographic and Health Surveys and Multiple Indicator Cluster Surveys websites for academic researchers.

  • The lead author (TR) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

http://creativecommons.org/licenses/by-nc/3.0/

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

References

  1. ↵
    GBD Compare. IHME Viz Hub. https://vizhub.healthdata.org/gbd-compare (accessed 30 Aug 2019).
  2. ↵
    WHO. UNICEF-WHO-The World Bank: Joint child malnutrition estimates - Levels and trends. WHO. www.who.int/nutgrowthdb/estimates/en/ (accessed 22 Aug 2019).
  3. ↵
    1. Almond D,
    2. Currie J
    . Killing Me Softly: The Fetal Origins Hypothesis. J Econ Perspect2011;25:153-72. doi:10.1257/jep.25.3.153. pmid:25152565
    OpenUrlCrossRefPubMed
  4. ↵
    Food and Agriculture Organization. Understanding the true cost of malnutrition. Rome, Italy: FAO 2014. www.fao.org/zhc/detail-events/en/c/238389/ (accessed 10 Oct 2017).
  5. ↵
    Food and Agriculture Organization. The State of Food and Agriculture. FAO 2013 www.fao.org/publications/sofa/2013/en/ (accessed 8 Jan 2019).
  6. ↵
    1. Nguyen PH,
    2. Headey D,
    3. Frongillo EA,
    4. et al
    . Changes in Underlying Determinants Explain Rapid Increases in Child Linear Growth in Alive & Thrive Study Areas between 2010 and 2014 in Bangladesh and Vietnam. J Nutr2017;147:462-9. doi:10.3945/jn.116.243949. pmid:28122930
    OpenUrlAbstract/FREE Full Text
  7. ↵
    1. Smith LC,
    2. Ramakrishnan K,
    3. Ndiaye A,
    4. et al
    . The Importance of Women’s Status for Child Nutrition in Developing Countries.International Food Policy Research Institute, 2003, http://ageconsearch.umn.edu/bitstream/16526/1/rr030131.pdf.
  8. ↵
    1. Akseer N,
    2. Bhatti Z,
    3. Mashal T,
    4. et al
    . Geospatial inequalities and determinants of nutritional status among women and children in Afghanistan: an observational study. Lancet Glob Health2018;6:e447-59. doi:10.1016/S2214-109X(18)30025-1. pmid:29454554
    OpenUrlCrossRefPubMed
  9. ↵
    1. Carletto G,
    2. Ruel M,
    3. Winters P,
    4. et al
    . Farm-Level Pathways to Improved Nutritional Status: Introduction to the Special Issue. J Dev Stud2015;51:945-57doi:10.1080/00220388.2015.1018908.
    OpenUrlCrossRef
  10. ↵
    1. Ruel MT,
    2. Alderman H,
    3. Maternal and Child Nutrition Study Group
    . Nutrition-sensitive interventions and programmes: how can they help to accelerate progress in improving maternal and child nutrition?Lancet2013;382:536-51. doi:10.1016/S0140-6736(13)60843-0. pmid:23746780
    OpenUrlCrossRefPubMedWeb of Science
  11. ↵
    1. Herforth A,
    2. Tanimichi-Hoberg Y
    . Learning from World Bank history: agriculture and food-based approaches for addressing malnutrition.World Bank Group, 2014, http://documents.worldbank.org/curated/en/497241468168227810/Learning-from-World-Bank-history-agriculture-and-food-based-approaches-for-addressing-malnutrition.
  12. ↵
    1. Bhutta ZA,
    2. Das JK,
    3. Rizvi A,
    4. et al.,
    5. Lancet Nutrition Interventions Review Group, the Maternal and Child Nutrition Study Group
    . Evidence-based interventions for improvement of maternal and child nutrition: what can be done and at what cost?Lancet2013;382:452-77. doi:10.1016/S0140-6736(13)60996-4. pmid:23746776
    OpenUrlCrossRefPubMedWeb of Science
  13. ↵
    1. Hossain M,
    2. Choudhury N,
    3. Adib Binte Abdullah K,
    4. et al
    . Evidence-based approaches to childhood stunting in low and middle income countries: a systematic review. Arch Dis Child2017;102:903-9. doi:10.1136/archdischild-2016-311050. pmid:28468870
    OpenUrlAbstract/FREE Full Text
  14. ↵
    United States Agency for International Development (USAID). Feed the Future Snapshot: Progress Through 2017. Washington, D.C.: USAID 2017. https://feedthefuture.gov/sites/default/files/resource/files/2017%20Feed%20the%20Future%20Progress%20Snapshot.pdf (accessed 10 Oct 2017).
  15. ↵
    1. Elliott K,
    2. Dunning C
    . Assessing the US Feed the Future Initiative: A New Approach to Food Security.Center for Global Development, 2016.
  16. ↵
    United States Agency for International Development (USAID). Feed the Future Guide. Washington, D.C.: USAID 2010. https://feedthefuture.gov/sites/default/files/resource/files/FTF_Guide.pdf
  17. ↵
    1. Du L,
    2. Pinga V,
    3. Klein A,
    4. Danton H
    . Leveraging Agriculture for Nutrition Impact through the Feed the Future Initiative. Adv Food Nutr Res2015;74:1-46. doi:10.1016/bs.afnr.2014.11.001. pmid:25624034
    OpenUrlCrossRefPubMed
  18. ↵
    Results. Feed Future. /results/ (accessed 22 Aug 2019).
  19. ↵
    1. Morris SS,
    2. Cogill B,
    3. Uauy R,
    4. Maternal and Child Undernutrition Study Group
    . Effective international action against undernutrition: why has it proven so difficult and what can be done to accelerate progress?Lancet2008;371:608-21. doi:10.1016/S0140-6736(07)61695-X. pmid:18206225
    OpenUrlCrossRefPubMedWeb of Science
  20. ↵
    WHO. WHO Anthro (version 3.2.2, January 2011) and macros. WHO. www.who.int/childgrowth/software/en/ (accessed 4 Jan 2019).
  21. ↵
    Briggs L, Vondal P, Vijayakumar C, et al. Feed the Future Global Performance Evaluation Report. Washington, D.C.: USAID 2016.
  22. ↵
    1. Barrett CB
    . Measuring food insecurity. Science2010;327:825-8. doi:10.1126/science.1182768. pmid:20150491
    OpenUrlAbstract/FREE Full Text
  23. ↵
    1. Hoddinott J,
    2. Alderman H,
    3. Behrman JR,
    4. Haddad L,
    5. Horton S
    . The economic rationale for investing in stunting reduction. Matern Child Nutr2013;9(Suppl 2):69-82. doi:10.1111/mcn.12080. pmid:24074319
    OpenUrlCrossRefPubMed
  24. ↵
    1. Hancioglu A,
    2. Arnold F
    . Measuring coverage in MNCH: tracking progress in health for women and children using DHS and MICS household surveys. PLoS Med2013;10:e1001391. doi:10.1371/journal.pmed.1001391. pmid:23667333
    OpenUrlCrossRefPubMed
  25. ↵
    The DHS Program - Data Quality and Use. https://dhsprogram.com/data/Data-Quality-and-Use.cfm (accessed 22 Aug 2019).
  26. ↵
    Tools - UNICEF MICS. http://mics.unicef.org/tools#data-processing (accessed 22 Aug 2019).
  27. ↵
    1. Basu S,
    2. Meghani A,
    3. Siddiqi A
    . Evaluating the Health Impact of Large-Scale Public Policy Changes: Classical and Novel Approaches. Annu Rev Public Health2017;38:351-70. doi:10.1146/annurev-publhealth-031816-044208. pmid:28384086
    OpenUrlCrossRefPubMed
  28. ↵
    World Bank. Indicators. https://data.worldbank.org/indicator
  29. ↵
    World Bank. World Governance Indicators. http://info.worldbank.org/governance/wgi/#home
  30. ↵
    World Health Organization (WHO). Diptheria tetanus toxoid and pertussis (DTP3) Immunization coverage estimates by country. 2017.http://apps.who.int/gho/data/node.main.A827
  31. ↵
    OECD. Detailed aid statistics: Official bilateral commitments by sector. OECD Int. Dev. Stat. Database. 2018.doi:10.1787/data-00073-en
  32. ↵
    1. Bendavid E,
    2. Holmes CB,
    3. Bhattacharya J,
    4. Miller G
    . HIV development assistance and adult mortality in Africa. JAMA2012;307:2060-7. doi:10.1001/jama.2012.2001. pmid:22665105
    OpenUrlCrossRefPubMedWeb of Science
  33. ↵
    1. Jakubowski A,
    2. Stearns SC,
    3. Kruk ME,
    4. Angeles G,
    5. Thirumurthy H
    . The US President’s Malaria Initiative and under-5 child mortality in sub-Saharan Africa: A difference-in-differences analysis. PLoS Med2017;14:e1002319. doi:10.1371/journal.pmed.1002319. pmid:28609442
    OpenUrlCrossRefPubMed
  34. ↵
    1. Lakkam M,
    2. Wager S,
    3. Wise PH,
    4. Wein LM
    . Quantifying and exploiting the age dependence in the effect of supplementary food for child undernutrition. PLoS One2014;9:e99632. doi:10.1371/journal.pone.0099632. pmid:24967745
    OpenUrlCrossRefPubMed
  35. ↵
    1. Bertrand M,
    2. Duflo E,
    3. Mullainathan S
    . How Much Should We Trust Difference-in-Differences Estimates?Q J Econ2004;119:249-75doi:10.1162/003355304772839588.
    OpenUrlCrossRefWeb of Science
  36. ↵
    1. Abadie A,
    2. Athey S,
    3. Imbens GW,
    4. et al
    . When Should You Adjust Standard Errors for Clustering?National Bureau of Economic Research, 2017, doi:10.3386/w24003.
    OpenUrlCrossRef
  37. ↵
    World Population Prospects - Population Division - United Nations. https://esa.un.org/unpd/wpp/ (accessed 9 Dec 2016).
  38. ↵
    1. Black RE,
    2. Victora CG,
    3. Walker SP,
    4. et al.,
    5. Maternal and Child Nutrition Study Group
    . Maternal and child undernutrition and overweight in low-income and middle-income countries. Lancet2013;382:427-51. doi:10.1016/S0140-6736(13)60937-X. pmid:23746772
    OpenUrlCrossRefPubMedWeb of Science
  39. ↵
    1. StataCorp
    . Stata Statistical Software: Release 15.StataCorp LLC, 2017.
  40. ↵
    1. von Elm E,
    2. Altman DG,
    3. Egger M,
    4. Pocock SJ,
    5. Gøtzsche PC,
    6. Vandenbroucke JP,
    7. STROBE Initiative
    . The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet2007;370:1453-7. doi:10.1016/S0140-6736(07)61602-X. pmid:18064739
    OpenUrlCrossRefPubMedWeb of Science
  41. ↵
    1. Masset E,
    2. Haddad L,
    3. Cornelius A,
    4. Isaza-Castro J
    . Effectiveness of agricultural interventions that aim to improve nutritional status of children: systematic review. BMJ2012;344:d8222. doi:10.1136/bmj.d8222 pmid:22251864
    OpenUrlAbstract/FREE Full Text
  42. ↵
    1. Huicho L,
    2. Segura ER,
    3. Huayanay-Espinoza CA,
    4. et al.,
    5. Peru Countdown Country Case Study Working Group
    . Child health and nutrition in Peru within an antipoverty political agenda: a Countdown to 2015 country case study. Lancet Glob Health2016;4:e414-26. doi:10.1016/S2214-109X(16)00085-1. pmid:27198845
    OpenUrlCrossRefPubMed
  43. ↵
    1. Monteiro CA,
    2. Benicio MHD,
    3. Conde WL,
    4. et al
    . Narrowing socioeconomic inequality in child stunting: the Brazilian experience, 1974-2007. Bull World Health Organ2010;88:305-11. doi:10.2471/BLT.09.069195. pmid:20431795
    OpenUrlCrossRefPubMedWeb of Science
  44. ↵
    1. Headey D,
    2. Hoddinott J,
    3. Ali D,
    4. et al
    . The Other Asian Enigma: Explaining the Rapid Reduction of Undernutrition in Bangladesh. World Dev2015;66:749-61. doi:10.1016/j.worlddev.2014.09.022.
    OpenUrlCrossRef
  45. ↵
    1. O’Donnell O,
    2. Nicolás ÁL,
    3. Van Doorslaer E
    . Growing richer and taller: Explaining change in the distribution of child nutritional status during Vietnam’s economic boom. J Dev Econ2009;88:45-58. doi:10.1016/j.jdeveco.2008.01.004.
    OpenUrlCrossRef
  46. ↵
    Gillespie S, Harris J, Kadiyala S. The Agriculture-Nutrition Disconnect in India: What Do We Know? Washington, D.C.: 2012. http://ebrary.ifpri.org/cdm/ref/collection/p15738coll2/id/126958
  47. ↵
    Development Initiatives. Global Nutrition Report 2017: Nourishing the SDGs. Bristol, UK: Development Initiatives 2017. www.globalnutritionreport.org/files/2017/11/Report_2017.pdf (accessed 23 Feb 2018).
  48. ↵
    Henry J Kaiser Family Foundation (KFF). U.S. Funding for International Nutrition Programs. KFF 2016. www.kff.org/global-health-policy/issue-brief/u-s-funding-for-international-nutrition-programs/ (accessed 31 Mar 2018).
  49. ↵
    1. Kim SS,
    2. Rawat R,
    3. Mwangi EM,
    4. et al
    . Exposure to Large-Scale Social and Behavior Change Communication Interventions Is Associated with Improvements in Infant and Young Child Feeding Practices in Ethiopia. PLoS One2016;11:e0164800. doi:10.1371/journal.pone.0164800. pmid:27755586
    OpenUrlCrossRefPubMed
  50. ↵
    1. Menon P,
    2. Nguyen PH,
    3. Saha KK,
    4. et al
    . Impacts on Breastfeeding Practices of At-Scale Strategies That Combine Intensive Interpersonal Counseling, Mass Media, and Community Mobilization: Results of Cluster-Randomized Program Evaluations in Bangladesh and Viet Nam. PLoS Med2016;13:e1002159. doi:10.1371/journal.pmed.1002159. pmid:27780198
    OpenUrlCrossRefPubMed
  51. ↵
    Action to protect the independence and integrity of global health research. BMJ Global Health. https://gh.bmj.com/content/4/3/e001746 (accessed 27 Aug 2019).
  52. ↵
    Theory-based impact evaluation: principles and practice, 3ie. www.3ieimpact.org/evidence-hub/publications/working-papers/theory-based-impact-evaluation-principles-and-practice (accessed 27 Aug 2019).
  53. ↵
    1. World Health Organization (WHO)
    . Global nutrition targets 2025: policy brief series (WHO/NMH/NHD/14.2).World Health Organization, 2014.
  54. ↵
    General Assembly resolution 70/1, Transforming our world: the 2030 Agenda for Sustainable Development, A/RES/70/1 (25 September 2015). undocs.org/A/RES/70/1.
Back to top