2018 Madagascar: Impact Evaluation of Fiavota Phase 1 Emergency and Recovery Cash Transfer in Madagascar
Author: Mitchell Morey & David Seidenfeld
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.
Madagascar’s « Grand Sud » houses a population of 1.6 million people. The region is extremely dry and only receives on average 500mm of rain per year. The progressive worsening of the situation has been noted since October 2015 with the onset of El Nino, which caused a drastic deficit in rainfall which continued to affect all districts in the South until February 2016. Crop planting in the off-season in June 2015 and the main planting in October and November both failed. In October 2016 a humanitarian response plan was put into place by the government in partnership with the Humanitarian Team. It targeted 850,000 people in need of humanitarian assistance including 221,400 children. One element of the multi-sectoral response was Cash Transfer through FIAVOTA programme (Joint UNICEF – World Bank programme).
The Fiavota programme has four primary objectives: (a) stabilise household revenue, (b) support households rebuilding assets, (c) strengthen access to nutrition services, and (d) support children’s continued school enrolment. The Fiavota transfer combined an initial large transfer called the recovery fund and subsequent smaller monthly transfers. Beneficiary households would first receive 180,000 Ariary (about US$ 60) that they were required to spend on a resilience-building item. Subsequently, households received 30,000 Ariary per month, which was paid on a monthly basis.
During the Emergency Response phase, nutritional aid accompanied the money transfer. The families with registered children received distributions of Plumpy’Sup, a nutritionally dense peanut-based product. In addition, each nutritional site received distributions of basic community nutritional items. Severely malnourished children received a referral to a health centre. All households had the opportunity to attend nutritional and health trainings.
UNICEF commissioned this Evaluation with the purpose of learning about the effects of “cash plus” in environmental humanitarian contexts. This Evaluation will help the government of Madagascar, FID, UNICEF, the World Bank, to understand how to address the acute needs of poor households in drought-affected southern Madagascar using cash transfers and nutritional support programmes. The evaluation occurs as the programme transitions from an unconditional cash transfer to a conditional cash transfer.
Based on discussions with UNICEF and other stakeholders, all parties agreed that the evaluation will cover the following three criteria (i) Relevance: measures the degree to which the programme is aligned with the larger goals of key stakeholders, (ii) Impacts: The evaluation addresses both the expected and unexpected effects of giving cash in a humanitarian context and (iii) Effectiveness: measures the degree to which the goal of the programme (distributing cash to vulnerable populations) was successful.
Fiavota’s impacts focuses on the changes we can attribute directly to the programme. We divided our impact research questions into two main categories: household-level impacts and child-level impacts. The household-level impacts focus primarily on food security and the households’ ability to withstand shocks. The child-level research questions focus on the nutrition, health, and child welfare of the beneficiaries.
The Phase 1 evaluation uses cross-sectional, quasi-experimental methods to estimate the impact of the Fiavota programme on the first phase of beneficiaries. We use propensity score matching (PSM) techniques to create treatment and comparison groups that are similar in terms of the characteristics that the transfer would not affect. With this approach, we estimate how the transfer affected outcomes compared with the counterfactual where they did not receive the programme.
We have a sample size of 2,976 beneficiary households and 2,381 comparison households across 199 treatment fokontany and 46 comparison fokontany. This sample enables us to detect an effect of 0.132 standardised mean difference in children’s stunting, one of the primary nutritional indicators for this study. We can detect this effect with 80% likelihood and 95% confidence.
To arrive at these estimates, we assume the intracluster correlation is 0.02. This value is drawn from a database of nutrition data from sub-Saharan Africa (Seidenfeld, Handa, de Hoop, & Morey, 2017). Further, we assume 25% of the sample are eliminated during the matching process. We assume that child-level covariates will explain 30% of the total variation and that village-level covariates will explain 15% of the total variation.
The evaluation relies on a single household survey instrument to collect household-level and child-level outcome data and anthropometric measurements to capture child growth. There was also a community-level survey that provides supplementary information on the areas.
Findings and Conclusions:
The programme achieves strong relevance with respect to the first goal of saving lives with a focus on food security and improved livelihoods. The programme targets households with children suffering from severe food insecurity due to drought, households that would have to turn to negative coping strategies such as selling off assets, reducing food consumption, and child labour to meet basic needs.
This impact evaluation highlights both the potential for programmes like Fiavota to improve food security and household livelihoods and the challenges of improving these outcomes in humanitarian crisis settings. We find that Fiavota produces impacts on the primary needs of households after 16 months of implementation. In the protective domain, the programme increases food security, overall consumption, diet diversity, and health for adults and children, and enables households to engage in positive coping strategies in the face of shocks. It further leads to improvements in the material well-being of children in terms of possession of clothes and blankets. In the social domain, Fiavota generates large increases in school enrolment. Indeed the impacts produced by Fiavota are some of the largest schooling effects reported for any cash transfer programme, whether conditional or unconditional, at close to 30 percentage point impact on enrolment. The combination of impacts thus improved the overall resiliency of households to manage and cope with shocks, an important goal of the programme.
Yet the programme were limited to move households into a position where they are no longer vulnerable to shocks and food insecurity. Although the programme leads to increased food consumption, beneficiary households remain at very high levels of food insecurity, with 89% reporting that they skip meals due to lack of money and 74% reporting that there is no food in the household due to lack of money, while 60% spent the night without eating and 55% spent the entire day without eating.
The recommendations primarily target implementers and policymakers who decide the programme’s goals:
- The programme demonstrates large impacts on productivity, especially on livestock ownership, signifying that the recovery fund lump sum transfer worked as intended. We recommend maintaining this component of the programme for future beneficiaries as they initially enrol in the programme as it seems to provide a good jump start toward building resiliency.
- Although we did not find programme impacts on young child nutritional outcomes, the evidence suggests that food consumption and access to healthcare rose. This result is common among many child-targeted cash transfers in sub-Saharan Africa. We recommend linking the programme with other services and programmes that also may affect child nutrition to leverage a multidimensional approach to child nutrition. Such services and programmes include improving access to clean water sources, education about water and sanitation practices, and counselling about optimal adolescent, maternal, infant, and young child feeding practices.
- Beneficiaries may have misunderstood key aspects that might affect their behaviour. Similarly, they may have falsely believed that the programme could end abruptly or that they may not know when they will receive their next payment, affecting their spending behaviour. We recommend clear communication about the programme to the community and beneficiaries that may improve programme operations for how people use the transfers.
- This study did not include baseline measures for the comparison group. The lack of baseline measures for the comparison group made us unable to demonstrate that they started at the same place. For these reasons, best practice is considered to include both the treatment and comparison groups in a baseline measure of an evaluation study, and we recommend future studies to follow these best practices.
The Fiavota programme provides several important lessons that stakeholders can apply to future humanitarian crises in Madagascar and other developing countries. We summarize the three primary lessons that we can draw from this evaluation.
- The Fiavota cash transfer and nutrition support programme demonstrates that aid can be mobilized quickly to address an environmental crisis. The government of Madagascar declared a state of emergency in early 2016 and the Fiavota programme was underway by December 2016. It is important that the programme was reaching beneficiaries in less than a year because they would have otherwise been coping with the effects of the drought on their own. In that scenario, they may have used some of the negative coping strategies described in this report. The rapid deployment of humanitarian aid shows that it is possible to address acute needs in a relatively short timeframe.
- The Fiavota programme shows that it is easier to address food insecurity than the nutritional deficits that food insecurity can cause. The report’s findings show that the transfer improved households’ access to food and reduced the frequency of households using negative coping strategies for food consumption. While the transfer reduced wasting and malnutrition measured by MUAC, there were no impacts on stunting and underweight.
- The Fiavota programme demonstrates that beneficiaries often struggle to understand how and why they are eligible for a cash transfer. Even though households were eligible just by registering at the nutrition clinic, beneficiaries often did not understand why they received the transfer.
Please find the attached labelled as follows:
- Evaluation Report - Report
- GEROS Evaluation Review - Part 2
- GEROS Feedback Summary - Part 3
- Evaluation Management Response (EMR) - Part 4
Full report in PDF
PDF files require Acrobat Reader.