2018 Kenya: Real Time Evaluation on Emergency Drought Situation Response in Kenya, 2017
Author: Peter Hailey, Nancy Balfour, Langdon Greenhalgh, Gert Venghaus and Margaret Bacon
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.
In 2016, Kenya began showing signs of an oncoming drought. Upon the official declaration of a drought emergency in February 2017, the UN issued a flash appeal and UNICEF scaled up their response that started in 2016. As such, the UNICEF Kenya Country Office requested that an RTE to be conducted to monitor and enhance UNICEF’s response efforts throughout the most affected areas of Kenya.
The prolonged exposure to drought and continuous, increasingly severe water shortages led to acute livelihood pressures which, in turn, brought about increasing safety problems and an exacerbation of the already critical food insecurity, particularly for the predominantly pastoralist population. The pressures on livelihoods eventually resulted in increasingly negative impacts on health status and alarming prevalence of malnutrition among the population in the affected counties, including global acute malnutrition (GAM) rates of over 30 percent in Mandera (MAN), Turkana (TK) and Marsabit (MAR) counties.
After the Long Rains Assessment in the middle of 2016, it was estimated that 1.25 million people were acutely food insecure and over 320,000 women and children required services for the management of acute malnutrition. In the first quarter of 2017, this had increased to an estimated 2.6 million people across 23 counties in Kenya who were acutely food insecure and over 380,000 women and children requiring nutrition services. This estimate increased to 3.4 million by August 2017 and more than 405,000 requiring nutrition services and was expected to increase further during the course of the last quarter of 2017 and into early 2018
The focus of this RTE is on UNICEF’s particular role in the drought response that took place during 2016 and 2017. UNICEF Kenya Country Office (KCO) responded to the disaster in collaboration with other UN agencies and national partners, including the Government's National Drought Management Authority (NDMA).
Global Emergency Group (GEG) and Centre for Humanitarian Change (CHC) conducted a Real Time Evaluation of UNICEF’s drought response in Kenya with the overall purpose of strengthening both UNICEF’s current and future programming based on real-time feedback and learning. This evaluation focused primarily on WASH, Nutrition and Health in the drought affected counties of Garissa, Tana River, Marsabit, Mandera, Turkana, and Kilifi. The RTE considered the extent to which the lessons learned were applied from the 2011 IASC RTE of the Humanitarian Response to the Horn of Africa Drought Crisis.
The primary purpose of this RTE is to inform and directly influence UNICEF’s ongoing WASH, nutrition and health response in the drought affected areas of Kenya with an emphasis on operational decision-making and effective collective learning. While the RTE primarily addresses the learning aspect, it also provides an important accountability tool (a) to supported communities by assessing the appropriateness, timeliness and relevance of the intervention, and (b) to donors by considering the timely release of funds. More specifically, the evaluation focused on the following OECD-DAC criteria for evaluation:
- Criteria 1 – Relevance/Appropriateness of the response and accountability to affected communities;
- Criteria 2 - Timeliness of action;
- Criteria 3 - Sustainability/Connectedness of the response to the development agenda.
This RTE was designed as a rapid and participatory evaluation. The intended audience for the RTE is the UNICEF Kenya Country Office. Though the RTE was conducted between August 2017 and February 2018, the RTE scope covered the UNICEF drought response from August 2016 to December 2017. The RTE process comprised a thorough desk review, field visits, three interim report presentations, a participatory workshop with key stakeholders, and a final report. The desk review included an in-depth analysis of early warning systems, situation reports, early response plans, briefings, appeals, and other evaluations concerning past drought crises in Kenya and the Horn of Africa. Three rounds of field visits included individual and focus group interviews with key stakeholders and communities affected by the 2017 drought crisis, totalling over 500 interviewees. Three interim report presentations and debriefing in the UNICEF zone offices allowed for real-time feedback from UNICEF and key stakeholders and for real-time adjustments to be made to the response. A final workshop was conducted with key UNICEF stakeholders from each sector as well as partner agencies, including government. The workshop allowed for detailed feedback and discussion with the aim of validating and prioritizing findings and recommendations that emerged throughout the RTE. Altogether, field visits, community discussions, interim presentations, and the final workshop built upon the desk review and served as the basis for the content of the final report.
Findings and Conclusions:
RELEVANCE/APPROPRIATENESS: The UNICEF response was relevant in most areas, particularly as related to health facility level nutrition interventions and integrated outreach (including the Catch-Up Campaign) but could have been even more relevant to addressing water stress if UNICEF had applied a more strategic and context specific approach, with greater emphasis placed on assessing and addressing the specific effects of the drought on water access in each area.
TIMELINESS: The UNICEF response started as early as August 2016 with advocacy and planning to address the deteriorating nutrition situation. RTE data collection demonstrated that the UNICEF nutrition response evolved to match the changing needs. This was prior to the Government of Kenya disaster declaration and the response used both available funding and additional, scaled up resourcing from donors. However, the scale-up of other sectors in UNICEF only started after the GoK declaration of emergency (February 2017). The timing of scaling-up the outreach response was, in part, limited by the reality of resource mobilization, stewardship by all stakeholders and lack of outreach strategy prior to the scale up of the response.
Connectedness and Sustainability: The RTE was able to identify many positive examples of national and county government structures taking the lead in managing the drought response. UNICEF has provided support to system strengthening for governance capacity in the nutrition and health sector and to a lesser extent in WASH, including technical assistance embedded in ministries (national and county levels) and NDMA, as well as building information management and supply management capacities. The RTE found that this assistance directly contributed to the government’s capacity to lead and manage the drought response, irrespective of whether this assistance specifically focused on drought response or on more regular governance capacity.
The recommendations below are listed accordingly to strategic priority and as ranked by the RTE workshop participants. The Sub-Recommendations, listed after each Strategic Recommendation, flow from the Strategic Recommendations and are also listed in order of priority (again as specified and ranked by the RTE workshop participants). Thus, the recommendations are a prioritized listing of the combined efforts of the RTE team and RTE workshop participants.
Strategic Recommendation #1: UNICEF should expand and deepen its system strengthening assistance to government. A specific objective of this support should be to ensure that governance systems are risk informed and that there is drought response capacity at all levels (from the national level to community level).
Strategic Recommendation # 2: UNICEF should further develop strategies for a phased and surged approach (surge up and down) to drought response based on the principle of supporting and strengthening rather than substituting government response.
Strategic Recommendation # 3: UNICEF should use a more demand based response, link with other resilience actors to build on existing programming and community plans, strengthen the information base, strengthen needs assessment and increase accountability to and accountability of communities for the monitoring, planning and response to a drought emergency.
The RTE team identified several areas of lessons learned throughout the process to inform future responses:
- UNICEF KCO and partners’ efforts to track and understand evolving drought impacts definitely improved the timeliness and relevance of response efforts. Though not all sectors were evaluated in this RTE process, it is important to continue these efforts across all sectors moving forward.
- Integrated Outreach programs proved to have predominately positive results and feedback.
- The UNICEF surge approach, while not always as nimble and adaptable as it could be, was generally successful in many elements of the response.
- Reflection on who is qualified as ‘most vulnerable’ is not common in an emergency response with the result that some groups in need may be overlooked (e.g. urban vulnerable and the interdependence between sedentary households in the ASALs and their pastoralist extended family).
- Several elements of the response proved to be timely and effective (i.e. advocacy, planning and assistance). However, communities still perceived the response to be late. This is applicable across sectors evaluated in this RTE but likely applicable to additional sectors and certainly relevant for future responses.
- For the most part, UNICEF was able to effectively strengthen and support the government-led response, increasing capacity to respond in a variety of sectors. This confirms that value of system strengthening for both regular and emergency service delivery.
- The divide between emergency and development program still exists within UNICEF and its partners and somewhat limits the connectedness and effectiveness of response efforts. The success of the surge approach in Nutrition suggests that this is a good model for overcoming this limitation.
- Accountability to rights holders of the response is not as strong as it could be, and is therefore an opportunity for improvement.
Please find the attached labelled as follows:
- Evaluation Report - Report
- GEROS Evaluation Review - Part 2
- GEROS Feedback Summary - Part 3
- Evaluation Management Response (EMR) - Part 4
Full report in PDF
PDF files require Acrobat Reader.