We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 EO: Evaluation of the UNICEF Level 3 response to the cholera epidemic in Yemen: A crisis within a crisis



Author: James Darcy, Christophe Valingot, Laura Olsen

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 6’ of the report, and the executive feedback summary here.

Background:

This evaluation, requested by the UNICEF Middle East and North Africa Regional Office (MENARO), was commissioned and managed by the Evaluation Office in New York. It was conducted between December 2017 and March 2018 by an experienced senior evaluator and an epidemiologist with cholera expertise, with the support and direct involvement of staff from the Evaluation Office. Its main purpose was (a) to help inform the organization’s approach to further potential cholera epidemics in Yemen, based on an analysis of lessons from the response to the 2016-2017 outbreak; (b) to provide some basis of accountability for that response; and (c) to add to the organization’s global learning on cholera prevention and response.

Purpose/Objective:

The purpose of the current evaluation is three-fold:

  1. To inform the current and future UNICEF response in Yemen by providing an evaluative analysis of the UNICEF response to the 2017 cholera/AWD epidemic, within the overall context of the Yemen conflict, the epidemiology of the 2017 outbreak, and the wider system response to that outbreak. The intention is that lessons should be identified from this analysis that can help inform current and future responses, and specifically to provide a set of actionable recommendations to strengthen the organization’s response to the current and potential future cholera outbreaks in Yemen. This is particularly pressing given very real prospect of further outbreaks from spring 2018,  and thus the intention has been to provide real-time feedback from the evaluation as it progresses. The interim report mentioned above is part of that ‘real-time’ process.
  2. To provide a (limited) basis for accountability in respect of the 2017 UNICEF response. The evaluation includes an analysis of what UNICEF did, when and where; whether the response was timely, appropriate and effective; and what were the key internal and external enabling and constraining factors. This includes a judgement of UNICEF performance overall, and some related analysis of its key management and other functions. However, the limited scope of this evaluation means that a fuller analysis of internal UNICEF functions and processes as they relate to the Yemen emergency programme is not included.
  3. To add to the organization’s wider institutional learning from its responses to cholera and other recent infectious disease outbreaks. As compared to a ‘standard’ evaluation, this report is in some respects a more technical review. It attempts to locate the Yemen response in the wider context of cholera prevention, preparedness and response, and should thereby have wider relevance for the organization’s approach to cholera globally

Methodology:

This evaluation represents a new approach for UNICEF to humanitarian evaluations, under which the standard evaluation process is accelerated with a view to producing real-time results that can feed directly into programme decision-making. Under this ‘rapid and timely’ approach, the intention is that the period from recruitment of the evaluation team to completion of the draft report should be not more than four months. In the case of this evaluation, due to the decision to expand the current evaluation process to include an interim management report and a workshop discussion of the full draft findings and recommendations, this period has extended to approximately six months.

In order to achieve results in this time frame, some of the standard phases of evaluation must be merged, shortened or undertaken simultaneously. Following the approach outlined above, the primary methods used in the evaluation were key informant interviews (KIIs), particularly with those directly involved in the cholera response, and documentary review, with a focus on planning, monitoring and decision-making. In order to provide a solid factual basis for the evaluation, the team constructed a timeline of key UNICEF decisions and programme delivery dates against the background of the shifting political context and the course of the 2017 cholera/AWD epidemic. Findings from KIIs conducted outside Yemen were triangulated with the results of partner interviews and focus groups discussions with beneficiaries and local volunteers, which were conducted in country by three Yemen-based consultants. Altogether, around 95 interviewees were consulted for the evaluation.

Findings and Conclusions:

Please refer to the full report.

Recommendations:

  • For the full recommendations please see the report. Vaccination campaign: vaccine supply.
  • Regional specialist capacity: epidemiology/cholera.
  • Cholera task force at RO level. 
  • UNICEF and WHO: harmonizing approaches and clarifying roles.
  • Clarification of coordination processes. .
  • Scale-up and securing of preventive WASH work.
  • Strengthen national cholera surveillance and reporting in Yemen.
  • Strengthen community-based surveillance and response capacities.
  • Enhance rapid response capacities.
  • Additional response preparedness measures.
  • Monitoring and quality control.
  • Invest in better understanding of behaviours and transmission contexts.

Lessons Learned:

Please refer to the report attached below.

Please click here for the marketplace poster on this Yemen evaluation report.

Here is the executive summaryE/ICEF/2018/21, EMR report, E/ICEF/2018/22 and the GEROS summary.

Please find the below attachments, labelled as follows:

  • Full evaluation report - Report
  • Management response to the evaluation report - Part 2
  • Summary for the Executive Board (English – Part 3
  • Summary for the Executive Board (French) – Part 4
  • Summary for the Executive Board (Spanish) – Part 5
  • GEROS Evaluation review - Part 6


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2018

Region:
HQ/Global

Country/Office:
Evaluation Office

Coverage:
Yemen

Type:
Evaluation

Theme:
Emergencies

Sequence #:
2018/001

Language:
English

New enhanced search