We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2016 Yemen: Evaluation of Humanitarian Assistance (EHA)



Author: Nadeem Haider (Evaluation Leader)

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report, and the executive feedback summary labelled as ‘Part 4’.

Background:

The current crisis in Yemen has its roots in the political instability in the country, which led to spiralling of violence in 2009. Yemen went through political upheaval in early 2011, when protesters – inspired by the Arab Spring  – rallied against the rule of President Ali Abdullah Saleh. The protesters forced the President (in office for the past three decades) to resign. However, the violence continued unabated as a result of the Government’s conflict with Al-Houthis, whose southward expansion resulted in the dislodging of the subsequent Government, led by President Hadi, in January 2015.

In February 2015, Al-Houthis declared themselves in control of the government and installed an interim Revolutionary Committee led by Mohammed Ali Al Houthi. The deposed president first escaped to the port city of Aden and subsequently left the country and took refuge in Saudi Arabia.

Yemen remains the most impoverished country in the region. The country was already going through a protracted humanitarian crisis, which became further complicated after the military strikes by the coalition. The escalation in hostilities led to multiplying the humanitarian needs. Reportedly, the crisis resulted in more than 21 million people in need of humanitarian assistance, including 10 million children.  In view of the worsening humanitarian situation, the United Nations declared an L3 emergency and launched a large-scale humanitarian assistance programme.

The declaration of an L3 or Corporate Emergency (as referred by UNICEF) prompted the United Nations agencies to revise and launch a bigger inter-agency Yemen Humanitarian Response Plan (YHRP) 2015. Significant priorities of the revised plan included saving lives, protecting civilians, building capacities for humanitarian response, reducing vulnerabilities and ensuring equitable assistance. 

Purpose/Objective:

The primary purpose is to ‘learn from the management of UNICEF's Yemen humanitarian response following the escalation of conflict in March 2015’. It is part of the regional learning initiative of multiple country humanitarian evaluations.
It is expected that the findings, analysis and learning shall form a critical review and consequent revision of UNICEF preparations and responses to future complex emergencies, particularly in the context of its operations. This undertaking also attempts to test the assumption or hypothesis –  i.e., if, and to what extent, do efficient and effective operations of UNICEF enable achieving response appropriateness and intended coverage in humanitarian actions.
The specific objectives of the evaluation were as follows:

  1. To assess and comment on the level of compliance with the standards of the  CCCs across six operational units and two cross-cutting units – i.e., Rapid Assessment, Monitoring and Evaluation (RAME) and Supplies and Logistics (S&L);
  2. To map and analyse the extent to which UNICEF’s humanitarian assistance was appropriate;
  3. To assess and comment on the ability of UNICEF to identify and reach out to the most vulnerable groups;
  4. To assess if and to what extent compliance of CCCs (or otherwise) enable or hamper UNICEF Yemen to deliver an appropriate humanitarian response and achieve the desired level of coverage – i.e., reach out to the most vulnerable, those facing life-threatening situations.

Methodology:

The evaluation draws on the qualitative methods. For the particular security situation in Yemen, this evaluation has largely been implemented from a distance. The evaluation relies heavily on secondary information available in the form of reports and other documents. The primary data collected complement the literature review. The primary data collection relied on semi-structured key informant interviews, with a wide range of stakeholders including UNICEF, United Nations agencies, implementing partners, media representatives, third-party monitors and community leaders. All interviews were conducted from a distance while using modern communication mediums and tools such as Skype, phone, emails and WhatsApp. The evaluators conducted 44 key informant interviews in total.

To make the compliance assessment process objective and evidence-driven, the evaluators developed and applied a ‘Three Dimensional Framework’, to assess and rate the level of compliance with the CCCs prescribed normative standards and actions for operations. The three dimensions include; i) performance, ii) documentation and iii) achievement/contribution to the intended result (more details available in the report). It was on this assessment that compliance was assessed or scaled from non-compliant to compliant.

The standard Organisation for Economic Co-operation and Development (OECD)-Development Assistance Committee (DAC) criteria have been used for assessment of the appropriateness and coverage. The influence of management of humanitarian response on appropriateness and coverage has been done while corroborating the degree of compliance with CCCs standards and the extent which they either positively or negatively affected the results  vis-à-vis appropriateness and coverage.

Findings and Conclusions:

Overall, UNICEF Yemen operations achieved a high level of compliance with the relevant CCCs standards, despite a challenging security environment. The assistance provided was found to be largely appropriate. UNICEF Yemen by and large did manage to achieve the desired level of coverage, while prioritizing the most vulnerable regions/groups. The continued presence on the ground, ability to form and manage effective partnerships (with public and non-public sector partners) and integration/twinning of allied services did help in achieving/exceeding the set targets.

The evaluation did not find sufficient evidence to conclude the relevance of the CCCs (for operations and except RAME) and, similarly, the level of compliance with making assistance more appropriate. However, the evaluators can certainly argue that these standards and their compliance do relate to the criteria of coverage. And the level of compliance with the assessed CCCs did enable UNICEF Yemen to achieve the desired coverage.

The operation did contribute to a range of planning/operational learning. This key learning included: twinning/integration of allied services (health and nutrition) for improved appropriateness, coverage and achieving cost-efficiencies. The diverse and informed partnerships (with a variety of stakeholders) helped achieve immediate scale-up, effective integration, and leveraging the spread and resources of partners to achieve desired coverage. There are lessons learned around proactive engagement with Saudis for de-confliction approaches/actions, which facilitated the safety and security of staff and supplies. The management of S&L by establishing a regional logistic hub (the one established in Djibouti) was proven useful. This is a good example not only for UNICEF but for others to consider/replicate in the future. UNICEF could not deploy surge capacities fully, hence there are lessons for country and regional offices to critically evaluate and re-think surge mobilization.

Recommendations:

  1. UNICEF may need to demonstrate leadership by taking relevant stakeholders (may be a level of Inter-Agency Standing Committee (IASC)) on board to critically review and revise/adapt CCCs for operation (units), particularly during complex humanitarian situations.
  2. For improved compliance and consistent understanding and use of CCCs, UNICEF is advised to produce a series of guidelines (may be unit-based) and related training packages (for basic users and trainers). For the start, the roll-out may entail nomination of unit-based regional and country-level focal points, who may then lead the implementation of training, adaptation/adoption of standards, and rigorous follow-up.
  3. To overcome the security challenges of undertaking immediate/structured post-disaster need assessments in security-constrained environments, as was the case in Yemen, UNICEF may need explore local-term partnerships with third-party contractors (with the ability to operate in insecure environments and capacities to deliver) for such undertakings. UNICEF Yemen demonstrated this by awarding Long-Term Agreements (LTAs) for media and communications (M&C) functions.
  4. UNICEF may need to either establish parallel mechanisms or embed CCCs operations (benchmarks and actions) within ongoing institutional preparedness planning – i.e., EWEA and Humanitarian Performance Monitoring (HPM). By this, the country offices could improve preparedness-level compliance, which by default would have positive impact on response compliance.
  5. UNICEF must promote and create wider organizational acceptance (mainly at the country level) for immediate shifting to SSOPs as soon as a corporate emergency is declared. For this to happen, UNICEF must evolve and implement operations-related humanitarian monitoring mechanisms, similar to those in place for programmatic monitoring.

*Please see report for full list of recommendations.

Lessons Learned:

Please see the report for lessons learned.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2016

Country:
Yemen

Region:
MENA

Type:
Evaluation

Theme:
Emergencies

Language:
English

Sequence #:
2016/009

New enhanced search