We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 Haiti: Evaluation of UNICEF response to Hurricane Matthew



Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report, and the executive feedback summary labelled as ‘Part 4’.

Background:

After cutting a deadly swath across the Caribbean region, Hurricane Matthew, a Category 4 storm with sustained winds of 235 km/h, violently struck south-western Haiti on 4 October 2016, causing widespread damage, flooding and displacement. Fierce winds knocked out communications with the southern part of the island for hours. Hurricane Matthew has caused the largest humanitarian crisis witnessed in the country since the 2010’s earthquake  at a time when the country is already facing a significant increase in the number of cholera cases, and severe food insecurity and malnutrition.

Following the passage of Hurricane Matthew, the most pressing needs included: shelter; food; access to safe drinking water and sanitation; health and nutritional services; education services; protection services, psychosocial assistance and access to critical lifesaving messages and information services. Women and girls have become vulnerable to gender-based violence due to temporary living arrangements. Children have been at increased risk of being placed in “orphanages” for lack of supports of families.

UNICEF response, based on the guiding principles laid down in its Core Commitments for Children (CCC), included providing clean and safe water (Water trucking in shelters, repairs of damaged water systems; potabilization, aquatabs, rehabilitation and equipment of affected schools to allow safe return to schools, health services (mobile clinics) in order to identify health and nutrition needs; protection services in order to minimize negative coping mechanisms; etc.

UNICEF estimated to 42.5 million USD the amount necessary to cover the needs of 1.4 million people in need of humanitarian assistance. UNICEF received close to 31 million USD from traditional and new donors in order to respond to the Emergency. UNICEF was also co-leading 4 emergency working groups (Nut, Edu, Protection and WASH).

Purpose/Objectives:

The purpose of this evaluation is to (i) evaluate UNICEF’s response in this specific humanitarian context, including multi-sectoral provision of services and aid, as well as coordination role, and (ii) to draw lessons learned for application in future emergencies and capacity building of UNICEF Haiti, where appropriate. Secondly, the evaluation is also intended to strengthen accountability and transparency to UNICEF’s partners and donors.

The evaluation will be formative in nature with the primary aim to identify strengths and weaknesses in UNICEF response. The specific objectives were to:

  • Assess the extent to which the preparedness activities have fed into the Hurricane Matthew emergency response
  • Take stock of the emergency response for the onset of the crisis to transition and recovery, highlighting the appropriateness, efficiency and effectiveness of the response
  • Assess both the positive and negative sides of UNICEF response
  • Identify what are the long lasting effect of UNICEF’s response
  • Identify lessons learned and develop recommendations on each separate phase of the response (preparedness, emergency, transition/recovery)
  • Facilitate learning and strengthen UNICEF Haiti’s accountability and transparency towards partners and donors

The three main expected audience are :

  1. UNICEF staff who are directly or indirectly involved in this emergency – including those at the field, national, regional and headquarters levels – who will use the results of this evaluation to fine-tune and calibrate UNICEF’s humanitarian efforts.
  2. UNICEF staff faced with similar emergencies in the future who may choose to use this evaluation as a reference document and use the results to inform their own strategies.
  3. UNICEF donors and implementing partners, both governmental and non-governmental, and beneficiaries, as a mechanism to strengthen accountability and transparency.

Methodology:

The evaluation uses the Core Commitments for Children in Humanitarian Action (CCC). The evaluation framework is based on the evaluation criteria as defined by ALNAP and the OECD / DAC: relevance / adequacy, effectiveness, efficiency and connectivity / sustainability as well as coordination / coherence, coverage / equity and respect for the humanitarian imperative to evaluate UNICEF's emergency response to Hurricane Matthew.

The evaluation uses a mixed approach, using both qualitative and quantitative methods, and used Humanitarian Performance Monitoring (HPM) indicators as a benchmarks. All survey questionnaires have been reviewed and approved by an External Review Board.

A sampling methodology, based on intentional sampling to better answer the evaluation questions, was used to select the areas to be visited and the interviewees. The collection of data was based on different techniques: i) document review, ii) observation during field visits, iii) online questionnaire, iv) key informant interviews, v) focus groups with affected people.

The evaluation team interviewed 107 key informants, including 39 women including UNICEF members (present in the country or remotely), members of UN agencies, the Government as well as at the central and departmental level, international NGOs and national NGOs as well as main donors. The team visited numerous structures in 11 municipalities in the departments of Sud and Grand'Anse, including schools, health centers, hospitals and drinking water supply networks. A total of 107 people, including 87 women, participated in seven focus groups held during field visits.

All deliverables from the evaluation team were validated by a multi-sector steering committee composed of members from UNICEF, OXFAM, OCHA and DINEPA. A dissemination video of the findings and main conclusions of the evaluation was also produced and information leaflets developed to better disseminate the findings and recommendations.

Findings and Conclusions:

For the most part, the response has been rapid, proportional, appropriate, responsive and with real progress towards achieving the goals set. This was facilitated by several key factors:

  • contingency plan, even if limited, and pre-positioning some stocks in the affected departments
  • leadership of UNICEF management,
  • commitment of staff to the emergency situation,
  • field offices set up very quickly in the two most affected departments
  • existence of four Contingency Project Cooperation Agreements (PCAs)
  • immediate availability of funding.

UNICEF and its partners responded to the needs of the different groups in the areas of UNICEF's mandate. UNICEF's response was generally appreciated by most of the key informants interviewed, including by the affected population, in terms of: (i) rapid mobilization of human resources and supplies, (ii) leadership and adequate coordination.

However, the evaluation identified some bottlenecks and areas for improvement for future disaster situations.

  1. During the first weeks,  geographical coverage of needs was insufficient, limited by financial, operational and logistical constraints. The initial estimate of needs was not always adapted to reality. (difficulty of access to certain zones, quasi-absence of partners in the area)
  2. Mobilized partners, available resources and adopted strategies have not always been adapted to achieve the planned objectives.
  3. lack of explicit planning for the transition phase and UNICEF longer-term programming
  4. lack of multi-year funding and of flexibility of some donors have been a major constraint to connectivity. Reluctance of donors to fund preparatory actions or the rehabilitation of infrastructure was problematic.
  5. weakness to engage with affected communities (CwC) / (AAP) and therefore to adapt response
  6. Limited use of cash transfers
  7. UNICEF commitments to respect the principles of the Humanitarian Reform (Grand Bargain) has been uneven

Recommendations:

The final report contains 37 recommendations organized around the evaluation questions’ theme: Preparedness, Connectivity, Coverage, Efficiency, Efficacy, etc. Recommendations are prioritized according to immediate/urgent, medium and long term action needed.

Lessons Learned:

The lessons learned must be based on the preparation aspects (need for a preparation proportional to the needs, the importance of the availability of the partners and the capacity for rapid mobilization), to the prior knowledge of the Context and the development of a local network to better manage interventions in rural areas of difficult access. Rural areas require a different approach to that deployed in urban or peri-municipal areas at different levels: coverage strategy, protocols and types of intervention. Examples of good practices to address this challenge have been highlighted, including the use of resource-specific resources for affected communities and the deployment of mobile teams in remote areas. In addition, the Haiti Country office (BPH/HCO) has developed an interesting experience regarding the deployment of teams, the installation of field offices and the integration and convergence of program mechanisms

Please find below leaflets of summary:


You can watch the 15-minute video of the Evaluation here.

Please find below the following attachments labelled as follows:
  • Full Evaluation Report - Report
  • Annex 2 - Part 2
  • GEROS Evaluation Review - Part 3
  • GEROS Feedback Summary - Part 4
  • Evaluation Management Response - Part 5


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2018

Country:
Haiti

Region:
LACR

Type:
Evaluation

Theme:
Emergencies

Language:
French

Sequence #:
2018/001

New enhanced search