We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2016 Global: Evaluation of UNICEF's response to the Ebola outbreak in West Africa, 2014-2015



Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 4’ of the report, and the executive feedback summary labelled as ‘Part 5’.

Background:

The Ebola virus disease (EVD) outbreak began in December 2013 in remote areas of Guinea and by late May 2014 had spread to Liberia and Sierra Leone. In August 2014, WHO declared a Public Health Emergency of International Concern. With high poverty rates, weak health systems and minimal levels of preparedness, the three most-affected countries were poorly equipped to respond. Although the outbreak’s impact was most evident at the local level, with 90 per cent of cases concentrated in 20 districts across the three countries,  it wrought serious humanitarian, economic, development and health consequences on populations across the region. 

National and international responses only brought the outbreak under control after thousands of deaths and widespread devastation had already occurred. As the outbreak spread, weak national health care systems struggled to cope and health care professionals became infected and died at alarming rates.

As the world became gripped by fear of Ebola, world powers strengthened their resolve to contain the outbreak. In 2014–2015, national governments, INGO partners and the UN system, led first by the World Health Organization and subsequently by the United Nations Mission for Ebola Emergency Response, took measures to coordinate the effort to stop the outbreak and prevent its further spread.

Within the larger response, UNICEF country offices in Guinea, Liberia and Sierra Leone participated in early efforts to detect and manage Ebola cases alongside their regular country programes. In September 2014, UNICEF declared an L3 corporate emergency and outlined its response in the Humanitarian Action for Children 2014–2015 Ebola appeal. UNICEF aimed to: 1) stop the outbreak through community-level actions; 2) prepare for outbreaks in additional countries; and 3) contribute to maintaining or building back better the primary health care and other social systems in the most-affected countries.

Purpose/Objective:

This report presents the findings, conclusions and recommendations of an evaluation conducted between November 2015 and September 2016 of UNICEF’s response to the Ebola outbreak in West Africa. The report is organized into five sections covering the response effectiveness, efficiency, internal coordination, external coordination and accountability. This Section describes: a) the evaluation purpose, objectives and scope; b) the evaluation methodology; c) the context of the Ebola outbreak and responses across Guinea, Liberia and Sierra Leone; and d) the UNICEF response in these countries in 2014–2015.

The purpose of this evaluation is “to provide an impartial assessment of UNICEF’s response to the needs of the affected populations and other challenges arising from the Ebola outbreak in West Africa”  towards the following three specific objectives: 

  1. Accountability: To provide an assessment of UNICEF’s response to the Ebola outbreak in West Africa, enabling defined stakeholder groups to offer feedback and recognize overall value;
  2. Learning: To highlight as conclusions the main lessons for UNICEF, to contribute to knowledge management and public health emergency preparedness; and
  3. Strategy: To provide strategic recommendations a) to UNICEF HQ on preparing for future public health emergencies; and b) to UNICEF’s regional and country offices on responding to emergency, recovery and reconstruction. 

The primary audiences for this report are UNICEF actors at both HQ and regional levels. The secondary audiences are country office actors in Guinea, Liberia and Sierra Leone.

Methodology:

The evaluation is focused on UNICEF’s corporate response to the Ebola emergency in Guinea, Sierra Leone and Liberia during the period declared a Level 3 (L3) emergency (August 2014 through the end of 2015). The assessment considers the effectiveness, efficiency, internal coordination, external coordination and accountability of the response.

The analysis is broadly focused on the strategic challenge of coordinating UNICEF’s levels, programmes and operational functions (i.e. how these elements combined to deliver an effective response). As such, it does not attempt to provide detailed information on implementation. As reflected in the Terms of Reference, the evaluation prioritizes programme elements related to: 1) health/case management, including the community care centres (CC centres); 2) Communication for Development (C4D), social mobilization and community engagement; and 3) child protection, particularly family tracing and reunification and care for separated and unaccompanied children.

Data collection and analysis was conducted using mixed methods, including qualitative and quantitative data collection techniques: a lessons review; a document review; data analyses; stakeholder consultations; stakeholder polling; and case studies of affected communities. Field missions were undertaken to Guinea, Liberia, Sierra Leone and Senegal during February and March 2016. In the three most-affected countries, analysts conducted consultations and polling among implementation actors and national leaders; and in two of the most-affected communities, analysts also performed case studies. The evaluation was limited by a number of factors, including the evolving strategies for the response both within UNICEF and externally; the ongoing adaptation of performance monitoring; and the difficulties involved in using humanitarian response standards, frameworks and tools in a public health emergency.

Findings and Conclusions:

  1. UNICEF’s public health response made a useful contribution to stopping Ebola transmission, most notably through community engagement, isolation and care activities, and large-scale delivery of supplies and WASH support.
  2. UNICEF’s response neither promptly nor adequately addressed Ebola’s secondary humanitarian consequences and specific effects on children.
  3. Although UNICEF worked to support, maintain and strengthen health systems in the three countries, the organization and partners struggled to reinforce basic services in the recovery without adequate funding. As a result, national health systems remain vulnerable to public health threats.
  4. UNICEF shared responsibility for critical delays in preventing and responding to Ebola. In some cases, its effectiveness was constrained by inadequate inter-agency strategic leadership and operational coordination.
  5. UNICEF’s contributions relied significantly on an innovative community-based implementation model that involved targeted actions to generate community behaviour change.
  6. UNICEF provided strategic direction needed to stop Ebola transmission, but its leadership was hampered by inadequate institutional arrangements, performance management and information analysis.
  7. UNICEF’s mobilization of financial, human and supply capacities enabled a large-scale response and made strong material contributions to effectiveness but struggled with Ebola-specific challenges and existing gaps in HR competencies.
  8. UNICEF’s response did not sufficiently rely on knowledge management and it remained only partially prepared for future public health emergencies (PHE). UNICEF did, however, make significant efforts to learn by doing.
  9. Although UNICEF’s response was aligned with the inter-agency strategy and epidemiological context, the lack of an accountability framework for PHE meant that the response was disconnected from the CCCs, which were neither fully appropriate nor relevant to a PHE.

Recommendations:

  1. UNICEF WCARO, country offices and partners in the three most-affected countries should ensure that: 1) health systems retain a rapid response capacity to prevent Ebola outbreaks and develop International Health Regulations core capacities; 2) community health systems are reinforced in the affected communities; and 3) children affected by Ebola receive adequate protection.
  2. UNICEF's Global Management Team (GMT) should develop a policy and accountability framework for responding to public health emergencies (PHE) that includes: 1) specific goals; 2) programme guidance; 3) global partnership objectives; and 4) assessment of broader humanitarian risks. It should build on rather than duplicate UNICEF’s existing emergency response policies and processes.
  3. The UNICEF GMT should recognize areas to improve and strengthen coordination, strategy and information capacities for PHEs. UNICEF should develop tools, guidance and mechanisms and strengthen capacities for: 1) global emergency coordination; 2) planning, programme support and performance monitoring; and 3) information and knowledge management.
  4. The UNICEF GMT should continue to strengthen capacities for rapid, large-scale deployment of financial, human and material resources in emergencies by: 1) applying lessons and protocols from the Ebola response on duty of care; 2) significantly increasing emergency HR capacities and emergency competencies in country offices; and 3) involving operational and administrative staff in strategy and programme management.
  5. UNICEF EMOPS and Programme Division should further develop the community-based approach as an implementation modality inclusive of strong AAP and community engagement components. Recognizing the role of communities in stopping Ebola, UNICEF should strengthen local capacities and systems for health and social protection at community level. Within UNICEF, capacity for community engagement and social mobilization should be increased.

You will find the reports/files below labelled as follows:

  • Final Evaluation Report - Report
  • Annexes - Part 2
  • Evaluation Summary - Part 3
  • GEROS quality rating - Part 4
  • GEROS executive feedback summary - Part 5


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2016

Office:
Evaluation Office

Region:
HQ

Type:
Evaluation

Theme:
Health

Language:
English

Sequence #:
2016/010

New enhanced search