We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2019 EO: UNICEF Response to the South Sudan Humanitarian Situation



Author: James Darcy; Hisham Khogali; Volker Huls, Ramlat Musa Ali Wani and Alimure Awuda Amena

Executive summary

 

Purpose/Objective

This evaluation of UNICEF’s response to the humanitarian emergency in South Sudan was commissioned by the UNICEF Evaluation Office in New York. It considers the UNICEF response to the South Sudan crisis between January
2016 and May 2018 and was designed to fulfil two functions:

  1. A summative accountability function, reflecting the need to account internally and externally for one of UNICEF’s largest and most life-critical country programmes, and an emergency designated a corporate Level 3 priority and now entering its sixth year; and
  2. A formative learning function, reflecting the need to capture lessons from a programme of this duration and significance, to inform the country programme and UNICEF’s global programming and practice.

Though the focus of the evaluation is on the humanitarian components of the UNICEF programme, it also considers resilience and the reduction of vulnerability in the medium term. The overall guiding question for the evaluation is therefore: How well did UNICEF respond to the short- to medium-term threats to children’s well-being and development in South Sudan between January 2016 and May 2018? How well placed is the organization now to respond to future threats?

Given the time and resource constraints associated with the rapid and timely evaluation approach, much of the focus of the evaluation is on child survival – i.e. on the nutrition, health and WASH components of UNICEF’s response to the Level 3 crisis. In addition to this child survival agenda, the evaluation explores a number of cross-cutting issues, including the balance of UNICEF’s emergency programmes between different groups and areas; efforts to promote the resilience of families, communities and systems; related attempts to localize the programme, working closely with communities and civil society organizations; the relationship between emergency and development agendas and approaches; and the current partnership model that UNICEF is using to deliver the bulk of its programming.  Related process issues are also explored, including UNICEF’s vital supply and communications roles, partnership processes and its coordination role within the cluster system.

Methodology

Overall, the team’s approach was determined by what it took to answer the guiding questions with a reasonable degree of confidence. The approach agreed diverges from the options set out in the TOR, which, upon further consultation, did not appear to provide an appropriate basis for conducting the evaluation. Given the somewhat conflicting demands of the accountability and learning purposes, the evaluation adopted a two-strand approach:

  1. The first strand (accountability) consists of a limited review of the entire UNICEF Level 3 programme during the period of January 2016 through May 2018. This includes an analysis of the programme as a whole and its component parts against a limited set of quality and performance criteria, including overall relevance, scale, coverage, balance, coherence, resourcing, target achievement, compliance with the Core Commitments for Children in Humanitarian Action (CCCs), equity and gender; and application of humanitarian principles.
  2. The second strand (learning) involves a more focused review of lessons emerging from those sectors related most directly to child survival within UNICEF’s overall response (nutrition, health and WASH). This includes both areas of apparent under-performance or challenge, and areas of good or innovative practice that could benefit UNICEF globally. Our starting point for this component was a review of existing learning and questions arising from earlier evaluations and reviews.

Following the approach outlined above, the primary methods used in the evaluation have been key informant interviews, both with UNICEF staff and with staff of other organizations; and document review, using mainly UNICEF materials. These methods were supplemented by field observations and limited consultations with affected communities and local partners. Altogether, six field locations were visited beyond Juba.  Short online surveys were conducted with UNICEF South Sudan staff and in-country partners, and the results helped inform the evaluation’s conclusions. A validation workshop was held at the UNICEF South Sudan office in Juba to discuss the findings and provisional recommendations of the evaluation, based on the first draft report.

Key Findings

  • On Strategy:  Weak or absent government capacity have meant that UNICEF’s role during the evaluation period was appropriately focused on helping to meet basic needs and mitigate the worst effects of the conflict through a mix of partnership-based programmes and direct service delivery. Given the limited service provision by the Government, the South Sudan people are increasingly dependent on international assistance and civil society support to meet their basic needs.
  • UNICEF and other aid actors are currently playing a more substitutional than auxiliary role in South Sudan; this is likely to remain the case in the short to medium term.  Yet ultimate responsibility for the South Sudanese people’s welfare, development and protection lies with the Government, and it is important to maintain this line of political accountability. This raises an important question: How does UNICEF help strengthen the sense of national and local political responsibility and accountability for services and outcomes, given its substitutional delivery role?
  • Humanitarian-development distinctions start to break down in a context such as South Sudan, where the development base is so weak and access to basic services is so limited. While tackling the most immediate threats to children and their families clearly remains a priority, UNICEF and others are rightly concerned with helping to build resilience – the ability of people and systems to withstand and recover from shocks. This bridges the humanitarian and development agendas but is too abstract a concept to be operationally useful. Work remains to be done to better define this more broadly, as well as for each individual sector. This is appropriately high on the United Nations Country Team agenda.
  • On Nutrition:  UNICEF’s record in this sector is strong: all nutrition CCCs have been substantially addressed in South Sudan. Effective leadership has been provided through the cluster mechanism and UNICEF has played a critical role in providing the core pipeline of commodities required for treatment of SAM. Community-based management of acute malnutrition and IYCF activities are undertaken wherever access is possible, as are vitamin A supplementation and deworming. Screening and referral activities conducted by community nutrition volunteers have provided a regular opportunity for children and women to access the information and nutrition programmes that UNICEF supports.
  • On Health: UNICEF’s efforts in South Sudan have substantially addressed the health CCCs, though apart from immunization, its contributions have been relatively modest in scale. UNICEF has coordinated its own work with that of the Health Cluster – including the work of NGOs funded from the Health Pooled Fund – and the South Sudan Ministry of Health. UNICEF has also made substantial contributions to life-saving interventions, including measles vaccination, micronutrient supplementation and the provision of treated bed nets.
  • UNICEF has also supported essential health services, targeting the three main morbidities in South Sudan: malaria, pneumonia and diarrhoea. Access to maternal health services has been facilitated through support for antenatal care, safe delivery and prevention of mother-to-child transmission of HIV. Key health education and promotion messages have been delivered both at facilities and at the community level through mother-to-mother support groups and integrated community case management (iCCM) activities. The evaluation team was unable to find evidence of any activities undertaken by UNICEF to address Commitment 5 during the evaluation period, however.
  • On WASH: Overall, during the evaluation period, UNICEF showed substantial delivery against all five WASH CCCs, though for reasons largely beyond its control, it has fallen short of full achievement. Commitment 1 on leadership now appears to be well achieved, with cluster leadership back on track after a period of deficits and appreciated by WASH Cluster partners. Commitment 2 has only been partially achieved as gaps remain in the coverage of the overall WASH response. Furthermore, the WASH Cluster targeted only a proportion of those identified in need in the 2017 HRP, explicitly referring to resource constraints as a factor. Not all people in need of WASH assistance were reached by the sector response. The same is true for Commitment 3 on sanitation, with an even larger gap between estimated needs, targets and delivery. While Commitment 4 is worded without clear reference to the depth of the messaging, there have been concerns from WASH partners about the effectiveness of UNICEF’s hygiene promotion activities. Finally, while there is evidence of UNICEF’s delivery on Commitment 5 on WASH in schools and child-friendly spaces, the extent of the coverage and gaps could not be established with the available data.

General Conclusion

Overall, the evaluation found that UNICEF and its partners performed well during the evaluation period, in an often hostile and challenging operating environment. Some areas of programming – notably nutrition – were stronger than others, and UNICEF needs to do more to test the effectiveness of its interventions across the board. Despite the best efforts of UNICEF and its partners, the concern remains that child survival and basic development indicators have continued to worsen and the related caseloads have continued to grow. Mass displacement, insecurity and access constraints continue to challenge the organization’s ability to assist children and their families; and UNICEF and its partners will need to continue to adapt their approaches to address these challenges.

The country team is already testing or proposing several new initiatives and course corrections; and this evaluation has attempted to identify additional opportunities and necessary improvements. The recent peace agreement may offer a greater prospect of progress, not just on the humanitarian agenda but on chronically neglected development priorities, including education – without which children’s life chances in South Sudan will remain seriously compromised for the foreseeable future.

Recommendations

  1. Nutrition: Extend and enhance the nutrition programme, including:
    • Ensure that target setting is more ambitious without compromising quality; advocate with partners, particularly WFP, to increase supplementary feeding coverage; and include school-aged children and adolescents, particularly girls, in the nutrition programme.
    • Improve the quality of sanitation facilities at feeding centres; consolidate the various cadres of community volunteers that work multi-sectorally; and integrate nutrition into mobile health outreach.
    • Advocate for addressing chronic malnutrition (with partners, particularly WFP and FAO), across sectors; and explore more strategic partnerships to address training and capacity building, among other areas.
  2. Health: Extend and enhance the coverage of UNICEF’s health programme, including:
    • Increase immunization coverage through mobile outreach; and explore other distribution mechanisms to increase the reach of insecticide-treated bed nets.
    • Work with other sectors to incorporate nutrition into mobile outreach and to consolidate the cadres of community volunteers that work multi-sectorally.
    • Increase the targeting of adolescent girls and better address their specific vulnerabilities.
  3. WASH: Ensure the quality and sustainability of UNICEF’s WASH programme, including:
    • Integrate humanitarian and development approaches, ensuring minimum standards of implementation; and where possible, ensure the sustainability of WASH facilities, in terms of the quality of construction, operation and maintenance, including the full implementation of planned engineering supervision.
    • Transition from hygiene campaigns to behaviour change work where possible; and work with other sectors to consolidate the cadres of community volunteers that work multi-sectorally.
    • Strengthen water source data collection and sharing through contractual obligations and advocacy.
    • Strengthen operation and maintenance through contractual obligations by more systematic teaching water committees about the value of water source data and management plans.
  4. Supply and logistics: Review and supplement UNICEF’s human resources capacity in supply and logistics as necessary.
  5. Programme balance: Review and progressively address the balance between camp, static and outreach programmes.
  6. IRRM: Review IRRM targeting, effectiveness, follow-up and reporting, including:
    • Review missions more consistently and schedule follow-up missions of partner visits to ensure continuity.
    • Systematically collate and share situational data from IRRMs.
    • Review how the results of IRRM missions are reported to ensure more coverage information can be substantiated;
    • and undertake a comprehensive stock take that analyses and learns from the specific model pursued in South Sudan
  7. Sector evidence base: Strengthen the sector evidence base, specifically in WASH and nutrition, including by investing in a WASH baseline (also as part of the REACH initiative) and strengthening nutrition information and analysis.
  8. Monitoring and reporting: Strengthen the programmatic evidence base, including through stronger monitoring and oversight processes, by further increasing the regularity and coverage of field monitoring visits by UNICEF staff and considering alternatives for triangulation that go beyond third-party systems.
  9. Efficiency: Take additional steps to increase efficiency through combined processes, both internal and shared, including through joint PCAs and a common cadre of community volunteers.
  10. Accountability to affected populations: Take additional steps to strengthen accountability to beneficiaries and support claims of programme effectiveness.
  11. Partnership model: Review the current partnership model and strengthen related business processes, including pursuing multi-year agreements where rules permit and accelerating internal review processes to reduce delays in contracting.
  12. Resilience in practice: Take steps to make the resilience agenda actionable and measurable within and across sectors, starting with clearly defined criteria for what constitutes resilient households, communities and systems in each area of intervention.

 

You will find the following attached:

  • Evaluation report - Report
  • Annexes - Part 2
  • Executive Summary - Part 3
  • Evaluation Management Response - Part 4


 



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2019

Region:
HQ, ESAR

Office/Country:
Evaluation Office, South Sudan

Type:
Evaluation

Theme:
Emergencies

Language:
English

Sequence #:
South Sudan/63/2019/16061

 

New enhanced search