We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 ESARO: Evaluation of Expansion and Scale-Up of HIV-Sensitive Social Protection in Eastern and Southern Africa



Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background

Under the initiative, UNICEF ESARO and UNICEF Country Offices provide technical assistance to the four priority countries, and document cross-country learnings. The initiative aims to effectively reach and impact vulnerable children and adolescents, with a focus on people who are at risk, living with and affected by HIV; strengthen government capacity for scale-up and operationalization of social protection systems; and increase vulnerable children’s and adolescents’ access to and utilization of social services.

Activities under this initiative differed in their design and execution, allowing for adaptation to the country context. In Malawi, activities focused on monitoring and evaluation of the National Social Support Policy, the design and implementation of a system to refer cash transfer beneficiaries to HIV-related social services and on creating demand for HIV services among adolescents. In Mozambique, activities focused on providing policy level support to the operationalization of the new social protection strategy, strengthening community-based and statutory case management and the implementation of social protection fairs. In Zambia, the government and UNICEF evaluated and scaled-up a package of services that aims to increase the utilization of HIV services by adolescents. In Zimbabwe, the initiative focused on strengthening the child protection case management system and ensuring linkages between the country’s flagship cash transfer programme and HIV-related services by using payment days to deliver services. In addition, the initiative’s regional component, led by UNICEF ESARO, focused on the documentation and dissemination of best practices and overall technical assistance to the country offices involved.

Purpose/Objective

The purpose of the evaluation is to better understand how and under what conditions the interventions implemented under the initiative’s grant are functioning and to assess the extent to which the initiative has met its objectives and achieved expected results. The evaluation also documents the successes, challenges and lessons learned in the implementation of the initiative and assesses the efficiency and effectiveness of UNICEF’s inputs. The evaluation covers the period from 2014 to 2018, which comprises of the first and second phases of the grant.

Methodology

The methods utilised to carry out this study include a structured desk research; a review and analysis of secondary data; key informant interviews with government and non-government stakeholders; and focus group discussions with beneficiaries, community workers and volunteers.

The study has several limitations. First, a maximum of two sub-national locations could be visited per country, limiting the generalizability of the evaluation’s field work findings. Second, limitations derive from the absence of secondary data to answer parts of the evaluation criteria on effectiveness, efficiency and impact. Although primary qualitative data collected was helpful in answering parts of these criteria, availability of secondary data from continuous monitoring and evaluation activities under the programme could have enriched the depth of the evaluation.

Findings and conclusions

Over the last years, the initiative has supported the agenda setting for social protection in the region, from delivering simple returns towards leveraging the transformative power that integration of sectoral interventions into social protection can have on achieving long-term developmental outcomes. Overall, the results of the initiative have been positive, although several of the ambitious targets were not met and future efforts are required to sustain the gains made.

Recommendations

Increase investments in monitoring and evaluation
At the initiative level, to increase the long-term impact of the initial investment and the sustainability of its results, time-bound initiatives in an emerging area such as ‘cash plus’ benefit from higher investments in monitoring and evaluation. At programme level, monitoring should also be used more continuously throughout implementation to identify measures that could enhance efficiency and effectiveness in a timely manner.

Enhance the strategic interaction with community structures
Programmes in all countries would benefit from more strategic interaction with community structures. At the policy level, there is a need to better regulate their involvement and comprehensively build their capacity. At programme level, better communication, mentoring and feedback channels between community structures and sub-national government structures can optimize implementation and keep community volunteers motivated.

More thoroughly design forthcoming interventions
In future programme design, attention should be drawn towards designing specific, measurable and attainable objectives and associated targets to be met within a realistic timeframe. Setting more realistic targets could have freed up more resources for quality-assuring support functions and developing a TOC can clarify objectives and impact trajectories.

Render initiatives more age-sensitive
Mainstreaming age in social protection programming, when done adequately, is an effective way to optimize the quality of interventions, increasing the relevance, efficiency and impact of programmes and strengthening the attainment of equity and human rights. Interventions in the focus countries can be sensitized further to the socially prescribed roles, power relations and legal conditions imposed on different age groups – as well as reach and interact with individuals in an age-sensitive manner.

Lessons Learned

  1. Social protection programmes can serve as platforms to deliver cross-sectoral returns, including HIV and SRH outcomes. Linking programmes more explicitly to social services through a ‘cash plus’ approach has the potential to strengthen outcomes more sustainably, delivering high returns on investments made.
  2. ‘Plus’ activities can be delivered as integral programme activities facilitated through the provision of linkages and referrals to existing services to drive health and protection outcomes.
  3. Synergies in service delivery can be exploited by bringing government actors to the table. Integration of a wide array of government stakeholders strengthens the quality of design, buy-in into efforts to improve coordination, cross-sectoral achievement of outcomes and ultimately the sustainability of results, if compared to service delivery through third parties.
  4. Placing communities at the forefront of implementing a ‘cash plus’ model can build local structures’ and actors’ capacities and knowledge and, if executed well, can be a low-cost and sustainable approach to implementation. For such a community-driven model to work effectively, community actors and structures need to benefit from integration with sub-national government structures which providing support, supervision and mentoring.
  5. Programmes and activities targeting adolescents must be designed to meet the target group’s unique needs and must reach out to the target group in a suitable and conducive manner. Working with peer educators and CATS, training health care workers in the provision of adolescent-friendly services, setting up adolescent-friendly corners and integrating service delivery into fun events can help to reach adolescents successfully.

Please find the attached labelled as follows:

  • Evaluation Report - Report
  • GEROS Evaluation Review - Part 2
  • GEROS Feedback Summary - Part 3
  • Evaluation Management Response (EMR) - Part 4


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2018

Country/Office:
ESAR

Region:
ESAR

Type:
Evaluation

Theme:
Social Policy (Cross-cutting), Social Policy themed sectoral or partnership

Language:
English

Sequence #:
2018/003

New enhanced search