We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 EO: Strengthening Child Protection Systems: Evaluation of UNICEF Strategies and Programme Performance



Author: Evaluation Office

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report, and the executive feedback summary labelled as ‘Part 4’.

Background:

This report presents the results of an evaluation of UNICEF’s strategies and programme performance in applying a child protection systems-strengthening (CPSS) approach in its work across country, regional and headquarters (HQ) levels. Child protection (CP) – defined in the Convention on the Rights of the Child as “preventing and responding to violence, exploitation and abuse against children” – is central to UNICEF’s mandate. Historically, UNICEF and others working in this field targeted their child protection efforts mainly at specific issues, or specific populations considered especially vulnerable to child protection threats. Over the past decade, however, the need for more sustainable outcomes produced a gradual shift towards a more holistic, comprehensive approach. As such, a CPSS approach has been a strategic objective of the overall CP work of the organization since the adoption of the Child Protection Strategy in 2008.

Purpose/Objective:

This evaluation is the first comprehensive attempt to gather and analyse evidence about the application of CPSS across UNICEF. It has two overall purposes: 1) to contribute to learning and decision-making with a view to improving UNICEF’s ongoing programming, and 2) to support accountability on CPSS. It assesses UNICEF’s global strategies and country-level programme performance to strengthen CP systems in order to bring about better protection for girls and boys. It examines CPSS from prevention through response and follow-up, involving formal and less formal actors, covering national and sub-national levels in middle income, low-income and fragile countries. While the evaluation traces UNICEF’s CPSS work over the past decade, the main time period under review is 2012 to 2018.

Methodology:

The evaluation was guided by ten key evaluation questions (see Annex 1) in combination with four of the standard evaluation criteria of the Development Assistance Committee of the Organisation for Economic Co-operation and Development (OECD-DAC), namely: relevance, effectiveness, efficiency and sustainability. While some evaluation questions specifically targeted one level— global (HQ), regional (RO), or country (CO) – others related to more than one tier but generated differentiated evidence and findings for each. Effectiveness at the country level was, in addition, assessed in relation to progress towards the six key elements of a functioning child protection system noted above.

Case studies of a purposive sample of 24 countries enabled the evaluation to bring detailed evidence from the country level to bear on the question of UNICEF’s performance in implementing CPSS, and to generate evidence to inform regional and global efforts in the future. The cases were divided among three levels of depth: Twelve were examined through documentary desk study, six drew additionally on remote interviews with key informants (UNICEF and partners), and six in-depth case studies also drew on data collected during field missions through face-to-face interviews, focus group discussions with adolescents and field observation. Two RO visits were combined with the country missions to collect additional regional-level data. Key informant interviews at global level, a document review and two online surveys completed the data sources – one with UNICEF country office teams, one with government and civil society partners. Qualitative comparative analysis (QCA) was used to examine which combinations of UNICEF investments most successfully contributed to CP systems-strengthening within the set of 24 countries.

Findings and conclusions:

The evaluation generated a total of 42 findings. Findings are presented in full in Chapters 3 and 4 of the report.
Conclusion 1: After a decade of work in this area, conceptual clarity on child protection systems strengthening in UNICEF is still incomplete. As a result, there is a lack of shared understanding around CPSS among UNICEF staff at various levels, and with partners.
Conclusion 2: Staff lack technical support on operationalizing the CP systems-strengthening approach in different programming contexts and against the backdrop of large issue-based initiatives and VAC as the unifying umbrella concept. 
Conclusion 3: UNICEF has had considerable success in advancing the child protection systems agenda at the national level and in raising awareness among national partners. However, by and large, this has not translated into adequate domestic investments in CPSS.
Conclusion 4: Donors have played a largely negative role in advancing national CP systems by distributing funding on a narrow issue-by-issue basis and using parallel monitoring and reporting systems. The organization has so far failed to present key donors with a compelling ‘business case’ for child protection systems strengthening.
Conclusion 5: UNICEF’s programme performance on CPSS has been moderate overall. Progress was uneven, with stark gaps in low-income, fragile and humanitarian contexts.
Conclusion 6: The evaluation has identified a number of interventions that are particularly effective for strengthening CP systems, including capacity-building/social workforce strengthening, leveraging public resources, evidence and research and policy advocacy. This provides some clarity on priority investments for UNICEF.
Conclusion 7: UNICEF’s ambitions for what a child protection system can be expected to accomplish in specific contexts, and in what timeframe, have not been realistic.
See chapter 5 for conclusions 8, 9 and 10.

Recommendations:

Recommendation 1: Clarify UNICEF’s definition of, and role in, child protection systems strengthening, and ensure that this approach is reflected in organizational strategies, policies and plans.
Recommendation 2: Define UNICEF’s niche in CPSS and invest in the most impactful areas to strengthen child protection systems.
Recommendation 3: Strengthen staff and partner capacities and learning on CPSS
Recommendation 4: Leverage partnerships and resources for CPSS
Recommendation 5: Address the CPSS data and measurement challenges 

Lessons Learned:

Lesson 1: A number of the findings, conclusions and recommendations from this evaluation echo those of other evaluations touching on systems issues, including the evaluation on rural water supply (2018) and early findings from the evaluation on health systems strengthening (2019). This suggests that some of the strengths, weaknesses and challenges observed in this report are not specific to child protection, but are relevant to UNICEF’s systems-strengthening efforts in general.
Lesson 2: UNICEF’s comparative advantage as a child protection actor is its tremendous reach and convening power, which allow it to influence the global agenda for children and work hand-in-hand with governments at the policy level to strengthen systems. In practice, however, UNICEF is often opportunistic when it comes to securing funding, which can lead to fragmented efforts that ultimately undermine systems-strengthening outcomes.
Lesson 3: UNICEF’s monitoring systems fall short of accurately capturing what the organization is doing, and whether it is achieving results, and for whom, with regard to systems strengthening. This is particularly worrisome from a gender and equity perspective.
Lesson 4: UNICEF’s relatively modest success in leveraging national financial resources for sustainable child protection systems partly reflects a lack of institutional expertise and partnerships in this area at country level. Creating ‘fiscal space’ for strengthened child protection systems within the national budget requires specific skills in child-friendly budgeting, and in general, that capacity is lacking is COs.

Please find here the Summary for "Strengthening Child Protection Systems: Evaluation of UNICEF strategies and programme performance"

Please find below the following for the "Strengthening Child Protection Systems: Evaluation of UNICEF Strategies and Programme Performance":

  • Main evaluation report - Report
  • Annexes - Part 2
  • GEROS Evaluation Review - Part 3
  • GEROS Feedback Summary - Part 4
  • Executive Board Summary - Part 5
  • Evaluation Management Response - Part 6



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year: 2018

Office/Country: Evaluation Office

Region: HQ

Type: Evaluation

Theme: Child Protection

Language: English

Sequence #: 2018/016

New enhanced search