We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 Bulgaria: Final Evaluation of the UNICEF 'Children at risk behind bars' project



Author: Dan O'Donnel

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background

The Children at Risk Behind Bars Project was prepared in 2011, and funded by the French National Committee for UNICEF. Implementation began in 2012 and ended in 2017, although it was originally expected to end in December 2014. The project was based largely on the ‘Concept’ on children and justice adopted by the Council of Ministers in 2011, which in turn was based in part on the Committee on the Rights of the Child’s 2008 comments on the report of Bulgaria on implementation of the Convention on the Rights of the Child.

Purpose/Objective

The purpose of the evaluation of the project “Assisting the reform of the juvenile justice system in Bulgaria by developing and implementing a model of closing correctional institutions for children and providing adequate services for children in conflict with the law and children at risk” (or the Children behind bars project) is to assess the achievement of the planned objectives and the overall impact of the project, identify lessons learned and based on the findings provide recommendations for the future  support to the reform in the area of juvenile justice for UNICEF, national and local stakeholders.


Methodology

The evaluation framework, as defined by the ToR, calls for the assessment of the project’s contribution to ‘achieving results for children in terms of their relevance to the child rights and equity agenda, effectiveness, efficiency, relevance, sustainability, and impact.’ However, the ToR also recognize that, ‘questions related to impact on children may be limited to the impact of the project at system level rather than the impact on individuals.’ The sentence that defines the framework in general terms is followed by a list of 36 questions concerning project relevance and design (7), efficiency (7), effectiveness (10), impact (4), sustainability (8), and two questions on ‘partnership and cooperation’.


Findings/Conclusions

The project was very much aligned to the government’s reform agenda, when it began. The Concept on justice for children adopted by the Council of Ministers in 2011 called for the development of community-based services for children involved in criminal activity, closure of the Social Pedagogical Boarding Schools, and increased inter-institutional cooperation. It also called for bringing the legislation into harmony with Bulgaria’s obligations under international human rights law, repeal of the Juvenile Delinquency Act, and giving a greater role to NGOs. The project also was aligned with the ‘Road Map’ for implementation of the Concept adopted in 2013. However, the commitment of the government to the Concept and Road Map varied during the duration of the project, and from one counterpart to another.

Differences between the positions of national counterparts became increasingly evident as the project unfolded. The Ministry of Labour and Social Policy and the Central Juvenile Delinquency Commission opposed the draft law on juvenile justice developed in 2015.  On the other hand, an important new partner, the Office of the Prosecutor General, joined the project in 2015.  The Office of the Ombudsman also began to play a strong supportive role, especially in the closure of the Social Pedagogical Boarding Schools and law reform. Differences also emerged on the regional level. The mayor of Sliven was strongly supportive of the project, but was replaced in 2015 by a mayor who was not. The regional government remained supportive throughout the duration of the project.

Recommendations

UNICEF should continue to support juvenile justice reform in Bulgaria. Every national counterpart that met with the evaluation team agreed that it should do so. Although most of the aims of the project were not met, or not met in an appropriate manner, and although the Ministry of Justice, at one point the main counterpart, seems to have lost interest in juvenile justice reform, the evaluation team agrees that UNICEF should continue to support juvenile justice reform. There are two reasons for this conclusion. First, to step back at this point would have an adverse impact on the reform process. In our view, the process has reached a point where there is considerable support for it in civil society, in some of the relevant professions, and amongst public officials, but there is not enough momentum for the process to continue without UNICEF support. Ending UNICEF support now could make the investment made during the last six years a waste. Secondly, the team believes that there are steps that can be taken to overcome the difficulties that have prevented the Children Behind Bars project from being more successful.

Lessons Learned

Both positive and negative lessons can be learned from the design, implementation and results of this project.
One lesson concerning the design of the project is that projects should contain an analysis of foreseeable risks that could have a negative impact on implementation. The period 2012 – 2017 was marked by frequent changes of government. The national government in place at the time the project was designed and implementation began left office in March 2013, the second year of the project. The succeeding government was in place a little over one year, until July 2014. An interim government was in place until November 2014. One year later, in November 2015, local elections led to a change of government in Sliven municipality. The national government resigned in November 2016, after a presidential election, and an interim government was in place from then until May 2017. The government formed in May 2017 remained in place for the remainder of the project. The changes in government led to changes in the identity of key counterparts in the Ministries that were partners in the project, and a change of government in the municipality of Sliven. Changes in the national government effectively stopped one of the main activities, law reform, and the change of government in Sliven had dramatic consequences for the piloting of community-based services.  It is difficult to say what changes in project strategy or implementation might have been made had these problems been anticipated, but this is an example of the kind of risk that must be taken into account in designing a project. This is an important lesson for UNICEF offices contemplating similar projects in other countries, and for UNICEF Bulgaria should it decide to develop another project on juvenile justice.  



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year: 2018

Office/Country: Bulgaria

Region: ECAR

Type: Evaluation

Theme: Child Protection; Juvenile Justice

Language: English

Sequence #: 2018/001 

New enhanced search