We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2016 Bulgaria: Evaluation of the Family for every child project in Bulgaria



Author: Joanna Rogers, Elayn M. Sammon, and Luba Devetakova

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

Bulgaria has been implementing a child care reform since 2000 leading to establishment of a family support services and alternative care across the country and a reduction of the number and share of children placed in residential care. The child protection reform started with the adoption of the Child Protection Act in 2000 and the establishment of a specialized child protection system at central and local level. The establishment of the State Agency for Child Protection (SACP) aimed to provide a focal point for the child protection policy, however, there are still other ministries involved. The Child Protection Departments at local level have been established with the intention to become single entry points for the child protection system. A number of strategic documents and papers were developed in the first years of the reform with the objective of addressing the vulnerabilities. Reducing number of children in institutions and establishment community based services, were some of the key strategic priorities of the child protection and welfare system.
Deinstitutionalization and development of a continuum of services has been at the heart of the UNICEF programme for Bulgaria, as it seeks to improve policies and planning, allocation of funds, inter-sectoral and institutional coordination and develop capacities at national, of regional and local level to better plan and deliver services to children and families. 

Purpose/Objective:

The main objectives of the evaluation were to:

  1. Assess the relevance, efficiency, effectiveness and sustainability and, to the extent possible, the impact of the Family for Every Child Project;
  2. Assess the level of coordination and complementarity between the Family for Every Child Project and other programs, services and projects implemented in the region of Shumen with specific focus on effectiveness for target beneficiaries – children and families;
  3. Identify and document lessons learnt and the contribution of UNICEF to the identified changes;
  4. Assess feasibility of the developed model and factors to be taking into account for its scaling up;
  5. Provide recommendations on focus areas and specific activities to be included in the UNICEF child protection programme for the next programme cycle.

Methodology:

In line with the UNICEF GEROS Methodology , the evaluation was based on OECD DAC criteria  that consider relevance and appropriateness of the project design and how the inputs, activities and outputs contributed to achievement of the outcomes (results) and wider impact. The evaluation methodology employed a mixed methods approach to capture a range of perspectives and ensure triangulation of data. These involved: key informant interviews, focus group discussions, document review, administrative data analysis and case files review.
Fieldwork was conducted during the period 03-14 October 2016 and comprised 46 KII and FGD involving 85 representatives of national and local government partners, NGOs, UNICEF as well as staff of the newly created services in Shumen (Annex 4). Due to time constraints, it was not possible to interview or consult with service beneficiaries in any meaningful way. This report was finalised after findings, conclusions and recommendations were discussed with interested stakeholders from Shumen and Sofia at two workshops held in January 2017.

Findings and Conclusions:

The project evolved over time from a focus on closure of the Shumen Homes for Medical and Social Care for Children (HMSCC) to an ongoing service development project. This introduced new services, as they were identified as necessary during project implementation and as factors driving family separation were identified.
The key services introduced, initially informed the design of the ‘Posoka Semeistvo’ project - Family Counselling Centres (FCC), Infant Foster Care and a Centre for Maternal and Child Health (CM&CH). The government aimed to pilot closure of eight HMSCC and create new services in the former HMSCC buildings. In designing the services initially, the emphasis was on preventing entry into the infant home and facilitating exit of children from the infant home.
The Shumen HMSCC has closed.
The infant foster care service was of key importance to enable the closure of the infant home.
The maternity ward service to prevent relinquishment was also important to prevent entry of newborns to the HMSCC by supporting mothers to take them home or diverting them into foster care.
The FCC appear to be working with around 100 family cases per year each and have contacts with other types of service users.
The CM&CH is offering a health visiting service that is valued by a range of national and local stakeholders and which appears to have considerable reach.
UNICEF Bulgaria’s government partners in Shumen and nationally, greatly value the support provided to develop services and to close the HMSCC in Shumen.
Key findings from child protection system data analysis  could indicate (with some caution as there are gaps in data) a general change in default behaviours by the CPD, in attitude and in default responses to referrals that theoretically could be attributable to the closure of the Shumen infant home and the development of the new services.

Recommendations:

The focus of UNICEF management interventions in the Shumen project in the next programme cycle should be to secure the sustainability of, and capitalise on, the investment made to date at both service levels and at the level of goals to inform national policy. The key areas for immediate action are:

  • Documentation to inform the process of deinstitutionalisation and replication or scaling-up of new services
  • Comprehensive financial analysis and costing of the child protection system in Shumen to support sustainability of project outcomes to date
  • Consultation with service users to test again, and correct if necessary, the assumptions on factors leading to separation that informed service design at the project outset
  • Implementation of an audit of the well-being and development of children who were moved from the infant home as well as the other infant homes that have closed during the implementation of the first Action Plan; this should be embedded in a quality methodology which utilises standardised assessment tools and which is approved by government
  • Implementation of a structured modular continuous professional development system for Shumen service and CPD staff (and the social services workforce in general) to further strengthen service delivery which can also inform replication
  • Review and revision of the national system for on-going monitoring of child well-being and development outcomes to inform wider national implementation
  • Development of standardised evidence-based instruments to support intersectoral working
  • Development of a simplified case management reporting interface that can provide government and partners with much needed data for monitoring and evaluation as well as planning purposes

Lessons Learned:

It has not been possible to assess improvements in child development outcomes in the region, although there are some indications that children from vulnerable communities are attending pre-school at a higher rate than prior to project intervention and that because of the interventions of the CM&CH, infant and child feeding practices are being established within WHO guidelines. If an assumption is made that remaining in family care or being placed in alternative family care is equivalent to improved child development outcomes compared to those achieved through institutional placements (especially for young children), then children in the Shumen region do have improved outcomes. This outcome is qualified, given that not enough is known about well-being in family placements and that the rate of referral to CPDs for children aged 4-6 years for child protection reasons is higher than for other age groups and that the rate of 1 and 2 year olds in formal care appears to be increasing.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2016

Country:
Bulgaria

Region:
CEE/CIS

Type:
Evaluation

Theme:
Child Protection

Partners:
Ministry of Labour and Social Policy, State Agency for Child Protection, Ministry of Health, Agency for Social Assistance, Regional government Shumen

Language:
English

Sequence #:
2016/004

New enhanced search