Author: Ana Redzic
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labeled as ‘Part 2’ of the report.
This is the report of final evaluation of the project “Transforming Residential Institutions for Children and Developing Sustainable alternatives”, implemented by UNICEF Serbia between 2008 and 2011. The main purpose of this document is to evaluate the final results and achievements of the project in relation to the project log-frame. The broader purpose is to:
- Evaluate contribution of the project to the implementation of the Social Welfare Development Strategy, including contribution to the development of new policies and legislation in the area of child care;
- Identify approaches that were vital for the achievement of results as well as lessons learned and good practice examples that can become a knowledge base for future programming;
- Provide insight into the current status of child care system and strategic recommendations for the next steps in the reform process relevant for all engaged stakeholders.
In 2005 Government of Serbia adopted the Social Welfare Development Strategy (SWDS), main strategic framework for the reform of social welfare system. One of the main goals of SWDS is “improvement of offer and quality of services in all forms of residential placement of beneficiaries”. The strategy foresees decrease of number of children placements in residential institutions and introduction and application of new methodological approaches, new organization of work and guaranteed quality of services, which are to greatest possible extent adjusted to beneficiaries’ needs. The strategy envisages development of new services and service departments which would support the life of children with disabilities or without parental care in the community, such as foster care, respite care, etc.
To respond to the needs of Government of Serbia and more specifically, Ministry of Labour and Social Policy, UNICEF has made a decision to assist MoLSP in their efforts to improve quality of services in residential placements and decrease the number of children placed in residential institutions, through their transformation. UNICEF, in close cooperation with MoLSP, designed the project “Transforming Residential Institution for Children and Developing Sustainable Alternatives”, which started in 2008, funded by the European Union. The intervention was built on results and recommendations from a previous external evaluation of deinstitutionalization process in Serbia (2006) and Assessment of Child Protection in Serbia, conducted in 2007.
The overall purpose of the project was to reduce the number of children in residential institutions in Serbia by 25% by 2011, through the transformation of residential institutions and development of local institutional capacities for provision of family support and family substitute services.
The implementation of the project was organized through four components, with the establishment of four multi-sector thematic Working Groups:
1. Transformation of institutions and inter-municipal and operational planning for children;
2. Strengthening accountability and monitoring mechanism;
3. General and specialized fostering;
4. Health-care family support in maternity hospitals.
Respective multi-sector Working Groups consisted of representatives of relevant ministries, experts, practitioners, representatives of professional associations and representatives of trade unions. The main role of Working Groups was to advise and oversee project implementation as well as to provide strategic direction and endorse policy documents and recommendations.
The technical and organizational support to these four Working Groups was provided through four UNICEF contracting partners, who worked closely with Project Management Unit and who were ultimately responsible for implementation of these project’s components.
The credibility of an evaluation’s conclusions rests on the quality of the evidence that supports them. This, in turn depends on the appropriateness of the evaluation design and methodology for data collection. In this section, evaluation design and methods that were used to conduct evaluation are listed:
1. An introductory briefing
2. A comprehensive desk review of external and internal documents
3. An extensive round of interviews with key stakeholders
4. Field visits to three types of institutions
5. A de-briefing on the preliminary findings of the evaluation (Preliminary Draft Report)
6. Presentation of main findings, conclusions and recommendations (Draft Evaluation Report)
7. A Final Evaluation Report
The evaluation was designed as a summative, using non-experimental approach. It was conducted in the period from 21st April to 10th of June 2011. Evaluation was designed to provide answers to the key questions under each of the evaluation criteria as outlined by the ToR.
The beginning of evaluation assessment consisted of analytical research (desk review) of all relevant documents and reports provided, to get familiar with the policy basis and expected outputs that need to be delivered within the proposed timeframe. Documents included project documents, previous monitoring reports, key national policies and legal documents. Special attention was given to the project log-frame and action plans against which project progress was supposed to be evaluated.
The second phase of evaluation was based on empirical research, which was performed through direct interviews with stakeholders and followed by the content analysis. The methodology used entailed a wide consultative process with various stakeholders/partners to enable appropriate data collection and generate evidence to substantiate all findings, ensuring that the data collection methods and data analysis is credible and accurate.
Lessons Learned (Optional):
The project’s progress over the period 2008 and 2011 generates number of lessons learned that could be used for future programming:
1. Importance of a broad participatory approach of all stakeholders, which was crucial in ensuring their ownership of project’s results;
2. Changing of legislative framework through which transformation process could be pursued needs to be followed by intensive capacity building of all important stakeholders, as to ensure adequate implementation of new legislation;
3. Investing efforts into creating a mechanism at the operational level of the leading ministry, which could ensure sustainability of the management of the reforms, beyond external support;
4. Investing sufficient time and resources into creating partnerships and functioning channels of communication between MoLSP and residential institutions, which could decrease the resistance of residential institutions towards transformation process and ensure sustainability of the reform;
5. Strengthening the cooperation with local self governments from the beginning of transformation process to ensure their support to development and sustainability of community based services.
PDF files require Acrobat Reader.