2013 Serbia: Expanding Early Learning Opportunities
Author: Camelia Gheorghe, Team Leader, Zehra Kacapor-Dzihic
"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labeled as ‘Part 3’ of the report."
The development objective of the Project “Expanding Early Learning Opportunities for Vulnerable Children’’ is to provide better access to quality preschool education for children aged 3-5, with the focus on most vulnerable. The Project Document specifies that the immediate objective is to increase coverage and improve quality of preschool education for 1,500 children 3-5 years old in 10 municipalities.
Project activities included assessment of needs and capacities for preschool education; adaptation of facilities for implementation of diversified programmes; development and piloting of two models of alternative (special) programmes; design and accreditation of training modules and instructive materials/manuals; training of trainers for capacity building of local partners; training for preschool staff, parents’ associations and parents. The target groups of the project is composed of 300 professional staff, 500 parents, preschool institutions and local self-governments in the project municipalities. The direct beneficiaries are 1,500 girls and boys aged 3-5 years, who are out of preschool education services, living in rural, economically-disadvantaged or remote areas, especially Roma, children from poor families and children with disability and other developmental difficulties. Indirect beneficiaries are 3,000 children benefitting from improved teaching and child development skills of trained teachers.
The Project under evaluation has a budget of 517,483 EUR, of which IKEA provided 409,483 EUR while UNICEF contributed complementary funds amounting to 108,000 EUR. The Project was implemented from May 2011 to October 2013.
The purpose of evaluation is to evaluate the project outcomes and achievements in relation to the project strategy. The specific objectives of the evaluation, as per ToR, are the following:
1. Provide feedback to UNICEF and its national counterparts (MoESTD, and other stakeholders) on the soundness (defined as relevance, effectiveness, efficiency and sustainability) and impact of the project approach in increasing coverage of children from 3 to 5.5 and improving quality of preschool services: a) Evaluate Project Impact following Project Outputs; b) Reveal good practices and gaps in approaches.
2. Provide the IKEA with information on impact of their specific support to expanding preschool services for vulnerable children in Serbia.
3. Based on the experience from the project implementation, extract general lessons learned and recommendations aimed at further enhancement of the preschool education and further UNICEF interventions in this area
The evaluation covers the entire implementation period (May 2011 – October 2013) and all selected municipalities, following the way how the project has been conceptualized in the Project Document, Theory of Change and the Agreement signed between UNICEF and the Center for Interactive Pedagogy (CIP Center), the main implementing partner. The evaluation has been carried out during by a team of experts from Promeso Consulting (Romania), selected following competitive tender.
In the Inception/Desk Phase, the team reviewed the entire project documentation and the national policy papers, legislation, strategies and action plans in the area of preschool education. The desk phase also included extensive review of European strategies and EU accession-related documents as well as reporting to international bodies and against human rights conventions ratified by Serbia.
The Field Phase has been devoted to the collection of data from key stakeholders at national and local levels, based on the evaluation instruments developed during the Inception/Desk Phase. The semi-structured interviews with UNICEF country office and representatives of national project partners took place in Belgrade in the period 11 -15 November 2013. For the site visits, which took place during the same week, the evaluation team has developed a set of criteria for representative sample selection based on rural/urban regional balance, territorial distribution and development level of municipalities, pilot-non-pilot nature of municipalities and representation of vulnerable parents in the governance structures of the targeted preschool institutions. Out of 10 project municipalities, the team has visited two municipalities (Smederevo and Aleksinac), two kindergartens (Lipogradic/Lipe and Green Bunch/Aleksinac) and two preschool institutions located in the cities of the two visited municipalities, conducting two discussion groups with preschool principals, regional school administrations and LSGs representatives as well as four focus groups with specialists (educators and expert associates) and beneficiary parents.
In the Synthesis phase, the team has applied the standard evaluation criteria analysis (relevance, efficiency, effectiveness, impact and sustainability) in combination with Human Rights-Based Approach in order to assess the achievement of planned results, draw informed conclusions and lessons as well as to provide meaningful and practical recommendations.
Findings and Conclusions:
The Project is highly relevant for Serbia’s preschool education reforms and national policies for improving the early childhood education and development. The Project remained relevant over its entire lifetime, as documented by the annual progress reports of the EC.
The Project was effective in achieving the results and objectives as outlined in the Project Document and Theory of Change. It had effective and coordinated implementation at municipality level and connection with legal and policy developments at national level.
Most of the planned Project activities have been delivered in an efficient and timely manner. The activities and management of the Project were conducted professionally and with high quality.
Considering its rather limited resources, the Project had a good impact level, making an important contribution to increasing the coverage of children aged 3-5.5 years benefiting from preschool education, especially of children living in rural areas of project municipalities. Self-evaluations conducted by the Project capture a very positive feedback from educators and parents on the developmental results of the newly-introduced special programmes upon beneficiary children.
Effects and outcomes of the Project are most likely sustainable. The capacity building tools, accredited training programmes, guides, practicums and self-evaluation methodologies developed by the Project’s partners are already in use or could easily be used for future establishment of similar special programmes in other municipalities.
The Project had a major contribution to the promotion and realisation of child rights through opening access to pre-school education services and early learning development to children left out of the system, empowering them, as rights-holders, to make informed choices and have their opinion heard.
The Report provides a number of key recommendations for the Serbian Government and LSGs, UNICEF, IKEA and other donors, based on the findings and conclusions of the evaluation, as follows:
Strategic Recommendations (S):
S1: Further invest in the expansion and diversification of country-wide, flexible early learning programmes for children aged 3-5.5 years to ensure higher and more inclusive coverage of vulnerable children in preschool education
S2: Ensure high quality standards of preschool education on the basis of a transparent national accreditation system and capacity building of teachers. Promote innovation in the development of preschool programmes to respond to the complexity of needs of their beneficiaries.
S3: Secure the financial sustainability on long-run of early learning childhood education programmes in least developed municipalities, based on the principles of equity, territorial cohesion and social solidarity.
Operational Recommendations (O):
O1: Carry out a country-wide identification of vulnerable children who are left out of the preschool education for evidence-based policy making and integrated support action
O2: Improve monitoring and reporting practices in future projects and the evaluability of impact
The evaluation of the Project led to the following Lessons learnt:
1) Investment in preschool education should be long-term, until the country gets closer to European benchmarks.
2) Local commitment and good cooperation between relevant local stakeholders is key to inclusive and sustainable results.
3) The importance of locally-generated solutions is not to be underestimated.
4) Expansion of early learning opportunities, especially in rural areas, requires both hard and soft investment.
5) Complex needs and scarce available resources require innovative coping strategies.
6) Any action targeting early childhood education and development must engage the parents and, to the extent possible, the grandparents of beneficiary children from start throughout the whole process.
Full report in PDF
PDF files require Acrobat Reader.