We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2014 Macedonia: Evaluation of the Early Childhood Development Programme in the former Yugoslav Republic of Macedonia



Author: Dr. Anise Waljee, John Wood and Jake Grout-Smith

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

The former Yugoslav Republic of Macedonia ranks 78th in the HDI, with 27% of its population living in poverty. The country has a population of about two million comprising of two main ethnic groups, Macedonians (64%) and Albanians (25%), and other smaller minorities such as the Roma (3%) who remain marginalized: poverty rates for Roma people are three times higher than the national average. Since conflict between the two main ethnic groups in 2001, political and constitutional amendments have been carried out to develop a multi-ethnic civil state, but implementation is challenging and political reform has slowed with worrying trends including the increasing segregation of education systems. Preschool enrolment in the former Yugoslav Republic of Macedonia has been gradually expanding from a low base, but enrolment levels remain some of the lowest in the region. In keeping with many of the Republics of the former Yugoslavia, the country has a strong tradition of state-funded support for the care and education of preschool children, although with relatively low national coverage and largely concentrated in urban areas. This service was often provided through the workplace and consisted of a full day of care in kindergartens that allowed mothers and fathers to work. Many of the institutions were well equipped for care, with facilities for children to sleep during the day and to be fed. Following the collapse of Yugoslavia, kindergartens became reliant on payments to compensate for state under-funding and low salaries, throwing up additional barriers to access for poor and marginalized groups.

This legacy established expectations of kindergartens providing full day care and education. During the transition from the collapse of former systems, this legacy biased efforts towards the maintenance and restoration of the old infrastructure of kindergartens, directing much reduced budgets on a programme model that is unaffordable for widespread, equitable access at scale.

Purpose/Objective:

This report presents findings for the former Yugoslav Republic of Macedonia drawn from the research and data collected as part of the multi-country evaluation (MCE) on early learning and school readiness (ELSR) that was commissioned by the UNICEF CEE/CIS Regional Office (RO) in 2013. The MCE provided findings and recommendations for UNICEF’s work on ELSR in the CEE/CIS region, built on research in six CEE/CIS country cases.  This report was commissioned by the RO and the former Yugoslav Republic of Macedonia Country Office (CO) in 2014 to draw out country-specific findings, conclusions and recommendations for the former Yugoslav Republic of Macedonia in more depth than was possible in the regional MCE report.
In line with the research carried out for the MCE, the specific objectives of this report, as set out in the
Terms of Reference, are to:

• Document and report on impact results (in terms of changes in children's life) and reduction of equity gaps in ELSR;
• Assess how such results were made possible through system changes (removal of system bottlenecks) at national and/or local levels; and
• Document the contribution of UNICEF to these system changes.

As a central aspect of the equity focus in these objectives, the evaluation has given specific attention to gender and human rights considerations in its analysis of both system changes and UNICEF contributions. The evaluation covers the period 2005-2012.
It is important to note that because of the multiple influences and actors involved in system level changes, direct attribution to UNICEF interventions could not be sought through this evaluation. The evaluation focuses on a contribution analysis of the part UNICEF interventions played in the processes and influences that led to the system changes identified.

Methodology:

This country evaluation draws on the data and analysis gathered for the Multi Country Evaluation on ELSR. The methodology used for the MCE is presented in detail in the MCE report volume 2, but in relation to the former Yugoslav Republic of Macedonia involved:

Documentation review: incorporating all UNICEF documentation relating to the design and implementation of ELSR programmes (including Situational Analyses, Country Programme Documents and Action Plans, Annual Reports and Annual Work Plans, evaluation and monitoring reports, policy and advocacy publications) and country policy and legislative documents that provide the framework for national and sub-national implementation of early childhood education services.

Data review: of available national datasets of ELSR indicators (including levels of disaggregation), time and geographical range, and data collection, preparation and analysis methodologies used. The system level focus of this evaluation meant that only those datasets that could show system level impacts would be appropriate for inclusion in analysis.
Country consultations: A 2-week visit by an international member of the evaluation team and a national consultant. This visit consisted of consultations with stakeholder groups that covered the different levels of the TOC, from UNICEF interventions to national ELSR system changes, their mediation into sub-national implementation and the actual experience of users of ELSR local services.

This MCE-related methodology was supplemented for this specific country report by a further documentation review (including additional documentation that had become available since the MCE), going into more depth and detail on national and UNICEF ELSR documents for the former Yugoslav Republic of Macedonia than had been possible under the MCE.

Findings and Conclusions:

The TOC has not captured the importance, at the system level, of decentralized responsibilities.  The capacity of critical institutions to fulfill their role has been identified as an important system determinant, which is not properly shown in the current TOC.

There has been a gradual but steady increase in access to ELSR across the evaluation period. There is still a long way to go, particularly to reach those most marginalized groups. However, whilst the majority of the population remain without access to preschool services and significant inequities exist, it is important to stress the progress that has been made in establishing the underlying frameworks, structures and systems for more accessible, equitable and quality ELSR.

Interventions have increased the numbers of children receiving preschool education but, have had limited relevance to children from the marginalized groups, including those with disabilities, from ethnic minorities and vulnerable families, who remain excluded.

UNICEF has started to engage with system determinants that are important for equitable provision for marginalized children – those for identifying, assessing, budgeting and administering special support but further work will be needed around this. UNICEF has also developed models to provide ELSR to members of the Roma and other traditionally isolated and stigmatized communities.

UNICEF has introduced the concept and practicalities of child-centered methodologies into all its work, including modeling, professional support, knowledge exchange and, importantly, the ELDS.

The ELDS offer a useful tool for recognizing and assuring quality, and provide a basis for a more coherent 0-6 range of services. The upcoming revision of the ELDS will need to reflect other emerging areas of consideration such as diversity and inclusion.

However, taking methodologies and standards into practice at scale will be challenging and is an increasing focus of UNICEF work.

Recommendations:

Recommendation 1: It is recommended that the TOC is amended to reflect system approaches to capacity development and the importance of system determinants at sub-national as well as national levels.
Recommendation 2: It is recommended that UNICEF’s future ELSR programming is developed closely in line with the emerging UNICEF Regional Strategic Guidance on ELSR.
Recommendation 3: It is recommended that UNICEF continues its support to reviewing and restructuring financial allocation and disbursement mechanisms for ELSR to incorporate more equitable and efficient use of resources to expand coverage in ELSR.
Recommendation 4: It is recommended that UNICEF maintain and deepen its focus on the quality of different forms of ELSR provision, particularly around the possible risks to quality arising from rapidly increasing access, with short national research studies. This should also include research into the teacher determinant in quality ESLR and improved learning outcomes for children.
Recommendation 5: It is recommended that UNICEF deepen its approach to teacher training and professional development and work on children’s ways of learning and respect for children’s rights throughout preschool (identity, integrity, participation and agency).
Recommendation 6: It is recommended that UNICEF strengthen its work in embedding diversity and inclusion within preschool environments and practice, including the representation and role models available within the ELSR sector and within teaching and learning materials that reflect gender, linguistic and ethnic diversity. This will also involve substantive work in addressing the social norms in communities around the inclusion and participation of marginalised groups in ELSR services.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2014

Country:
Macedonia

Region:
CEE/CIS

Theme:
Early Childhood Development

Type:
Evaluation

Language:
English

New enhanced search