We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2017 Montenegro: Evaluation of the Programme “Montenegro – Investment case on Early Childhood Development”



Author: Camelia Gheorghe and Ajsa Hadzibegovic

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report, and the executive feedback summary labelled as ‘Part 4’.

Background:

This document represents the Evaluation Report of “Montenegro - Investment Case on Early Childhood Development”, part of a broader collaboration between UNICEF Headquarters (HQ) and the H&M Conscious Foundation (HMCF) entitled “Unleashing Children’s Potential”. The evaluation was conducted between August 2016 and March 2017.

Purpose:

The purpose of the evaluation is multi-fold: a) to improve accountability for the achievement of results and use of resources; b) to provide recommendations for the scaling-up of the tested services for children aged 3-6 at national level, based on documentation of good practices and lessons learnt; c) to help ensure that the focus is kept on issues of equity, inclusion and quality in the process of implementing the Strategy on Early Childhood and Preschool Education 2016–2020; d) to inform the potential replication of Montenegro investment case in other countries, including by way of capturing the changes at the policy level.

Methodology:

The evaluation applied a mixed-method approach, including: stakeholder mapping; mapping of situation and contextual analyses, barriers and bottlenecks; in-depth documentary review and structured desk analysis of Programme design, implementation approaches, documenting of results and processes; structured desk analysis of policy documents and legislative frameworks; analysis and testing of the revised Theory of Change (ToC); analysis of results from M&E systems and data at national and municipality level; aggregation and analysis of data collected via the Data Sheet and various mapping templates; contribution analysis to determine factors which promoted or impeded the progress against intended results and attribution analysis to the extent possible; systems analysis of management, monitoring, quality control and assurance strategies; mapping of risks analyses and mitigation measures; financial analysis; analysis of sustainability strategies and systemic barriers to sustainability; in-depth interviews and focus groups; direct observation and guided discussion with children during site visits to municipalities.

Findings and Conclusions:

The Programme has won an argument in the public space on the need to invest in ECE by steering informed policy debate and building support in the society for ECE for which UNICEF CO and the MoE, the key driver of the Programme, deserve high praise. The Programme implemented new ECE services; it campaigned to create demand; and it diversified the ECE service menu. The investment in improving the service provision was supported by a strong and tireless policy advocacy of UNICEF CO, grounded in solid evidence and based on robust and easy-to-understand arguments. The Programme was fully in line with the national priorities and the European and international human rights commitments. Implementation of the Programme delivered valuable results: a) developed strategic building blocks for ECE expansion country-wide; b) increased access of 613 children (3-6) to early learning opportunities in target municipalities, especially of children living in rural areas; c) improved awareness of parents on the importance of ECE; d) strengthened capacity of professionals to run new ECE services and apply child-centred methodologies in their work with children. The Programme increased the overall number of children enrolled in public ECE programmes in target municipalities by 7% to 55% in 2015/2016, the highest impact being recorded in the areas which needed it the most. The Programme was implemented in a professional and culturally-sensitive manner. The current policy and legal framework governing ECE in Montenegro is supportive for the development and expansion of ECE services for children aged 3-6, but their financial sustainability is under risk in some locations. The learning infrastructure in some locations is not adequate, impacting on the quality of the educational process. The evidence provided by the Programme is a powerful tool in advocating for universal coverage of children in ECE and corresponding public finance.

Recommendations:

The evaluation provides a number of strategic recommendations (SP) and operational (O) recommendations in line with the ToR and the need to engage all major stakeholders in a concerted effort for the continuation of reforms on the advancement of ECE in Montenegro.

  • Strategic Recommendations (SR)
    SP1: Ensure sustainable equity prerequisites for expanding the ECE services for children aged 3-6 at national level and breaking intergenerational cycles of exclusion (Addressed to: MoE with the support of UNICEF and in partnership with PSIs, Centres for Social Work and Health care providers)
    SP2: Ensure that quality safeguards are embedded in the expansion of ECE service provision (Addressed to: BfES, MoE, UNICEF, PSIs, pre-service and in-service training providers and local communities)
    SP3: Encourage participation of children and empower parents to become active and efficient partners of the education system in the process of early childhood education and development (Addressed to: UNICEF in partnership with the MoE, PSIs and CSOs)
  • Operational Recommendations (O)
    O1: Carry out a quick quality and financial sustainability assessment of outreach interactive services in the target municipalities supported by UNICEF (first wave) (Addressed to: UNICEF, MoE and BfES)
    O2: Develop the Monitoring and Evaluation Framework of the new ECE strategy 2016-2022 (Addressed to: UNICEF and MoE)
    O3: Improve the results architecture and evaluability of future UNICEF initiatives (Addressed to: UNICEF CO)

Lessons Learned:

The Programme provided useful policy and programmatic lessons to the countries which would like to improve the enrolment of children aged 3-6 in inclusive and quality pre-school education. In this respect, the replication of Montenegro experience in other countries would need to consider the following lessons learnt:

  • Development of an enabling framework for universal coverage of children in ECE requires a systematic, comprehensive and concerted approach of all determinants of system change and of all key stakeholders at various levels of intervention.
  • Expansion of ECE for children aged 3-6 requires both hard and soft investment.
  • Any action targeting ECE must engage the parents of beneficiary children from start throughout the whole process of change.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2017

Country:
Republic of Montenegro

Region:
CEE/CIS

Type:
Evaluation

Theme:
Early Childhood Development

Partners:
Ministry of Education, Bureau for Education Services, Preschool Institutions 

Language:
English

Sequence #:
2017/001

New enhanced search