We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 Montenegro: Programme-Informing Evaluation of the 2014-2017 Child Rights Monitoring System in Montenegro and Planned Approach to CRM under the 2017-2021 Country Programme



Author: Carolyn Hamilton, Awaz Raoof, Ruth Barnes, Jorun Arndt

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

This report presents the findings, conclusions and recommendations stemming from an evaluation of the child rights monitoring (‘CRM’) system in Montenegro at the national and local levels for the period from January 2014 to October 2017, in light of the Government of Montenegro and UNICEF’s planned approach to strengthening the CRM system under the 2017-2021 Country Programme. The evaluation was commissioned by the UNICEF Country Office in Montenegro and conducted by Coram International at Coram Children’s Legal Centre.

Purpose/Objective:

The overall purpose is for UNICEF and key national stakeholders (notably the Government of Montenegro; Parliament; the Ombudsperson and particularly the Deputy Ombudsperson for Child Rights, Social Protection and Youth (‘DO’), civil society organisations (CSOs) and academia) to use the recommendations, knowledge and insights from the evaluation to inform the development of UNICEF-Government joint programming under the new Country Programme 2017-2021. Secondary or indirect beneficiaries include children, parents, caregivers, professionals and practitioners who work with children, and others who may benefit from the use of the evaluation by the primary target audience.

Methodology:

The conceptual framework for this evaluation is rooted in the CRC and international standards relating to CRM. Further, the evaluation criteria, questions and indicators for analysis were developed based on the Organisation for Economic Co-operation and Development/Development Assistance Committee (‘OECD/DAC’) Principles for Evaluation of Development Assistance: relevance; effectiveness; efficiency; and sustainability, with the addition of two additional criteria (coordination and mainstreaming of cross-cutting equity issues) which were considered important in light of the objectives of the evaluation. Given that this is a programme informing evaluation, an evaluation of impact or unintended impact was not considered a priority or possibility at this stage, although the evaluation sought to produce knowledge and insights to develop recommendations on how UNICEF interventions may be developed to contribute to the desired impact.

This evaluation adopted a non-experimental design and was primarily qualitative, although it included analyses of secondary quantitative data to facilitate triangulation and interpretation of qualitative findings. Data collection techniques included: a desk review; semi-standard interviews, and focus group discussions with children, in three research sites (Podgorica; Bijelo Polje and Herceg Novi); a review of a random sample of Ombudsperson case files; and a validation meeting to refine and validate findings and recommendations. The evaluation was carried out according to UNICEF’s and Coram International’s ethical standards.

Findings and Conclusions:

Likely impact: The activities of the key CRM stakeholders and mandates of the CRM Mechanisms are broadly relevant to the needs of boys and girls in Montenegro.

Inter-Sectoral Coordination: Despite the clear mandate of the National Council on Children’s Rights (CCR) to conduct CRM, it is not currently achieving effective inter-sectoral coordination amongst the different CRM mechanisms. 

Relevance: Of the CRM Mechanisms, periodic reports issued by line ministries and Parliamentary committees are less relevant to children compared to the more specific CRC and CEDAW periodic reports or the annual reports on the implementation of the NPAC 2013-2017, which are targeted at improving children’s rights.

Effectiveness: Data collected across all Government bodies on child rights is primarily quantitative and does not provide sufficiently disaggregated data (by gender, age, ethnicity, rural/urban location, disability or other status) to enable identification of the gaps or inequities in the enjoyment of child rights across the country.

Efficiency: All CRM stakeholders, including the Ombudsperson and the DO, Government ministries and departments, Parliamentary committees, CCR and NGOs, face financial resource challenges.

Sustainability: Knowledge and skills retention in the CRM system appear to be threatened, not so much by staff turnover, but rather by funding and resource constraints that lead to uncertainty and burn out among staff. Other contributory factors include the absence of tools or guidelines to support stakeholders in carrying out their CRM activities or in systematising CRM practices, and limited opportunities for ongoing training on CRM and research on child rights.

Cross-Cutting Equity Issues: Different CRM stakeholders, particularly line Ministries and departments, the Ombudsperson, public bodies and CSOs, collect quantitative data concerning children, however not disaggregated by age, sex, disability, ethnic or social origin or urban/rural areas.

Recommendations:

The Government should consider:

  • Prioritising CRM and children’s issues on its agenda, in particular, by integrating this within its activities and strategies towards EU accession; 
  • Reforming the institutional structure of the CCR by establishing a multi-sector committee chaired by the Deputy Prime Minister and including representatives from all line Ministries involved in children’s matters.

The Parliament, including all Parliamentary committees that engage with children’s matters, should consider:
Making full use of their powers to question Government on its CRM record, using the findings of the Ombudsperson’s reports, amongst other sources, to inform its deliberations.

It is recommended that the Ombudsperson and DO should develop a costed strategy and work plan outlining their vision in the area of CRM for the next five years, including concrete goals and activities, within the broader objective of attaining ‘A status’ by the GANHRI.

UNICEF should consider providing technical assistance to the Government and relevant professional training bodies, such as the Institute for Social and Child Protection, to develop a knowledge and skills-based capacity-building training-of-trainers course, focusing on a broader education on children's rights; the aims, objectives and purpose of CRM and the CRM Mechanisms; child rights budgeting; and the effective use of data to monitor child rights, with a view to working with professional training academies to integrate this course into existing training programmes for civil servants involved in CRM.

UNICEF and the Government should consider extending its planned CRM activities under the Country Programme to strengthening national systems for data collection beyond SWIS and PRIS, including MEIS, the new judicial information system, ‘ISP’, and the collection of data by the MoH, to establish a harmonised data collection system for CRM bodies with the inclusion of disaggregated indicators.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2018

Country/Office:
Republic of Montenegro

Region:
ECAR

Type:
Evaluation

Theme:
Social Policy (Cross-cutting)

Institution:
Coram International at Coram Children’s Legal Centre

Language:
English

Sequence #:
2018/001

New enhanced search