We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2014 Montenegro: Evaluation Report for the Justice for Children Initiative



Author: Professor Carolyn Hamilton, Awaz Raoof and Jorun Arndt

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report.

Background:

Coram Children's Legal Centre was engaged to conduct the final, summative evaluation of the Justice for Children Project (J4CP) - an initiative of the Montenegrin Ministry of Justice (MoJ), Ministry of Labour and Social Welfare (MoLSW), supported by UNICEF Montenegro and with the financial assistance of the European Union (EU) under the Instrument for Pre-Accession Assistance 2011. The total budget for the J4CP was €550,000, comprising €500,000 from the EU and €50,000 from UNICEF. The Contribution Agreement for the J4CP between the EU and UNICEF was signed on 9 July 2012, and the project implemented from November 2012 to 9 July 2014.

The overall aim/intended impact of the J4CP pursuant to its Logical Framework (LogFrame) and Theory of Change (ToC) is to ensure that children benefit from the full application of international norms and standards when they come into contact with justice and related systems as alleged offenders, victims and witnesses of crime or for other reasons where judicial, state administrative or non-state adjudication is needed. To achieve its desired impact, the LogFrame and ToC specified a series of 17 activities and 9 outputs to achieve the following three outcomes:

  • The development and implementation of a legislative and policy framework for juvenile justice, including the adoption of an appropriate juvenile justice data collection and information system upgraded and harmonised with internationally recognised standards;
  • The strengthening of institutional, administrative and human capacity for the implementation of the newly adopted Treatment of Juveniles in Criminal Proceedings Act 2011;
  • The increased public awareness of juvenile justice matters through the active participation of boys and girls in conflict with the law in the promotion of their rights.

Purpose/Objective:

As per the Terms of Reference (ToR), the Evaluation covered the entire project implementation period (i.e. from July 2012 to July 2014) at both national and local levels and all project components, following the way that the J4CP was conceptualised in the Grant Proposal, LogFrame and ToC. This Evaluation is aimed to:

  • Provide feedback to EU Delegation to Montenegro, UNICEF Montenegro, the MoJ and MoLSW, and other stakeholders and partners, on the relevance, effectiveness, impact, efficiency and sustainability of the J4CP, including by revealing good practices adopted as part of J4CP and identifying lessons which can be learnt from the implementation of the J4CP; and
  • Inform the development of recommendations aimed at furthering the juvenile justice system reform process in Montenegro.

It is envisaged that the findings and recommendations generated from this Final Evaluation will be used by the MoJ, MoLSW, EU Delegation, UNICEF and independent oversight bodies and NGOs to further develop the area of juvenile justice, strengthen the implementation of the new Act, strengthen monitoring and advocacy efforts, and inform further reform efforts.

Methodology:

The methodology for the data collection has been developed using an ‘Evaluation Matrix’ encompassing the criteria against which the J4CP was evaluated (relevance; effectiveness; impact; efficiency; and sustainability), the evaluation questions for each criteria, indicators for each evaluation question, and a plan for the data collection corresponding to each evaluation question. As per the TOR, the Evaluators used a mixed-methods approach utilising quantitative and qualitative data collection techniques and non-experimental design to gather rich and accurate data and improve the validity of results through triangulation. This consisted of a desk review and in-country data collection visit to four regions (Podgorica; Bijelo Polje; Tivat; and Nikšić) where a series of semi-structured individual interviews, focus group discussions and observations were conducted. The Evaluators adopted a participatory and process throughout the Evaluation by, among other things, submitting the draft methodology, data collection tools and evaluation report to UNICEF for feedback from the EU, MoJ, MoLSW and other key stakeholders, and via a validation conference with stakeholders held in Podgorica on 3 July 2014.

Findings and Conclusions:

Relevance: The design of the J4CP is extremely relevant to the needs of stakeholders and beneficiaries.
Effectiveness: The Evaluators conclude that the J4CP was, overall, effective in meeting the outcomes as defined by the LogFrame and ToC.
Efficiency: From the information available, the Evaluators conclude that the J4CP was on the whole cost efficient, although certain activities could have been designed in a more-cost effective manner.
Impact: The J4CP is contributing to achieving its desired impact.
Sustainability: The J4CP is broadly sustainable.

Recommendations:

The evaluation report outlines recommendations for stakeholders, including:

  • The MoJ, MoLSW and UNICEF explore further opportunities to support the relevant training bodies in providing continuous training to juvenile justice professionals and to ensure an adequate geographical coverage and numbers of trained personnel throughout Montenegro
  • The MoJ, Judicial Training Institute and MoLSW provide further cross-sector practical training in the work of the PSS
  • The MoJ explores opportunities to provide financial and logistical support to the PSS to assist it in procuring the equipment and strengthening the infrastructure it needs to conduct its services effectively
  • The High Court of Podgorica moves its child interview room to a location that is more conducive to holding child-friendly hearings
  • The MoJ and UNICEF reconsider how to organise the implementation of the criminal sanction of referrals to closed-type institutions for children in conflict with the law
  • MoJ, MoLSW, PSS, Ljubovic Centre and Institution for Execution of Criminal Sanctions ensure that all NGO/third party programmes selected to provide services for children in conflict or in contact with the law have been designed with equity and HR & GE considerations as priorities
  • The Judicial Training Centre, Police Academy and Bar Association:
    - approve and accredit the final specialist training programmes for judges, prosecutors, police and defence lawyers respectively;
    - incorporate into the specialist training a specific module on the needs of children of different genders, children with disabilities and children from ethnic minority groups; and
    - provide continuing professional development programmes focused on the treatment of juvenile victims and witnesses to refresh the skills of trained professional and to train up new professionals.
  • The MoLSW and MoJ:
    - work with the Ljubovic Centre to ensure that it is financially and logistically able to implement its amended Methodology and Programme of Work; and
    - establish geographically accessible day treatment centres that could provide similar support services to the Ljubovic Centre across the country.

Find below:

"Report" - Evaluation Report
"Part 2" - Annexes
"Part 3" - GEROS



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2014

Country:
Montenegro

Region:
CEE/CIS

Theme:
Child Protection

Type:
Evaluation

Language:
English

New enhanced search