We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2016 Macedonia: Evaluation of UNICEF Child Protection Programme



Author: Pluriconsult Ltd.

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as 'Part 3'.

Background:

The object of the evaluation was the UNICEF supported child protection programme 2010-2015. This programme had two main components. One component was related to Justice for Children and also included issues affecting children victims of violence. The activities under this component addressed three main areas of systemic change: policy and legislation, capacity building and prevention. The second component focused on the reform of the social protection sector targeting, in particular, Centers for Social Work. This was addressed by introducing of standards of work, an in-service training programme, licensing of social work and establishment of a data base.

UNICEF supported the reform at both central and local levels, provided policy advice and technical assistance to key legislative reforms, assisted in demonstrating how the provision of family support services can be translated in practice, provided capacity development for improving quality of service and exchange of experience with other countries to move forward the reform agenda.

Purpose/Objective:

The present document is the Evaluation Report (ER) for the project entitled Evaluation of UNICEF Child Protection Programme in the Former Yugoslav Republic of Macedonia 2010-2015, under Contract No. 43182510, between the United Nations Children’s Fund (UNICEF) Country Office Skopje and Pluriconsult Ltd.

This evaluation covered the two key programme components, justice for children and social protection sector. The purpose was to:

  • Assess the relevance, efficiency, effectiveness, sustainability and to the extent possible impact of UNICEF's programme interventions;
  • Identify and document lessons learned and the contribution of UNICEF to these systems or impact changes; and
  • Provide recommendations to guide the child protection programme for the next programme cycle.

This was a formative evaluation undertaken at the end of the UNICEF supported programme for 2010-2015, signed with the Government, and covered this programming period. The knowledge generated by the evaluation will be used by the Government and UNICEF to inform the planning and implementation of the new country programme 2016-2020.

Methodology:

The evaluation followed internationally agreed evaluation criteria of relevance, efficiency, effectiveness, impact, and sustainability. Stakeholders participated in the evaluation through discussions, consultations, provision of comments on draft documents and some of them will be responsible for follow-up of the recommendations. In gathering data and views from stakeholders, the evaluation team considered a cross-section of stakeholders with potentially diverse views in order to ensure that the evaluation findings are as impartial and representative as possible. The evaluation was based on a mix of quantitative and qualitative approaches, and utilised a participatory methodology to establish triangulation of information through a variety of means. The evaluation was based on analysis of secondary data and on primary data collection.

The evaluand was an intervention of a rights based organization, therefore the evaluation mainstreamed gender and human rights considerations throughout. Whenever possible, disaggregation of data by gender and age was made.

Findings and Conclusions:

Relevance
UNICEF CP programme has been embedded in the political, social and economic context of the country. UNICEF CP programme has shown flexibility and ensured a balance between planned interventions and adjustments to the changes in the political agendas.
UNICEF aligned its programming with Government’s priorities and has been able to strengthen its programming by its close links to national objectives. However, the frequent changes of government/ministers have been challenging, and UNICEF has invested significant time of its staff to respond to these changes and to establish strong relationships with new decision makers and different perspectives.
Efficiency
Over the reference period UNICEF has demonstrated a good capacity to make the best use of the available resources in the interest of the final beneficiaries. The quality of UNICEF management of activities is highly appreciated by donor and partners.
Effectiveness
Overall, the programme was successful in achieving the planned results. However, demonstrating the progress in terms of quality seems to be more of a challenge compared with showing results in terms of quantity. There are difficulties in the diffusion of results from central to local level and in ensuring a good coordination among the stakeholders.
Impact
Both components aimed at achieving better results for children by enabling the environment for better care and protection of children (i.e. the legislative reform and strengthening institutional capacities), as well as by piloting the diversification and improvement of services. Nonetheless, tracing the direct influence of the programme on changes in children’s well-being has been challenging for reasons of difficulties in establishing a direct causal relation between the interventions within the respective component and the final beneficiaries and for reasons of limited sustainability and monitoring of the cases.

Recommendations:

  1. More emphasis on prevention: The Government should develop more family support services in the country, in terms of variety and geographical coverage and adopt measures aiming to ensure that social work practice is reoriented on outreach service based on professional workers.
  2. More focus on the reduction of equity gaps: All stakeholders working in the J4C and SP should avoid ‘creaming’ of cases (i.e. approaching those case more easy to work with and to report progress
    about) in the pilot projects and in general. The most vulnerable groups of children should be the
    priority in terms of resources allocation.
  3. Inter-sectoral coordination among the authorities: The decision makers should develop a vision of what CP should be in the country and based on this overarching desiderate to develop a more coherent system.
  4. Strengthen the capacity of social workers and of the staff working in the SP system: MoLSP should reduce staff turnover by motivating them to remain in the system and by reducing employment based on political criteria. Clearer system of reward based on functional workloads need to be introduced together with careful workload management and the development of a range of incentives for specialized social work interventions.
  5. MoLSP and ISA should increase the capacity of the managers of the CSWs, including by measuring the efficiency of the CSWs in order to increase the accountability of the CSWs’ managers. In addition, the system of services’ quality assessment should become fully implemented and operational.
  6. There is a need on behalf of the public authorities to strengthen the stability of the legal framework. When legislative changes are needed, there should be more time allocated to consultations with a wider variety of stakeholders at different levels (including with people who will implement the law, i.e. practitioners). UNICEF should advocate for ex-ante impact assessments of future laws.


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2016

Country:
Macedonia

Region:
CEE/CIS

Theme:
Child Protection

Type:
Evaluation

Language:
English

New enhanced search