We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2017 Macedonia: Evaluation of the training programme for continuous professional development of social protection staff



Author: Dessislava Ilieva, Natalia Mihaylova, Stoyan Mihaylov, Keti Jovanova

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

FYR of Macedonia has ratified most of the international conventions on children’s rights and protection of children. A key process is to reform and strengthen the capacity of the social protection system. DI and the development of non-institutional services are a key priority of the reform. Social support and services are administered at local level by 30 Centres for Social Work (CSWs) run by the Ministry of Labour and Social Policy (MoLSP) in 30 municipalities. MoLSP is responsible for the development of policies, strategic planning and inspection of entities providing social protection services. The Institute for Social Activities (ISA) is responsible for the assessment of social problems and possible solutions, continuous professional development, supervision of professional social work, setting standards and procedures.

As stated in the 2010-2015 CPD, UNICEF CO addresses three main development challenges faced by the children: child poverty, disparities and social exclusion. UNICEF supported the introduction of a comprehensive Training Programme for continuous professional development of social protection professionals, as part of a wider effort to reform the country’s social and child protection systems. It results in the development of ISA in-service training curriculum, training for ISA and CSW staff based on the new curriculum, and support the licensing process and training of professionals from social services. The main goal of the programme is to improve the performance and the quality of social services and contribute in the process of licensing of the social protection professionals.

Purpose/ Objective:

The main objective of this evaluation is to provide evidence of the impact and the results achieved by ISA Training Programme, identify bottlenecks and barriers in implementation and provide recommendations for future activities. Therefore, the objectives of the evaluation were to:

  • Assess the relevance, efficiency, effectiveness, sustainability and, to the extent possible, the impact of the ISA Training Programme;
  • Identify lessons learned and UNICEF contribution to these systems or impact changes; and
  • Provide recommendations to guide UNICEF involvement.

Methodology:

In the absence of outcome-based M&E mechanisms and a logical framework with objectives, expected results, indicators could not be developed. The evaluation team developed a framework of sub-questions to all evaluation questions. It provides a basis for identification of bottlenecks at individual and system level change. Evaluation was, therefore, parametric i.e. performed along parameters, or changes, at three levels: system level (an enabling environment); organisational level (in services); and individual level (within professional staff).
The methodology includes two groups of methods – tools for data collection and analysis of the information:
Data collection methods: Desk review and Primary data collection nationally and locally  (In depth Interviews (IDIs); Focus group discussions; Observation of social services for children; Online research of national coverage.)
Data analysis: Quality and quantity analysis; Comparative analysis; Stakeholder analysis; Cost-effectiveness analysis; Case Study.
The secondary data sources are not up to date, have limited focus and are sometimes not comprehensive; there is lack of coherent statistical data and ISA aggregated data. Quality assurance is achieved by triangulating the information collected via different instruments and sources.
The evaluation followed the UNEG Norms and Standards as well as the UNEG Ethical Guidelines for Evaluation.  Key features of the ethical code that were applied include: Respecting gender and human rights principles throughout the Evaluation process; Maximizing the degree of participation of stakeholders; Disaggregating data by gender, geography, and social groups, where feasible; Ensuring that outputs use human-rights and gender-sensitive language; Confidentiality was maintained; The quality of interviews and focus groups was ensured; Ensuring objectivity and independence of evaluation.

Findings and Conclusions:

Relevance
Overall, the ISA Training Programme is partly in line with the main strategic directions for the reform of the social protection system in the FYR of Macedonia. The ISA Training Programme is partly aligned with the CRC principles, gender mainstreaming and HRBA. The programme is insufficiently oriented to all children’s rights as it fails to address most vulnerable groups.
Effectiveness
The ISA Training Programme has been effective in ensuring core training needed for obtaining license but less effective in providing continuous professional development due to lack of customisation. The ISA Training Programme has been not effective in improving the service capacity for some of the most vulnerable groups: children with disabilities and children in residential facilities.
Efficiency
UNICEF has been highly efficient in providing support for the minimal package of requirements. However, to the extent to which licensing and the introduction of the minimal standards in the social protection sector do not directly lead to better quality of services, UNICEF financial support has not been highly efficient in improving service quality for children.
Impact
The quality of service provision for children has not changed for the last 5 years. In some cases the prospects of improving quality are diminishing. Foster care has not been targeted at all.
Sustainability
Generally, as presently designed, the ISA Training Programme demonstrates higher sustainability in ensuring licensing, lesser sustainability in ensuring standardisation, and weak sustainability in ensuring continuous professional development process.

Recommendations:

High priority recommendations

  • The ISA Training Programme for continuous professional development must be made a function of the national reform agenda, specifically in terms of case management promotion, deinstitutionalisation and inclusion of children with disabilities. (MoLSP/UNICEF)
  • The ISA Training Programme must be redesigned in a way that reflects all rights of all vulnerable groups of children with special attention to the most vulnerable: children with disabilities, children in contact with the law and children living without parental care, including foster placements. (ISA/UNICEF)
  • Disconnection needs to be made of the direct link between licensing and training for continuous professional development. Design new capacity building interventions on system level and service level in addition to classical training. (MoLSP/UNICEF)
  • Training needs to be made available to all child protection staff extending to those who work directly with children but are not subject to licensing. (ISA)
  • The ISA Training Programme needs to strategically link to other capacity building efforts targeting social protection. (MoSLP/ISA)
  • Redesigning the training content (along with conceptualising the programme as process as well as content) to promote a care management approach. (ISA/UNICEF)


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2017

Country/Office:
Macedonia

Region:
CEE/CIS

Type:
Evaluation

Theme:
Social Policy (Cross-cutting)

Language:
English

Sequence #:
2017/001

New enhanced search