2015 Armenia: Evaluation of Family Support Services and Stakeholders Contribution to Related Services/Systems
Author: VISTAA Plus LLC (Armenia) and Mathematica Policy Research (U.S.)
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report.
This Evaluation Report presents the findings, conclusions and recommendations with respect to the object of the evaluation, namely family support services/systems provided to respond to child poverty, needs of vulnerable families with children and child care in the context of related interventions implemented by UNICED Armenia in 2010-2015 and stakeholders contribution to related services/systems. The scope of the evaluation covered the period from 2010 to present and covering Armenia’s 10 regions as well as the capital city Yerevan.
Currently, Armenia is implementing comprehensive reforms in the area of social protection, including child protection. Introduction of the Integrated Social Services (ISS) is the main pillar of these reforms emphasizing a “one window” model for provision of cash and non-cash benefits, co-location of different service providers (responsible for pensions, social assistance, employment and disability certification), introduction of the institute of case management and project management through local social planning efforts.
In 2010-2015 UNICEF Armenia has been heavily involved in the GoA’s social services reform guided by its belief that family vulnerability including issues related to child rights protection should be addressed holistically by a single system. UNICEF Armenia’s efforts in support of the reform ranged from policy advocacy and legislative advice to capacity building to public awareness of the reform.
The purpose of this formative evaluation is to assess the relevance, effectiveness, efficiency and sustainability of UNICEF Armenia’s interventions to child focused social services reform in Armenia.
The specific objectives of the evaluation were to:
- Assess the relevance, effectiveness, efficiency and sustainability of UNICEF Armenia’s support to selected state statutory services (Integrated Social Service Centres and Territorial Offices of Social Services);
- Validate UNICEF Armenia’s efforts to integrate ISS and CP functions in responding to the needs of vulnerable families with children including through improved mechanism of referring to in-depth family assessments when making a decision by GTC on child separation/alternative placement;
- Identify/document possible linkages, coherence and institutional cooperation between these state statutory services and other child related services;
- Assess advocacy for child-care reform, including methodological assistance in relation to de-institutionalization reform, reunification of children with their families; prevention of further admissions and promotion of alternative services.
The report was prepared based on the analysis and triangulation of data received through mixed data collection methods, including:
- desk review of the secondary documentation, including UNICEF Armenia reports and UNICEF Armenia supported research studies, evaluation reports, policies, laws, regulations and other relevant materials addressing reforms in social services and child protection as well as the impact of UNICEF Armenia’s interventions on child focused social services;
- quantitative census of 109 case managers working at ISS/TOSS, who were involved in UNICEF Armenia supported trainings or trained with UNICEF Armenia developed materials;
- in depth interviews and focus group interviews with over 100 stakeholders and key informants representing national level decision makers, donors, social and child protection system representatives, and most importantly beneficiary families.
The feedback, opinions and positions of respondents and key informants were triangulated with the information received from the desk review of materials to feed into the findings, conclusions and recommendations included in this report.
Findings and Conclusions:
- UNICEF Armenia decision to support social services reform in Armenia has proved to be worthwhile in terms of its relevance, efficiency, effectiveness and sustainability (relevance, efficiency, effectiveness and sustainability).
- Social services system in Armenia is in transition and transformation.
- While Integrated Social Services and Case Management are perceived as the adequate response mechanism to social needs, several gaps and challenges slow down effective implementation of the system.
- While training of Case managers has been used as a key intervention to build case management capacity, there are many gaps in case managers’ knowledge and skills.
- Application of case management practices (including tools taught through trainings) is controversial and inconsistent; common understanding of case management methodology is lacking.
- Consistently challenges to application of effective case management practices are explained by lack of resources that include operational resources and resources required for project implementation.
- Family, Women and Children Rights Protection Units are the main institution in Armenia dealing with child rights and child care related issues.
- While beneficiary families with children appreciate any support to overcome their social vulnerability, there are largely unaware of their rights and entitlement to any assistance other than cash benefits as well as the changes in the system.
- UNICEF Armenia should continue its support to social services reforms in Armenia based on such consideration and realities as expressed GoA political will to support the reform, enhanced legislative framework in support of social services reform and the pending regulations.
- UNICEF Armenia’s support should be balancing between sustaining its previous efforts and emphasizing child protection aspects of ISS and LSP. Increased awareness of beneficiary families about availability of new services and opportunities should be emphasized.
- With respect to case manager professional development and training, UNICEF Armenia should work with the GoA on the following: 1) separating case management and benefit administration functions, 2) putting in place a comprehensive capacity building program, 3) improving case manager involvement in development and implementation of Local Social Plans, 4) improving case managers’ involvement in child care/deinstitutionalization reform and 5) emphasizing the need of a senior position at ISS responsible for cross cutting aspects of case management.
- UNICEF Armenia needs to define and articulate its position on: 1) three tier child protection system and its linkages to the wider social protection system, and 2) community social worker institute and its relationship to case management.
- LSP ownership and sustainability should be linked to its funding base and implementation mechanisms. Currently it is not clear where the funding for LSP projects should come from, while its ownership is mixed since the LSPs are developed on regional level and implemented on local level.
- UNICEF Armenia needs to further enhance external cooperation with partners and internal coordination among its programs.
- UNICEF Armenia may consider concentrating its efforts around an innovative center of excellence project idea, which will demonstrate in practice how all aspects of ISS and child care/child protection could be integrated and coordinated.
- Success of UNICEF Armenia’s interventions with respect to case manager capacity building or LSP have been challenged by gaps in regulatory framework, lack of operational resources, overlap of case managers’ and social inspectors’ functions. In addition, certain aspects of the reform may move faster than others thus creating a misbalance between its parts. This was the case with LSP, which progressed more dynamically compared to ISS and case management resulting in LSP ownership on marz or community level, but not at ISS level.
- Within a five-year time frame developments could affect some of underlying assumptions and positions articulated at the start of the reform. This was the case both with the three tier system and social worker institute. As noted throughout the report, several factors and realities made UNICEF Armenia to revisit its initial position with respect to these two.
- Consistency in all aspects of agency programming is needed for ensuring the best benefit for children. Success of UNICEF Armenia’s work in child care and deinstitutionalization is to great extent reliant on existence of effective ISS and case management. While in several instances UNICEF Armenia has managed to build necessary linkages on planning level (i.e. by integrating a social service component into its deinstitutionalization project), their practical execution is yet to be ensured.
- Adherence to UNICEF’s requirements for results based management and Theory of Change driven efforts with sound underlying assumptions and risks are critical during the planning stages of any intervention to ensure their higher relevance and effectiveness and to be able to manage occurring diversions and interruptions, especially in multifaceted reform support areas like the one in question.
Full report in PDF
PDF files require Acrobat Reader.