We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 Armenia: Evaluation of Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services as Part of Child Care Reform Project



Author: VISTAA Plus LLC

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

The USAID funded Project of UNICEF Armenia has started in September 2014 with an end date of September 2019. However, in September 2017 USAID Armenia made a decision to terminate its agreement with UNICEF thus suspending some of the Project activities.

Before the termination, both UNICEF and USAID agreed to proceed with a mid-term review/evaluation of the project. As a consequence of project termination, the assignment that had been initially designed as a formative mid-term evaluation evolved into more of a final evaluation of the Project activities.

Purpose/Objective:

The main goal of the Mid-Term Evaluation of the “Toward Social Inclusion of Vulnerable Children: Expanding Alternative Care, Family Support and Inclusive Education Services as part of Child Care Reform’’ project (the Project) is to assess the relevance, effectiveness, efficiency as well as impact and sustainability of interventions with specific objectives defined in the ToR. The time frame covered by evaluation is September 2014-June 2017 with geographical coverage focusing on Lori, Syunik and Yerevan.

Methodology:

Mixed (quantitative and qualitative) data collection methods were used by the evaluation team for data collection from relevant groups:

  • Expert interviews were used for data collection from national level key stakeholders (ministries, NIE, etc.) as well as the Office of Human Rights Defender, legal and child protection experts, etc. In total, 25 experts were interviewed as part of this stakeholder group.
  • In depth interviews (including group interviews) were used to collect data from Project beneficiary/stakeholder groups identified through stakeholder mapping such as parents/family members, school principals, staff of RPPSC, CFSC, regional authorities, service providers and others. In total, in-depth interviews (including 3 small group interviews) were conducted with over 40 key informants.
  • Focus groups were conducted with teachers and parents trained under the inclusive education component, trained foster parents, staff of institutions and services. 14 Focus groups were conducted with a total of 104 participants.
  • Desk review and observations were used as additional qualitative data collection methods.
  • In compliance with the ToR for this assignment, trained school teachers under the Inclusive Education component (implemented by the BoH) were the target respondents of the quantitative data collection. As agreed with UNICEF during initial meetings, the evaluation prioritized Syunik for the quantitative data collection given the lengthier experience (2 years) of the schools in this region of involvement in inclusive education. Quantitative interviews were implemented with 312 teachers of Syunik Marz.

Findings and Conclusions:

The evaluation findings are presented using the evaluation criteria, i.e. relevance, effectiveness, efficiency, sustainability and impact. As prescribed by the ToR, “Given that this is a mid-term evaluation, two of the evaluation criteria this project will be evaluated against, namely sustainability and impact, are to be considered to the extent possible”.

Recommendations:

Recommendations included in the report build up on the Findings and Conclusions resulting from this evaluation and are also grouped by relevant evaluation criteria.

Lessons Learned:

Some lessons learnt that can be derived from the result of this evaluation include the following:

  • Putting two reforms and directions under on one Project umbrella has complicated the Project structure and coordination creating pressures with respect to communication with partners and stakeholders, involving them in advocacy in different areas, undertaking responsibility for ensuring results in too many directions.
  • UNICEF in Armenia’s role as a major international organization involved in policy advocacy on child rights protection is very important and as much as possible should not be mixed up with the role of implementer, thus challenging its involvement with national policy makers as a policy leader and a watchdog.
  • A national level public awareness campaign should have been incorporated in the design of the Project and its standalone components especially given its focus on sensitive reforms touching upon thousands of lives, including children and their families, as well as requiring involvement and coordination of different stakeholders from different levels in society.
  • Acceptance of a model for reorganization of the institutions should have been more rigorously pressured and promoted to the Government of Armenia allowing for a specific approach to closing down the institutions and establishing alternative services across the Project-targeted institutions. Similarly, community based alternative services should have been established in the communities of origin of returning children. UNICEF in Armenia could have worked more intensively with the NGOs present in the regions to involve them in the follow up of the reunified children and their families (as was the case with SOS villages, which was mobilized at later stages of the Project to support its implementation throughout its ongoing activities).


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2018

Country/Office:
Armenia

Region:
ECAR

Type:
Evaluation

Theme:
Child Protection

Language:
English

Sequence #:
2018/003

New enhanced search