We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2017 Guatemala: Evaluation of UNICEF support to the National Strategy on the Protection of Human Rights of Girls and Boys 2014-2016



Author: Universalia (Management Consulting Firm)

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

Migration has been one of the key problems faced by Armenia since independence due to poverty and unemployment. While labour migration (primarily to Russia) has been long considered a coping mechanism that allows families overcome economic challenges posed by unemployment, its rather serious social consequences have become more apparently visible throughout the years. Those include weakened family ties and increased rate of divorces, lack of adequate parental care and subsequent psychological problems, threat of sexually transmitted diseases and several others that affect the overall family in general, and children in particular.
From December 2012 to September 2016, UNICEF in Armenia has been implementing the European Union-funded “Mitigating Social Consequences of Labour Migration and Maximizing Migrants’ Involvement in Local Development” project (hereinafter the Project) in four target marzes (regions) of Armenia - Lori, Tavush, Shirak and Gagharkunik. The purpose of the project was reducing the social vulnerability of labour migrants' families and communities through provision of tailored social services and best use of migrants’ resources in the development of their communities of origin/return.
The project was designed as a three year intervention with 5 interrelated components and a budget of 1,103,160 Euros (80% of which was contributed by the EU, while the rest was covered through UNICEF contribution). Five key components of the project included Case management, Territorial/Local Social Planning, Institution Cooperation, Parental Care, and Diaspora Engagement. The first three project components built up on UNICEF in Armenia’s previous efforts to support policy setting and reform agenda with respect to case management, local social planning and institutional cooperation, and as such were designed to ensure “the practical application of the ongoing reform of Integrated Social Services (ISS) in the context of migrants’ families”.

Purpose/Objective:

The purpose of this evaluation was to assess “the relevance, effectiveness, efficiency, impact and sustainability of interventions under the EU-funded “Mitigating Social Consequences of Labour Migration and Maximising Migrants’ Involvement in Local Development Project”. The special focus of the evaluation, due to the design of the project, was on social services responding to the needs of socially vulnerable and migrant families in Armenia.

 
The specific objectives of the Evaluation included:  

  • Assessing whether the Project has successfully achieved its key outputs and outcomes (established in its Action Plan and the Log Frame) with respect to their coherence and suitability to addressing the social costs of migration;
  • Assessing the level of project sustainability with respect to changes in policies and practice, and
  • Assessing project achievements for migrant families within the overall ISS reform and UNICEF’s work in this regard as well as migration and development nexus.

Methodology:

This evaluation applied a combination of qualitative and quantitative methods for data gathering.  The qualitative methods included reviewing documents, semi-structured interviews and discussion with focus groups.  The evaluation team undertook field visits to nine municipalities located in five provinces.  In selecting the provinces, vulnerability criteria were used to compare the situation of each province in the following areas:  malnutrition; violence against women; indigenous population; and illiteracy and poverty.

Data collection was carried-out in May and June 2017.  These methods allowed the collection and triangulation of information to respond to a series of questions established in the evaluation ToRs.

The evaluation team faced some methodological challenges.  The main limitation was that the initial sampling proposed by the evaluators had to be adapted to the availability of partners in the selected provinces and municipalities. While it was possible to respect the sampling at provincial level, the team had to substitute some of the municipalities initially chosen due to the unavailability of some participants for the interviews.  To complement the documents provided by the UNICEF Guatemala Protection staff, the evaluation team made additional efforts to identify credible secondary sources (newspaper articles, reports on the internet) to triangulate perceptions

Findings and Conclusions:

The project supports the Government to comply with its national and international commitments to provide special protection for children and adolescents who are victims of violence, abuse, exploitation and neglect.  UNICEF interventions include political advocacy at the national and subnational levels, technical assistance to strengthen institutional capacity, training for officials and staff, and emergency interventions.  These interventions have been positively assessed by the beneficiaries, particularly UNICEF Guatemala’s technical assistance and political incidence.

Despite the political and institutional instability, UNICEF maintained its technical assistance at national and municipal levels.  The authorities of the institutions acknowledged UNICEF support, not only that which was planned support, but the support provided for emerging issues when necessary.  In addition, the municipal protection system for children and adolescents promoted by UNICEF Guatemala is highly valued by the mayors, municipal stakeholders, and the communities using the protection services.  The project expanded from 80 municipalities in 2013 to 130 in 14 provinces by the end of 2016.

In terms of effectiveness, UNICEF Guatemala and its partners supported over 19 institutions at the national and decentralised levels.  UNICEF actively engaged in political advocacy with Congress to influence the political agenda and Guatemala’s regulatory framework. 

Three main strategies were the key to achieving sustainability within State institutions:  a) technical support (diagnostics, consultancies, creation of protocols, among others); b) training for officials; and c) advocacy to promote changes in policies and institutional processes.

The evaluation concludes that the project has been relevant for the majority of its beneficiaries at the national and municipal levels, and that it is clearly aligned with national priorities

Recommendations:

The recommendations were developed based on the findings of the evaluation and the analysis of the criteria by relevance, effectiveness, efficiency and sustainability.  The UNICEF protection component should:

1) continue promoting the institutionalisation of the municipal protection system throughout the country;

2) develop a strategy to enhance the integration of gender and cultural relevance in all work with the State institutions;

3) document the protection work at community level in order to validate and scale up its interventions at the municipal level in the next phase of the project;

4) together with implementing partners, consider alternative strategies to ensure the sustainability of the municipal protection system, in the case the proposed Law on the Protection System is not approved; and

5) hold mid-year evaluation meetings with its implementing partners and entities benefitting from training and institutional capacity development interventions.

6) intensify advocacy with Congress in order to influence budgetary allocations to allow scale-up and decentralisation of actions carried out with key stakeholders. 

7) establish coordination mechanisms with other cooperation actors working in child protection in the field.

8) identify the strategic priorities and the most suitable partners to carry them out

Lessons Learned:

The collected and analyzed data revealed the following lessons learned:

- the PINA Law adopted in 2003 provides a legislative framework for comprehensive child protection which complies with international standards, and is a necessary condition but not sufficient for its implementation nor for effective protection of the human rights of girls and boys;
 
- long-term planning and investment are necessary to successfully adopt a systemic approach in a national context characterized by political instability and weakened State institutions;

- collaboration and partnerships with NGOs and other development agencies are essential to achieve results for a greater number of beneficiary institutions in different administrative areas of the country; and

- political advocacy has a greater impact when it is carried out with higher level authorities.







Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2017

Country/Office:
Guatemala

Region:
LACRO

Type:
Evaluation

Theme:
Child Protection

Language:
Spanish

Sequence:
2017/008

New enhanced search