2014 Colombia: Joint Evaluation of the Cooperation Programes of UNICEF Colombia with Canada (2009-2013) and Sweden (2011-2013)
Author: Gabriela Byron, Eleanor Douglas, Alejandra Bravo and Rhian Williams
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 4’ of the report.
The evaluation covers the results achieved by UNICEF Colombia with the support provided by Canada and Sweden. Canada’s total support from 2009-2014 was CAD 17,000,000 and covered all areas of UNICEF’s program (Early Childhood Development, Quality Education, Adolescent Development, Child Protection, and Evidence Based Public Policies). Sweden provided USD 9,046,849 (2011-2014) to support the following more specific areas within the overall area of Child Protection: prevention of recruitment by illegal armed forces of children and adolescents, support to children and adolescents affected by conflict and improved protection through strengthened implementation of the recommendations of SCR 1612. The lens of strengthening protective environments for children and adolescents was used as an overall shared thematic approach for the evaluation. As of June 2014, UNICEF had implemented 96.5% y 86.06% of the funds from Canada and Sweden, respectively.
The Specific objectives of the evaluation were to assess the results achieved, as well as the effectiveness, efficiency, relevance and sustainability as outlined in the Framework of the support provided to UNICEF by Canada and Sweden; to evaluate the incorporation of gender into the program; to assess the degree of coordination and harmonization between Sweden and Canada; to identify the principle lessons learned; and to provide programmatic and strategic recommendations. The key stakeholders for this evaluation are Canadian and Swedish cooperation, and UNICEF Colombia. The evaluation took place as UNICEF’s agreements were coming to a close, and shortly before planning a new agreement with the government of Colombia. The evaluation is to serve as an input to these decisions. A workshop was also planned to share findings with a group of key partners and strategic allies.
An external team composed of a team leader, a senior and intermediate evaluators, and a research assistant carried out the evaluation between March and July of 2014, including 3 weeks in country with extensive interviews in Bogota, followed by field visits to programs in La Guajira, Chocó and Putumayo to assess the effects of programs on children and youth at the local level, as well as the effectiveness of UNICEF’s upstream- downstream approach.
The Team carried out interviews with approximately 100 individuals representing key ministries and government departments, local governments, NGOs, church based organizations, communities, educational institutions, health institutions, and organizations of international cooperation including other UN agencies. It also held focus groups with children and youth in all three territories using age appropriate tools and sensitivity to the effects of conflict. In addition, the team carried out an extensive review of documents from UNICEF, the government of Colombia, and partner reports and studies to ensure both depth and breadth of the data sources. The evaluation was based on standard OECD criteria of effectiveness, efficiency, relevance and sustainability, and also included a special consideration of gender and of coordination and harmonization, according to key questions in the Terms of Reference.
Findings and Conclusions:
UNICEF’s strategies have been a major factor for success, particularly when they are used to reach the same objective.
The strategy of combining ground level practice with national level policy allows for more effective and credible advocacy, although UNICEF’s limited presence in the territories affects its ability to take full advantage of this potential. UNICEF’s integration of a gender mainstreaming approach remains in need of more attention.
UNICEF is efficient in addressing ongoing crises such as the consequences of armed conflict, short-term emergencies (emergency nutritional support for young children isolated by conflict or threatened by drought), and long-standing inequalities and exclusions. Central to its approach is empowerment and resilience which allow girls, boys, adolescents, their families and their communities to address a range of issues that threaten them in the long or short term.
Relevance in the Colombian context is high, as UNICEF is aligned to national government strategies and contributed to their development and implementation. Implementing partners and UNICEF have closely aligned objectives, complementary strengths, and respectful, horizontal relationships.
Sustainability of results is built into the program as projects are designed to be replicable by others, and systematized to that end. The strategy of promoting policy change based on evidence, and then providing technical support to implement it, has been successful over the years, and is likely to be so with the newer initiatives in recent years.
Harmonization and Coordination
The Evaluation Team found there were common grounds for harmonization and coordination between Canada and Sweden in the work with UNICEF, but this evaluation may be the first concrete coordinated activity. However, support to UNICEF’s coherent overall program ensured efforts were not duplicated. There is evidence of complementary activity.
UNICEF must increase its field presence. It should design and implement monitoring systems that allow it to report on its results at the departmental level. The evaluation recommends that UNICEF further develop and adapt the organization’s draft Theory of Change to the Colombian context.
In order to properly monitor the overall results of working with consultants, as well as their cost effectiveness for UNICEF Colombia, the evaluation recommends that UNICEF develop an information management system for consultants that would allow it to track the kinds of consultancies, length of contracts, results in relation to “outputs/outcomes”, etc.
Given the eventuality of a Peace Accord, the Evaluation recommends that UNICEF facilitate the participation of children and adolescents in local peace-building, ensuring their proposals are heard and incorporated into national processes of Truth, Memory, Reparation and Reconciliation, including gender needs and the concerns of ethnic minority children and adolescents.
Given UNICEF’s strategy of supporting innovative pilot projects, the Evaluation Team recommends that UNICEF develop more explicit strategies and provide sufficient resources to ensure successful transfer or replication of projects.
As gender results are inconsistent, the Evaluation Team recommends that UNICEF fully implement its comprehensive corporate gender policy, adapting it to Colombia and fully resourcing it, specifically:
Hire a full time gender specialist to work with the staff and partners
Carry out an organizational needs assessment in gender capacity, as the basis of a capacity development plan
The governments of Sweden and Canada should consider supporting the overall UNICEF program or specific parts of it, to reduce the requirements on UNICEF to report separately to each donor, and to increase opportunities to develop a joint monitoring process.
"Report" - Evaluation Report Part 1
"Part 2" - Evaluation Report Part 2
"Part 3" - Annexes
"Part 4" - GEROS
Full report in PDF
PDF files require Acrobat Reader.