2010 Philippines: UNICEF Philippines Country Program Evaluation
“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report.”
This report provides the results of an evaluation of the UNICEF’s fifth and sixth country programmes in the Philippines (CPC 5 and CPC 6) stretching over the period 1999 to 2009 and involving over US$111 million. The evaluation was commissioned by the UNICEF Philippines Country Office (PCO) and is intended to inform PCO management decisions on the CPC 6 extension (2010-2012) and the next country programme (2012-2016), as well as contribute to UNICEF organizational learning (and good practices) on promoting children’s rights and well being globally.
The evaluation was framed by existing measurement frameworks for CPC 5 and CPC 6, as well as key evaluation issues, negotiated with and approved by the Evaluation Advisory Committee1 and UNICEF’s Senior Management Team during the evaluation inception phase. Volume II, Appendix II provides the final evaluation matrix which defines the scope and focus of this evaluation.
Profile of UNICEF Support to the Philippines UNICEF’s support to the Philippines began in 1948 and since that time has broadened considerably, as exemplified below:
• UNICEF Roles evolving from an initial focus on emergency aid to children to encompass service delivery, capacity building and advocacy.
• UNICEF Primary Targets increased from children and families (in the 1950’s) to include national and local government agencies and civil society organizations (since the 1990s).
• UNICEF Program Approach initiated in CPC 1 (1979) represented a departure from previous project-based initiatives, guided by analyses of the needs of children. CPC 3 (1988) and CPC 4 (1994) introduced area- and sector-based programming respectively.
Compared to previous UNICEF programs, the major distinction of CPC 5 and 6 was the focus on strengthening national and local government capabilities. UNICEF’s development and pursuit of a Child Friendly Movement (CFM) promoted a more holistic and sustainable approach that addressed children’s needs in local, regional, and national contexts.
CPC 5 (1999-2004) sought to “implement national and local government capacity building to implement the CRC and to contribute to a massive mobilization in support of a CFM, at all administrative levels, and local communities and within the family.” This was a multi-sectoral approach, crossing and integrating seven programs or sectors (Communications, Local Policy and Institutional development, Health and Nutrition, Education, Children in need of special protection, Gender and Coordination, monitoring and evaluation), through 19 defined projects.
CPC 6 (2005-2011) defined “reduction of disparities among children in national and focus areas by at least 50%” as its key objective and pursued 18 defined projects within four sectoral components – Health and Nutrition, Education, Child Protection, HIV/AIDS; and two multi sectoral components – Communications and Local Policy and Institutional Development.
CPC 5 and 6 shared the same 24 focus areas, and many similar programs and projects.2 They are differentiated in two ways. Firstly, CPC 6 objective emphasized disparity reduction among children in national and focus areas; CPC focused more on capacity building. Second, program components were defined differently. In CPC 5 there were three interrelated multi-sectoral components, while CPC 6 had four sectoral and two multi-sectoral components. (CPC 6 included HIV/AIDS, a new component that did not exist in CPC 5.)
The evaluation is expected to assess UNICEF’s contributions to the GoP Medium Term Development Goals and capture lessons learned to inform the next round of the Common Country Assessment – UN Development Assistance Framework (CCA-UNDAF) 2012-2016 and the formulation of the UNICEF CPC 7.
The overall objective of the evaluation is to assess the results obtained in CPC 5 and CPC 6 and to capture lessons learned to inform the new country program (2012-16).
The primary audiences for the evaluation are UNICEF, including the UNICEF Philippines Country Office (PCO), UNICEF HQ, and the UNICEF Regional Office (EASPRO). Secondary audiences include UNICEF’s key
development partners in the Philippines, including the National Economic Development Authority (NEDA); agencies that fund UNICEF, including the Asian Development Bank (ADB), Australian Government Overseas Aid Program (AusAID), and the Canadian International Development Agency (CIDA); NGOs; and the UN Country Team.
An earlier version of this report was submitted to the Advisory Committee and to the UNICEF Management Team. Their comments and suggestions have been incorporated into this final report.
Evaluation methods included extensive documentation review, secondary data analysis, in-person and telephone interviews, focus sessions and site visits to six focus areas, where UNICEF had concentrated their efforts over this period. The methodology and analytical process were comprehensive, providing rigor and validity through triangulation (of both sources and methods). The Evaluation Team consisted of international and national consultants, who were committed and adhered to ethical principles and standards of evaluation.
The evaluation was undertaken for UNICEF Philippines by a team of international and national consultants under the direction of an Advisory Group of key CPC stakeholders including UNICEF and the National Economic and Development Authority (NEDA).
During the Inceptions Phase, the Evaluation Team developed the methodology for the evaluation (see sidebar). The Advisory Group provided guidance and feedback on all interim deliverables and approved the final Inception Report (February 2010) that served as the basis for conducting the evaluation. Data collection was carried out between February and April 2010 in the Philippines by a team of six international and Filipino consultants.
The evaluation conforms to the principles, standards and practices set out in the Norms and Standards for Evaluation in the UN System (2005), DAC Principles for evaluations (1991), and the DAC Evaluation Quality Standards (2007).
Conclusions and Recommendations:
Over the last decade, UNICEF, through and with its strong partnership with the GoP, made notable contributions to enhance government capabilities to implement the Convention on the Rights of the Child in the Philippines – the overarching goals of both the CPC 5 and CPC 6. CFM espoused an integrated multi-sectoral approach and made valued and recognizable contributions, most notably in stregthening the insitutional capacities of LGUs in focus areas and in national policy development. This evaluation revealed some issues for UNICEF to address as it plans its next country program and some lessons that could inform programming for children.
The following recommendations provide direction for UNICEF PCO going forward:
1. The UNICEF PCO should articulate a clear theory of change for the next country program and define its role within the change process to maximize its comparative advantage and resources in the Philippines.
2. In its upcoming programming in the Philippines, the UNICEF PCO should define sustainability strategies. These should include exit strategies, identification of risks and mitigation strategies to achieve sustainable results, and more sustainable approaches to capacity development.
3. In clarifying its program theory and future roles in the Philippines, the UNICEF PCO should increase its efforts to more systematically leverage resources for children.
4. The UNICEF PCO should develop a single clear results framework and a monitoring system to track and report on its performance in realizing planned objectives.
5. The UNICEF PCO should develop a more holistic approach to revenue generation that complements its program strategy and ensures adequate provision for its operational and administrative needs.
Lessons Learned (optional):
Lessons learned from this evaluation that may be of benefit to programming for children (notably
within UNICEF and the UN) are:
1. An organization that operates in a complex, multi-stakeholder context needs a single, clear results framework that speaks to all stakeholders.
2. Organizational change requires strong leadership, commitment, as well as the necessary incentives, support mechanisms, and checks and balances.
3. Innovative programs that aim to realize societal change take time and require a long term vision, commitment, as well as mechanisms to support program learning and adaption.
4. To be perceived as credible and trustworthy partners, organizations must demonstrate and account for their performance to stakeholders or risk reputational and/or financial risks.
5. Over reliance on one or a few sources of revenue reduces the viability of an organization.
6. The results of a capacity building initiative are more likely to be sustained if the initiative is mainstreamed within the targeted entity, if it supports one-time rather than ongoing costs, if exit strategies are defined and used, if sustainability issues are identified and addressed in timely ways, and if individual capacities are supported in tandem with (as opposed to in isolation of) institutional capacity building.
Full report in PDF
PDF files require Acrobat Reader.