2010 Global: Progress Evaluation of the UNICEF Education in Emergencies and Post-crisis Transition Programme (EEPCT)
Author: N. Boothby, P. Buckland, et al.
“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report.”
Education is a fundamental right for children. However, it is estimated that 72 million children remain out of school, 54% of whom are girls. UNICEF’s Education in Emergencies and Post-Crisis Transition (EEPCT) programme began in 2006 as a four- (later extended to five-) year, US $201 million dollar partnership between UNICEF and the Government of the Netherlands. Additional support for the EEPCT Programme was provided through a contribution of €4 million from the European Commission (EC). The EEPCT Programme aims to “put education in emergency and post-crisis transition countries on a viable path of sustainable progress toward quality basic education for all.” It seeks to accomplish this through four principle goals:
1. Improved quality of education response in emergencies and post-crisis transition countries;
2. Increased resilience of education sector service delivery in chronic crises, arrested development, and deteriorating contexts;
3. Increased education sector contributions to better prediction, prevention and preparedness for emergencies due to natural disaster and conflict; and
4. Evidence-based policies, efficient operational strategies and fit-for-purpose financing instrumentsfor education in emergencies and post-crisis situations.
UNICEF sees the EEPCT programme as the centrepiece of its education-programme activities in humanitarian crises, post-crisis and transition situations. EEPCT funds support UNICEF education programming in 39 countries and territories and are also used to advance the global agenda for education in crisis-affected contexts.
The purpose of this Progress Evaluation (PREV) is to identify and assess progress in the strategic goals of the Programme and to enable systematic reflection that results in concrete programme improvements. The EEPCT Programme was examined at global, regional and country levels through quantitative and qualitative methods that combined comprehensive coverage with in-depth analysis.
Data collection took place June-August 2010, and included primary and secondary source literature reviews, key informant interviews, staff and partner surveys, global surveys and blogs, UNICEF self-assessments and field visits, observation during site visits, focus groups, and interviews. The evaluation reviewed the 39 EEPCT countries and territories.
Findings and Conclusions
The original programme proposal was both ambitious in scope and imprecise in laying out the programme’s expected results. The visionary rhetoric and lack of precision in the proposal allowed for rapid start-up in activities, but the delay in clarifying specifics of the proposal undermined overall programme coherence and effectiveness.
Funding was directed through well-established UNICEF channels, which track resources by donor, generating a good record of the flow of funds. While this mechanism provides for tracking of funds allocated and expended according to donor and country programme, it does not permit easy analysis of expenditure by goal or activity. The evaluation identified difficulties related to the flow of funds to countries. Funds are received late in the fiscal year, which leads to a scramble to allocate them in the year received. The process by which funds are allocated to countries was substantially improved in 2009 and 2010.
Communication within UNICEF was not sufficient for country offices to understand EEPCT’s aims and objectives. EEPCT has been used more as a fund to support existing country programmes than to support the programme’s global objectives. In 2009, UNICEF undertook significant steps to address the lack of clarity regarding the objectives of EEPCT at the country level, and understanding at the country level has improved.
Besides the overall findings described above, the evaluation also reviewed the programme against its four specific goals.
Goal One seeks to support improved quality of response in education systems in emergency and transition countries. Overall, EEPCT funds have enabled UNICEF to work in a more flexible, timely and responsive manner with partners and governments to promote a more coordinated, higher-quality education response in emergencies and post-crisis transitions. Moreover, the flexibility of EEPCT is in line with OECD-DACs’ Principles of Engagement in Fragile States and Situations to “act fast and stay engaged to give success a chance.” EEPCT funding is especially well-suited to support elements of education in situations where predictable funding for education is lacking, and coordination amongst actors – including donors -- is weak.
Almost all the 39 countries or territories supported by EEPCT funds are involved in various child-friendly schools initiatives (CFSI). Briefly stated, child-friendly schools spring from a rights-based approach to education, and aim at an environment in which children are motivated to learn, with friendly and welcoming staff. The breadth of coverage is an indication of the extent to which the concept has brought a measure of coherence to UNICEF’s efforts to change the quality and conditions of learning in all countries. However, the extent and manner of implementation varied substantially and there appeared to be confusion between reporting on the number of child-friendly schools (CFS) and the number of schools that were not CFS as such, but were involved in child-friendly school initiatives (CFSI).
Goal Two is to support the increased resilience of education service delivery. Resilience is a key concept in post-crisis reconstruction. Globally, resilience is defined as the “capacity of a system to absorb disturbance, undergo change and still retain essentially the same function, structure, identity, and feedbacks.” However, at an operational level, the concept of resilience is not well understood. This confusion undermines the coherence and effectiveness of efforts to build resilient education systems.
At both the global and country levels, EEPCT support for the Education Cluster system is enhancing coordination and coherence as key elements of resilient education service delivery in emergency and post-crisis-transition contexts. Sub clusters, round tables and other alternative coordination platforms are enhancing system resilience across these contexts. In nine EEPCT countries reviewed, Accelerated Learning Programmes (ALPs) have enabled over-age children to re-enter or complete their education. This was found to be an effective and impactful resilience building practice that could be taken to scale. However, the evaluation findings raise concerns about the relevance and sustainability of the Learning Along Borders (LAB4LAB) programmes in West Africa. These programmes aim to provide access to education for all children, including refugees and the displaced, in areas bordering the civil war-ravaged countries of Côte d’Ivoire, Liberia, and Sierra Leone. The concerns identified include the cost of maintenance and upkeep, programmatic support, UNICEF’s long-term engagement and expectations of into programming at the global, national and local levels. However, some DRR initiatives, which have effectively established capacity at the central level of governments, were not widely understood or owned at school and community levels.
However, the evaluation findings raise concerns about the relevance and sustainability of the Learning Along Borders (LAB4LAB) programmes in West Africa. These programmes aim to provide access to education for all children, including refugees and the displaced, in areas bordering the civil war-ravaged countries of Côte d’Ivoire, Liberia, and Sierra Leone. The concerns identified include the cost of maintenance and upkeep, programmatic support, UNICEF’s long-term engagement and expectations of the communities’ capacity to support the schools long-term. One school has opened in Liberia and five remain under construction in Côte d’Ivoire but they are not connected as a cross-border regional programme as intended.
Goal Three is to increase education-sector contribution to better prediction, prevention and preparedness for emergencies caused by natural disaster and conflict. This goal supports countries in the fulfilment of the Hyogo Framework. Traditional examples of Disaster Risk Reduction often focus on natural disasters. EEPCT is forward thinking in including post-conflict countries that remain vulnerable to a recurrence of violence years after the cessation of the conflict.
Progress towards Goal Three objectives is being achieved in the majority of 29 EEPCT countries reviewed. A number of promising policy initiatives and good-practice examples are also emerging; however, the results of these endeavours are not being tracked. Steps to improve school and child safety were also identified in a number of case-study countries as making an important contribution to risk reduction. However, case-study country assessments of school construction projects found that high percentages of child-safety elements are absent.
Goal Four relates to evidence-based policies, efficient strategies and fit-for-purpose financing. Realization of Goal Four can help countries emerge from the emergency or post-crisis transition phase and start on a path of long-term development through a combination of evidence-based policies, systems development, research and analysis and fit-for-purpose financing modalities.
Country case studies found inconsistent implementation of good programme practices (situational assessments, monitoring and evaluation and programme-learning feedback loops). While Education Monitoring information Systems (EMISs) exist, data collected is still of poor quality. Limited progress was also noted in regards to innovative and fit-for-purpose financing instruments, with only a few examples (the Liberia Pooled Fund, Zimbabwe Education Transition Fund) identified by this evaluation.
The Liberia Pooled Fund emerged as the strongest example of a “fit-for-purpose financing mechanism”. While there are reservations about the efficiency of the management of this fund, and relevance of communication concerning its performance, the establishment of the fund represents a significant step forward in experimenting with a novel approach to addressing a problem that has produced many ideas and documents, but few practical initiatives, over the past five years.
OECD-DAC Aggregate Review
Four OECD-DAC criteria (relevance/appropriateness, effectiveness, efficiency and coherence/ coordination), supplemented by two additional criteria (impact and sustainability), were employed in the evaluation.
Relevance/Appropriateness: The distribution of EEPCT funding suggests that UNICEF targeted relevant contexts. The countries receiving the largest share of funds were those in Transition, Deteriorating or Chronic Crisis. These countries suffer from shortage of funds in the gap between humanitarian intervention and development, and EEPCT funding helps fill that gap. In addition, EEPCT funds supported work in the neediest countries and addressed issues that largely reflect local needs.
Effectiveness: In many cases it was only possible to observe activities and review programme outputs, since, with many country-level interventions, it is too early to assess their effectiveness. Nonetheless, as the findings for Goal One indicate, EEPCT funds have enabled UNICEF to work in a more flexible, timely and responsive manner with partners and governments to promote a more coordinated, higher-quality education response. In addition, there is some evidence of effective contributions to resilience emerging in Goal Two, at least with respect to the Accelerated Learning Programme, the most frequently supported activity. The successful continuation of Disaster Risk Reduction (DRR) activities under Goal Three also points to significant potential for effectiveness, which can be realized if local-level ownership and engagement is ensured through continued work and follow-up.
Efficiency: Many of the interventions reviewed in this evaluation have yet to produce significant outputs, and/or what outputs have been delivered have not been consistently captured by the reporting system. Nonetheless, it appears that the greatest challenge to the efficient use of EEPCT resources was the substantial disbursement lag in the first two years of implementation. The recent administrative measures, such as improved communication and reallocation of unspent funds, have had a very positive impact on expenditure rates.
Coherence and Coordination: Programme coherence was limited by inadequate internal communication about EEPCT objectives and strategic intent, particularly in the first two years of implementation. The findings also identify significant progress in coordination through the Cluster system, which is enabling greater coherence, effectiveness and efficiency between operational partners and government authorities at the country level.
Sustainability: The strength of the EEPCT programme is that the transition from needs-driven response to strategy-driven programmes is inherent, thereby enabling UNICEF to play a more active role in this transition phase. Within this context, UNICEF has undertaken significant efforts to integrate sustainability into programming at the global, national and local levels. However, some DRR initiatives, which have effectively established capacity at the central level of governments, were not widely understood or owned at school and community levels.
As a fund, EEPCT has contributed to UNICEF’s work in emergency education, and to notable achievements at global, regional and country levels. EEPCT has therefore helped UNICEF establish a niche for itself as a leading partner in the field of supporting education in emergencies and a significant player in post-crisis transition. As a programme, however, EEPCT’s impact has been limited by a lack of clarity regarding its identity, purpose and goals.
Full report in PDF
PDF files require Acrobat Reader.