Base de données d'évaluation

Evaluation report

2009 Global: Children and the 2004 Indian Ocean Tsunami - Evaluation of UNICEF's Response in Indonesia, Sri Lanka and Maldives (2005-2008) Overall Synthesis Report



Executive summary

Background

This report is a summary of an extensive evaluation undertaken by the United Nations Children’s Fund (UNICEF) of its response to the 2004 Indian Ocean tsunami. It focuses on Indonesia, Sri Lanka and Maldives—countries that had the most serious damage and received 84 per cent of UNICEF tsunami country-level funds. The evaluation mainly examines the recovery and early development phases and assesses outcomes and impacts of the response from 2005 to 2008 in the four major sectors of UNICEF involvement: child protection; basic education; water, sanitation and hygiene (WASH); and child and maternal health and nutrition. In addition, the evaluation provides lessons for each sector and for crosscutting issues related to recovery programming.

Detailed evaluation findings, lessons and recommendations can be found in the following evaluation products: reports of each of the four sectors for each country (12 sector reports); three country synthesis reports; and this overall synthesis report. The overall synthesis draws upon the sector and country synthesis reports to present a broad set of findings, lessons and recommendations to strengthen ongoing programmes and future efforts in disaster preparedness planning and humanitarian and recovery response.

Methodology

The evaluation was conducted by an international team of consultants who were supported by national teams. In each country, the focus was on changes and trends occurring in the conditions of children and women in tsunami-affected areas compared to the pre-tsunami situation. In Indonesia and Sri Lanka, variables with respect to the impact of conflict were taken into account. In some sectors and countries, data trends were readily available. However, where reconstruction was still underway, only predictive outcomes could be discerned.

Data collection methods included: desk reviews of pre- and post-tsunami documents and records from UNICEF and governments, field visits for observations, key informant interviews, and brief household and facility surveys. The evaluation confirmed findings in validation workshops in each country.

Findings

A challenge for the evaluation has been to set a suitable baseline from which to measure progress over the five-year period. Particularly in Aceh, pre-2005 data was sparse, and in Sri Lanka and Maldives, coastal and island data pertaining to those areas hit by the tsunami was also incomplete. The benchmarks for adequate response in an emergency of this kind are those set out in international guidelines and standards (for example, the Sphere standards), but the recovery-development phase is often new territory, not easily measured against predictive targets. The mantra ‘build back better’ anticipated that the well-funded international response would accelerate and improve upon nascent development plans in many areas, as was the case in some sectors.

Child protection: In Indonesia and Sri Lanka, the emergency response was an opportunity to build upon rudimentary systems of child protection, providing the impetus to develop or reactivate policies and approaches. Human capacity and budgetary resources improved, putting child protection issues on the national policy agenda and significant systemic development in Indonesia and Sri Lanka. The impact of these initiatives remains to be determined given their nascent development. In the Maldives, child protection systems remain particularly weak. The value of non-governmental organization services in child protection should not be underestimated.

Education: The Core Commitments to Children in Emergencies were largely met in the post-tsunami emergency phase as children were brought very quickly into safe learning spaces with teachers sensitized to dealing with trauma. The recovery focused on large-scale construction, especially in Aceh and Sri Lanka, and the majority of children in all three countries returned to better-built permanent schools within the first year. Innovation in teacher outreach training was introduced in the Maldives. In all three cases, enrolment rates have recovered, and in Aceh they have improved beyond pre-tsunami levels. All three countries extended the child-friendly school concept by introducing child-centred physical structures, school management and teaching practices. Conflict mattered: the peace agreement in Aceh had a significant influence on all aspects of education recovery, especially for girls; and persistence of war in Sri Lanka appreciably delayed recovery for large numbers of children. The peace agreement in Aceh provided a conducive climate for improvements in the education sector and UNICEF was able to capitalize on this. There were gaps: greater attention to pockets of exclusion and community involvement in schools remain key to ensuring fully effective building back, as do strategies for consolidating and institutionalizing piloted child-friendly innovations.

Health and nutrition: Improvements in the sector included greater access to health services through new facilities and the encouragement of governments to expand their policies with respect to wide agerange immunization campaigns. UNICEF was a relatively small contributor to the overall health and nutrition response in all three countries, but its contributions to these issues resulted in the prevention of disease outbreaks and some improvements in district- and island-level capacity development. However, the tsunami response was a missed opportunity to more effectively address the underlying, pre-existing causes of malnutrition and maternal mortality in all countries primarily related to child and maternal health.

WASH: Across all three countries, the UNICEF emergency response in the WASH sector provided access to safe water and sanitation for the resettlement of internally displaced children and their families.  In the recovery phase, it contributed towards the restoration of water and sanitation facilities for tsunamiaffected families and improvements in water supply, particularly rural water supply. Some improvements can be seen in water security, basic sanitation and hygiene. However, efforts to build back better may have been over-ambitious, introducing concerns about maintenance and sustainability in new and sophisticated facilities.

Recommendations

In addition to its sectoral focus, this report provides an analysis of recovery and transition related issues and draws lessons. The following are general lessons and recommendations distilled from the overall evaluation findings and conclusions.

1) One of the key constraints for UNICEF was its focus on capital-intensive infrastructure rather than building institutional capacity. Capital-intensive projects, such as construction and the purchase of significant assets entails high transaction costs for materials as well as staff and can distract from wider development concerns. This was particularly the case with school building in Aceh and WASH hardware installation in the Maldives. The pressure to spend large sums of public money raised for the tsunami would inevitably lead towards choosing these projects over others.
Recommendation: Management of large-scale construction programmes should not be undertaken by UNICEF—this should be outsourced in its entirety. UNICEF country staff should not be responsible for day-to-day management of contracts, procurement, etc. UNICEF’s comparative advantage in policy development, and in working with governments to improve public finance management with respect to key sectors, would then become the focus, matched by appropriate incountry capacities.

2) The move from emergency to recovery and development requires different sets of skills, and UNICEF can do better in managing human resources in this respect. For example, strong contextual analysis (political economy, institutional analysis, etc.) can significantly improve the relevance, effectiveness and sustainability of interventions. Likewise, investments in planning and preparedness pay dividends economically, socially and in terms of speed of recovery. In the health sector, for instance, a clear lesson emerged particularly from Sri Lanka and the Maldives that the tsunami response had little impact on underlying chronic trends in nutrition and maternal mortality. However, incremental changes can be induced through judicious use of time-bound funds.
Recommendation: Senior staff continuity over the transition period should be assured. Effective exit strategies for emergency programmes are unlikely to occur unless more developmentoriented staff are in place at an early stage of the recovery.

3) Building communication channels and dialogue with civil society (including community groups) can often fill a gap where decentralization, for instance, is still relatively new. UNICEF has not always matched ‘encouragement’ of these mechanisms with actual funds. For example, promoting child-friendly school concepts through school and parent committees requires predictable and sustained funds to build these bodies into sustainable entities. Likewise, in child protection, there needs to be a systematic process for consultation with local communities to ensure long-term viability and, above all, ownership.
Recommendation: Technical assistance offered to government ministries—and by extension to subnational and local authorities—should include means whereby a public communications strategy and community dialogue could increase demand and community ownership of any new services being offered. This should be complemented by assistance and resources offered to viable community groups. UNICEF should thus encourage community linkages with schools, children’s centres, disaster preparedness planning processes and risk reduction exercises.

4) UNICEF-supported programming has in several instances jump-started favourable government policy. However, the capacity of national and local institutions has held back progress in policy development and implementation. For instance, in child protection, UNICEF’s early linkage of the dual objectives of responding to immediate needs, while helping build the welfare and legal systems for children has had positive and lasting results, notably in Indonesia. However, weaknesses continue to be apparent at subnational levels.
Recommendation: A capacity needs assessment should be undertaken as soon as the acute emergency phase is over. In particular, this should include analysis of capacities at provincial and district levels, cross-referenced with an understanding of work being undertaken by other agencies in relevant sectors. This is important not only for intervention strategy but also for sequencing of priorities in UNICEF’s capacity work.

5) The significant investment made in systems and equipment needs to be matched with an analysis of how best to maintain these in the long run. Considerable concerns about the sustainability of infrastructure funded by UNICEF remain. The full and appropriate use of quality infrastructure in the education and WASH sectors was sometimes undermined by poor planning with respect to maintenance agreements.
Recommendation: Investments in assets and infrastructure should be accompanied by longerterm capacity development and a clear outline of responsibilities towards maintenance. Greater attention should be paid to how recurring costs, in terms of management and maintenance, are to be met through either government or community budgets.

6) Evidence-based data collection systems have been extremely useful for planning, implementation, and monitoring and evaluation. UNICEF support to the DevInfo systems has been exemplary. Continued support is critical to ensure that information management and data collection systems are used to strengthen monitoring and evaluation, and to inform decision-making and strategic planning.
Recommendation: Continue to support evidence-based systems to inform planning, implementation, and monitoring and evaluation. Greater investment is needed in information management and data collection systems. Gender disaggregated data is one area of uppermost importance. Partners should be trained in the usage, search and analysis of such data and taught how monitoring and evaluation informs decision-making and strategic planning.

7) Emergency response and assessment tools tended to be too generic, sometimes missing the inclusion of the most vulnerable children or population groups. Country and area-specific methods for identifying the most vulnerable children, including strategies for reaching them and incorporating them into programming, were not developed fully. Although domestic and local safety nets will always be of paramount importance, adherence to human rights principles and international standards will only be assured through institutionalizing participatory methods of recognizing and measuring inclusion and exclusion.
Recommendation: The collection and analysis of disaggregated data on vulnerability—both qualitative and quantitative—should be promoted. This is called for in the emergency preparedness and response planning that is currently evolving and is an explicit requirement of a human rights-based approach to programming.

8) The sometimes greater efficiency and sustainability of national NGO and community-based organizations in comparison to international NGOs requires more than ad hoc use of these institutions. UNICEF has demonstrated how community linkages can be strengthened by engaging communities and non-governmental and community-based organizations in schools, children’s centres, disaster preparedness planning and risk reduction exercises. In Sri Lanka in particular, women’s grassroots organizations were identified as potentially useful partners.
Recommendation: UNICEF should positively discriminate in favour of grassroots and advocacy organizations as implementers and provide enabling and appropriate capacity assistance. This would counter inherent cultural and gender bias in the selection of partners. Resource commitments should, however, recognize that capacity provision, as well as capacity development, is appropriate in some cases to get programmes underway.

9) The development of inter-agency sectoral guidelines in emergencies has advanced considerably in the years since the tsunami, and UNICEF has contributed to this. However, there are some sectors—notably WASH facilities within new housing schemes and childfriendly schools—where the preparation and dissemination of such guidelines in advance of construction work would ensure adherence to standards. These could be included in preparedness plans and incorporated into existing long-term agreements.
Recommendation: Support the early development of guidelines appropriate to the hardware provided in emergencies, especially where expensive equipment is linked to other sectors.

10) Organisation for Economic Co-operation and Development - Development Assistance Committee (OECD-DAC) guidelines1 encourage the development of indicators appropriate to assessing the extent to which recovery and development programmes are ‘conflict aware’ and how measuring outcomes in this respect might be attained. The evaluation found, for instance, that there was no ‘do no harm’ analysis apparent in the UNICEF analysis in Sri Lanka and Indonesia, despite the fact that the effects of conflict pervaded all programme areas.
Recommendation: UNICEF should develop in-house capacity to assess and incorporate a peace-building (including a ‘do no harm’) perspective into all its existing planning vehicles in conflict countries.

11) Overarching recommendation: UNICEF should strengthen and revise its post-emergency recovery and transition strategy and guidelines to better address both strategic planning (including information needs assessment in various phases, targeting and capacity development) and management (human resource and operations) aspects. The lessons and detailed recommendations provided in this report offer considerable inputs for use in formulating revised guidance.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Recherche