Evaluation database

Evaluation report

2009 Thailand: CHILDREN AND THE 2004 INDIAN OCEAN TSUNAMI: Evaluation of UNICEF’s Response in Thailand (2005-2008)



Author: Alastair Ager

Executive summary

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”

Background:
There is very little systematic documentation of the Thailand post-tsunami experience. December 2008 marked the four year anniversary of the Indian Ocean tsunami and a natural milestone to reflect upon strategy and achievements during this period. This includes reflecting on the effectiveness of UNICEF’s immediate response to the tsunami and the process of transitioning toward ‘mainstreamed’ programming work within three core areas: child protection, education and capacity building. It also involves consideration of progress in the chosen programming areas and the extent to which, with hindsight, such progress represents a valuable investment – in terms of programming outcomes and policy development - in promoting child well-being in Thailand.
Such reflection is clearly of relevance to the development of UNICEF’s country strategy for Thailand (for example, in advance of the scheduled mid-term review of 2009). But it also potentially informs wider consideration of the role of UNICEF in middle-income country settings, where natural disaster may provide opportunities for differing forms of engagement than possible during a period of regular (non-emergency) cooperation.

Purpose/Objective:
The evaluation was structured according to the following objectives:
1. Determine the extent to which UNICEF’s overall response to tsunami was adequate and relevant considering UNICEF’s Core Commitments to Children and comparative advantage in Thailand.
2. Examine the overall achievement and effectiveness, efficiency, coverage/impact, sustainability, and replicability of UNICEF’s response (including advocacy, communication, and partnership/social mobilization aspects) with main focus on education, child protection and local/district capacity development components.
3. Provide recommendations for further strengthening UNICEF’s on-going Programme of Cooperation in Thailand to further achieve the Millennium Development Goals (MDGs) and child rights in Thailand focusing particularly on tsunami affected and similar disadvantaged and vulnerable areas and population groups.
4. Contribute to the building of organizational knowledge and learning linking emergency response to the long-term recovery and development by identifying lessons learned and good practices during the tsunami response, especially focusing on recovery/transition programming.

Methodology:
The evaluation utilized both secondary analysis in the form of document review and primary data collection. Primary data collection was used to investigate the perspective of key beneficiaries of programmes in the education, child protection and local capacity building sectors since the relief period. Fieldwork was conducted in three of the six tsunami affected provinces: Ranong, Phang Nga and Krabi. The provinces were selected to be broadly representative of the affected area, although each of the provinces faced distinct challenges. Phang Nga and Krabi were two of the provinces most directly and immediately impacted by the tsunami, and were selected for inclusion on that basis. Ranong, while not as severely affected by the physical impact of the tsunami, contains a high number of migrant children and was the major site for the pilot provincial model for child protection monitoring and response.
Focus group discussions, group interviews and one-on-one interviews were the qualitative methods used in the fieldwork in the south, as described in more detail at the beginning of the appropriate section in this report. Each sector was included in the fieldwork for each province, with stakeholders drawn from the provincial, district, sub-district and community level. In total, between August 2008 and January 2009, 19 focus group discussions, 17 group interviews and 4 individual interviews were conducted across 40 tambons.

Findings and Conclusions:
Detailed findings and conclusions can be found in the body of the report. To highlight a few:
- - UNICEF’s approach to child protection systems strengthening is appropriate and coherent.
- The CPMS potentially provides a sound, rigorous, effective basis for monitoring child protection concerns.
- Case managers represent a key resource for child protection recognized by all stakeholders.
- There are opportunities to link the three child databases (school, TAO (CPMS) and OSCC case records).
- Working through a CFS framework to re-establish schools in the tsunami-affected provinces was the right thing to do. Progress on building a more permanent child-friendly education “system” in the south, however, is proving a rather more tenuous task to get right and it is unclear if schools and communities are fully aware of what CFS means or what actions they can take to make it happen.
- Strategies for engaging teachers in the CFS process are limited by a cascade model that typically fails both to complete the learning cycle and to make sufficient use of CFS as a “whole school” approach to change.
- Support to excluded children through the EDC project is a significant step to moving forward UNICEF commitment to reaching the most excluded; so, too, are the workshops to train ESAO and schools to account for missing children. However, without a widening and targeting of action to find and support these children on a broader scale and without consistent follow-up of the training, there is little to suggest significant progress on the BBB focus on “ vulnerable children and women” in the south.

Recommendations:
For Government and Partners:
a) Establish a fixed annual schedule for disaster preparedness training. This should be planned and coordinated in line with established cluster roles and Department of Disaster Preparedness and Mitigation priorities, and include codes of conduct governing treatment of and access to children in both the general and sector-based context of emergencies.
b) Select a sub-group of schools for particular and comprehensive action on issues of inclusion, focusing of children at risk from marginalization both within the school and in being excluded from it. It should build, where possible, from the base already established by the ECD programme.
c) Strengthen review processes regarding local planning for children and youth. The MoI, with support from UNICEF, should establish a formal annual review of tambon plans and approved expenditure for children’s and youth activities, and provincial office audits of submitted Children and Youth Plans.
For the Thailand Country Office:
a) Conduct a review of the model of the child protection system to be promoted for national roll-out, involving both stakeholder feedback and an organizational systems perspective that incorporates an understanding of administrative and managerial processes and HR issues of recruitment and retention.
b) Establish operations research capacity in a small number of tambons. The focus should be on establishing ways for enabling greater synergy between child protection, and local capacity building and education; and identifying how existing systems and processes at sub-district level can be optimally augmented and revised to support planned activities.
c) Promote the value of, and methods for, engagement with children and youth as a more explicit element of UNICEF planning processes and the planning processes of government and other partners. This would build on some of the best practice demonstrated in the context of commissioned work on placement conditions of separated and migrant children and in BCC projects.
d) Confirm directives and develop skills for conducting RALS in emergency situations, with a particular focus on differentially affected children. This should include analyses of the overall situation for children and availability of safe and learning-oriented spaces, give attention to children who may be specifically at risk, and generate baseline measures against which progress on the relief-recovery-development transition can be assessed.
e) Consider means to strengthen influence in the south if the intention is to do more than advocate CFS. This recognizes that mentoring at school and ESAO level will be needed until coherent action is established and that closure of the Phuket office may require alternative mechanisms for active, sustained engagement.
f) For all work targeting ‘systems development’, more clearly identify necessary mechanisms of monitoring and technical support. Where these are not within current government capacity, set in place clear arrangements need for contracting such support.
g) Continue and extend efforts to strategically identify partners outside of government (particularly with regard to the strategic goal of securing more inclusive education) as a means of enabling UNICEF’s more active reach to excluded children and support to development of more effective approaches for eventual system uptake.
h) Adopt a more strategic approach to putting issues of gender on the programming agenda, including tracking and extending development of existing initiatives and their adaptation within the policies and programmes of other ministries.
i) Develop a more proactive communications strategy for field-partner. For example, through briefing papers shared on a regular basis updating partners on UNICEF’s work and evolving commitments.
j) Establish clearer mechanisms for integration across sectoral teams within the Country Office by establishing an internal cross-sectoral working group to consider common challenges and potential strategies for establishing new or revised systems in public service provision.
For UNICEF Headquarters and Partners:
a) Share learning on systems development by commissioning distillations of best practice in systems innovation and scale-up across middle-income settings to inform country programming strategy in such work.
b) Recognize the inherent responsibility and capacity of national governments, systems and communities to manage their own relief and recovery process. This implies that UNICEF’s responsibility for the CCC focus on ensuring that its own structures and systems facilitate effective and efficient local action and avoid undermining, contradicting or duplicating effective and efficient local action.
c) Plan a more complete emergency response to include tailored rapid assessments and strategic interventions, as well as a context specific development-oriented strategic analysis and action plans. Such a response should be congruent with, and make use of, the technical resources and experience of UNICEF’s broader country strategy, but evolve separately from these on the basis of locally relevant and appropriate action and local participation.
d) Avoid the temptation of efficiency over effectiveness in promoting CFS, focusing on a capacity development, participatory and whole-school approach, tailored to and with the children concerned rather than the creation of a sturdy, standardized and easily delivered “package”.

Lessons Learned (Optional):
Specific lessons can be found in the report. To highlight a few:
- The CCCs are a relevant framework in middle-income settings, although governments may play a strong role in enabling their application.
- Efforts to implement the cluster approach in emergencies should avoid some of the coordination problems that were experienced in the context of the tsunami response.
- Disaster preparedness training is a valuable and necessarily on-going investment.
- Codes of conduct should be in place during emergencies to govern access to children in emergencies.
- The overall strength of government response may mask important weaknesses with respect to particularly vulnerable groups.
- The potential viability of a Child Protection Monitoring and Response System has been established but the current model requires review.
- The goals of CFS-based education will be at risk without stronger monitoring and evaluation at an organizational level in UNICEF.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search