2009 Thailand: CHILDREN AND THE 2004 INDIAN OCEAN TSUNAMI - Evaluation of UNICEF's Response in Thailand (2005-2008)
“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”
The Indian Ocean Tsunami of December 2004 caused widespread devastation along much of Thailand‘s 400-kilometre southern coastline, directly affecting 407 villages and completely destroying 47 of them. About 1.9 million people, including 600,000 children, were affected in six southern provinces—Satun, Trang, Krabi, Phuket, Phang Nga and Ranong. In January 2005, the UN Inter-Agency Flash Appeal for the tsunami response requested support for Thailand‘s disaster response and rehabilitation operations for a 6 month period. As part of this effort, UNICEF Thailand appealed for USD $4.8 million to cover short-term priorities. However, with growing awareness of the needs in tsunami-affected provinces and the potential to "build back better" appeals were increased and the Global Appeal for Tsunami Response eventually allocated the Thailand country office USD $21.2 million. There is very little systematic documentation of the Thailand post-tsunami experience. December 2008 marked the four year anniversary of the tsunami, and a natural milestone to reflect upon strategy and achievements during this period. This includes reflecting on the effectiveness of UNICEF‘s immediate response to the tsunami and the process of transitioning toward "mainstreamed" programming work within three core areas: child protection, education and capacity building.
The evaluation was structured according to the following objectives:
1. Determine the extent to which UNICEF‘s overall response to tsunami was adequate and relevant considering UNICEF‘s Core Commitments to Children and comparative advantage in Thailand.
2. Examine the overall achievement and effectiveness, efficiency, coverage/impact, sustainability, and replicability of UNICEF‘s response (including advocacy, communication, and partnership/social mobilization aspects) with main focus on education, child protection and local/district capacity development components.
3. Provide recommendations for further strengthening UNICEF‘s on-going Programme of Cooperation in Thailand to further achieve the Millennium Development Goals (MDGs) and child rights in Thailand focusing particularly on tsunami affected and similar disadvantaged and vulnerable areas and population groups.
4. Contribute to the building of organizational knowledge and learning linking emergency response to the long-term recovery and development by identifying lessons learned and good practices during the tsunami response, especially focusing on recovery/transition programming.
The evaluation will be based mainly on secondary data sources but there will be need for some primary data collection which will be determined and planned as part of the scoping/evaluability mission. The evaluation will be based on the following methods: Document review (both in-country, on web, and on UNICEF intranet): There is considerable literature on tsunami in Thailand and this includes UNICEF‘s periodic review reports and many studies. In addition, the latest MICS survey provides province-level data which can be an important source for the evaluation. Scoping/evaluability mission: The evaluation is planned in two phases. The first phase will include a scoping mission and evaluability assessment which will include detailed planning for the evaluation and field data collection methodology, instrument development. Primary data collection: Primary data collection will be organised with the help of a national team who will gather information (qualitative and quantitative) through key informant interviews, focus group discussions and through field observation and rapid surveys (if necessary). There might be a need to engage a national institution for conducting survey but this will be known only after the scoping mission. Field-level data collection and any surveys will also entail proper use of sampling and statistical methods. Data analysis using qualitative and quantitative methods: Given the qualitative nature of much of the investigation, findings should be triangulated from a wide range of sources. Validation of findings will take place during the field presentation, followed by comment and feedback on the draft reports.
Relevanc:e UNICEF actions during the post-tsunami period were generally consistent in showing commitment to determining evidence of need and strategic purpose. Most were preceded by situational assessment and defined within the Country Plan. Engagement with children in shaping strategic agendas and programmes was not prominent, suggesting greater attention could be paid to identifying the expressed priorities of children. Effectiveness: The Core Commitments for Children were clearly met and mobilized in an effective transition to recovery response. Subsequent programmatic activity focused on infrastructure, training and systems development. Infrastructure investments established considerably improved school sanitation facilities but raised questions of cost-effectiveness with respect to maintenance and replicability. Training activities were well executed and received based on their generally strong contents, but were weakened through use of cascade methods. Models adopted within a systems development framework were potentially effective mechanisms to address identified needs; however, there was as yet limited evidence of their institutionalization. Impact: There was as yet no strong evidence of impact due to the relatively recent introduction of programmes, the time required to institutionalize systemic change and the absence of routine data collection. Proxy indicators, however, indicated progress toward impacts: parents valued the CFS initiative, reporting positive change in children‘s capacity; tambon officials noted increased awareness of child protection issues; and case managers appreciated the effectiveness of monitoring and response systems. Taken together, such data suggest progress is being made toward targeted changes in children‘s well-being. Specification of indicators and systematic data collection will be needed to provide confirmation of their being achieved. Coverage: Appropriately, UNICEF and partners focused on areas directly affected by the tsunami in the emergency phase. Equally appropriate was the policy decision to designate all communities within the six provinces as ‗tsunami-affected‘ as a means of broadening coverage to the entire population and reinforce a ‗systems development‘ strategy toward scaling-up interventions. In practice, however, ‗pilot‘ and ‗staged‘ approaches to programming have resulted in partial coverage, though province-wide adoption of the CPMS system in Ranong indicates how successful demonstration can encourage wider adoption. Efficiency: Spending was greatest across all sectors in 2007, suggesting the length of time it took to fully establish programme strategies and modalities. The basis of inter-sectoral prioritization of expenditure was somewhat unclear. Funding to education was the largest across the sectors, with potential absorptive capacity seemingly a key consideration. Increased allocation to work in child protection and local capacity development as innovations in approach and targets, however, would have supported more robust institutionalized change at the local level and thus more efficiently established a proven systems model for national adoption.
Sustainability: There is reasonable evidence to suggest achieved gains will prove sustainable. UNICEF strategies have connected with appropriate ministries and national institutions; explicitly engaged provincial and sub-district levels; and have been generally consistent with governmental policy priorities. Momentum has also been achieved through commitment and reliability of key individuals within partner agencies and units. Strengthening the cadre of such leaders will be key to supporting sustainable change in policy, strategy and services.
For Government and Partners:
a) Establish a fixed annual schedule for disaster preparedness training. This should be planned and coordinated in line with established cluster roles and Department of Disaster Preparedness and Mitigation priorities, and include codes of conduct governing treatment of and access to children in both the general and sector-based context of emergencies. b) Select a sub-group of schools for particular and comprehensive action on issues of inclusion, focusing of children at risk from marginalization both within the school and in being excluded from it. It should build, where possible, from the base already established by the ECD programme. c) Strengthen review processes regarding local planning for children and youth. The MoI, with support from UNICEF, should establish a formal annual review of tambon plans and approved expenditure for children‘s and youth activities, and provincial office audits of submitted Children and Youth Plans.
For the Thailand Country Office:
a) Conduct a review of the model of the child protection system to be promoted for national roll-out, involving both stakeholder feedback and an organizational systems perspective that incorporates an understanding of administrative and managerial processes and HR issues of recruitment and retention. b) Establish operations research capacity in a small number of tambons. The focus should be on establishing ways for enabling greater synergy between child protection, and local capacity building and education; and identifying how existing systems and processes at sub-district level can be optimally augmented and revised to support planned activities. c) Promote the value of, and methods for, engagement with children and youth as a more explicit element of UNICEF planning processes and the planning processes of government and other partners. This would build on some of the best practice demonstrated in the context of commissioned work on placement conditions of separated and migrant children and in BCC projects. d) Confirm directives and develop skills for conducting RALS in emergency situations, with a particular focus on differentially affected children. This should include analyses of the overall situation for children and availability of safe and learning-oriented spaces, give attention to children who may be specifically at risk, and generate baseline measures against which progress on the relief-recovery-development transition can be assessed. e) Consider means to strengthen influence in the south if the intention is to do more than advocate CFS. This recognizes that mentoring at school and ESAO level will be needed until coherent action is established and that closure of the Phuket office may require alternative mechanisms for active, sustained engagement. f) For all work targeting "systems development", more clearly identify necessary mechanisms of monitoring and technical support. Where these are not within current government capacity, set in place clear arrangements need for contracting such support.
g) Continue and extend efforts to strategically identify partners outside of government (particularly with regard to the strategic goal of securing more inclusive education) as a means of enabling UNICEF‘s more active reach to excluded children and support to development of more effective approaches for eventual system uptake. h) Adopt a more strategic approach to putting issues of gender on the programming agenda, including tracking and extending development of existing initiatives and their adaptation within the policies and programmes of other ministries. i) Develop a more proactive communications strategy for field-partner. For example, through briefing papers shared on a regular basis updating partners on UNICEF‘s work and evolving commitments. j) Establish clearer mechanisms for integration across sectoral teams within the Country Office by establishing an internal cross-sectoral working group to consider common challenges and potential strategies for establishing new or revised systems in public service provision.
For UNICEF Headquarters and Partners:
a) Share learning on systems development by commissioning distillations of best practice in systems innovation and scale-up across middle-income settings to inform country programming strategy in such work. b) Recognize the inherent responsibility and capacity of national governments, systems and communities to manage their own relief and recovery process. This implies that UNICEF‘s responsibility for the CCC focus on ensuring that its own structures and systems facilitate effective and efficient local action and avoid undermining, contradicting or duplicating effective and efficient local action. c) Plan a more complete emergency response to include tailored rapid assessments and strategic interventions, as well as a context specific development-oriented strategic analysis and action plans. Such a response should be congruent with, and make use of, the technical resources and experience of UNICEF‘s broader country strategy, but evolve separately from these on the basis of locally relevant and appropriate action and local participation. d) Avoid the temptation of efficiency over effectiveness in promoting CFS, focusing on a capacity development, participatory and whole-school approach, tailored to and with the children concerned rather than the creation of a sturdy, standardized and easily delivered ―packaged.
Full report in PDF
PDF files require Acrobat Reader.