Author: Jon Bennett (Team Leader) and Jenny Reid Austin (Co-author)
“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”
UNICEF commissioned an evaluation in 2008 to assess its humanitarian response to the three countries hardest hit by the 2004 Indian Ocean tsunami: Indonesia, Sri Lanka and the Maldives. The evaluation was conducted between August 2008 and July 2009. The focus is primarily on the tsunami recovery and early development responses, and on UNICEF’s interventions in four major sectors of the agency’s programme involvement within the country, namely water, sanitation and hygiene; basic education; child protection; and child and maternal health and nutrition. These sectors financially accounted for the majority of UNICEF’s tsunami response (89 percent) in the Maldives. In examining sector issues, the evaluation has taken a broad-based approach often focusing on national and sub-national data and drawing conclusions that can be relevant for the sector as a while. The evaluation also addressed findings in relation to cross-cutting issues through the recovery to development transition, including: national and local capacity development; partnerships; disaster preparedness; targeting the disadvantaged; human rights-based approach to programming; and gender issues.
The purpose of the evaluation in the Maldives has been to determine outcomes and impacts of UNICEF’s response to the tsunami, and draw lessons and recommendations—both for the UNICEF and the sectors as a whole—that will be useful for strengthening ongoing programmes or policies to improve the wellbeing and rights of children. In addition, the evaluation draws lessons for recovery/transition programming that will be useful in informing future responses to disasters of this nature.
A thorough literature review, including data not always in the public realm (e.g., country-level NGO reports and academic studies), comparing and contrasting approaches undertaken in the recovery phases was carried out. Extensive interviews with senior and technical government ministry staff was done to determine overall progress within each sector and to assess the relative contribution UNICEF has made to developments in the country over a 4-5 year period. Additional methods included: interviews with previous and current UNICEF programme staff to nuance existing documented lessons; field surveys (per sector, though in some cases combining sectoral questionnaires): teams were responsible for conducting primary, field-based data gathering that included focus group discussions, questionnaires and transect walk methods. Field survey teams were also responsible for collating the data.
Each sector evaluation also looked at a number of important cross-sectoral issues. In each sector report reference is made to UNICEF Core Commitments for Children in Emergencies, the extent to which UNICEF took part in inter-agency needs assessments and/or other surveys, and how it reported on the general situation of children and women. Likewise, for the recovery and early development phases, each evaluation refers to UNICEF’s MTSP and issues related to human rights-based approach to programming, gender mainstreaming and national/local capacity development. Finally, each sector evaluation examines the extent to which UNICEF has contributed to disaster preparedness and risk mitigation efforts.
Findings and Conclusions:
This is very much a ‘work in progress’. In returning to a regular—and much reduced—programme after almost five years of unusually high financial inputs, a key question is the extent to which UNICEF’s strategic approach complements, enhances and influences efforts of both the Government of Maldives and of development agencies which, like UNICEF, will remain in the country.
The emergency was heavily resourced, with the Red Cross and NGOs having significant independent financial reserves and direct implementation capacity. UNICEF’s greater impact in all sectors was through investing in sector leadership—identifying gaps, addressing systemic issues and setting priorities. Yet it is apparent that the pressure from UNICEF and donors to spend money within a certain time frame affected decision making on programme strategies.
In the WASH sector, positive outcomes were achieved in terms of the population’s access to safe water, and other gains. The weakness in the early response was in-country capacity, with UNICEF struggling to meet the initial surge requirements within the context of a multi-country response. The sanitation infrastructure being built after the tsunami is radically changing from household-managed systems to community systems, though the cooperative management systems required to maintain this have been slow in developing. Most importantly, the choice to invest in capital-intensive programming was not balanced with appropriate national capacity development; policy and support frameworks were less developed, and the capacity to maintain the systems was still basic. UNICEF’s ten NGO implementers were preoccupied with complex engineering systems while the need for government and community capacity building, as well as long-term sustainability, was overlooked and relatively under-resourced.
In the health and nutrition sector, notwithstanding a lack of attention to technical capacity development, the disaster response to a large extent contributed to subsequent developments. In most areas, today’s quality of new or rehabilitated facilities (many funded by UNICEF) is excellent, and the decline in mortality among children, for instance, is a major achievement. UNICEF’s support to MaldivInfo and the development of the national Online Nutrition and Child Health Surveillance System has helped strengthen primary health care and preventive care actions.
The use of highly skilled, more clinically oriented, foreign health workers has to some extent curtailed capacity development of national staff. Meanwhile, some of the specific disaster-related objectives (such as emergency preparedness) and groups in need (such as IDPs) were less than fully addressed. In terms of nutrition, the 2008 food crisis may end up impacting the atoll population more seriously than the tsunami. There continue to be inequalities in access to food, and these are consistent with a need for strenuous health promotion to assure diet diversity and adequacy among children in remote atolls.
In child protection, there have been substantial GoM budgetary increases over the last four years, though it is difficult to gauge actual disbursements to individual sectors since there is little disaggregation of budgetary expenditure. The establishment of the NDMC has been a significant positive step. However, psychosocial emergency responses mirrored the centralised approach by the GoM: they were top-down, demonstrated a predominantly male perspective, and rarely reflected realities and perspectives outside of the capital. UNICEF’s overall protection programme was diminished by the lack of realistic planning and in-house sector expertise. Nevertheless, UNICEF’s child protection programme has maintained a close working relationship with key government agencies. Further, the “Wake Up” Drug Abuse Prevention and Advocacy Campaign has proven to be a positive new departure for UNICEF and one worth pursuing further.
In education, the inventive use of student ‘tsunami teachers’ was a success that helped to meet the needs of schools, provided unexpected support to students and parents, and helped to provide on-the-job training that would contribute to the teachers’ development and teaching abilities. Immediately following the tsunami, UNICEF focused on bringing children quickly back to secure learning spaces, and to do so in ways that laid the basis for the transition to the recovery period. UNICEF’s work with Child-Friendly Schools and Child-centred Teaching-Learning was also highly relevant. The reach of the CFS approach was good, increasing the knowledge of teachers, school administrators and parents. The challenge, however, is in attaining critical mass: while rollout of the training by EDC and UNICEF is proceeding, there are still relatively few teachers involved. Likewise, with the TRCs, relatively few teachers are receiving the breadth and depth of support needed to constitute effective professional development.
The Maldives is currently undergoing a rapid—though largely peaceful—governance transition. The evaluation coincided with a change of government and an accelerated reform process that may yet see greater levels of atoll-level ownership of development initiatives; the tsunami response has contributed to greater awareness and demand at sub-national levels. There has been an encouraging increase in national NGOs.
Strengthening the capacity of the civil service will be critical if the country is to provide public services at a level consistent with the country’s upper-middle income aspirations. Despite a gradual improvement in the education and skills of senior civil servants, the civil service still lacks policy, implementation and administrative capability. Over half of the civil service is concentrated in three ministries: education, health and defence services.
The evaluation covers only certain sectors, but it confirms the findings of the 2008 UN Human Development Report that vulnerability is outpacing the country’s ability to provide safety nets. Two particularly vulnerable groups are women and unemployed youth. The concentration of poverty in the Northern atolls and among particular population groups is of particular concern.
For UNICEF itself, strategic planning for the transition from relief through recovery to development was compromised by the transaction costs involved in protracted infrastructure work and a related lack of capacity available to the Country Office. At the same time, the Government of Maldives has been engaged mainly in succinct sector-specific thinking, limiting holistic planning. Nevertheless, the division of responsibilities under the UNDAF enabled UNICEF to pioneer the flagship MaldivInfo database, which has proved particularly valuable to the health and nutrition sector. Capacity development generally remains centralised, however.
Attributing change to any one agency would be dishonest, but UNICEF’s sizeable interventions over four years, and the encouraging manner in which it has assimilated lessons from these interventions, have given it a unique opportunity to guide and influence national and sub-national government.
1) Capacity needs assessment should be undertaken very early in the recovery phase. In particular, this should include analysis of capacities at atoll/island levels, cross-referenced with an understanding of work being undertaken by other agencies in relevant sectors. This is important not only for intervention strategy but also for sequencing of priorities in UNICEF’s capacity work.
2) The management of large-scale construction programmes in all sectors should not be undertaken by UNICEF – it should be outsourced. Capital-intensive projects, such as construction and the purchase of significant assets entails high transaction costs for materials as well as staff, and can distract from wider development concerns. Nevertheless, new facilities present unique opportunities to develop and promote quality issues, for example in education.
3) Where investments in assets and infrastructure are undertaken, these should be complemented with longer-term capacity development and a clear indication of responsibilities towards maintenance.UNICEF should ensure that longer-term, comprehensive planning for capacity building and coordination are built into the project planning to allow for greater sustainability. This should include support for maintenance, continuity and human resources development, and anticipate needs after agencies and donors exit.
4) Continuity of senior staff over the transition period should be assured. Effective exit strategies for emergency programmes are unlikely to occur unless more development-oriented staffing is in place at an early stage of the recovery.
5) UNICEF should ensure that technical assistance offered to government ministries is complemented by a public communications strategy that increases demand and community ownership of any new services being offered.
6) UNICEF should continue to support national NGO/CBOss and promote ownership and participation among stakeholders to sustain and strengthen the effectiveness of initiatives and interventions. Mobilise and leverage existing community resources and local knowledge, and provide support to stakeholders—such as families, children, youth and communities—and civil society, community-based organisations and national NGOs. The sometimes greater efficiency and sustainability of national NGOs and CBOs over INGOs has been noted. NGOs should be recognised for their important contributions. UNICEF should strengthen community linkages by engaging communities and these organisations in schools, children’s centres, disaster preparedness planning, and risk reduction exercises, among other areas.
7) UNICEF should promote the collection and analysis of disaggregated data on vulnerability—both qualitative and quantitative. Although domestic and local safety nets will always be of paramount importance, adherence to human rights principles and international standards will only be assured through institutionalising participatory methods of recognising and measuring inclusion and exclusion.
8) Prioritise developments in disaster preparedness and disaster risk reduction. Due to the severe vulnerability of the Maldives, it is critical to ensure that disaster preparedness and disaster risk reduction are among the highest priorities. This will require planning, advancing existing and future legislation, and rapidly implementing necessary provisions.
9) Through inter-agency collaboration, support human resource capacity development in education, local health care and child protection, preceded by a coordinated needs assessment. This includes enhancing the capacity of the existing civil service (policy, administrative and implementation capability), as well as developing skills in more specialist areas such as socio-legal and health training.
Recommendations per sector are included in the report.
Lessons Learned (Optional):
The following are the overarching lessons for ongoing efforts and for future responses to humanitarian emergencies:
1) Planning for emergency response through recovery and development should balance both longer-term capacity development with investments in assets and infrastructure, all of which should be reflective of local need.
2) UNICEF should protect the investments made in the relief, recovery and development phases by planning for sustainability, such as maintenance support and continuity of human resources capacity development.
3) There must be a high priority placed on disaster preparedness and disaster risk reduction in this highly vulnerable country.
4) Disaster preparedness planning should include needs development assessment tools for each sector and trainings that have been pre-tested in various contexts.
5) Through inter-agency collaboration, there is a need to support human resource capacity development in education, local health care and child protection, preceded by a coordinated needs assessment.
6) There should be support for and maintenance of evidence-based systems of data collection and management to better inform policy, strategic planning and allow for strengthened monitoring and evaluation.
7) Given challenges in implementation capacity and decision making in an environment with high staff turnover and competing priorities, UNICEF should support and develop guidelines for the provision of needed items, equipment and materials during emergencies.
8) UNICEF should support the meaningful participation of communities, CBOs and national NGOs to leverage their knowledge and encourage their commitment to sustaining and strengthening outcomes of tsunami interventions.
PDF files require Acrobat Reader.