Evaluation database

Evaluation report

2011 Bangladesh: UNICEF’s Disaster Risk Reduction and Preparedness at School: A Report on Evaluation of Approaches and Capturing the Lessons



Author: Khurshid Alam; M. Shameem Siddiqi; Khaled Hossain; Abdullah Al Rashed; Md. Ashraful Haque; Sumaiya Kabir

Executive summary

 

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”

 

Background:

Disasters pose a serious risk to children’s access to quality education. Disasters threat life and wellbeing of students and education staff as well as limit their access to school, deteriorate quality of education, delay recovery of school and increase drop-out. The situation is likely to exacerbate by the impact of climate change as well as threat of earthquake.

The government of Bangladesh (GoB) undertook a number of initiatives e.g. improvement of physical infrastructure and inclusion of disaster in the school curriculum. Such initiatives reduced the physical risk, but challenges remain in the alignment education and disaster risk management system of the country. These challenges were highlighted in the discussion after the two massive disasters in 2007. The Education Cluster, led by GoB and UNICEF, piloted a set of approaches to find an effective strategy to protect primary education from disasters. With funding support from UNICEF Bangladesh US$ 2.5 million, the project was implemented by two consortiums led by Save the Children UK (SC-UK) and ActionAid International Bangladesh (AAIB). The project worked with 1,400 schools in flood and cyclone affected areas in 2009 and 2010.

Purpose/Objective:

The objectives of the evaluation are to examine the effectiveness, efficiency, sustainability and replicability of the various institutional and programmatic approaches to recommend a set of approaches that can be replicated at scale.

Methodology:

The methodology of this evaluation was designed on the basis of:
(a) The methodology, process and overall outcomes of the study outlined in the TOR from UNICEF Bangladesh entitled, “Effectiveness study and process documentation of the projects on disaster risk reduction and preparedness at school level”;
(b) Brainstorming meetings among UNICEF, the project consortium partners and the consultant team to work out the key aspects of the study; and
(c) Comments received on the inception report presented to UNICEF and the two consortiums.

The primary audiences who have been kept in perspective are the GoB and the Education Cluster. However, there are several secondary audiences including the school community, education managers, civil society, and others. Emphasis was put on the critical lessons based on experiential learning of the project partners.

Sample Design and Sample Size:

A multi-stage purposive sampling technique was used in the study. The central unit for sampling process was the primary schools in different agro-hazard zones where the UNICEF funded pilot project has been implemented. In line with the geographic areas of the pilot project, the samples were selected from the following zones:

i. From the flood zones, schools (a) very close to rivers; (b) located within 10 kilometres of river; and (c) in typical flood plain; and
ii. From cyclone zones, schools (a) physically close to the recent cyclone paths from both the south and east coast; (b) located in central Bangladesh that is exposed to both cyclones and flooding; and (c) adjacent to the Sundarbans.

Efforts were made to include districts affected by disasters during the project’s life. As for school selection, three schools were selected from each zone with one government primary school, one registered government primary school, and one community school. Thus, the study covered a total of 18 schools. One district was purposively selected from each of the above-mentioned zones. In each selected district, the study involved the following institutions:

• Government education department at district and upazila level;
• Education committees at district, upazila and union level; and
• Disaster management committees at district, upazila and union level.

In each school, information was collected from: (a) School Management Committee (SMC); (b) Parent-Teacher Association (PTA); and (c) the students (boys and girls) in the selected schools. In addition, the study team interacted with and collect information from selected representatives from the project consortiums, implementing partner organisations, key policy makers, and UNICEF.

Study Methods and Tools:

The following tools were applied for the collection and analysis of information:
1. Literature review for identification of key approaches and lessons;
2. Individual in-depth interviews for evaluating programme approaches and institutional arrangements, and collection of lessons;
3. Group discussion for evaluating programme approaches and institutional arrangements, and collection of lessons;
4. Institutional analysis / case study; and
5. Workshop for collective analysis for generating recommendations and collection / interpretation of lessons.

Findings and Conclusions:

Results/Conclusion:
• Past disaster preparedness projects were conducted taking into consideration the most recent disasters and the areas affected by those disasters. Since the country had not experienced a major earthquake in the recent past, the project did not take it as a major consideration in the design. As a result, earthquake preparedness did not feature significantly in the project design which can be seen as a missed opportunity for developing an approach to deal with earthquakes.
• The vulnerability of primary education to disasters in Bangladesh is multi-pronged and multi-faceted calling for responses at various levels. It requires the engagement of different types and levels of stakeholders as well as changes in policies, institutions and the mindset of stakeholders. The project demonstrated understanding of this complexity and designed strategies, approaches and actions accordingly but the input needed for the required depth and intensity of the engagement was not adequate for disaster risk reduction.
• Despite the limited duration and inadequate financial input for field operations, the project has been able to create noticeable impact at political, strategic and operational levels. The related consequences and necessary measures to address education in emergencies for boys and girls were assessed through workshops and dialogues with various stakeholders, from union to district level. Changes in this regard were evident in all districts under study. At the national level, through studies, information sharing and dialogue with government and civil society, the project instilled giving a higher degree of importance to the effects of disasters on education and the urgent need to address these issues.
• At a strategic level, the major value addition of this project has been the generation of knowledge, lessons and possible approaches to address flood and cyclonic risks to primary education. Such knowledge includes gaps in the systems and policies, and the potential stakeholders to deal with these along with their possible roles and capacity building needs. While the approaches were reasonably understood and acknowledged at the school level with external facilitation and resources, the appropriate and cost-effective institutional approaches require further research and reflection for replication and scale up. At the same time, it should be noted that understanding impact requires longer duration in a project of strategic nature like this one. Moreover, there was no disaster during the project life to put the approaches under test to understand the appropriateness and efficiency of the programmatic and institutional approaches. In order to address these two challenges, the evaluators adopted two assumptions that the targeted schools would be less affected by future disasters if: (a) school communities were able to identify all the main causes of the vulnerability, and (b) adequate inputs were provided to mitigate those causes.
• It is clear from the evaluation that all the sample schools were able to identify, analyse, document and plan on the core causes of vulnerability of the schools which in turn translated into action plans or contingency plans. However, the limited budget of the project influenced the planning process and content, resulting in prioritisation of the ‘affordable’ actions. In most cases, the planning process itself led to the creation of local leadership and relationships among the stakeholders which has a potential impact in the implementation of some or all of the activities in the plan. On the other hand, the process itself was influenced by the adults and the popular thinking, priorities and the ideas of children (boys and girls) were not sufficiently reflected.
• The capacity for local level advocacy and leadership multiplied as a result of the project. However, support from district and upazila administration could not be galvanised sufficiently for a number of reasons It may not be unlikely that a local good practice can influence national policies and the ways institutions operate; but the time and other resources required to nurture this was not sufficient in this project. The agenda and initiative for the national level advocacy came towards the end of the project life resulting in inadequate efforts for such influencing work.
• The project was able to identify the right institutions for engagement from school to district levels e.g. SMCs, UPs, district and upazila education offices, disaster management committees, etc. However, the engagement of the wider community was not adequate. In addition, the project’s engagement with the identified institutions was not sufficient mainly due to late start of the field implementation. The project also did not establish any new institutional arrangement to review and implement the plans at the school level. International experience in the region shows that new institutional structure can be effective to implement disaster management plans.
• In terms of coverage, the project accomplished a massive task by identifying 1,400 schools, and conducting necessary exercises for analysis and planning in those schools and related institutions in a relatively short period of time. Both the consortiums were able to utilise the existing research and knowledge on DRR in Bangladesh to apply in the context of education. Closer engagement of the implementing local partners could speed up the process of developing the guidelines and tools of the project.
• The project had one set of objectives, but two consortiums were engaged in the implementation where the two sets of implementing organisations applied different approaches, tools and techniques. While such diversity created the potential for testing a wide variety of approaches and methods, the limited coordination and sharing between the two consortiums beyond national level missed the opportunity to create synergy and shared which, in turn, could result in the development of concrete approaches and methods.
• Both consortiums built partnerships with implementing organisations, both with those with presence in the project areas and with those not presently covering the area. In the cases where partner organisations were taken from other geographic areas, the impact and potential for continuity were less evident. In order to test such new approaches, existing local presence of partner organisations is essential for building and capitalising on relationships with relevant stakeholders, building on and deepening the understanding of the local context, and creating possibilities for greater local ownership and continuity.

Recommendations:

1. Design and implement a follow-on phase of the pilot project in order to consolidate the interventions in preparation for wider replication and scaling-up. The context of earthquake should be factored in the main design of the project.
2. In selecting the field level implementing partners, preference should be given to those who are already present in the respective geographic locations. This will help ensure easier access to the communities and other stakeholders, and contribute to the likely continuity of the interventions beyond the duration of funding.
3. UNICEF and the consortium partners should work with the government to identify the policy and institutional barriers to facilitate the local level initiatives.
4. Adopt child-centeredness as the overarching principle of future initiatives.
5. The involvement of the local government must be prioritised in order to create local political agenda and leadership for resilience building in education.
6. A conceptual synergy can be built on three important concepts which exists in Bangladesh and elsewhere in South Asia i.e. DRR though school (mostly used by ActionAid), EiE, and school safety in the context of earthquake. Based on these concepts, three overall objectives can be set for the next phase of the project with a goal to build resilient primary education. These include: (i). protection of life and reduction of injury among children, teachers and school community; (ii). continuation of education in emergencies, and (iii). investing in children to ensure they grow as future leaders in disaster risk reduction and to climate change adaptation.
7. Consider the development of a resilience building strategy for education on the basis of the learning of this pilot project. Considering the multi-sectoral nature of the problem, engagement of all relevant sectors, departments and other stakeholders should form integral parts of the development and implementation of the strategy.
8. Risk factors to education should be included in the existing disaster management systems, tools and procedures. This is expected to resolve the conflicting priorities of the institutions related to education and disaster management.
9. Examine the potential of the existing institutions (e.g. National Academy for Primary Education-NAPE, PTI, infrastructure facilities, etc) to implement the capacity building components of the strategy. A separate contingency fund at the district level can also be considered to improve the timeliness of disaster response.

Lessons Learned (Optional):

1. Community initiatives and leadership to protect their school from disaster are highly evident in line with their capacity. Externally facilitated projects produce effective result if they are built on such leadership.
2. Success of school-based DRR initiative depends on a multi-stakeholder partnership within the school community (i.e. students, parents, teachers and SMC) and wider stakeholders (e.g. local government, elected representatives and government departments).
3. Systematic risk analysis using participatory methods helps school community to generate collective analysis and actions. Children add significant value to the process, but their participation in the planning can be limited by cultural norms.
4. School communities can translate their analysis into contingency and long term plans, and such a plan can sustain when it is integrated with school improvement plans (where it exists) along with appropriate institutions and leadership.
5. External facilitation to school community has been helpful but may not be feasible at national scale. Further research should be conducted on the possible integration of DRR in the regular education process.
6. Effective coordination is yet to exist between education and disaster institutions at sub-national level which is an influence of national policy and institutional mind-set. Further initiatives should be taken to examine and address the existing gaps.
7. Capacity is not yet adequate (e.g. human, knowledge/skills and finance at the education institutions) to adopt a disaster risk management approach. More discussions are required within the Education Cluster to integrate necessary capacity building initiatives in the existing programmes.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search