2013 Sri Lanka: Support to the Education Sector in Conflict affected Northern Province in Sri Lanka: Summative Evaluation
"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report."
With the end of an almost 30 year conflict in Sri Lanka in 2009, reconstruction and rehabilitation was initiated by the Government of Sri Lanka with the support of the international community. The conflict resulted in the full or partial destruction of school infrastructure in the Northern Provinces. UNICEF and the Ministry of Education (MoE) identified key gaps in the education sector. These included the rehabilitation of education facilities. A project to support ‘The Education Sector in Conflict Affected Areas in Northern Sri Lanka’ (SESCAANSL) was initiated by UNICEF in May 2010, through a request for funding for an integrated package of assistance to children affected by conflict. The Department of Foreign Affairs and Trade of Australia (DFAT) funded some of the education component from June 2010 to July 2011 (Phase 1) for the reconstruction of three selected schools in the northern Kilinochchi district.
In February 2011, a multi-sectoral assessment of the immediate needs in the Northern Province was jointly undertaken by the Government of Sri Lanka (GoSL), the UN agencies, and other organisations. The assessment was reported in the Joint Plan for Assistance (JPA) for the Northern Province – 2011. To assist the Government in response to the assessment, and based on the success of Phase 1, DFAT agreed to Phase 2: The project proposed to rehabilitate schools to nationally-accepted Child Friendly Schools (CFS) standard, providing access to quality learning for children in the North in line with the priorities of the Ministry of Education. The CFS requirements included water and sanitation facilities (WASH), facilities for children with disabilities, and adherence to Disaster Risk Reduction standards. UNICEF was responsible for the management of all phases of the school reconstruction and rehabilitation. Under DFAT funding, 23 primary, secondary and mixed schools were rehabilitated or reconstructed in two districts of Northern Province.
On completion of Phase 2 of the Support to the Education Sector in Conflict Affected Areas in Northern Sri Lanka (SESCAANSL), and in accordance with the quality processes of both organisations, DFAT and UNICEF agreed to undertake a joint summative evaluation of both phases of the project. The purpose of the evaluation is to assess the relevance, effectiveness and efficiency, and sustainability of project implementation. The assessment includes value for money, quality of implementation and the construction, as well as compliance to the Child Friendly School requirements.
The evaluation will review and analyse what the project has achieved, what has worked, and what did not work and why. The evaluation will not examine impact as it is too early to evaluate the longer term effects of school construction. This analysis will inform and shape DFAT and UNICEF Sri Lanka current and future programming in the education sector post humanitarian response phase. The evaluation was led by an external consultant with DFAT and UNICEF supporting an impartial and independent process.
The specific objectives of the evaluation are to assess:
I. the extent to which SESCAANSL achieved its objectives
ii. whether the schools were constructed to meet CFS and other specified requirements
iii. the quality of the school construction and any maintenance schedules
iv. the appropriateness of the delivery approach for the context (i.e. the selection of schools, monitoring and engagement of stakeholders)
v. whether the overall project represents value for money in the re-construction of schools
vi. whether the project aligns with the policies and strategies of DFAT, UNICEF and GoSL
vii. lessons learned and recommendations that will inform and shape future programming in the education sector
viii. whether the project has improved access to education for boys and girls.
The consultant met with DFAT and UNICEF staff and discussion included the draft evaluation plan, selection of schools based on selected criteria, draft key informant questions, logistics support prior to undertaking the field mission activities.
UNICEF utilised three modes of implementation for the construction and rehabilitation works of the 23 schools. Firstly, the 13 schools requiring major construction works (i.e. works valued at over USD50,000) .
Secondly, six of the 10 minor construction and rehabilitation works (i.e. less than USD50,000) were implemented by a local construction contractor procured under an open tender process following MoE procedures administered by the Provincial Education Office (PEO). Thirdly, the rehabilitation of the remaining four schools was undertaken by the School Development Society (SDS), with funding provided to the school via block grant from UNICEF. For the evaluation 9 schools were selected, 6 major, 2 minor and 3 rehabilitated.
The main tools were observations of the completed schools using a construction assessment form to compare, to the extent possible, the actual situation with that shown on the as-built drawings and in the operation and maintenance manuals and focus group and key informant discussions (FGD) with stakeholders, particularly the school staff and SDS members, students and parents and Zonal Education Office officials. In addition, discussion were held with District and Provincial level managers of the Education Department responsible for the reconstruction program.
Key informant interviews
The main groups / organisations contacted are:
- School staff, teachers, students,
- School Development Societies (SDS), Parents, district and zonal education officials
- Construction contractors and supervision consultants
Findings and Conclusions:
Overall, the project has achieved the aim of restoring 23 schools in areas where people have been displaced, supporting the success and sustainability of the returnee process. Schools help restore the community to a sense of normalcy after decades of multiple displacements and help reduce the trauma of children.
School re-construction was very relevant at the outset of the project to address the urgent infrastructure needs at that time. As the backlog of classrooms to be refurbished is eliminated, classroom reconstruction will be of diminishing relevance going forward. Sector support is transitioning from humanitarian to longer-term development programming in line with the DFAT Sri Lanka Country Strategy objectives.
Feedback indicated that stakeholder engagement throughout the reconstruction process was strong and effective, and that teachers and students were very positive about their new school buildings. The objectives and scope were poorly defined in the grant agreements.
The project was completed on time, with value for money demonstrated for the majority of schools. However, there were quality and procurement risks with six minor-construction schools and structural defects on major and minor construction schools (the latter can be fixed).
DFAT Disaster Risk Reduction requirements relating to environmental issues (e.g. unexploded ordnance, tsunami and storm surge) were considered during the design process. The main concern is the lack of a periodic maintenance budget for the MoE for major maintenance works. Meetings suggested that small routine maintenance could be managed at the school level by the staff supported by the School Development Society.
Women were active participants in decision-making both through their representation on the staff and the School Development Society. Separate WASH facilities were provided for girls and boys in most schools.
Recommendation 1: In the case of the six minor-construction schools where new classrooms were built and/or significant work was undertaken on structural elements, and the construction was not monitored by a qualified structural engineer, UNICEF should arrange for a structural engineer to review the quality records and engineers’ logs, and certify in writing that the elements/classrooms have adequate strength and are safe, and if not, list the corrective actions required to make them safe.
Recommendation 2: That UNICEF ensures all defects identified and listed at the hand-over inspection, plus other defects discussed with the UNICEF field office engineers during the evaluation visits, are rectified before the final inspection.
Recommendation 3: At completion of the defects liability period for each school, UNICEF should provide DFAT and the MoE with certification that “the buildings as constructed are safe and satisfy the relevant standards, codes and building regulations in Sri Lanka, and do not contain any asbestos containing materials”.
Recommendation 4: That UNICEF provides a detailed acquittal of the balance of the funding (27.5% - approximately USD2,772,802) that was not directly allocated for the design and construction of the 23 schools.
Recommendation 5: That all future construction activities supported by DFAT in the education sector comply with DFAT’s Disability Inclusive Design criteria, and the Ministry of Social Services Sri Lanka policy “Promotion of Accessibility to the Built Environment for Persons with Disabilities” and include, inter alia, toilets for people with disabilities as well as the associated ramps, handrails and doorways suitable for wheelchair access.
Key lessons learned include:
• detailed project design documents should be prepared to define the scope, cost, implementation schedule, reporting and monitoring arrangements, preferably before the commitment of funds
• an independent and critical assessment of the government’s school infrastructure investment decisions will ensure that resources are properly targeted and avoid over-capacity in any locations
• if the recipient government procurement systems are used, appropriate checks and balances should be undertaken
• all construction activities need to be independently and systematically monitored to ensure compliance with the design and safety of final structure
• the modality of community-based contracting is good for small reconstruction works provided appropriate technical support is given.
Some Lessons learned for Australian aid initiatives
This evaluation has found a number of lessons learned which are transferable across initiatives and countries for the Department of Foreign Affairs and Trade, Australia and relate to: project design and planning; procurement and construction processes; quality assurance procedures; school facilities; and reporting.
Full report in PDF
PDF files require Acrobat Reader.