We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2017 Afghanistan: Evaluation of WASH in Schools



Author: Viju James

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report, and the executive feedback summary labelled as ‘Part 4’.

Background:

The school WASH situation mirrors that of Afghanistan in general. MoE data show that although considerable progress had been made by 2011 much more needs to be done. The main problems are an insufficient number of schools, inadequate WASH facilities, lack of O&M, and insufficient behavioural change to promote hand washing and MHM.

The WinS Programme from 2012 aims to make visible the value and impact of school sanitation as perceived by the community and thereby raise the level of ownerships; promote the importance of WASH in schools at national, state and district levels; improve hygiene practices among school children, their families and communities; develop, test and improve the curriculum, teaching methods, teaching aids and teaching programmes, with a view to children learning the value of hygiene and health-promoting behaviour; and promote family and community involvement, and partnership in the sustainability of school WASH facilities.

UNICEF wished to conduct an evaluation of the WinS programme, whose findings and recommendations are intended to (1) guide UNICEF, the GoA and other stakeholders to improve the WinS programme, and (2) contribute to evidence-based policy making in the field of WASH and maximize the impact of the programme. Primary users of the evaluation analysis, conclusions and recommendations are the UNICEF Afghanistan WASH Team, the WinS implementing partners in the government, and other NGOs and UN agencies working closely with UNICEF.

Purpose/Objective:

The purpose of the evaluation is to evaluate the implementation of the hardware component (quality of construction, design appropriateness, cost effectiveness and sustainability of the WASH facilities at schools, etc.) and the software component (hygiene education including MHM, O&M arrangements, etc.). The four sets of Evaluation Questions are: Relevance (the extent to which the programme is suited to the needs of the target population and aligned with WASH strategies and national priorities); Effectiveness (the extent to which programme interventions attained intended results); Efficiency (Qualitative and quantitative measures of programme outputs relative to inputs); Sustainability (the extent to which interventions are likely to continue without direct UNICEF support). Impact was not part of the TORs.

The final sample surveyed is 106 schools, comprising 75 WinS and 31 comparison schools.
A mixed methods approach was used, collecting both qualitative and quantitative information, through a desk review; semi-structured interviews of the staff of MoE and UNICEF Afghanistan; Key Person Interviews with province-level and district-level officials from MoE and MRRD, and school Principals; and Focus Group Discussions with teachers, SMC/shura, school girls & boys, and differently-abled students. The Quantified Participatory Assessment (QPA) was the method used to collect and analyze the qualitative and quantitative information from the field. A QPA uses standard PRA tools but transforms qualitative information into numbers using different methods including ordinal scoring.
The Evaluation followed the Norms and Standards as well as Ethical Guidelines for Evaluations of the UNEG. Checks to ensure the quality of information collected included intensive training, field supervision, telephone checks to school principals, consistency checks in the custom-built database and internal reviews of all reports by UNICEF.

Methodology:

The final sample surveyed is 106 schools, comprising 75 WinS and 31 comparison schools.

A mixed methods approach was used, collecting both qualitative and quantitative information, through a desk review; semi-structured interviews of the staff of MoE and UNICEF Afghanistan; Key Person Interviews with province-level and district-level officials from MoE and MRRD, and school Principals; and Focus Group Discussions with teachers, SMC/shura, school girls & boys, and differently-abled students. The Quantified Participatory Assessment (QPA) was the method used to collect and analyze the qualitative and quantitative information from the field. A QPA uses standard PRA tools but transforms qualitative information into numbers using different methods including ordinal scoring.

The Evaluation followed the Norms and Standards as well as Ethical Guidelines for Evaluations of the UNEG. Checks to ensure the quality of information collected included intensive training, field supervision, telephone checks to school principals, consistency checks in the custom-built database and internal reviews of all reports by UNICEF.

Findings and Conclusions:

Although WinS was a good response by UNICEF to the GoA’s Joint Call for Action, the hurried start affected preparations and course correction based on feedback was insufficient.

There was a misunderstanding over UNICEF’s role in implementation: MoE felt UNICEF should have provided more support staff; UNICEF felt MoE did not adequately own WinS.

Little coordination between Education and WASH Sections within UNICEF, and within MoE section responsible for hardware and software components of WinS.  Providing basic WASH facilities will be a continuing focus of WinS, given the continued lack of basic WASH infrastructure in schools.

A major drawback for future WinS programming is that WASH in schools is still not a priority for MoE policy and strategy.

Contracting larger construction companies may have provided economies of scale and infrastructure that was better adapted to local contexts.

Better planning and design, based on lessons learnt, needed for effective WASH facilities.
The ineffective hygiene education and MHM trainings need to be integrated in the standard school curriculum and into the teacher training curriculum.

Cost must be monitored better, with competent and decentralized database management
Contracting procedures should be reviewed and revised to tackle reasons for delays and cost escalation (including over-design and disputes with contractors), include O&M, and attract larger companies (for economies of scale).

Effective O&M arrangements (e.g., trained personnel with tools, protocols and adequate O&M budgets for preventive maintenance and repairs) for all schools are vital to sustain WinS infrastructure
Innovative planning is needed to provide O&M budgets (since local stakeholders cannot contribute more) to maintain the newly-created and expensive WASH facilities.

Recommendations:

- Focus more on ensuring service delivery with specific objectives (e.g., reducing incidence of water-borne disease or girl drop-outs due to poor WASH) not just infrastructure provision.

- Involve local stakeholders (especially women) more in planning, designing and construction of school WASH facilities, and provide budgets and capacity building for effective O&M.

- Provide specialized WASH training for teachers and principals prior to implementation as part of regular teacher training curricula, with innovative techniques to make WASH trainings interesting, relevant and therefore useful and effective for school children of different ages.

- Build a cadre of good-quality professional WASH trainers to train teachers on how to train.

- Make greater efforts to recruit and train women teachers for school WASH and MHM counselling, with innovative approaches (e.g., training local women in MHM and ensuring that every school with girls has a designated set of local women trainers)

- Increase MHM interventions e.g., awareness generating activities like classes and seminars; informative materials like books and pamphlets; and facilities like sanitary napkins, incinerators and dustbins, etc.

- Involve religions leaders such as mullahs and imams of local mosques to lead the community effort on improving school WASH facilities may be a useful option.

- Use social and individual incentives for effective School WASH such as competitions within districts and provinces for innovative WASH training, or for schools whose boys and girls have performed well in WASH-related activities, or for the cleanest toilets, or for teachers voted as Sanitation Ambassadors.

- Cease the construction of double vault composting toilets for schools and return to the old single vault toilet designs implemented by the MRRD. These are much better adapted to the local situation with respect to ease of emptying and use of stones as anal cleansing material.

Lessons Learned:

- Future focus of WinS should be on ensuring effective delivery of school WASH services.

- Integrated planning and implementation of hardware and software improves effectiveness.

- School principals and teachers need to be well oriented prior to implementation. 



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2017

Country:
Afghanistan

Region:
ROSA

Type:
Evaluation

Theme:
WASH

Partners:
Ministry of Education, Ministry of Public Health, Ministry of Reconstruction and Rural Development

Language:
English

Sequence:
2017/003

New enhanced search