Evaluation database

Evaluation report

2010 Ethiopia: Mid-Term Evaluation of EU/UNICEF Supported WASH Programme



Executive summary

 

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”

Background:
The WASH project extends to all regions of the country. It also includes capacity building components, undertaken with the Federal Ministry of Health and Federal Ministry of Water Resources. A total of 1.4 million people in 78 Woredas are targeted for project interventions. The overall objectives centre round poverty reduction and health improvement (with particular reference to vulnerable people living in marginal conditions and People Living with HIV and AIDS). The programme also centres on promoting women’s empowerment and girl child school enrolment. In accordance with the project proposal and logical framework , a Mid Term Evaluation (MTE) has been arranged to assess what the project has achieved against agreed outputs and to guide the remainder of the project in terms of its focus, direction and staffing within the current policy and socio-economic environment.

Purpose/ Objective:
The MTE is an independent assessment of the performance of the project to date, based on the project proposal, log frame, activities and budget, and in the current policy and economic environment as this affects the WASH Sector in Ethiopia. The MTE will identify key lessons and propose practical recommendations for follow up actions. In line with the project proposal, the UN Evaluation Group ‘evaluation’ criteria and OECD - DAC, the MTE will focus on five evaluation criteria: relevance, effectiveness, efficiency, sustainability and impact. Each is associated with a number of key evaluation questions that are to be addressed and explored.

Methodology:
Document reviews, review of programme theory, focus group discussions and key informant interviews. The mid-term evaluation team has visited 10 woredas (districts) in 2 regions and held an extensive range of interviews with stakeholders at all levels using a range of tools and checklists including a household survey which reached 300 households. With UNICEF working in 78 Woredas in all regions, the 10 woredas visited represent a snapshot but a chance nevertheless to appreciate the scale of the WASH challenge and the role played by the UNICEF team in supporting government in its pursuit of universal access.

Findings and Conclusions:
Despite the funding delays, hold ups in procurement, inexperienced staff, low absorption of woredas and the scale of the WASH challenge project achievements have been substantial.

Successful women’s empowerment, rehabilitation, self-supply, CLTS but schools WASH is slow.

Successful capacity building (technical training), new partners, inventory BUT private sector development is slow.

- Outputs are generally on track but funding streams slow. The multiple partnerships mean a diversity of approaches and the project is considered cost-effective at $20 per head. The innovative CDF has proved a successful conduit for funds and a means to ensure sustainable community WASH management.
- The innovative inventory is an important tracking device and is recognized a useful resource across the board by key stakeholders interviewed.
- Sustainability – WASH teams, WASH Coms, the case for WSGs & CFTS, Spare part supply chains, environmental challenges – drying wells and behavior change.
- Although capacity building is moving in the right direction there needs to be a stronger focus on inter-sectoral, multi-stakeholder planning and implementation.
- The added value – dedicated, well-qualified team willing to take risks, learn lessons and take to scale e.g. self supply, CLTS and the WASH inventory
- The time is ripe for full alignment with the WASH programme – National Programme Implementation manual.
- Minimal attention to environmental issues and long term sustainability of water sources and latrines is at stake.

Recommendations:
- Establish a minimum WASH package (not less than $50,000 per woreda). UNICEF should align more closely with the national WASH programme.
- Maximize women’s role in monitoring and evaluation of all WASH facilities as they are the primary users of water points and can provide daily checks on functionality. In the case of sanitation, place a stronger focus on educating men about the negative health impact on women unable to complete ablutions during daylight hours, emphasizing that the availability of a safe, private, convenient and hygienic latrine is a high priority for women on many levels.
- Convergent planning and implementation for a more integrated, comprehensive WASH package must be strengthened to build the potential school/community synergies. UNICEF should consider expanding their work through partners acting as woreda support groups and community facilitation teams. Agree and set woreda target service levels in each of the EU-UNICEF operational woredas including the preparation of detailed plans for each woreda identifying all work required towards achieving the woreda targets.
- For each technology options identified it is good to estimate per capita investment cost and operation and maintenance (O & M) cost; thus determining the affordability level of the community particularly for covering cost of O & M. UNICEF should adopt the principle of plausible inference and push to achieve >80% access and use of latrines, soap when hand washing and compliance with the safe water chain.
- Take the WASH inventory and report cards to scale in all EU-UNICEF supported woredas. Continue to support the rolling out of the WASH inventory and the WASH report card providing close supportive supervision to ensure high standards and good credibility.
- It is important that communities choose the technology that will give them the highest service level they want, afford, and can maintain. Ensure support to woredas addresses priority problems in each woreda based on the woreda’s strategic plan. UNICEF should ensure that the process of evaluating impact (outcomes) is integrated into the regional workplans and includes members of government and other key sector stakeholders.
- Ensure greater integration of all WASH actors at woreda level by strengthening woreda WASH teams’ capacity to plan and implement together within a common framework (e.g. WASH Manual). Conform with the national WASH implementation procedures.
- Apply a modified version of community led total sanitation to include a stronger focus on wider more sustained behavior change (focus hand washing) in line with the Task Force recommendations. The Community led Total Behavior Change for Improved sanitation and hygiene which combines existing strategies within a more holistic approach needs to be scaled up with appropriate supportive supervision.
- It is essential that the EU-UNICEF project continues to work in line with UAP, PASDEP, HEP, HSDP; to build consensus on the national WASH manual; and to align with WASH programme community contribution (5% cash and 5% in kind). In line with government strategy, UNICEF staff should be assessed on key performance indicators which actually encourage delegation of activities e.g. sustainable/localised systems for CLTS training.
- UNICEF should work closely with Catholic Relief Services to promote Arborloos in the community and fossa alterna/skyloo ecological latrines in schools and other institutions. The EU-UNICEF project could do more to encourage regional/woreda/community efforts (including partnering with environmental focus programmes and projects) to assure sustainability of the water sources.

 



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search