We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 Republic of Mozambique: Final evaluation of the project: “Rehabilitation of two piped water systems in Mozambique”



Author: Bruno Valfrey, Martina Rama, and Augusto Razulo

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

 

Background:

In 1998 the Government of Mozambique introduced the Delegated Management Framework (DMF) for urban water supply services, a public-private partnership whereby assets are owned by the government’s agencies FIPAG (for main urban areas) and AIAS (for small towns), and operations are managed by the private sector under a concession, lease or management contract. Regulation is ensured by the regulatory commission CRA. UNICEF’s support to the implementation of the DMF started in 2012 with the introduction of its Small Towns WASH programme. The programme is implemented in partnership with public counterparts and aims to support AIAS in the development of an implementation model to leverage funds for broader implementation of the DMF.

The project “Rehabilitation of 2 piped water supply systems in the towns of Ulónguè in Tete Province; and Espungabera in Manica Province” was part of the Small Towns WASH programme, amd implemented by UNICEF in collaboration with AIAS, DPOPHRH Tete and Manica and SDPI Mossurize and Municipality of Ulongue from early 2012 to mid-2017. The project was co-funded by the Government of the Netherlands (82%), UNICEF (15%) and the Government of Mozambique (3%). The project’s main objective was to reduce the relative poverty levels through improved access to water supply, adequate sanitation and hygiene promotion. It included 3 components: rehabilitation of piped water supply systems, sustainable management models for the water systems, and peri-urban sanitation. The projects expected outputs were the following: 36,000 new users of 2 rehabilitated systems, with 16,000 users in Ulónguè and 20,000 users in Espungabera,

Purpose/Objective:

The objective of this end-of-project evaluation is to evaluate the performance of the UNICEF-managed project “Rehabilitation of Two Piped Water systems in Mozambique” (Espungabera and Ulónguè) based on the OEDC-DAC methodology for evaluating development assistance.

Methodology:

The project’s performance was evaluated based on the OEDC-DAC methodology for evaluating development assistance, following the standard evaluation criteria of relevance, effectiveness, efficiency, impact and sustainability. In addition to these 5 standard criteria, the following 3 additional criteria were used: added value, equity and inclusion, respect of the environment. Based on these 8 criteria, 12 key evaluation questions have been generated, together with the relevant sub-questions, sources of information, measurement methods, tools.

Evaluators conducted a series of interviews with most relevant partners and stakeholders at national, provincial and district / small town level and a field visit to ascertain conditions of the water supply and sanitation systems, together with interviews and focus group discussions with local stakeholders and beneficiaries.  Prior to the field work, methodology and approach for the evaluation were discussed with representatives from national, provincial and district level partners through a consultative meeting.  Preliminary results were also reviewed and discussed at national level through a similar consultation process.

Findings and conclusions:

The project was very relevant in responding to sector priorities and needs. It was aligned and consistent with national priorities and strategies. Though done in a consultative manner, a significant weakness of the project design was the lack of Logical Framework nor Theory of Change.

Project’s effectiveness is low for water but higher for sanitation. Water systems were correctly dimensioned for the expected users and the quality of infrastructures is good. However, current number of beneficiaries is below the target declared in the proposal, referred to water systems design population and not users at end of implementation. The monitoring quality was average, considering the absence of initial targets and indicators, compensated by systematic monitoring during implementation.

Project’s efficiency is low with respect to timeliness as project was extended for almost two years, in agreement with Donor.  Though this influenced negatively its efficiency, such extension is not uncommon for this type of intervention. Based on number of beneficiaries considered for the evaluation framework, average cost per capita is 270 USD for water and 200-240 USD for sanitation, which is high compared to international literature for projects in Mozambique (230 USD/capita), but in line with estimates by Mozambican public institutions (260-400 USD).

The project impacted positively access to water and - to a lesser extent - to sanitation, however the coverage of water networks limited the project’s impact. Both water systems are in use, with private household connections functioning well, but public standposts not functioning as expected and many closed down.

Projects’ sustainability strategy was sound and gave good results: national stakeholders feel responsible for the services, sanitation promotion activities have been (partially) taken on board by local authorities, and private operators are managing the water systems with good performance and profit perspectives.

Recommendations:

For UNICEF: for future projects, develop of a logical framework including a detailed and equity-lensed M&E framework; proceed to a more accurate cost-estimation for budgeting and better definition of beneficiaries and unit-cost; budget for sufficient secondary and tertiary network to cover the whole towns in order to maximize the immediate direct impact of the investment; develop baseline studies  at the beginning and at the end of the project in order to evaluate changes in access to service; consolidate all sanitation promotion activities and interventions into a consistent strategy or approach.

For AIAS: Increase monitoring efforts and analyze data received from operators to provide feedback to them on how to improve service levels; work with DNAAS to formally agree on the role to be played by the regional and district technical authorities in the DMF and consequently train staff; develop a strategy to better involve municipalities in the monitoring of operators’ performance; provide for independent auditing of the operators; improve clarification over responsibilities for network expansions; ensure investment in expanding the secondary and tertiary network; improve cooperation with CRA on the monitoring of the regulatory framework.

For CRA: Improve cooperation with AIAS on the monitoring of the regulatory framework; assess the utility and relevance of the CORAL mechanism; provide support to AIAS to revise the operators contract to increase their duration and better clarify responsibilities and targets for network expansion.

For DNAAS/DPOPHRH/SDPI : Sign a cooperation agreement with AIAS to better clarify the roles within the DMF; increase the budget allocation of deconcentrated bodies for monitoring activities; collaborate with administrative authorities to introduce a system of sanctions for users who are not respecting correct hygiene / sanitation practices.

Lessons Learned:

For UNICEF: for future projects, develop of a logical framework including a detailed and equity-lensed M&E framework; proceed to a more accurate cost-estimation for budgeting and better definition of beneficiaries and unit-cost; budget for sufficient secondary and tertiary network to cover the whole towns in order to maximize the immediate direct impact of the investment; develop baseline studies  at the beginning and at the end of the project in order to evaluate changes in access to service; consolidate all sanitation promotion activities and interventions into a consistent strategy or approach.

For AIAS: Increase monitoring efforts and analyze data received from operators to provide feedback on how to improve service levels; work with DNAAS to formally agree on the role to be played by the regional and district technical authorities in the DMF and consequently train staff; develop a strategy to better involve municipalities in the monitoring of operators’ performance; provide for independent auditing of the operators; improve clarification over responsibilities for network expansions; ensure investment in expanding the secondary and tertiary network; improve cooperation with CRA on the monitoring of the regulatory framework.

For CRA: Improve cooperation with AIAS on the monitoring of the regulatory framework; assess the utility and relevance of the CORAL mechanism; provide support to AIAS to revise the operators contract to increase their duration and better clarify responsibilities and targets for network expansion.

For DNAAS/DPOPHRH/SDPI: Sign a cooperation agreement with AIAS to better clarify the roles within the DMF; increase the budget allocation of deconcentrated bodies for monitoring activities; collaborate with administrative authorities to introduce a system of sanctions for users who are not respecting correct hygiene / sanitation practices.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year: 2018

Office/Country: Republic of Mozambique

Region: ESARO

Type: Evaluation

Theme: WASH; WASH - Household water security: systems, projects, strategies

Language: English

Sequence #: 2018/001 

New enhanced search