Evaluation database

Evaluation report

2010 Iraq: Evaluation of Water Quality Control and Surveillance in Iraq



Executive summary

 

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”

Background:

Water supply systems in Iraq sustained major damage brought about by wars, economic sanctions, unabated waves of violence, looting of equipment and failure to carry out regular maintenance and quality control testing and monitoring. As a result, morbidity and mortality from water-borne infections had increased to unprecedented levels especially among children the under 5 years of age.

According UNICEF1, less than half of Iraq‟s population can claim reliable access to potable water and less than 10% of urban households outside Baghdad are connected to sanitary sewage systems, and where they do exist, there are frequent failures operating on limited electricity, ideal sewage pumping stations and treatment plants flood neighborhood sides and discharge raw waste water into Iraq‟s rivers.

In order to ameliorate the situation and help the most affected families, UNICEF launched a water tankering service in the aftermath of 2003 war, which was part of rehabilitation activities of many water treatment projects and represented a life line for more than 200,000 Iraqi people.

The project “Water Quality Control and Surveillance in Iraq (phase two) is a joint initiative of WHO and UNICEF in collaboration with the Ministry of Environment and Ministry of Municipalities and Public Works of Iraq and, the Mayoralty of Baghdad and Ministry of Health and Ministry of Environment/KRG Erbil.

The first phase of the project was funded at USD 6.26 million and aimed at expanding the water quality monitoring network, include waste water treatment plants and perform extra quality analysis to meet Iraqi national and WHO standards for drinking water. A network of 15 governorate, 30 district and 220 water quality laboratories were established at water treatment plants. The second Phase of the project aimed at expanding and strengthening what has been achieved during the first phase by procurement of equipment and supplies for additional water and some waste water quality control laboratories, capacity development of national staff and launching of public awareness campaigns.

The key developmental goal of the project was to ensure further reinforcement of the water quality monitoring programme with the overall purpose of supporting the provision of safe drinking water to the consumers according to Iraqi Standards and WHO guidelines for potable drinking water by establishing an expanded network of quality control laboratories, development of human capacity among laboratory managers and technicians ,establishing a laboratory information system and launching public health education campaigns in high risk areas.

The project was funded under the UNDG-ITF, Cluster E (Water and Sanitation) at USD 4,297,643. The original project duration was 12 months starting 22 December 2006 and was extended four times, the last of which was until 31 December 2009.

WHO has operationally completed all planned activities and the project has been closed from WHO‟s side meanwhile, UNICEF has developed and installed the Laboratory Information Management System (LIMS) at central laboratories in Ministry of Environment, Ministry of Municipalities and Public Works and Mayoralty of Baghdad. The software was installed at 30 laboratories of Ministries of Municipalities and Environment at 15 governorates. Also LIMS was installed and is operating at 8 projects in Mayoralty of Baghdad. There is however some minor issues related to the operation of the software, which needs to be overcome.

Equipment, laboratory supplies and reagents have been procured and received by the laboratories. This has enhanced the water testing procedures and facilitated the analysis of many tests which were not previously possible in these facilities, as well as increasing new water quality parameter tests by almost 60%. The three rehabilitated wastewater laboratories have contributed to strengthened wastewater testing capabilities in Najaf, Ninewah and Diwaniyah.

In addition to the 50 on-the-job trainings, the 12 Train-of-Trainers has made a positive effect on the performance of staff by exposing them to modern technology and procedures in water quality testing, monitoring, awareness campaigns and community mobilization in 8 governorates. Twelve wastewater laboratory staff was trained in Amman on testing of basic parameters and Quality Assurance & Quality Control in wastewater sampling and analysis. Another 23 laboratory staff was trained in Basra on Water Quality control and testing.

Following installation training for 61 LIMS users for governorate laboratory staff in three batches at the Basra, Najaf and Kirkuk training centres during April – May 2009. Also training of users of 19 laboratory staff was conducted in November 2009.

In late April 2009, UNICEF obtained ITF approval for utilizing savings achieved during project implementation to undertake the development of LIMS in the KRG, as well as the construction of the Central Wastewater Quality control laboratory in Baghdad.

Work related to the construction/expansion of the Central Sewerage Laboratory building has commenced and is currently operational.

The hygiene awareness campaigns targeting 4,000 community and religious leaders have had a direct impact on raising the awareness and are expected to improve hygiene practices of the communities leading to the reduction of water-borne diseases among children.

However, regarding the procurement and delivery of 7 water quality monitoring mobile laboratories (3 for Ministry of Environment, 2 for Ministry of Municipalities and one each for Mayoralty of Baghdad and Erbil) and had become operational or in the process of being operational.

The project was in-line with the priorities identified in the National Development Strategy (NDS), 2005-2007 with regard to improving quality of life of the Iraqi people with special emphasis on water and sanitation. Also the project contributed towards attaining the MDG goals in particular to Goal # 4 = Reduce Child Mortality, Goal # 7 = Ensure Environmental Sustainability and Goal # 8 = Develop Global Partnership for Development.

According to the evaluation results, the main challenges encountered during the project implementation were:
 The unstable security situation affecting movement of staff.
 Delays by the Authorities in formulating their needs and articulating their requirements in terms of type, parameters, layout and frequency of reports generated by the system;
 Frequent changes in the Authorities‟ requirements leading to repeated re-work. This was further delayed by the protracted review and provision of comments by the Authorities to the procurement contractor;
 Lengthy programming process requiring the Contractor to transfer the comments received either on structural layout or statistical calculations of report format to programming language;
 Varying levels of staff skills and differing needs for LIMS structure among the three Authorities;
 The Contractor‟s inability to manage multiple Authorities with their differing needs simultaneously and a number of holidays during the last two months of the year also severely affected the implementation.
 Lengthy UN procurement procedures.

Purpose/ Objectives:

The immediate objectives of the project were:
1. Strengthen the capacity of Ministry of Environment, Ministry of Municipalities and Mayoralty of Baghdad at central, governorate and district levels in terms of planning, implementation and monitoring, main concentration will be improving technical and managerial capacities of laboratories and the staff responsible for water and waste water quality control.
2. Empower the Ministry of Environment, Ministry of Municipalities and Mayoralty of Baghdad to ensure the provision of potable water and the enforcement of the rules and regulation related to the protection of the water resources from pollution.
3. Raise public awareness among communities for the protection and preservation of water resources.

The original duration of the project under evaluation was 12 months starting 22 December 2006, but four extensions were approved, the last of which until 31 December 2009.

The original budget of this project was US$ 4,297,643, of which US$ 2,700,895 earmarked for WHO and 1,596,748 for UNICEF.

This evaluation exercise is part of the UNDG-ITF project evaluation where specific criteria were applied to select some projects for evaluation purposes. This independent evaluation comes at the end of the implementation cycle of the project and aimed to assess the overall contribution of the project towards water safety and quality control programs in Iraq while distilling lessons and good practices to feed into future programming. The evaluation provided recommendations to enhance operational and programmatic effectiveness of similar initiatives in comparable situations. In addition, the evaluation assessed how WHO and UNICEF have contributed towards an enhanced partnership with GOI in addressing critical issues affecting water quality in Iraq.

The evaluation findings will be disseminated to all stakeholders at different levels including decision makers both within the Government of Iraq and the UN to support future policy development especially in the areas of environmental health.

The evaluation findings will serve as an advocacy tool to demonstrate the impact and feasibility of water quality control and surveillance programs. The project evaluation will also provide donors with a comprehensive assessment of the impact and utilization of their investment in these programmatic areas. In addition, the evaluation will support WHO and UNIECF own capacity for programming, project management and accountability towards donors, GOI and the target population. The lessons from the evaluation and the evaluative evidence will also feed into the upcoming UNDG ITF lessons learned process as well as the proposed UNDG ITF project evaluations.

Methodology:

The evaluation addressed the Organization for Economic Cooperation and Development –Development Assistance Committee (OECD-DAC) evaluation criteria including relevance, effectiveness, efficiency, and sustainability. In addition, the evaluation also look at the contribution of the project towards partnership building within UN, GoI and civil society. Specifically, the evaluation had been guided by the following key objectives:

1. To assess and showcase the achieved progress and results against stipulated project objectives and outputs for improved water quality control and surveillance in Iraq;
2. To assess the efficiency and effectiveness of the interventions included in the project;
3. To assess the relevance of project components in strengthening the water quality control and surveillance in Iraq vis-à-vis needs of the target population at the catchments area;
4. To understand the extent to which this project has contributed to forging partnership with at different levels including with GoI the Government of Iraq, Civil Society and UN/donors;
5. To appreciate the management arrangements in place by the GoI and/ or the beneficiary communities towards the sustainability of various project-initiated services and benefits;
6. To assess the management arrangements (Including procurement procedures. coordination, monitoring) in place by GoI and/or the beneficiaries towards the sustainability of various programs/ project initiated services and benefits;
7. To generate lessons on good practices based on assessment from the aforementioned evaluation objectives and to provide recommendations to GoI , UNICEF and WHO on how to maximize the results from similar initiatives in comparable situations

A. Evaluation methodology:
The evaluation process comprised of the following:
Desk review
The evaluation team reviewed the project document, progress reports with focus on UNDG ITF and other documentary materials generated during project implementation to extract information, identify key trends and issues, develop key questions and criteria for analysis, and compile relevant data during the preparatory phase of the evaluation. The evaluation team also reviewed relevant national strategies to see the links between the project objectives and national priorities.

In consultation with WHO and UNICEF, related governmental departments, the evaluation team identified all stakeholders to be included in the evaluation exercise. Once stakeholders were identified, the evaluation team devised participatory approaches for collecting first-hand information. These included interviews, focus group discussions, observations, end-user feedback survey through questionnaires, etc.

Field visits to target districts
Field visits and meetings were held with partner institutions. To the extent possible, beneficiary populations were engaged in the evaluation process to get their feedback and reflection on project benefits.
 Focus group discussions were held with the beneficiaries;
 Questionnaires were used for beneficiaries from the different capacity building activities.

The evaluation teams interviewed and met more than 250 stakeholder and beneficiaries to obtain their feedback and assess their role during the project implementation.
Evaluation Guidelines
In preparation of the evaluation report; due consideration was given to the UNEG evaluation guidelines and the UNDG-ITF guidelines on Development Effectiveness and Operational Effectiveness.
Moreover, the evaluation provided a brief description on the following:
• Key implementing agencies;
• Intended outcome(s) and output(s);
• Underlying logic of project design;
• Key assumptions that guided the design and implementation strategies;
• Any major divergences in the design and/or implementation strategy;
• Monitoring arrangements;
• Lessons learned.

Key Findings:

The Programme related to the UN Assistance Strategy with regard to “Rehabilitating and developing the country‟s social, economic, financial, physical and institutional infrastructure to ensure sustainable livelihoods and durable solutions to displaced populations in the country".
The project also contributed to attainment of the Millennium Development Goals (MDGs), in particular;
Goal #6: Combat diseases, by reducing the risks of contracting diarrhea, infectious hepatitis and enteric group fevers.
Goal #7: Ensure environmental sustainability, by controlling water quality and reducing environmental pollution.
Goal #8: Develop global partnership for development, by sustaining the close partnership between UN organizations and the GOI.

Furthermore, the project related to the National Development Strategy, through Goal 6 which is to “Achieve universal access to safe drinking water and sanitation” and to Target 8 in the Strategy which is to “Ensure that all people have sustainable access to safe drinking water and improved sanitation.”

The project also contributed to article 4.4.1.5 of ICI Benchmarks (as per the Joint Monitoring Matrix 2008) - Reduce population without access to safe drinking water to 10% in Urban areas (Baseline: UNDP Unmet Basic Needs Survey – 40.4% population without access to safe drinking water of which 20% in Urban and 60.4% in Rural areas); and also contributes to the following water and sanitation sector outcomes –
1. Increase sustainable access to safe water for urban and rural populations in 10 governorates, especially for the vulnerable.
2. Increase sustainable access to improved sanitation for urban and rural populations, especially for the vulnerable.
3. Capacities for effective management of water and sanitation enhanced.

Recommendations:

Recommendations for the main stakeholders:
1. Notwithstanding that the GoI is currently embarking on a large scale development programme of the country‟s infrastructure nevertheless a comprehensive approach to the water quality problem should be pursued based on adopting a twofold developmental strategy to expand and rehabilitate the country‟s basic infrastructure of water and wastewater facilities on the one hand, and to reinforce regulations and implement an effective system for prevention and control of high risk practices that contribute to contamination of natural water resources on the other hand.
2. Maintain the joint WHO/UNICEF efforts to support GOI for Enhancing Water Quality Control and Surveillance in Iraq, addressed the consequences of water unsafely and insecurity.
3. Related GOI departments should develop, plan and implement proper maintenance of water and wastewater quality laboratories. Experience from many developing countries suggest that in many instances, sophisticated equipment could easily be exposed to damage and become beyond economic repair if not utilized and regularly maintained. This is even more likely in countries plagued by ongoing violence and instability.
4. Public awareness and community participation in protection of natural water sources, abiding by good hygienic environmentally-friendly practices make a significant contribution towards maintaining water and sanitation facilities. Concepts of hygiene and clean environment currently integrated within the school curriculum should be further developed and more teachers trained on promotion of environment-friendly towns and villages.
5. The MoEv and MMPW in KRG should ensure that an effective monitoring system is developed and implemented to control hazardous practices of dumping domestic, industrial, agricultural and medical waste into rivers and other natural water sources.
6. The MoEv and MMPW should establish a Task Force comprising representatives from the Ministries of Health, Education, Higher Education, NGOs and civil society organizations in order to develop and implement an ongoing health educational awareness programme to raise public awareness on the importance of maintaining hygienic behavior and avoiding practices that endanger the environment and contribute to disease transmission. The assistance and support of WHO and UNICEF in this respect would be indispensable. The technical assistance of WHO and UNICEF would be desirable in this respect.
7. The process of national capacity building that has been started through phase one and phase two of this project should be continued by GOI in close cooperation with WHO and UNICEF in order to develop national ownership of the water quality control and surveillance activities and reverse the current trend of morbidity and mortality from water-borne diseases.

Lessons Learned (optional):

1. The urgent needs and priorities in such projects should be clearly defined and agreed upon between major partners ahead the start of project in order to avoid delays and setting incompatible priorities during project implementation.
2. Inter-ministerial coordination at the national level is fundamental for implementation of developmental activities and achievement of the contemplated objectives. Disputes and disagreements within national partners affect their credibility and pose serious threats for the future sustainability of projects implemented with the support of UN agencies.
3. Public awareness and community participation in activities aiming at promoting hygienic practices and avoiding hazardous practices that involve serious risk to human life and natural resources should be an integral component of any development project seeking to improve water and sanitation.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search