Evaluation database

Evaluation report

2011 Pakistan: Mid Term Evaluation Report: Early Recovery Programme on Rural Sanitation in Nineteen Flood Affected Districts of Pakistan (Phase II)



Executive summary

 

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”

 

Punjab. There are now other districts in KP and Sindh where people have returned and have started reconstruction efforts. The project has four major components i.e. a) institutional linkages & capacity building, b) campaigning for improved hygiene behaviors, c) attaining total sanitation, and d) monitoring, evaluation and learning.

The overall purpose of the mid-term evaluation was to assess the degree to which the program (Phase II) is achieving the objectives and results as outlined in the PCA. The evaluation will particular emphasize on evaluation criteria endorsed by UNICEF i.e. relevance, effectiveness, efficiency, impact, and sustainable of the interventions and extract the lessons/recommendations to enhance the quality of remaining part of the program.

The team consisted of one Team Leader assisted by one Deputy Team Leader, five Team Coordinators and team members. 10 districts from all five regions were selected for mid-term evaluation. Two villages from each district were selected. 5% of the HHs were selected on randomly basis for in-depth interviews with households and 5% children were selected for interviews in the schools located in the villages selected.

 Meeting with IPs

 19 FGDs conducted (11 with men, 8 with women)

 266 in-depth interviews with HHs

 150 structured interviews with children

 41 interviews conducted with IPs staff and government officials

 Field observations

The findings of the evaluation are assessed on relevance, effectiveness, efficiency, impact, sustainability, and recommendations to enhance the quality of remaining part of the program.

 

Relevance:

 

 The project is complimenting the Millennium Development Goals 6 and 7. The project is contributing to National Environment Policy 2005, National Sanitation Policy 2006 and Pakistan‟s Approach to Total Sanitation.

 

Mid-term Evaluation Report-Early Recovery Program on Rural Sanitation in Nineteen Flood Affected Dist. of Pakistan (Phase II) UNICEF, Pakistan Page viii Plan International, Pakistan

 

 The project approach is also flexible as the project is based on Pakistan Approach to Total Sanitation (PATS) using Community Led Total Sanitation (CLTS) and School Led Total Sanitation (SLTS)) as tools to achieve its objectives.

 The flood affected communities lost their sanitation infrastructure and the project is working on the graved need of the communities related to sanitation. The project objectives are relevant with the needs of health and environmental conditions in the target areas. Moreover, the project is also promoting self-help approach among its target beneficiaries through rigorous hygiene promotion and demand creation in the community needs.

 Identification of villages should have been carried out on the basis of open defecation practices or overall sanitation practices of the population. Almost 12 out of 22 villages already had latrines in 60-100% households.

 The project approach on total sanitation from the beginning of early recovery and reconstruction was a good strategy. The project is very effective at early recovery stage as it discourages the dependency syndrome soon after the disaster.

 The project proposal is well designed with good clarity of the project approach. The structured logical framework matrix has not been developed. However, the project‟s progress indicators have been developed which particularly deal with the output. The impact indicators, pre-conditions, and means of verification are not available in the PCA. The project plan lacked buffer time for the project.

 

Efficiency:

 

 All the IPs selected for the implementation of this project has vast experience and expertise in WASH and Social Mobilization. The project management system at Plan Pakistan and at IPs level is well organized with clear roles and responsibilities.

 The project staff interviewed was well aware of the project objectives, implementation approach, activities and target communities. Community Resource Persons (CRPs) are the important project link in the community and evaluation team observed that most of the CRPs in DI Khan and Gilgit did not have the required level of experience and expertise.

 There is good coordination among UNICEF, Plan Pakistan and IPs. The regular reporting and coordination exists for smooth implementation of the project.

 The project activities were generally found well communicated to government officials and line departments. However, in DI Khan (KP) and Thatta (Sindh) it was observed that the project coordination with the respective departments remained less effective. The project review

 

Mid-term Evaluation Report-Early Recovery Program on Rural Sanitation in Nineteen Flood Affected Dist. of Pakistan (Phase II) UNICEF, Pakistan Page ix Plan International, Pakistan

 

committees consisting of district administration and line departments were yet to be formed in some of the project districts.

 The evaluation team observed during the evaluation and discussion with the respective IPs staff that the project activities were not conducted as per the plan. The IPs shared that the delay in activities were mainly due to the late disbursement of the project funds.

 

Effectiveness:

 

 The project effectively achieved 87% of advocacy consultation/orientation targets in the respective districts. However, concrete support from the local administration yet to be ensured through a formal MOU that can provide a solid basis for stronger coordination with local administration.

 The data collected from the household interviews highlighted that most of the villages identified by the project already had latrines. 12 out of 22 villages already had latrines in 60-100% households.

 The project overall has achieved 118% targets for village and cluster mapping. Village wise triggering plans have been prepared for 2856 villages against target 2,244 villages.

 92% respondents of in-depth interviews participated in health & hygiene awareness raising activities and were able to share one-three messages received on health & hygiene.

 IEC material developed was relevant and easy to understand. However, the IEC material was delayed and activities conducted without IEC material. Whereas conducting the awareness raising activities with IEC material might have resulted in better understanding of the illiterate people.

 The children are very active in sanitation and hygiene activities in their schools and WASH Clubs are formed and active in 19 visited schools during the evaluation. 79% of the children interviewed participated in health & hygiene awareness raising activities in their schools. 70% of the children are practicing hygiene and washing their hands after latrine use, and before eating.

 The idea of using street theatres for health and hygiene promotion was generally found effective and appreciated by the community.

 89% of the HHs respondents had the latrines at the HH level. 86% of the HHs with latrines responded that they are using latrine and 3% said that they are not using the latrine

 94% of the HHs responded that they wash their hands after latrine use, before & after eating.

 98% people said that they use different methods to treat their drinking water (boiling, staining, chlorinating, sun method, fountain, covered).

 

Mid-term Evaluation Report-Early Recovery Program on Rural Sanitation in Nineteen Flood Affected Dist. of Pakistan (Phase II) UNICEF, Pakistan Page x Plan International, Pakistan

 

 1The project has been successful in declaring 24% villages ODF by the IPs against the target and 2.16% of ODF villages have been verified as ODF. Moreover it was observed that a small number of people having latrines are still practicing OD in the visited villages.

 The village sanitation committees are active in all the villages visited during evaluation. The VSCs are clear on their role and have established close relationship with CRPs and Social Organizers.

 In all 22 villages visited by the evaluation team had functional village sanitation committees

 Beside strong awareness level of school children towards sanitation the team noted in some of the schools that they are compelled to practice open defecation, either due to lack or non functionality of latrine facility.

 277 latrines constructed by the communities at HHs level after the triggering among the visited communities

 The sanitation marts have not been established yet. As per discussion with the project staff it is in process.

 The evaluation team observed that the laying of sewers and WW collection not initiated in any of the model villages yet. The outflow of the toilets at the moment, in all areas visited, is going into open drains in the streets. With the construction of new latrines, the number will increase tremendously thus increasing the waste. Currently the required attention has not given to the waste management aspect of sanitation interventions. Open drains may become a health hazard with the magnitude of the latrines constructed in any area.

 7 out of 22 villages have the demo latrine constructed. This may have remained a limiting factor to effectively promote the idea of household level latrine to achieve ODF status.

 The demo latrines were located at common places and the priority was given to the places near the residence of the poor people. Most of the target population has limited financial resources and therefore require linkages with the resources available in the respective areas. The communities are following different latrines models not limited to the demo latrines. In some low lying areas the construction of demo latrines and latrines construed by the community after triggering are following pour flush latrine design that is not fully supported with the proper drainage and thus may become a health hazard for the communities.

 

1 Source: Sit Reps of Progress Matrix of All 10 Districts 2 Source: Sit Reps of Progress Matrix of All 10 Districts Mid-term Evaluation Report-Early Recovery Program on Rural Sanitation in Nineteen Flood Affected Dist. of Pakistan (Phase II) UNICEF, Pakistan Page xi Plan International, Pakistan

 

 Project effectively gathered the best practices/case studies and human interest stories. However it appears that documenting key learning, wrong practices, and negative stories were not duly included in the monitoring systems.

 

Impact:

 

 In few areas like AJK and GB the Government institutions are mobilized on sanitation problems in their areas. They are cooperating with the project staff and are in regular contact with the project staff. There is no participation of the government in monitoring of the project activities.

 Awareness level of the people on health and hygiene has been enhanced. The communities were able to share the messages on health and hygiene and were practicing hygiene practices at household level. The communities are also applying different water purification methods to treat their drinking water.

 The children have formed WASH clubs and are regular participating in the activities of the WASH clubs. The health and hygiene practices of children are improved and they are taking care of their personal hygiene i.e. wearing clean clothes and are properly washing their hands after defecation.

 The communities are sensitized and mobilized on sanitation problems and now the people are constructing latrines at household level. The open defecation is reducing in the villages and communities are also identifying solutions to their sanitation problems on self-help basis. In some areas like Bagh and Neelum, the community is mobilized and is disposing garbage properly.

 

Sustainability:

 

 Awareness level of the people on health and hygiene has been enhanced to greater extent Since the project mainly focuses on awareness raising and capacity building shall be sustained and transferred to peers and future generations. The active participation of the students in SLTS will ensure sustainability at personal, school, household and community levels.

 In most of the areas the community is willing to continue the project interventions and to construct latrine with their own resources. The village sanitation committee (VSCs) are capacitated and mobilized to sustain the impact of activities beyond the project period.

 The project approach has been appreciated by different respondents since it discourages the dependency syndrome observed among the communities after disasters.

 The mandate of line departments including PHED, Health Department, Education Department, and Municipal Administration are complimented by the project, therefore, such linkages would ensure sustainability of the project.

 

Mid-term Evaluation Report-Early Recovery Program on Rural Sanitation in Nineteen Flood Affected Dist. of Pakistan (Phase II) UNICEF, Pakistan Page xii Plan International, Pakistan

 

The key recommendations based on the analysis of the findings are as following:

 

 Identification of villages should have been carried out on the basis of open defecation practices or overall sanitation practices of the population. Almost 12 out of 22 villages already had latrines in 60-100% households.

 The design of the latrine needs to be customized suitable for environmental conditions of the areas i.e. water table too high for dry pit latrine in Kohat. The issue of waste water of pour flush latrines needs to be addressed to support this with the appropriate drainage systems according to the area requirements.

 Children will prove to be the major force behind the project sustainability. Availability of latrines at school needs to be ensured to either build latrines or make the existing latrines functional.

 Community Resource Persons (CRPs) are the important project link in the community and evaluation team observed that most of the CRPs in the field did not have the required level of experience and expertise. The capacity of the CRPs needs to be enhanced to a better level.

 The project‟s activities started with some delay due to some strategic activities at Plan Pakistan and now the project still has to complete its targets. The evaluation team is of the view that keeping in view the current progress of the project, the project needs at least 3-4 months to complete its all activities successfully.

 Project monitoring systems needs to be strengthened to ensure that field issues and area of improvements are reported, documented along with the project outcome/impact on regular intervals.

 Media needs to be mobilized and motivated to play their social responsibility towards the issues of national and human interest.

 Further follow up with the government is required at district and provincial level to strengthen the relationship to ensure their active participation as envisaged in the project proposal. Proper MOUs need to be signed with the respective government to ensure clarity of roles and responsibilities of the stakeholders at all level.

 A clear project exit strategy needs to be developed. The community should also be informed about the project exit strategy.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search