We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 Vietnam: Evaluation of UNICEF Viet Nam Rural Sanitation and Hygiene Programme (RSHP) 2012 – 2016



Executive summary

 With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

Sector Context: Viet Nam is a mid-income country located in Southeast Asia. With a population of 92 million , it is the thirteenth (13th) most populous country in the world. In recent decades, the country has made impressive progress in reducing poverty and improving access to public services including water and sanitation. For instance, access to improved water supply increased from 50% in 1990 to 94% by 2011. Similarly, access to improved sanitation has surged twice i.e. from 30% to 67% for the same time period . The significant policy actions taken include: approval of ‘National Rural Clean Water Supply and Sanitation Strategy 2000-2020’ (revised in 2016); three (03) rounds of ‘National Target Programme’ for water and sanitation (from 1998 to 2015); and approval of standards concerning water quality and hygienic toilets. There are evident inequities in poor  and ethnic minorities (in Central Highlands, Northern Mountains and Mekong river delta) still deprived of services. It is in these areas and groups that UNICEF prioritised to work with . The country envisions to achieve ‘country-wide ODF’ by 2025, and ‘universal access to safe drinking water’ by 2030 respectively. 

Rural Sanitation and Hygiene Programme (2012–2016): Object of Evaluation: A total of 10 national and sub-national (provincial) projects (2 national and 8 sub-national) constitute the UNICEF supported Rural Sanitation and Hygiene Programme (RSHP). Of the two national projects one focuses on rural water and the other rural sanitation, whereas all provincial projects are multi-sector (called Child Friendly Projects) of which WASH is a part. For rural water, the key partner is Ministry of Agriculture & Rural Development (MARD), whereas Ministry of Health (MoH) is the lead partner for rural sanitation. The provinces where projects were implemented include Dien Bien, Lao Cai, Ninh Thuan, Gia Lai, Kon Tum, An Giang, Dong Thap, and Ho Chi Minh City.

Purpose/Objective:

For a balanced focus on both ‘accountability’ and ‘learning’ purpose, the evaluation has been tagged as ‘Summative-Formative’ evaluation. More specific purposes were to: ‘generate evidence of success for RSHP (2012-16), and document learning to inform UNICEF-GOVN upcoming cooperation programme’ . 

The objectives included to: i) generate evidences of Programme’s successes ii) inform UNICEF management of continuity and direction of future WASH assistance iii) contribute to regional and global discourse and learning and iv) map out the extent of slippage i.e. reversal to open defecation free (ODF) communities and assessment of social norm creation of latrine use. On the request of UNICEF management, the evaluators added a commentary on (mostly based secondary sources) WASH-Malnutrition Nexus (for children) in Viet Nam, and the extent to which UNICEF provincial projects have affected child malnutrition. 

Methodology:

Evaluation Design, Methodology, Approach and Limitations: Keeping in view the evaluation objectives and questions, an overarching ‘Theory Based’ evaluation approach has guided the evaluation. Within that a ‘Hybrid Evaluation Design’ comprising ‘Descriptive’ and the ‘Social Norms Theory’ designs have been applied to asses Programme’s contributions. The ‘before and after’ sub-design was used to assess supply side interventions. The standard principles of ‘Social Norms Theory’ have guided the social norms assessment.

‘Mixed-method’ and ‘Participatory’ approaches were applied for data collection, analysis and evaluation management.  Besides the secondary data both qualitative and quantitative methods were used for data collection. The key methods applied included: household survey (Post-KAP), key informant interviews (34 KIIs), focus group discussions (24 FGDs including men, women, boys and girls), field observations, and field photographs. Stratified-random and purposive sampling techniques were applied for respondents’ and stakeholders’ selection. A series of processes were evolved and implemented to ensure compliance with established research and ethical standards.  A Hanoi based research institute and think-thank, DEPCON, was contracted for field work as local partner. The process was participatory as it sought to take views from all key stakeholders (over 600 people including public officials, donors, UNICEF, NGOs, and communities were consulted). An Evaluation Reference Group (ERG) comprising key stakeholders was formed to provide steerage and approve deliverables. 

The evaluation faced some delays subject to some issues with regards to accessing secondary data, securing visa for team, securing approvals for field data collection, changes in UNICEF team, and multiple rounds of feedback to finalise report.

Findings and Conclusions:

Relevance: The RSHP is assessed to be relevant in terms geographic targeting (of under-developed regions and marginalised groups); addressing priority needs of the intended beneficiaries; coherence with the national and UNICEF’s global WASH strategic priorities; and focusing on the institutional capacity gaps of relevant public stakeholders. 

Effectiveness:  Overall, the Programme was assessed to be largely effective vis a vis evolving participatory mechanisms to identify needs, causes and solutions e.g. Bottleneck Analysis exercises undertaken with provinces. The support and approach used to advocate, and support adoption of policy shift for rural sanitation (in terms of replicating the CLTS) proved relatively more effective than one for rural water. 

Efficiency: The RSHP implementation is assessed as efficient in terms of timely completion of planned activities, funds utilisation, and leveraging public sector funds for shared results. The provinces did well to implement activities despite having lag time of 6-18 months between signing of contract and approval of Action Plans. 

Sustainability:  For rural water, the formulation of ‘Water Safety Plans’ is a step in the right direction and bodes well for sustainability. For rural sanitation, the Programme has demonstrated the success of CLTS as an approach which in view of the planned/anticipated scale-up of CLTS is likely to sustain. The other results that may sustain include: ‘National WASH Strategy’ (as guidance document); the redefining of MoH/VIHEMA mandate as the lead rural sanitation implementer; availability of tested training manuals; and improved standards for ODF verification and certification. The enhanced capacities of MoH/VIHEMA are likely to sustain, however they may require continuous support.  The issue of ‘Slippage’ (communities reverting to open defecation in ODF certified communities) is very much present, amounting to 15% of certified communities. 

Recommendations:

  1. Monitoring System Support (Water): Align and harmonize the current monitoring indicators, tools and reporting system for water safety and quality with SDG 6.1 targets and indicators.
  2. Water Safety Products and Promotion (Water): Continue advocacy with MARD to support local businesses and entrepreneurs for innovation and cost reductions for localised water-safety products and additional public-sector investments for BCC campaigning to influence people’s behaviour. 
  3. Research and Innovation (Water and Sanitation): Enable relevant public-sector partners to explore, establish and support partnerships with private sector research entities, academia, and businesses for technology innovation and contextualisation. 
  4. Coordination, Partnerships, Advocacy (Water and Sanitation): Encourage and where required extend technical support to PO MARD to play its due role effectively in public advocacy, sector coordination, increased resources allocation and research and evidence creation.
  5. Resource Mobilization and Provision of Equitable Services (Sanitation): Continue advocating and lobbying with national and provincial governments for approval of ‘Provincial ODF Plans’ and allocation of adequate resources for implementation of plans; also provide technical support to VIHEMA/DoH where required for effective CLTS/CATS implementation. 
  6. Social Norm Integration (Sanitation): Review the successes, bottlenecks and opportunities in the implementation of CLTS/CATS for creation of ‘social norm’ (for hand washing with soap and latrine use) considering the existing capacities of local government tiers and the communities as well. 
  7. Strengthening Monitoring System (Sanitation): Extend technical assistance in revamping of rural sanitation related Monitoring System to enable sector partners draw relevant, usable and timely (monitoring) information and evidences to facilitate informed decision-making.
Lessons Learned:

The key lessons learned centre around: i) The community-based development interventions must attempt to leverage the existing community platforms and the representatives, to benefit fully from their good-will and influence; ii) Social entrepreneurship is critical to achieving social impact; iii) The ‘Integrated (multi-sector) Approach’ for provincial projects has evidently created value in leveraging thematic overlaps, complementarities, and optimisation (of resources); iv) Evidence creation holds key to effective advocacy; and, v) The availability of adequate and qualified team is critical to successful implementation of technical assistance initiatives.

You will find the attachments below labelled as follows:
  • Report - Evaluation report
  • Part 2 - GEROS Assessment
  • Part 3 - GEROS Summary
  • Part 4 - Evaluation Management Response



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2018

Country/Office:
Vietnam

Region:
EAPR

Type:
Evaluation

Theme:
WASH

Language:
English 

Sequence #:
2018/003

New enhanced search