Evaluation database

Evaluation report

2013 Sierra Leone: Evaluation of the WASH Sector Strategy “Community Approaches to Total Sanitation” (CATS)



Author: Cédric Estienne & Rebecca Scott

Executive summary

"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report."

Background:

In Community Approach to Total Sanitation (CATS) the goal is not to convince households to construct latrines, with the possible help of a subsidy, but to obtain a community commitment towards behaviour change around defecation (stopping open defecation and keeping a clean and healthy environment) and leave the lead to the community on how to reach that goal. It usually requires that households construct latrines, but they have to find the most suitable way for them to solve what has been acknowledge by the community as an issue for all.

CATS include a number of approaches such as Community-Led Total Sanitation (CLTS), School-Led Total Sanitation (SLTS), Total Sanitation Campaigns (TSC), etc.

Initially developed in South Asia countries, most common CATS were more recently introduced in Africa (around 2008), but spreading quickly and with very interesting perspectives in many countries. Over the last five years, many sector stakeholders (including central governments and partner NGOs or agencies) have been engaging with or are aware of the “total sanitation” approach and some countries are in a position to adopt CATS principles as their sanitation sector strategy (at least for rural areas).

CATS are now one of the 14 focus of 2012-2013 corporate evaluation priorities for UNICEF. As part of UNICEF’s evaluation policy, 2012-2013 scope of work The UNICEF Executive Board now wants to evaluate CATS and its possible roll-out in more UNICEF-supported country programmes. At this stage an evidence-based evaluation is needed to acknowledge the efforts made to date, assess the impact and contribute to global learning to be fed into the design of future interventions. After 5 years of developing the CATS concept and implementing CATS programmes, this corporate evaluation is a very important moment to look back at what has already been done, draw all the possible lessons and prepare the ground for a possible standardization of CATS.

Purpose/Objective:

The main purposes of the evaluation were:

1. To enable evidence-based decision-making: to link attained CATS results back to the inputs, activities, and performance by UNICEF and other stakeholders, and thereby to determine any changes needed to make national partners and UNICEF more effective at CATS programming and to guide decisions about scaling up or not of the strategy;

2. To contribute to global learning: to make available to the global communities interested in WASH, Education, and other topics the understanding about effective hygiene programming that will emerge, in order that they may alter the programs they support in light of the CATS evidence.

3. To promote accountability: to verify the accuracy of claims made about CATS performance, and to examine the reliability of data used to assess CATS performance, in order to assure internal and external stakeholders of the accuracy of the evidence that is presented and the efficiency of the program they support.

Methodology:

At global level, the evaluation methodology included various complementary tools:

1. Online survey, aimed at UNICEF staff and other actors who are directly involved in CATS implementation in the 50 countries where the approach is used, to build a global view of CATS implementation and challenges.

2. Webinars, aimed at UNICEF’s staff involved in CATS implementation. These will focused on group discussions on selected topics that either arise from the evaluation findings or appear to lead to different point of views between countries or specialists.

3. Semi-structured interviews, aimed at sanitation specialists outside of UNICEF and having a particular experience with CATS.

In country, the methodology included

1. A vast document review related to UNICEF’s CLTS programme in Sierra Leone, as well as national documents linked to sanitation.

2. Individual meetings with key partners and stakeholders at national and sub-national level.

3. Community visits in the field. Five communities that were targeted by the programme were visited. Two of them were ODF, another one was triggered but not yet ODF, and the last one had been ODF but is no longer.

4. One subnational workshop was organised in Bo. Local Government agents from Pujehun, Kenema and Moyamba Districts and a number of implementing partners attended the meeting and provided their feed-back on their experience with CLTS. A significant part of the workshop was dedicated to group working of the participants on specific topics .

5. One national workshop was organised in Freetown on August 21st. It enabled the evaluation team to make a presentation of the early findings and have a discussion with national stakeholders to confirm or amend our findings.

Findings and Conclusions:

1. CLTS is achieving results at-scale in Sierra Leone
In Sierra Leone, CLTS is not only the approach used by UNICEF as part of their WASH programme, but it is now used in major WASH programmes in all the 12 rural Districts of the country.

2. Sustainability of behaviour change is still uncertain
Even if the monitoring system put in place does not give sufficient data on the proportion of households reverting back to open defecation in the post-ODF period, all actors agree that sustainability of CLTS outcomes is far from being guaranteed. Several missing key ingredients have been collectively identified (durable and affordable materials for latrines, continuous monitoring/pressure on households, better linkage with other WASH activities and structures). UNICEF has made interesting and swift adjustments to at least partially address these issues (in terms of extended PCAs, VSLAs, WASH Committees, WASH Task Force at District level, etc.) but most of these adjustments have still to be proven beyond pilot stage, while CLTS is already at scale.

3. Access to safe water is a key missing component
The initial concept of CLTS focuses on a clean and healthy environment for the entire community, including access to safe water as a key ingredient. While CLTS has been used by UNICEF as part of their integrated WASH programme, the different pace and constraints of water supply activities often results in CLTS being applied in communities where a safe water point is not yet functional.

4. Stronger District-Level engagement is needed for future sustainability
Stronger engagement between an operational team of UNICEF staff and IPs and MPs, District Councils and Chiefdom structures is recognized as an ingredient to both enhance coordination within the Districts and to secure clearer roles for all local-level actors beyond the life of the programme.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2013

Country:
Sierra Leone

Region:
WCARO

Theme:
Water and Environmental Sanitation

Type:
Evaluation

Partners:
WEDC, UNICEF, Government of Sierra Leone

Language:
English

New enhanced search