Evaluation database

Evaluation report

2002 ZAM: Mid-Year Review of School Sanitation and Hygiene Education Project



Author: Dooley, T.; Michelo, L.

Executive summary

Background:

The SSHE project evolved out of the previous 5 years work in the target Districts under the WASHE programme. This programme laid the foundations for SSHE, which sought to strengthen the impact of the WASHE interventions and increase the focus on Children. During the implementation of the WASHE programme, conditions of sanitation facilities in primary schools were found to be very poor. Most of the schools lacked appropriate sanitation and hygiene-enabling facilities, and there was little or no emphasis placed on health and hygiene education within the school.

In the context of this challenge, UNICEF supported the Government of the Republic of Zambia to develop guidelines on the integration of health and hygiene education in the school curriculum and the training of teachers on the use of these integrated guidelines. In addition to this, support was provided for the construction of latrines and other hygiene-enabling facilities within the schools and surrounding communities. The project has five main strategies for implementation and they are: to increase access to safe water; to increase sanitary means of excreta disposal; to improve sanitation and hygiene practices in schools and communities; sustainability and capacity building strategies. While, direct support is provided to promote sanitation and hygiene in schools in the four districts, focus is also placed on villages and communities surrounding the schools. The project is implemented through the D-WASHE committees at district level, while NGOs are also actively involved in the process.

Purpose/Objective:

  • Assess project relevance
  • Assess cost effectiveness
  • Assess project sustainability
  • Assess project impact
  • Assess the programme management of the two approaches (UNICEF/D-WASHE and UNICEF/NGO/D-WASHE) to draw lessons for replication in similar future projects
  • Assess efficiency of service delivery (including the training and training materials)
  • Assess the effectiveness of the monitoring and evaluation system
  • Determine whether all the activities are necessary, are they sufficient, and is there an efficient monitoring and evaluation system?

Methodology:

The review was formative in nature and utilised a synthesis of epidemiological, anthropological, communication and participatory approaches aimed at providing both qualitative and quantitative information on the project progress and outcomes. Initially, information was gathered through a desk review of relevant documents and consultations with government departments at national level, UNICEF, Ireland Aid and various NGOs working in the sector. Following the selection of sites, meetings and discussions were undertaken with district teams, extension workers, school bodies, school children and recipient communities.

Data collection techniques included, key informant interviews at national, province, district, school and community level, focus group discussions with different stakeholders, observations and utilisation of various participatory techniques to assess knowledge and behaviour patterns. Additionally, to the extent they were available, health statistics were reviewed to assess possible epidemiological differences over the life of the project.

Findings and Conclusions:

Achievements
A lot has been achieved during the first two years of the project. In addition to identifying key partners, sensitising schools and communities, developing planning, monitoring and reporting structures, training and capacity building and engaging in national level dialogue on SSHE, the project is progressing very well in achieving its intended outputs and, in some cases, exceeding its planned outputs.

The project has a number of strategies and activities aimed at achieving the overall project objective. The majority of these activities and strategies are very relevant and are vital to the project. A well-defined implementation strategy with a good planning component ensures that the project has a very solid foundation. Combined with this is the development and strengthening of capacities at all levels to ensure that the necessary skills are available for all the various components of the project. While, a service delivery component that provides for the delivery and/or availability of basic services and materials for the construction of sanitation and hygiene enabling facilities is also very positive.

Relevance
However, the review team felt that some of the activities currently being promoted and implemented, particularly at community level, are not immediately relevant to the achievement of the project objectives. The original project design is good as it aims to ensure that basic WASHE needs are met in schools and surrounding communities. These basic WASHE needs are described as: sanitation facilities; hand washing facilities; drinking facilities; refuse pits and dish racks. But, currently, the project is also promoting tree planting, composting, fuel-efficient stoves, insecticide-treated bed nets and nutrition gardens. There is real evidence of an overload of activities, particularly at household level, and it would be better to streamline and encourage the prioritisation of the basic WASHE activities at school and community levels. It would be more appropriate to identify specific activities and steps to be taken within a given time frame; once these activities have been undertaken and achieved, a new series of activities could then be promoted, planned for and implemented.

Cost Effectiveness
In terms of district-level activities, the project is very efficient; project costs, support costs and service delivery have all been defined. One issue, however, which should be considered, particularly in relation to the provision of sanitation facilities, is that the cheapest option is not always the best option and longer-term sustainability of the structures must be considered. This is particularly relevant to school sanitation facilities due to their high level of use; sanplats may well be the cheapest option for sanitation facilities but, within schools, the superstructure is also vitally important. Thought should be given to increasing the level of support for the construction of sanitation facilities within schools so as to ensure that they are safe and durable, and can be properly maintained.

Project Management
Overall, the project is very well managed, particularly in relation to service delivery. Systems and strategies are in place to ensure effective and timely delivery of services at all levels. However, its major weakness lies in developing community management skills. In most cases, there appears to be a reluctance to allow the communities to manage the project themselves. The processes currently used are quite controlled, with the communities and schools just planning for pre-determined facilities. The role of the PTAs also remains unclear, particularly outside the school environment; there is currently little or no evidence of their role in village-level activities and this is an area that needs to be strengthened so as to strengthen overall community participation and management in the processes.

Capacity Building
The project has a very detailed capacity building component, much of which was started under the WASHE Programme. There are many levels and types of trainings included in the project and these range from the training of local masons and pump minders, through training of ACOs and EHTs to building the capacity of the D-WASHE. Training covers many topics at all the various levels and includes technical training on the construction and maintenance of water and sanitation facilities, communication and mobilisation skills, and planning and monitoring of activities.

The major gaps identified, both in terms of capacity building and the tools being used, relates to the omission of a number of steps in community capacity development. The project has a tendency to start identifying solutions within a community/school before the opportunity is given to these groups to identify and analyse the problems.

Technology Choices
The predominant technology choice for water supplies at district level is Boreholes, and some well rehabilitation. This is understandable given the current lack of rainfall in the region; however, where possible and practical, other options should also be investigated. The issue surrounding sanitation options has been widely discussed and researched under the project and the standard option is that of a Sanplat. At school level, however, more widespread discussion is needed so as to ensure that the structures are adequately completed and this may involve the provision of additional cement. What is important within the school environment is that the structures are well built, safe for use and capable of being thoroughly cleaned internally.

As malaria poses a very large threat, particularly to children within the target communities, mosquito nets are provided and sold under the project. This is a very positive step towards reducing the incidence of malaria within the area and should be expanded where possible. However, this component of the project requires a very defined strategy so as to ensure that the nets are both initially treated and that re-treatment takes place. Currently, various strategies are implemented in different areas and it is difficult to determine whether the nets are being treated and properly used or indeed if they are reaching the intended target groups. In order to address this issue, it is vital that a strategy be developed that also has a community education component.

Hygiene Education
The role of hygiene education in the overall project context needs to be more clearly defined. Fundamentally, this is a behaviour change rather than a service delivery project. The differences between knowledge and behaviour change needs to be addressed and the new mechanisms being put in place by the project should help to address this issue. The ACOs and EHTs need to be strengthened and supported in this area, especially in how they approach communities in terms of hygiene education. Without a behavioural focus, the project runs the risk of moving from assessment to action without providing the communities with the opportunity to analyse their situations and relate their practices to health improvements.

Monitoring and Evaluation
Extensive baseline studies have been undertaken as part of the overall process; there is a wealth of data available on schools and communities in each of the four districts and such information is exceedingly beneficial for planning and advocacy purposes. However, the information urgently needs to be decentralised to the districts so that they can use it for planning purposes and for reinforcing the reasoning behind site selections during DDCC meetings.

There is a very detailed monitoring system in place and UNICEF has a very comprehensive financial and supplies monitoring process at all levels. The reporting structures and data gathering techniques have also been well developed. However, given the fact that the project is reaching the conclusion of its initial phase, now might be an appropriate time to verify some of the reports provided by the districts and DAPP. Community-based monitoring systems should also be developed so as to enable communities to determine their own progress and monitor their own activities.

In order to assess impact, it would also be important in future areas to include some epidemiological data in the baseline surveys. This could be in the form of disease statistics from the clinic/health centers or through self-reporting during the household surveys. Additionally, school records on sickness or absenteeism among students would be beneficial. Such information would greatly assist during project monitoring and evaluation exercises in helping to determine impact.

Sustainability
Currently, the project utilises existing structures such as D-WASHE, V-WASHE, EHTs, ACOs and PTAs for project implementation purposes and this is a very positive strategy, which will help to strengthen these structures and encourage greater sustainability. In some instances, such as the interventions being supported through NGOs, extension agents are supported to implement the activities at community level; while this helps with implementation, its impact on longer-term sustainability is questionable.

While advocacy was undertaken during the initial phases of the project, due to high staff turn over and changed structures, this activity should now be reinforced at all levels as it will have an impact on project planning and commitment. Given the lessons learned over the past two years, such advocacy activities should be addressed to a multi-sectoral group and incorporate some of the positive health impacts that can be achieved through SSHE and not solely be directed at improvement in girls' participation in school.

Constraints

  • Late release of funding to districts is causing difficulties in relation to the availability of communities for participation in the project due to seasonal and agrarian seasons.
  • Transport at district level remains a problem. Project personnel encounter major difficulties in accessing transportation for monitoring purposes.
  • Delays are experienced in finalising annual work plans within UNICEF and this has a knock-on effect at district level.
  • There is a need for greater consultation with the districts concerning funding allocations and budget changes, and district feel that they should be given flexibility concerning the allocation of resources within their district.
  • Capacity building and training is seen as a once off event and the quality and impact of the training is not monitored.
    - Heavy rains in some area during 2000 resulted in many of the latrines collapsing and this undermined community confidence in the structures.

Recommendations:

The project should consider consolidating its current activities rather than expanding too much at this point in time. It should be remembered that this is a pilot project and the lessons learned and experiences gained through its implementation are vital for future expansion, both within the target districts and beyond. The lessons learned (both positive and negative) need to be reviewed and, where necessary, changes made during the final phases of implementation so as to strengthen the overall process and provide direction for future development of SSHE in Zambia.

The project should strive to be more health- and community-focused, therefore, the primary steps should be hygiene education and community mobilisation, with the actual provision of facilities being secondary and seen as complementary and supportive of the overall hygiene initiatives. With decentralisation, community-based planning (VAPs) for water and sanitation now operative in the districts, it is vital that school sanitation and hygiene initiatives are not developed separately, but integrated as part of the overall process at community level.

Schools should not be separated from their surrounding communities and complementary activities should be carried out both within the school and the community. Existing structures and extension agents should be utilised during implementation and emphasis should be placed on strengthening these. A major focus of the project should continue to be on skills transfer at local level, to ensure expansion and sustainability of the resultant behaviour changes. School and community activities must be co-ordinated and undertaken simultaneously and not separately as is sometimes the case with the current NGO/D-WASHE strategy.

In line with this, the overall process should be more integrated and multi-sectoral at sub-district level for SSHE is not just an education or health issue but rather an overall developmental issue. Therefore, all resources available within the target community should be utilised to the maximum possible extent and all sector agencies within an area should be aware of the resources and capacities of their sister agencies. School sanitation and hygiene education should be inclusive in the overall DDCC and D-WASHE plans in a district. Sub-D-WASHE extension agents such as those involved in community development, social welfare and agriculture should be oriented to the process.

Major opportunities exist for the development of school communication strategies for hygiene and overall health behaviour change. Experiences gained in the use of participatory methods and materials for behaviour change through the WASHE initiative should be built upon and combined with the Child-to-Child and Life Skills approach. Activity suggestions for teachers provide a great opportunity for really impacting not just on behaviour change, but the overall health of the school population.

Health and hygiene education, while included in the curriculum, should be developed into more than just an academic subject. Bearing in mind that teachers already have a very busy teaching schedule, teaching aids or activity sheets dealing with health and hygiene issues should be developed for use within the school. These should focus on a pupil-cantered approach and stimulate action based on learning.

IA/UNICEF should continue their support to this initiative as not only is it impacting on the target communities, but it is also proving to be a very valuable learning and developmental process for SSHE in Zambia. The lessons learned through the implementation of this initiative are extremely important for the whole sector.

Lessons Learned:

  • Baseline information on the target areas is essential. Such information, in addition to proving coverage and access data, should also try to include health profiles at schools and within communities.
  • Process and output indicators should be developed at the beginning of the project and be routinely monitored.
  • Project strategies should clearly relate to the overall project objectives.
  • A clearly-defined SSHE policy is essential and implementation guidelines should be developed that relate to overall policy.
  • Advocacy and information exchange is an essential component of the process at all levels.
  • A multi-sectoral approach is necessary and all key players should be identified, orientated and mobilised. Partnerships should be strengthened and SSHE should be integrated with other ongoing activities.
  • Existing structures and personnel should be used at all levels. The role of the DEOs should be clarified.
  • Training and capacity building needs should be determined and appropriate training provided to the identified groups/individuals. There should be an effective system for follow-up and support following training.
  • The quality and impact of training and capacity building should be closely monitored.
  • Appropriate training and orientation packages should be developed for all the various components.
  • SSHE should have a behaviour change focus where the provision of facilities is seen as complementary.
  • Hygiene education should be strengthened and appropriate materials and tools developed for both school and community levels.
  • Community Capacity Development is an important component of the process and mechanisms are needed to ensure active community participation in project planning, implementation and management.
  • A variety of technology choices and options should be provided for water and sanitation facilities while standard designs should also be developed.
  • Quality of construction should be closely monitored and facilities should not be recorded by the project until they are complete.
  • Action plans should be developed at school and community levels, and these should be used for project planning and monitoring purposes.
  • Community-based monitoring of both the processes and impact is necessary.
  • Specific materials appropriate for children should be developed and used, while school-specific and school-based activities should be clarified.
  • Guidelines should be developed on SSHE.
  • Activities should be prioritised in terms of the identified risk behaviours within an area.


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Date:
2002

Region:
ESARO

Country:
Zambia

Type:
Evaluation

Theme:
WES - Schools and Institutional

Partners:
Development Cooperation Irland (Irish Aid)

PIDB:
YW603-02

Follow Up:

Language:
English

Sequence Number:
2002/008

New enhanced search