Evaluation database

Evaluation report

2005 Timor-Leste: Assessment of Democratic Republic of Timor-Leste and UNICEF Water and Environmental Sanitation Project 2003-2005



Author: Wan, P.; Cruz, M.

Executive summary

Background

UNICEF Master Plan of Operations (MPO) 2003-2005 incorporates a Water and Environmental Sanitation (WES) Project as part of its Health and Sanitation Programme. The project is implemented in four of the thirteen districts of the country, and targets both selected rural communities and primary schools. The fund allocation was US$ 425,000 for 2003 and US$ 250,000 for 2004. The expenditure was about 70 percent and 90 percent respectively.

Purpose/Objective

The 2005 is the last year of the MPO. It is vital to conduct an evaluation of the project to verify the achievements and to design for the project for the next Country Programme Cycle based on the lessons learned. The objective of the present study is to assess the extent to which the Project has been effective in attaining the set objectives, and to define a roadmap for the future role of UNICEF in the sector.

Methodology

The assessment was conducted based on intensive review of literature, complemented by interaction with key partners in the sector and field studies. Literature review included the various Policy and Planning documents, Plans of Operation and Action, and Technical Guidelines. Discussions were held with staff in UNICEF, development partners including the relevant Government departments, major donor agencies, and NGOs. Seventeen days have been spent in the field, covering the four project districts. The areas were selected by UNICEF to reflect villages with good and less good performance. The initial field visits provided the team with a good basis for the development of Questionnaires for more in-depth field evaluation undertaken subsequently. Eight sets of Questionnaires were developed to elicit information from schools, village leaders, village entrepreneurs, households on use of water and sanitation facilities, and shops in Dili selling water and sanitation components. A total of 15 schools were visited in the four project districts. Eight villages were visited where the village leaders and entrepreneurs were interviewed and the community water systems were observed. In addition, 40 families provided with pour-flush latrines, 15 with water jars and 6 who built their own pit latrines were interviewed. In Dili town, 5 shops were visited in connection with the sale of water and latrine components.

Findings and Conclusions

UNICEF’s Capacity to deliver
The downsizing of UNICEF WES staff in mid-2003 from a 2-person team to one national Assistant Project Officer was highly detrimental to the organisation’s ability to properly develop and manage a good WES programme. This has inevitably affected severely its intention to build the capacity of CWSD and help to build a good foundation to the young national WES programme.

Influencing national and sub-national policies and strategies
UNICEF has been one of the key donor partners in developing guidelines on Community Water and Hygiene and Sanitation Promotion through Schools since their inception in 2001. Some of the technologies supported by UNICEF have been replicated by other agencies, e.g. the use of same moulds for construction of concrete latrine parts. However, with the reduced staff in UNICEF since 2003, the impact of UNICEF in influencing policies and strategies has been minimal.

Effectiveness of the Project
About 6,100 families have been reached through the construction/improvement of dug-wells and gravity-piped water supplies, against a planned target of 6,000 families. Against the same target, 4,184 families constructed sanitary latrines. The facilities provided with UNICEF support have generally been utilised, although some shortcomings were evident. The sanitary latrines provided to the individual household are effectively used where water for ablution and cleaning of the latrine pans is readily available. However, the facilities are not used or under-utilised where collection of water is problematic.

Effectiveness of technologies
Dug-wells. Since the wells are dug manually, the depth to which they penetrate the water table is limited. Hence, at the peak of the dry season, some wells can be dry or have inadequate flows to meet the needs of the users. As dug-wells are open, and water is drawn by buckets, the risks of well water getting contaminated are high.

Rainwater jars
Rainwater jars can be considered as increasing access if the supply from rain meets the overall demand. Providing each family a rainwater collection jar of limited capacity which meets only a 7-day requirement, as observed from the field, has not significantly reduced the drudgery of fetching water since the dry season is very long. For a prolonged duration of 120 days of dry weather, it is estimated that over 15 jars would be required per household, for a domestic demand of 20 litres per person per day. However, if only drinking water requirements are to be met, two jars should be provided for each target family.

Gravity-piped water supply
The cost of gravity-piped water can be high, as described earlier at US$ 330 per household. Another disturbing aspect is the fact that some systems are not functioning because there is no ownership, and community members do nothing when systems break down.
Concrete pour-flush latrine. The cost of providing a latrine at US$ 53.7 per household is quite substantial. With 81 percent of the rural families without a sanitary latrine, or some 45,000 families in the four project districts, the investment required is US$ 2.4 million. The approach is not sustainable. In addition, once the pit is filled up, a similar pit would cost another US$ 24.3. An alternative strategy has been proposed in this assessment.

Village Entrepreneurs and Field Monitoring Assistants
The Village Entrepreneurs have been instrumental in introducing technologies at the community level. Their workmanship is generally of good standard; With the Project shifting from the concept of introducing two VEs per village to two VEs per hamlet since 2003, the cost of training larger numbers increased sharply. Furthermore, after the completion of the activities of the Project, the VEs returned to their past employment, mainly farming, and much of the know-how is not fully applied thereafter. The concept of Field Monitoring Assistants (FMAs) to monitor the activities at the community is good. However, it has been reported that FMAs do not undertake regular field visits to the project areas. The interviews of village leaders also revealed that some FMA were found inadequate. As the FMAs are the only key Project presence at the sub-district levels, they play a critical role in the project implementation and monitoring.

People’s participation
Both males and females have been involved in the Project. The quality and extent of their participation have been determined by the design of the different inputs. In both water and sanitation, people’s contribution was largely in providing labour and locally available materials. They were involved in the initial mapping of the needs of their communities. Their continued involvement is crucial for the sustainability of the inputs and reaping beneficial outcomes.

Promotion of hygiene education
The promotion of hygiene education in the rural communities is very limited. The Village Entrepreneurs are the ones primarily promoting hygiene education in the community, and this activity comes to an end once the physical activities for which they are paid are completed. In the communities, the focus is largely on environmental cleanliness to combat mosquitoes. The development of IEC materials in UNICEF WES is still in the early phase, and requires increased attention. CWSD does not have the capacity to effectively promote a hygiene education programme. However, CWSD can and should play an effective role to coordinate the efforts of other partners, who are better equipped for the tasks.

Intersectoral collaboration
The project has been designed to enhance collaboration among relevant sectors in order to maximise the impact. It has both achieved a good level of integration, including the promotion of WES in primary schools and the planned de-worming of children, as well as convergence in districts with heavy focus on child survival. As the implementation of the Project progresses, it becomes increasing important to monitor the impact of the interventions. Hence, the collaboration should result in joint efforts by the WES, Health and Education sectors collecting regular data on key elements including behaviour and health indicators, such as the functionality and use of WES facilities in schools, hand-washing practices, use of sanitary latrines and incidence of diarrhea.

Storage and distribution of materials
The supplies provided by UNICEF and stored in warehouses before delivery to the project sites are not being properly monitored, as described earlier. Urgent actions should be taken to ensure that the materials are safely stored and accounted for. It is important that the manager of the warehouse be properly trained.

Number of schools provided with WES facilities
Against a target of providing 80 primary schools with water supply and sanitation facilities by end 2005, UNICEF has supported implementation in only 24 schools, with significant reduction in 2004. The reason given was that the budget is limited. The data from EMIS, 2005 shows that over half of the 213 primary schools in the Project districts do not have adequate water and sanitation facilities.

Assessment of water supply technologies
Rainwater collection. Given the long dry season, storage capacity to provide the needs of the students to last the long spell has to be large. If properly designed, the rainfall amounts and roofing areas can supply the requirement. However, this system is generally costly, and has to be considered after careful analysis of alternative sources. In addition, the application of rigid rules on using water and minimising wastage is crucial to avoid returning back to trekking for water. The present norm of providing a tank of 4.5 cubic meters is very inadequate.
Piped water supply. In the present context, the water is highly unreliable as it is from gravity-piped system serving the whole community, and the school has no control of its operation and maintenance. Cases of cutting of pipes on the upstream reaches by farmers to irrigate their lands or water their animals, or by users to facilitate collection of water are not uncommon. These practices are further enhanced when the flow is reduced during the peak of dry seasons.

Dug-wells fitted with power pumps
With the use of an electric pump, the reliability of power supply is a major consideration, as well as the cost of electricity. As rural electrification has reached only 13 percent of the population, diesel or petrol operated pumps would have to be used. However, the regular running costs would be a burden on schools.

Solar pumps drawing from dug-wells
This high-level technology is high cost, but efficient and requires a low level of maintenance. Due to cost consideration, only twenty units were brought in. Solar pumps are expensive, and the non availability of spare parts can disrupt the water supply for a long time.

Design of sanitation facilities
The three type designs are reasonable, with separate units for girls and boys, and urinal for boys. It is noted that the drawings provided to field staff do not indicate the dimensions, although the units have been priced; presently, only verbal instructions are provided to the implementers for construction.

De-worming of school children
The initiation of the de-worming scheme has been delayed. With the expected finalisation of the guidelines in 2005, and the approval of the action plans in the pipeline, the earliest start would be in the latter half of 2005.

Environmental Sanitation and Hygiene practices
The disposal of garbage is generally well taken care of, with the use of garbage pits. Regular hygiene education classes are given; this can be further supported by appropriate IEC materials. The practice of hand-washing is largely determined by availability of water.

Role of the Parent-Teachers Associations
Where PTAs have been proactive, they have played an effective role in improvement the environment of the schools. However, minor repairs to some WES systems that could be undertaken using local skills were not attended to.

Recommendations

Capacity building
• UNICEF should significantly strengthen its staffing capacity in order to play an effective role in influencing national and sub-national policies, guidelines and strategies, collaborate with other aid agencies for harmonisation of programmes, and build the capacity of the government as well as the private sector. It is recommended that at least one international staff and one national staff be recruited at the earliest. The added capacity will also facilitate external resource mobilisation for programme/project funding.
Planning
• It is advised that a district WES workplan be formulated for each district, with the active participation of relevant district officials, jointly led by the district Administrator and CWSD to generate intersectoral collaboration and joint ownership. The workplan should integrate both WES interventions in the primary schools and the community.
Community Water supply and environmental sanitation
• Groundwater. Re-introduce the strategy of exploiting groundwater resources for rural water supplies through the drilling of tubewells fitted with hand-pumps. The collaboration between UNICEF and the government could be on the basis of shared contribution, and progressively with larger inputs by the Government as the programme expands beyond the Project districts to other districts.
• Maintenance of pumps. Building from the past experiences in the country, it is necessary to develop a good maintenance system for the hand-pumps, involving community participation and ownership, ensuring reliable supply of spare parts and good back-up service by CWSD for major pump repairs.
• Rehabilitation of earlier tubewells and pumps. It might be worthwhile investigating the extent to which the large number of tubewells and hand-pumps can be rehabilitated, and the cost effectiveness of such a venture.
• Gravity systems. The design of gravity-piped systems, which is generally quite costly, should be limited to situations where the more cost effective tubewells with hand-pumps are not feasible. The transfer of the system for community management should include transfer of necessary skills and basic tools for maintenance and repairs.
• Rehabilitation of gravity-piped systems. A reconnaissance survey should be conducted on the non-functioning gravity-piped systems with a view to rehabilitate them and empower the communities to manage the systems.
• Rainwater jars. It is recommended that the provision of water jars of 1200 litres capacity be considered in limited situations where no other options are feasible. In such cases, two jars could be provided for drinking water only, and properly managed to last the long dry season. The provision of community rainwater tanks can be explored, provided it can be managed well by the community.
• Concrete latrine parts. It is recommended that the present approach of providing concrete parts to build a sanitary latrine at US$ 53.7 per household be phased out. In order to provide the needs of some 45,000 families without adequate sanitation facilities in the four Project districts alone, about US$ 2.4 million will be required, and this is not sustainable.
• Proposed sanitation approach. Building on the earlier experiences when families were motivated to build their own pit latrines, using local building materials and their own resources, while water supplies were brought to the villages and hamlets, it is recommended that a good social mobilisation and hygiene education drive be undertaken in the communities to motivate families to build their own latrines, and improve hygiene practices.
• Hygiene education. As part of the social mobilisation drive, hygiene education should be given high priority. The motivation of the local leaders as well as community volunteers is necessary to sustain the activity. The change agents should be well orientated, and supported by well tested IEC materials.
Water and environmental sanitation in primary schools
• Survey. A systematic survey should be conducted in schools lacking good WES facilities to analyse critically the rehabilitation needs, prepare estimates and gauge the interest of the headmasters.
• Water supply by tubewells. Tubewells fitted with hand-pumps are recommended where exploitation of groundwater is feasible, thus making the school self sufficient. Otherwise, in cases where water is taken from existing community piped system, this has to be integrated into the rehabilitation of the community system. Where rainwater harvesting is the only alternative, a reduced water requirement can be met by providing a large storage tank.
• PTAs. The Parents Teachers Associations, which are generally quite active, should be motivated to give adequate attention to maintenance and repairs of water and sanitation facilities, and recognise the importance of lifestyle education, beside the academic requirements.
Monitoring and evaluation
• The best formulated programme can prove disastrous if implementation and quality monitoring are poor. A Plan should be developed to facilitate the monitoring of Project implementation, as well as information gathering, by sampling studies, case studies, or evaluations of project components, in order to analyse trends in behavioural changes and health benefits.
Storage and distribution of materials
• Significant amounts of materials have been supplied by UNICEF. Urgent attention should be given to ensure the security of the materials and their proper management, including proper storage and regularly inventory.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Date:
2005

Region:
EAPRO

Country:
Timor Leste

Type:
Evaluation

Theme:
WES- Programme Review

Partners:

PIDB:
YI301-03

Follow Up:

Language:
English

Sequence Number:
2005/001

New enhanced search