Evaluation database

Evaluation report

2010 Lao: Hygiene Education Toolkit Evaluation



Author: Robert McLaughlin

Executive summary

Background:

The population of Lao PDR suffers from low levels of access to improved water supply and sanitation. Data from 2008 indicates a national access rate of 57% and 53% respectively (UNDP 2010), well below the targets set by the Government of Lao PDR (GOL) of 70% of rural communities with access to improved water supply by 2010 (CPI 2006) and the Millennium Development Goals (MDGs). However, progress has been made since the adoption of the National Socio-economic Development Plan (NSEDP) 2006-2010, which places a strong emphasis on the provision of water and sanitation – especially in rural and remote areas.

The lack of water and sanitation infrastructure coincides with a high prevalence of poor hygiene practices, such as open defecation, minimal hand washing with soap, and open waste disposal. A recent study shows the negative impact of unimproved household sanitation on the country’s economy (World Bank 2009). It estimated that in 2006, Lao PDR lost an estimated USD 293 million, or 5.6% of gross domestic product (GDP), with much of this attributed to the negative impact on health.

Overall levels of access to improved water and sanitation are reflected in the countries schools. Only 39% of the nation’s 8,871 primary schools have access to water supply  and 41% have access to a latrine. Less than one third (29%) have access to both water supply and latrines; rates are as low as 13% to 15% in several provinces (UNICEF 2009). The MOE, in collaboration with the Ministry of Health (MOH) has developed a National School Health Policy (NSHP) since 2005, and further revised in 2010, which aims to promote the health of both students and teachers through school-based interventions to improve water and sanitation provision in schools and promote the development of positive hygiene behaviors and related life skills.

The development partner response in support of MOE’s efforts to improve the provision of improved water supply and sanitation is led by UNICEF, though its Water, Sanitation and Hygiene (WASH) program.

UNICEF has been active in supporting water, sanitation and hygiene interventions in schools for well over a decade. The main strategies have included: policy development – as manifested in the National School Health Policy; provision of improved water supply and latrines; and the development of participatory methodologies and learning materials for primary schools to promote positive behavioral change focused on hand washing with soap, water and latrine use, personal hygiene related to the prevention of disease and environmental sanitation. The school is seen as a key entry point for spreading hygiene and sanitation messages to families and communities through children.

The methodologies and learning materials are presented in the form of the “Learning with Joy Kit”, which is popularly known as the “Blue Box”. The Blue Box contains a variety of learning materials, including posters, story books, games and songs. A teacher’s manual is included with instructions as to how to organize activities and use the materials. It also suggests core curriculum lessons to which the activities can be linked. In addition, a reference book for teachers is included, with more in-depth information on hygiene, sanitation and disease prevention.

The first edition of the Blue Box was developed in 1996. Since then, several revisions have been. Its current form was finalized for use in the 2004-2005 school year with substantial inputs from the World Health Organization (WHO). It was at that point officially recognized by the MOE as standard supplementary teaching material toolkit. Since its inception, the over 3,000 Blue Boxes have been distributed to more than 1,700 primary schools. This distribution has been linked to a three-day teacher in-service training course to introduce teachers to the materials and methodologies.

Purpose/Objective:

Since its introduction, the only assessment conducted was in 2001. The assessment recommended areas for improvement of the Blue Box, which were taken into consideration during the final revision. In addition, several human interest stories and a case study have been developed over the years that have served as a means of showcasing its effectiveness. However, the Blue Box has never been subject to a dedicated evaluation.

UNICEF is currently working with GOL counterparts to develop its next Country Program Action Plan (CPAP).  The WASH program sees it as an important opportunity to evaluate the effectiveness of the “software” component of its support to schools in order to incorporate recommended changes into the new CPAP. The policy environment as also changed, necessitating a shift in strategy for UNICEF. The new National School Health Policy has a much broader focus, incorporating the whole cycle of general education rather than just primary schools, whereas the Blue Box targets only primary schools. The MOE has also adopted the Schools of Quality (SOQ) approach, supported by UNICEF’s Basic Education and Gender Equality (BEGE) program. SOQ takes a holistic approach to school development that incorporates improvement of the physical environment – including hygiene and sanitation – with a comprehensive in-service teacher training program to promote participatory, activity-based teaching and learning and a strong focus on developing school management capacity and promoting a participatory approach to community involvement. SOQ has also developed a teaching and learning supply kit for each classroom, which brings into question whether the Blue Box – as a stand-alone toolkit – should remain as such or be incorporated into the more comprehensive package of learning resources provided as part of the SOQ package.

The overall objective of the evaluation is to provide information on the effectiveness of the implementation of UNICEF-support hygiene education in primary schools under the current CPAP, specifically related to the Blue Box and accompanying teacher in-service training. The evaluation is expected to help create a better understanding of the quality issues and needs related to the implementation of hygiene education in schools and provide recommendations for future improvement of both the Blue Box and supporting interventions in order to bring about positive behavior change.

Methodology:

The evaluation used a mixed-method qualitative and quantitative approach to data collection in an attempt to gather information from a variety of perspectives and in a variety of ways in order to address the range of evaluation objectives. Most of the data collection methods were employed in a purposively selected sample of 15 schools, with a mix of intervention and non-intervention schools surveyed. Telephone interviews were conducted with school directors and teachers from a separate list of 88 intervention schools.

A desk review of key water, sanitation and hygiene policies and programming was also conducted along with informal interviews with key informants on policy-related issues. In addition, a workshop was organized in which MOE and MOH officials participated in a structure exercise to compare the hygiene information and teaching methodologies presented in the Blue Box against the newly revised pre-service textbooks and primary school teacher manuals for the World Around Us subject. Finally, a visit to a Teacher Training College (TTC) was conducted during which informal discussions were held with trainers and trainees, along with a brief demonstration by the trainees on use of the Blue Box materials.

Findings and Conclusions:

In many ways, the introduction of the Blue Box has resulted in significant achievements. Many attractive teaching and learning materials that are popular with teachers and students have been distributed to targeted schools throughout the country. Development partners have joined together with UNICEF and the MOE to increase coverage to areas not targeted for UNICEF support.

Teachers reported high levels of satisfaction with the materials and use them often, both for lessons in the textbook and as extra-curricular activities or during free time. Competencies in using the materials are often related to overall teaching practice, which in general remains traditional, rather than the materials themselves. Teacher observed mostly engaged in whole class lecture and question and answer techniques. Many also incorporated group work, but this was generally limited to having students write down answers found in the textbooks to questions written on the board.

Students are rarely given the opportunity to handle the materials themselves. For the most part teachers were observed calling a small number of students to the front of the class to conduct an activity while the rest of the class observed. In some cases this is due to limited materials. For example, the Snakes and Ladders game is not provided in sufficient quantities to be used in group work.

Many of the Blue Box activities are suggested for extra-curricular use, which many teachers find difficult to make time for. Also, many of the activities are not leveled, i.e. are not designed for a specific grade level; rather, they are suggested for use in all grades.

The most popular materials are the posters, which were seen widely used, the “Do and Don’t” cards, Snakes and Ladders and the various story boards and books. However, most schools report that over the years, many of the materials have been damaged or lost and there has been limited resupply.

Though teachers report improved personal and environmental hygiene practices among students, interviews with students and observation of their fingernails indicated that hand washing with soap is not common. Use of sanitation facilities by students is also rare, with most preferring to use the bushes instead. Student’s knowledge of hygiene-related content varies but in general is still rather limited.

A major problem is management of the school environment. Many schools have litter and feces in the school yard. Very few have adequate fencing to keep out animals. Toilets in general are not well-maintained. School clean-up days are often held on a weekly basis rather than teachers promoting daily practices to keep the schoolyard clean. Many teachers point to the lack of equipment and supplies for maintaining personal hygiene (e.g. soap, buckets, and ladles) and environmental hygiene (waste bins, cleaning supplies for the toilets). However, schools implementing the Schools of Quality (SOQ) approach have much a much cleaner school environment and more organized, daily hygiene routines.

An issue often reported by teachers is lack of demand from communities for hygiene promotion. Though schools and communities work together on a variety of issues, hygiene is not “on the agenda” – with the exception, again, of SOQ schools. Teachers reported needing support from district education officials to learn how to mobilize the community in support of school health and hygiene activities. In one school, a local association, Nor Mai, had provided support to enhance school and community efforts in this area. A teacher at the school said their methodologies reminded him of the SOQ, which he had experienced in a
previous school he had worked in.

Perhaps the main success is the degree to which the Blue Box has influenced the primary curriculum. Experience shows that supplementary materials are generally under-utilized in an environment in which teacher have limited qualifications, resources, and time necessary to devote to activities outside the core curriculum. The approach of introducing the Blue Box as a supplementary material, but with links to the core curriculum, has facilitated its eventual incorporation into the core curriculum.  At the same time, the approach taken has had its limitations. These limitations are mostly related to the overall conditions of schools and the limitations of teachers in terms of pedagogical abilities, material resources and time available for preparing teaching and learning materials and co- and extra-curricular activities.

The Blue Box has been introduced to teachers with a “one-off” training with little or no follow-up support or replenishment of supplies as they are worn out or lost. It has been a discreet activity, for the most part not integrated with a wider system of support for classroom teaching and learning as well as whole-school – and whole-community – management of hygiene education and systems for ensuring the school environment supports the behavior change which the teaching and learning activities promote.

The challenges moving forward will be the integration of hygiene education with the school curriculum – not only in terms of what is taught in the classroom, and how it is taught – but how schools and communities can be supported to join together in promoting a school environment that is clean, healthy, safe and protective – and thus conducive to the development of positive hygiene and sanitation behaviors among students.

Recommendations:

The section is organized around specific issues arising from the findings linked with recommendations for future UNICEF WASH in schools programming.

Teaching and learning materials:
• Conduct a thorough review of all Blue Box materials in terms of appropriateness and coverage of relevant topics. Consider: level of difficulty; presentation format; material used for printing and quantities provided.
• Revise Manual and Handbook to reflect elements incorporated into pre-service training textbooks and primary teacher manuals. Incorporate guidelines for school management to improve the supporting conditions necessary to improving hygiene and sanitation practices in schools.
• Consider possible additions related to other subject areas into which hygiene and sanitation messages can be integrated, such as Lao Language, first using the supplemental material strategy but with the eventual aim of inclusion into the next round of curriculum revision.
• Consider developing additional materials for use in secondary schools in support of the new broader focus of the NSHP.
• Consider integration of the Blue Box materials into the teaching and learning kits provided in support of SOQ.

Training and follow-up support:
• Integrate teacher in-service training in hygiene and sanitation with the SOQ teacher training package.
• Review current supervision guidelines for PES and DEB staff – in particular pedagogical advisors – in support of classroom teaching and learning, school management and community participation in support of WASH and revise as necessary to ensure water supply and sanitation issues are monitored and issues within the capacity of the pedagogical advisors while those technical issues requiring MOH support are directed to the responsible district and provincial CNS officials.

School-community linkages:
• Future WASH programming should take a more community-based approach which includes the school a part of the community.
• Any WASH in schools activities should be linked with UNICEF support to the education sector and/or development partner inputs that can ensure more intensive and long-term support to schools.
• Increased partnership with a broader range of stakeholders such as non-government organizations, local associations working with schools and the private sector should be explored in order to provide on-going technical support to schools in WASH-targeted communities that are not covered by the SOQ initiative.

Water supply and sanitation construction and maintenance:
• Develop new designs for water supply and latrines that are appropriate to the geographical environment in which the school is located to ensure that adequate water supply is provided throughout the school year and that water points are located at the point of use.
• Develop guidelines for maintenance of water supply and sanitation facilities at schools and procedures for reporting problems to the DEB so follow-up support can be provided by the district CNS.

Supplies and equipment:
• Consider expanding the package of materials to include basic supplies required for maintaining a healthy school environment.
• Advocate for the inclusion of expenditure line items in planned community grants to be managed by the Village Education Development Committees (VEDCs) for hygiene and sanitation activities including installation and maintenance of water supply and sanitation facilities.

Demand creation and advocacy:
• Increase focus on IEC campaigns aimed at both schools and communities to increase demand for improved water supply and sanitation infrastructure and support improved hygiene and sanitation practices. Consider the use of traditional entertainment-oriented programs to reach rural and remote communities.
• Continue advocacy efforts to ensure that the provision of improved water supply and sanitation is explicit in MOE policies and development partner support programs.
 
Institutional arrangements:
• Ministry of Education: The current School Health Taskforce is based with the DPPE, and includes officials from Department of Teacher Training (DTT) and the National Research Institute for Educational Science (NRIES). This is a result of the predominantly classroom-based approach of current interventions to promote behavior change through interventions in primary schools. The new NSHP includes pre-schools and secondary schools in the policy framework. Key departments and centers with roles related to both the implementation and monitoring of WASH in schools – from pre-schools through secondary schools –
are currently not a member of the Taskforce, such as the Department of Secondary Education (DSE), the Department of Planning and Coordination, the ECDM, the Educational Statistics and Information Technology Center and the Educational Standards Quality Assurance Center. These should be considered for inclusion in the Taskforce.
• Ministry of Health: The Information Education Communication and Monitoring Evaluation Section of the Administrative Division should be responsible for devising and supporting approaches for IEC and community participation in WASH in schools. As such, the section should take a lead technical role in the development of IEC materials for use in communities and schools, in consultation with the MOE, in particular with RIES with regards to materials related to the school curriculum, and the DPPE, DSE and DTT for pedagogical issues, i.e. their use in the teaching and learning process. The section should also take the lead in monitoring and evaluating WASH in schools, in coordination with the relevant MOE departments. This includes providing input into standards for data collection on hygiene and sanitation for the education Management Information Service (EMIS). The Rural Water Supply Division should take the lead in setting technical standards for water supply systems for schools in coordination with the MOH, specifically, the ECDM, and development partners. The Environmental Health Division should take the lead in developing appropriate school sanitation systems and ensuring water quality and waste water management standards are maintained.
• Mass organizations: It is also recommended that relevant mass organizations such as the Lao Women’s Union and the Lao Youth Union be considered for inclusion in composition of the revised School Health Taskforce due to their key roles in community mobilization.
• Taskforce profile: The wider range participation of participation within would result in a review of who the Taskforce reports to: possibly a department or center with a broader mandate than the DPPE, or a vice-minister. This would entail the two ministries entering into a Memorandum of Understanding (MOU) to clarify institutional arrangements and elaborate on respective roles and responsibilities outlined in the NSHP. The MOU should include mechanisms at the central, provincial and district levels to ensure coordination between the two ministries and set out procedures for, among others: targeting of schools; review of school construction and renovation plans to ensure water supply and sanitation is adequately addressed; monitoring and feedback mechanisms to ensure schools receive adequate technical support for both “software” and “hardware” components, particularly related to the establishment and implementation of systems for the maintenance of water supply and sanitation facilities.

Raising the profile of the Taskforce may result in increased coordination between the MOE and MOH as well as providing a platform for mobilizing increased support from development partners.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2010

Country:
Lao People's Dem Rep.

Region:
EAPRO

Theme:
Water and Environmental Sanitation

Type:
Evaluation

Partners:
Ministry of Education, Lao PDR

Language:
English

New enhanced search