2011 Kyrgystan: Evaluation of the Welcome to School Initiative
Author: Dr. Birgitte Woel
“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”
The main objective of this consultancy was to conduct an evaluation of UNICEF Welcome to School Initiative (WTS), define strengths, weaknesses and challenges of the implemented programme with the view to identify which WTS activities will be relevant for the 2012-2016 Country Programme (CP) or for other partners. The report will be used by UNICEF, MOES and other implementing partners who addressed education issues during and after the emergency.
A total funding of US$3,861,931 was received from seven different sources being:
- Basic Education with US$283,775
- Emergency programme funding with US$748,256
- Thematic with US$256,987
- SIDA with US$350,663
- DFID with US$597,676
- Russian federation with US$450,000 and
- The Government of Netherlands with US$1,174,574
The ethnic violence between Kyrgyz and Uzbek people in Osh and Jalal-Abad June 2010 resulted in direct barriers to children’s access to school. Three schools were destroyed – two in Osh oblast and one in Jalal-Abad oblast – and several other schools in the affected areas suffered damages. Refugees from Barack, a Kyrgyz enclave in Uzbekistan, were informally settled in Kyrgyzstan’s Ak-tash village where no school is available.
Displacement caused by the conflict had adverse affects on school enrolment. Families worried about challenges in enrolling children in new schools, particularly for the many internally displaced children. A rapid assessment carried out before launching of the WTS Initiative showed that out of around 500,000 school children in Osh and Jalal-Abad provinces around 83,000 children were affected.
The violence caused an exodus of teachers to other parts of the country and to the Russian Federation, which worsened already existing teacher shortages. Further, many teachers suffered from post-traumatic problems like the rest of the population.
Children witnessed unimaginable physical violence and burning of homes and other properties and severely needed a daily moratorium away from the stressed parents and caregivers and a generally desperate situation. Families lost income and could not provide basics for their families, even less the costs of sending children to school.
There is a range of factors complicating improving of governance and economy in Kyrgyzstan. The country is land-locked and mountainous, with a geopolitical neighborhood best described as difficult. Kyrgyzstan has a relatively well-educated citizenry, a near-universal literacy rate and relatively acceptable resources for health care1. Public expenditure on education has remained between 3% and 4% of GDP, close to 20% of the government’s budget2. Some sources state that the education budget is around 5%3.
PROJECT STRATEGY AND OBJECTIVES
A number of conventions concern human and children’s rights, which have to be addressed also under challenging circumstances. The Kyrgyz Republic (KR) as the signing partner and UNICEF as the UN representative for the mentioned conventions had an obligation to react.
Rooted in the above and in Core Commitments to Children in Humanitarian Action, and grounded in UNICEFs IASC Cluster responsibilities. The overall goal of the WTS Initiative was to minimize the impact of the emergency on children’s education by supporting efforts to ensure that all children have access to quality preschool and basic education through:
ensuring the return to school of all children and adolescents affected by the June violence and advance equity of access to quality basic education, especially for those most marginalized;
providing psychosocial support to children affected by the June violence in Jalal-Abad oblast and Osh city;
promoting safe and tolerant schools, preschools and communities through peace education;
building the capacity and preparedness of schools and preschools to respond to emergencies;
coordinating humanitarian response for Education and ensure that cross-cutting linkages are established with Child protection and WASH clusters to strengthen joint results, reduce duplications and avoid gaps.
The activities was meant to help children, families and teachers recover from the impact of the crisis by creating a certain level of normalcy and create risk reduction and social cohesion through Peace Education (PE). Hence the Initiative was a multi-layer, inter-sectoral and multi-partner initiative.
Acknowledging that the obstacles to school enrolment go beyond the education sector, the one-year strategy took a comprehensive, inter-sectoral approach.
The strategy is reflected in the five objectives, which were implemented through a wide range of activities over three phases: Phase 1 from August – 30 September, 2010; Phase 2 from 1 October to 31 December, 2010 , and phase 3, which was running from 1 January to 31 July, 2011.
POSSIBLE INCLUSION OF WTS ACTIVITIES INTO FUTURE UNICEF ACTIVITIES
UNICEF is planning for a new five-year Country Plan (CP) for 2012-2016 which aims to promote equity. UNICEF works at policy level with the government to improve the social system, as well as on the ground to make sure that these systems reach all children, with a particular emphasis on the most vulnerable. The government strategy aims to include children with various disabilities.
The WTS as concept aiming at transitioning from emergency into development within a frame of complex psychological, social and cultural challenges should be accommodated in CP 2012-2016. The findings from this report will inform the planning work to ensure response to such special needs.
EVALUATION PURPOSE, OBJECTIVE AND SCOPE
In accordance with Terms of reference4 (TORs)The purpose of the evaluation was to find out how some of the successful activities can be mainstreamed into the UNICEF 2012-2016 country programme (CP).
The main objective of this consultancy was to conduct an evaluation of UNICEF Welcome to School Initiative (WTS), define strengths, weaknesses and challenges of the implemented programme.
The evaluation should address issues of project design, implementation, management, lessons learned, replicablility and provide recommendations for current and future projects. The questions to be addressed in the evaluation (provided below) are organised to provide an assessment of relevance, efficiency, effectiveness, sustainability, and impact on the target population.
The methodology consisted of the following considerations and components:
Documents: The consultant was well in advance provided with all relevant documents.5 Figures were used for entry into this report, where other sources did not exist. Finally the documents were used as one of the sources for triangulation of data.
People: The evaluation included the major groups of stakeholders such as:
- MOES from State Ministry to Department of Education, Principals and teachers
- The beneficiaries in terms of students and parents
- UNICEF staff
Site visits: The data collection in Bishkek involved State Ministry, UNICEF Country Representative and UNICEF Communication Department, Foundation for Education Initiatives Support (FEIS), while Department of Education.
In Osh four schools were visited, namely: Filial Lenin, Altybaev School, Sydyk Alaychi School and 3rd. International School6.
It was attempted to apply different methods and meet various stakeholders to allow for data validation.
All interviews were based on the same semi-open questions, which made it possible to compare different experiences with the same activities at various levels.
The same principle was applied for the Focus Group Discussions (FGD). The issue of collaboration was not discussed with the group of beneficiaries. Discussions with beneficiaries included: 2 groups teachers, 2 groups parents (who were all mothers or grand-mothers), 2 groups of students and 1 group of implementing NGOs.
The preliminary findings were presented in a three-hour debriefing meeting7 to stakeholders in Bishkek. The meeting had representatives from key stakeholders, inclusive of MOES, who were active in discussion of findings and recommendations.
The commented debriefing paper, notes and documents received from UNICEF formed the basis for the report writing. UNICEF and MOES gave comments, which were incorporated.
The consultant faced some constraints, which in different ways affected the data collection, control and thereby the quality of the report. These were:
- UNICEF Documentation
- MOES documentation
- Ethics in data collection
Multiple lessons were learned from this evaluation. They concern a wide range of issues and include:
The root-cause of the violence, namely ethnic differences, still exists and causes a stress which affects normalcy and contributions towards peace building, preparedness and conflict prevention.
The general learning situation is still hampered by the low level of teacher education, migration of teachers and parent desire to enrol children in mono-ethnic schools which
MOES’ counterproductive interventions in terms of withdrawing Uzbek learning materials without having sufficient materials in Kyrgyz language, sudden transfer of students to neither tally with the human and children’s rights to which the government is committed, neither to the intentions in EDS 2020.
UNICEF – Project design and management
Data collection: Traditional data collection tools are not able to capture the essence of such post-conflict situations without either being scanty or causing pain to the target group. If not well prepared the evaluation can have adverse effects.
Design: The different types of impact given by the Focus Group members indicate that there is need for a holistic approach in projects addressing the impact of humanitarian crises
it was proved that initiatives without a results matrix and RBM can deliver as planned. This should provoke some thinking around the present EBM requirements and their relevance in emergency projects – in the present design.
Better emergency preparation of government and partners is required as timely preparation of all key partners seems to be a precondition for early, streamlined and focused interventions
Education opportunities for youth were in high demand, as quite a number8 does not attend school and not sufficiently addressed.
Conclusion on the findings against the above objectives show achievements and draw-back as accounted for below.
The analysis of findings showed that the WTS Initiative overall delivered to all objectives.
The Initiative had overall based project design and all operations on the required ethics.
The combination of activities in the One Year Education Strategy and within the Initiative itself were complementary and had the desired synergy effect.
The WTS Initiative proved fully relevant, overall effective and efficient with intended impact. Due to design, timeframe and the context it cannot be regarded as sustainable, but replicable if extracting a few, actually relevant activities.
The inclusion of a wide and diverse scope of implementers made the WTS Initiative deliver while creating a sense of ownership and ability.
All implementers engaged whole-heartedly and delivered each their parts despite continued stress.
The Initiative did not have rigorous use of RBM practices;
The Education cluster was not ready and had problems in organising itself;
The present composition of the Education Cluster will not serve the required holistic approach to education.
The political uncertainty in Kyrgyz resulted in continued fear during the implementation because of the upcoming Presidential election;
The brittleness of the peace shows in the extensive humiliations of members of the “other” ethnic groups. Constant humiliation generates fear, but also anger, which can result in new outbreaks of unrest;
Pre-schooling comes to have adverse effect in situations where there is no space or enough teachers to accommodate all children in pre-school classes. Upon start in 1st. class the fresh students fall behind due to lack of immediate readiness for learning and the learning environment;
Traumatised parents and teachers are not the best support of likewise traumatised children in development.
It is overall concluded that the WTS Initiative was well designed responding to criteria for rights-based approach in choice of activities, approach and timeliness. Despite short-comings in application of RBM the Initiative overall delivered as planned. The work was well coordinated and the implementation through a wide and diverse scope of partners made activities be people-owned.
If incorporating recommendations the Initiative can continue with the one year extension primarily based on past activities with exemption of rehabilitation of schools.
Due to the challenging learning and ethnic situation in the country there is dire need for continued, extended and coordinated activities under the CP 2012-2016.
The following changes and interventions are strongly recommended:
a) Acute and immense need for extra-curricular education, which need urgent attention
b) Inclusion of youth through above activities
c) RBM in emergency may need re-design
d) Ethics in data collection should influence data quality requirements
e) Planning of transition should be integral part of emergency project designs
Government of Kyrgyzstan/MOES
a) Integration as practiced should be reviewed not be counterproductive to ongoing
emergency and development activities
b) Teacher training and learning materials need to be reviewed and re-designed to serve
the purpose of building people, who can develop a nation. This includes full coverage
of pre-schooling and modernised vocational training.
MAINSTREAMING OF WTS ACTIVITIES INTO CP 2012-2016
It is assumed that all activities will be rights-based and be implemented applying full RBM norms and standards. With that in mind the following is recommended:
1) The CP 2012-2016 should consider a transition period of maybe one year in the affected areas allowing for (new) special post-crisis interventions in the affected areas. Such new interventions should be based on thorough need’s assessments in the affected areas. Based on the above the composition of the Education Cluster should be reviewed and have a composition reflecting the findings in the needs assessment.
2) The governmental institutions repeatedly expressed their interest in supporting future UNICEF activities, which caused some concern. For sustainability purposes it is imperative that governmental institutions take the lead when past activities turn into development activities. The capacity in all partner institutions may need boosting
3) With regard to continued provision of materials and equipment to schools UNICEF could support in a manner encouraging the government to take the lead.
4) The extensive downstream inclusion of local governments, formal and informal authorities, staff of schools and maybe other institutions, and community members should continue. It should be streamlined through formalised structures and relevant training. Building capacity is also building self-esteem, which helps overcoming trauma.
Such design in combination with the proposed skilled training of youth would also deliver to broader impact considering psychological, social and economical impact at micro, meso and macro level.
5) Acknowledging that many children live a vulnerable live at home it might be relevant to secure protection of children from neglect, abuse and violence through establishment of community monitoring committees and adequate reporting facilities. Establishment of post-school activities e.g. scout activities and sport activities for young children aged 6-12 years and who cannot attend youth clubs, would also create a protection function.
Full report in PDF
PDF files require Acrobat Reader.