We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2014 AFG: Evaluation of the Let Us Learn (LUL) Formative Evaluation – UNICEF Afghanistan Country Office



Author: Kerrin Ann Barrett, Ph.D

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

The Let Us Learn (LUL) initiative (formerly known as the Basic Education & Equity initiative) is a unique partnership that allows for flexible and innovative approaches to address inequities in education access and outcomes. LUL programmes are rolled out in five countries: Afghanistan, Bangladesh, Liberia, Madagascar and Nepal. Challenges and barriers to education, particularly amongst excluded and marginalized children and youth have been identified in diverse country contexts. The foundation of the LUL programme rests on three “equity pillars”: reaching Out of School Children (OOSC); expanding girls’ education; and improving the quality of outcomes for learners.
 
Each country is targeting the hardest to reach children by sharpening the equity focus in both programming and monitoring of results. Twenty million USD have been designated for the period 2011-2014.

The aim of the LUL programme in Afghanistan is to provide a primary school education (Grades 1 to 6) for OOSC aged 9 to 15, primarily girls, who otherwise would not have a chance to obtain an education.  The programme began in 2011, focusing on the 12 provinces of the Central Region.  The programme is innovative in that students attend two grade levels per year, attending school year round, studying the same curriculum as that of formal schools.  In addition, the teachers are drawn from the local villages where Accelerated Learning Centers (ALCs) are located, thus community members are more trusting to let their girls attend classes.

ACO carried out a country-level evaluation to inform a scale-up of the LUL programme and a better programming to equity for females and OOSC. This evaluation report describes the findings and recommendations for the LUL programme in Afghanistan, which focuses on ALCs. LUL contributes to UNICEF’s focus on equity, innovation and outcomes for learners through support for pre-school education, learning about health and hygiene, and education of adolescent girls and OOSC.

Purpose/ Objective:

Purpose: To provide internal lessons learned for this pilot project in order to further understanding of how this type of educational innovation is best managed and whether or not it is relevant and effective for the target population.  Additionally, lessons learned will aid in determining whether or not the model can be scaled up and applied in other geographical areas suffering from similar barriers to equity for females and OOSC.

Objective: From a country perspective, the objectives of the evaluation of LUL in Afghanistan are threefold: 1) to examine the extent to which LUL is achieving intended outcomes in Afghanistan; 2) to systematically document new learning, and to document new efforts in monitoring for equity, in particular; and 3) to contribute to the global LUL evaluation and analysis of cumulative impact. The evaluation is focused on learning to be shared within the organization.

Methodology:

The methodological approach used was a self-evaluation exercise with a primary focus on learning and a secondary focus on accountability, using a common evaluation/learning framework developed for the global LUL evaluation.  The methodology is primarily qualitative in nature, using content analysis and constant comparison of narrative data to validate emerging themes.   As this was a pilot project, only the DAC criteria of relevance, effectiveness, sustainability and scalability (as a sub-criterion of sustainability) were used to provide a general framework and were adapted to the LUL programme in Afghanistan.

Five provinces out of the 12 provinces in the Central and Central Highland Regions, served by UNICEF, were highlighted for this evaluation: Bamyan, Daikundi, Kabul, Panjshir, and Paktya. The focus was on ALCs located specifically in 14 districts and 30 villages already selected and identified by Probability Proportional to Size (PPS). 

The National Evaluator and enumerator team conducted over 300 participant (teacher, community members, hub school principals, students) interviews and focus group discussions. Supporting the innovations in LUL’s approach, data gathering included the use of mobile phones.  Concept mapping aided in the data analysis to further refine relationships between findings.

Findings and Conclusions:

The LUL initiative in Afghanistan has met its objectives, reaching intended participants and increased learning outcomes, and has even shown significant effectiveness (impact) in communities for transformational learning.
An increase in community support for ALCs, and in particular for girls’ education, was found across all sampled provinces. There was a significant shift in perspective, from being against girls’ education to encouraging girls to continue on to university. In addition, ALCs address the needs of refugee children, most of whom are older, OOSC returning from Iran and Pakistan.
Increased learning outcomes were observed, primarily in the areas of literacy and numeracy, and students seemed to be ahead of their formal school counterparts. 

Effectiveness/impact: The building blocks of a civil society begin in early grades. The life skills curriculum teaches respect for oneself, one’s family, and for elders in the community. Most promising is the girls’ increased awareness of their own rights, and human rights in general, as a result of the curriculum. Many conservative areas formally against girls’ education now actively encourage their girls to attend the ALCs.
 
Relevance and effectiveness: Although overall the LUL initiative is successful, there are still significant challenges that need attention and resolution. The primary issue is weak capacity in the Ministry of Education (MoE). Although a CBE policy is in place at MoE, CBE has been primarily under the purview of NGO community. CBE is funded solely by donors and implemented by NGOs. Reliance on the MoE as an implementing partner has meant that essential aspects of the programme are severely deficient. Effecting both equity and sustainability, particularly in the more rural and conservative areas of the Central Region, is the lack of an adequate number of female teachers. Monitoring was an issue, especially in remote and insecure areas, which impacts both effectiveness and sustainability.

Recommendations:

Recommendations revolve around the lack of capacity in the MoE to fully implement the LUL programme. With TLM essential for ALC effectiveness, key stakeholders were all in agreement on this point: if the MoE is unable to deliver, UNICEF should organize distribution to ALCs until such time as the MoE has built enough capacity in order to ensure timely distribution to all Centers.  Subcontracting to an INGO would be one way to resolve this ongoing problem.

In order to ensure proper monitoring of the ALCs, especially in remote and insecure locations, the recommendation is to contract with a local (third party) NGO experienced in M&E, both qualitative and quantitative techniques, and use mobiles for real-time/GPS tracking.  A total 800 numbers and call centers should be made available so that any community member can call for free and report on the ALCs (e.g., delays in teacher salaries, etc.).

In order to improve sustainability of the ALCs, the recommendations are to: 1) strengthen MoE ownership of the ALC programme through more frequent collaboration with government officials; 2) collaborate with MoE’s Planning Department to strengthen monitoring and pilot mobile M&E; 3) continue supporting female teachers through encouraging MoE/TED to provide flexible teacher training programming; and establish a tracking system for the graduates.

The challenge to the future of the ALCs lies in the hands of the new government.  Though the MoE is a strong partner in CBE implementation, capacity and budget are nearly non-existent to support the current programme.  Continuing support from UNICEF/ACO will be needed to ensure the most marginalized girls and older children have an opportunity to learn and contribute to their families, communities and the future growth and stability of Afghanistan.

Lessons Learned:

Lessons learned from the LUL implementation in Afghanistan highlight the importance of community participation and good practices of processes to operationalize, monitor and engage communities and other stakeholders to make LUL a success.  Consideration should be given to a lengthy timeline of innovation integration to give social mobilizers time to prepare communities for CBE, especially educating girls in conservative areas.

The ramifications of a programme that successfully covers two classes in one year in terms of formal education bring up an important strategic issue for the MoE and donors. If funding were made available, the ALCs could be expanded to fully cover the remaining 3.5 million OOSC, as well as thousands of children projected to enter the school system in the next decade.

The ALCs in Afghanistan represent a promising way forward to improve equity for girls and OOSC in remote areas of the country. Most encouraging are the findings relating to the creation of civil society, wherein the finding showed an increase in respect for elders, a decrease in violence and an increase in awareness of human rights, particularly amongst girls.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2014

Country:
Afghanistan

Region:
ROSA

Theme:
Education

Type:
Evaluation

Language:
English

New enhanced search