We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2014 Afghanistan: In-depth Evaluation of Female Literacy Program



Author: Anne Bernard

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

The project aims to contribute to national literacy targets through support to a 9-month Literacy Centre (LC) training course for rural women; delivered in 2 hour per day classes in private homes and mosques; by purpose-trained female teachers; using a gender-tailored curriculum. It expects to enrol and graduate 306,000 learners with skills in basic reading and numeracy and, eventually, a 50% increase in literacy rates among females between ages of 15-24, the empowerment of women in the country and a better life for both women and children. As part of the Government system, and within the framework of UNICEF’s more focused Country Programme, LCs continue to be opened and monitored by District Education Offices, Literacy Managers and Monitors in ten most disadvantaged provinces.

Purpose/Objective:

Purpose: To assess project relevance to, and impact on, the literacy needs of Afghan women; To explore the likelihood of its uptake by government and other actors should UNICEF withdraw; and To inform UNICEF and government of strategies for project improvement.

Objective: To conduct a detailed and an in-depth assessment of the project that will lead to a well-defined strategic direction and action in the next Country Programme when the current cycle ends in 2014; To provide other literacy actors with up to date evidence-based lessons learnt that can inform decisions in the area of literacy training.

Methodology:

The evaluation was managed through a two-tier team structure: an independent external consultant  contracted by UNICEF as team leader to design, manage and ensure overall quality of the evaluation and write the final report; and a local Afghan national team of chiefly female researchers with direct experience of the selected sample provinces, contracted through a competitive process to conduct the fieldwork and coordinate the summary presentation of criteria-grouped English-language interview and FGD data to the team leader.

Five provinces were selected for the sample: Zabul, Paktya and Badghis (with active Centres) and Laghman and Balkh (with completed projects). Based on criteria of accessibility, security and project relevance, data were collected from four Centres in each of 9 districts following guidance of Provincial Education Directors, Literacy Managers and UNICEF officers. Three data collection methods were used: analysis of policy, project and evaluation documents and statistical reports; interviews or FGD with national and local stakeholders; and observation of literacy classes. Tools, translated into Pashto and Dari, generated context-specific data on stakeholder experiences, perspectives and opinions.

Findings and Conclusions:

Effectiveness The project appeared to be generally effective insofar as has enrolled the “right” people i.e. those adolescent girls and women denied the opportunity of a formal education. It has worked as intended in exposing learners to reading, although younger learners were progressing fairly quickly, while older ones struggled. Nevertheless, the majority were positive about all aspects of the LCs, the quality of teachers most frequently cited. Good links with communities appeared to be fostered.

Relevance was nuanced. While consensus opinion confirmed project purposes well aligned to stakeholders, both implied and expressed concerns suggested the scope should be broadened, for learners 15-20, to include a curriculum equivalent to grade 5 to support their “right age” entry to grade 6 and for those above age of 25 and married, to include rudimentary income generating skills.
 
Efficiency assessments were ambiguous. Cost per student of US$61 for the 9-month course compared favourably with US$67 for general education and US$98 for national literacy provision. Critically, learner and teacher satisfaction levels confirmed cost-benefit for them and the community; dropout and absenteeism were non-issues. However, a one-size-fits-all curriculum, with teachers not readily able to adapt it, has limited realizing optimal learning outcomes for the variety of participants involved.

Sustainability remained dubious. While the project has provided learners a first step toward literacy, it has overall been insufficient to consolidate capacity. While there continued to be strong expressions of interest among communities in what the project was providing, where UNICEF had withdrawn, the gap has not been filled locally or from Kabul.

Recommendations:

1. Reframe and advocate for a concept of literacy that is more explicitly cross-sectoral as a catalyst for women’s wider development, and put women at the centre.
- Put women, gender equality, protection and peace-building at the centre of all literacy programming.

2. Reconceptualize literacy (and so the project) within the broader framework of the principles and practices of an emerging nonformal Education sector.
- Include concepts, delivery modalities and methods of adult and community learning in an expanded preparation of teachers. 

3. Professionalize the teaching of adults as a legitimate and central area of teacher education within the wider education sector.

4. Increase, stabilize and diversify the resource base of literacy within the wider development-and-equity mandate - including, but not restricted to, education.

5. Provide more intensive support and training to the development of M&E theory and practice, toward both a better understanding of the purpose of evidence-based planning and decision-making and a more consistent application of the full triple-A (act-assess-adapt) cycle.
 - Consider in medium-term introducing use of RBM framework, through systematic, hands-on training at all levels.

6. Extend the range of content and duration of delivery channels for LC teacher training

7. Allow and enable greater flexibility of programming that is more user-driven and locally managed with respect to options and resources. Three elements are suggested for this:
- Where feasible, extend the duration of the single course format from 9 to 12 or more months and enable streaming within the period. 
- Where numbers allow, divide the program into various stand-alone courses based on learner goals.
- Provide a modular textbook format and training teachers in development and use of locally-made teaching-learning materials.

8. Support capacity building for outreach by and among local stakeholders.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2014

Country:
Afghanistan

Region:
ROSA

Theme:
Education

Type:
Evaluation

Language:
English

New enhanced search