We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2016 Guinea Bissau: Evaluation of the UNICEF Supported Adult Literacy Programme in Guinea-Bissau



Author: Paul Musker

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as 'Part 3'.

Background:

UNICEF Guinea Bissau during the last country programme (2008-2015) has been actively engaged in supporting adult female literacy in the country.

Adult literacy rates (population aged 15-24), particularly for women, were very concerning in 2006 (28.6%) and 2010 (39.8%), showing a slow but promising increase in 2014 (50.5 for females and 70.4 for males).

Evidence from previous studies namely ‘Relatorio de Avaliacao dos Centros de Alfabetização – Estratégia Alfa TV undertaken in July, 2012 show the great value of this support and the work that the Ministry of Education (MoE), through its Literacy department, has been developing during the years.

After independence, few efforts to increase literacy rates in the country were undertaken. Later on, around 2002/2003 discussions around the introduction of new approach to adult literacy and aimed at improving literacy training methods, started. As a result, in 2004 three officers from Guinea Bissau travelled to Cuba to receive training on methodology, literacy and long distance education for adult and youth. In 2006 a new approach to adult literacy started through the implementation of Alpha TV, a partnership among the Guinea Bissau MoE, the Cuban Cooperation and the Brazilian Cooperation. This approach Alfa TV ‘Sim eu posso’ was previously implemented in several Latin American countries with the main purpose to eradicate illiteracy. For Guinea Bissau it had the target that by the end of 2015 the country would be the first African country in the region to be free of adult illiteracy.

In light of increasing educational needs in the country, non-consistent increase in budget dedicated to the education sector by donors and the new strategic view of the country programme 2016-2020, UNICEF has planned, for 2015, to undertake an evaluation of its contribution to the adult literacy efforts in the country, in order to measure its effectiveness and sustainability, among others.

Purpose/Objective:

The overall objective of this evaluation is to assess whether the UNICEF GB adult literacy support effectively responds to the need of young adults in the country (particularly women) and has achieved the expected results. The evaluation results will inform UNICEF and its partners, including MoE and UNESCO, on the best approach to adult literacy in the country and will inform new possible interventions for the next UNICEF country programme (2016-2020), particularly regarding the out of school children component.
This evaluation has also a second main objective, to present a possible sustainability plan for the UNICEF supported adult literacy centers.

The evaluation will fulfill the following specific objectives:

  1. Determine the relevance of the UNICEF supported literacy programme vis a vis the needs of the intended beneficiaries;
  2. Determine the extent to which the expected results were achieved;
  3. Determine the extent to which the programme reached the hard to reach girls and women in the country;
  4. Identify the key bottlenecks in planning, implementation and monitoring (including quality of teaching) of the UNICEF supported literacy centers;
  5. Identify the added value of the literacy centers within the concept of Child Friendly Schools +;
  6. Analyze the extent to which the UNICEF supported literacy centers are sustainable and identify possible activities to ensure their sustainability;
  7. Identify lessons learnt from the UNICEF supported literacy centers project and make recommendations for future interventions, both for UNICEF and for partners;

Methodology:

Desktop review

A desktop review of ALP documentation was undertaken in the inception phase of the evaluation. This enabled the development of an ALP theory of change

Fieldwork

The evaluation fieldwork was conducted over eight working days in ten ALP centres, several of which were located in difficult terrain and hard to reach. A 100% success rate was obtained with respect to the intended research events. The evaluation used both quantitative and qualitative instruments  to support:

  • ten in-depth interviews with ALP facilitators (all male with a minimum of secondary education);
  • ten in-depth interviews with village chiefs;
  • ten focus groups of female ALP students (totalling 90 students);
  • ten focus groups of male ALP students (totalling 63 students);
  • ten students who were considered ‘outstanding achievers’ by the evaluator (in consultation with the INE fieldworkers), based on their level of achievement in literacy and numeracy and their level of confidence in our interaction in the course of the focus group discussions (7 female student and 3 male students, reflecting the proportion of female and male students in the ALP overall);
  • ten site observations; and
  • recording of student enrolment, drop-outs and pass rates per level (all by gender) in all ten centres.

Two fieldworkers from the Instituto Nacional de Estatística  (INE) facilitated the interviews and focus groups. The INE was also responsible for capturing and securely storing the data.

Lesson observation was not possible as the centres that were operational were on holiday. Some centres had discontinued classes as a result of a MoE instruction issued in August 2014.

Interview and focus group data as well as quantitative data were captured in Microsoft ExcelTM and coded for subsequent analysis. In the analysis stage triangulation was effected across instruments and across respondent types.

Findings and Conclusions:

The ALP has been well received and well supported in the sample villages. Disadvantaged students have been encouraged to participate in the programme through a variety of effective strategies at village level; women have been very successfully encouraged to participate despite the short-term opportunity cost of their reduced participation in agricultural activity and despite historical discrimination against female participation in education – women make up the majority of ALP students. Pass rates are high and the drop-out rate is low.

Literacy levels are higher than those reported in the 2013 evaluation of the ALP. For example, students’ level 3 exercise books contain evidence of their ability to write sentences well, not only ‘sign their names’.

There is a very high level of enthusiasm among students to become literate and numerate and an evident sense of pride in their achievement. In many cases this leads to practical applications that improve the quality of their lives and their sense of agency. ALP students appreciate the importance of education and an important result of the programme encourage or insist on their children’s attendance at school. This changed attitude was reported equally by female and male ALP students. ALP facilitators are very motivated and most have continued to teach without being paid since August 2014.

Despite concerns within UNICEF about the cost of the ALP, the cost is low at under US$22 per student over five years. Although the programme has been effective it has been cost-inefficient as over 50% of expenditure is attributable to supervision and support by the DGAENF, which deploys supervisors and methodologists to the ALP centres from Bissau. The low cost suggests that the ALP can be sustained, and recommendations are presented in section 12 below regarding enhanced cost-efficiency.

Recommendations:

  • The DGAENF should be provided with shoulder-to-shoulder capacity building TA to support programme management (including but not only strategic planning, monitoring and record-keeping capability).
  • UNICEF should ensure close collaboration with the DGAENF and maintain parallel records of programme progress and achievement in electronic and hard copy. When there is turnover of UNICEF and/or DGAENF staff every effort should be made to support incoming staff and ensure a smooth transition.
  • Technical assistance for the ALP should include the development of greatly improved monitoring instruments and record keeping, and a monitoring and evaluation plan for use in programme improvement.

Lessons Learned:

Literacy is power, as Paolo Freire would say. Among the 153 students interviewed in the course of the evaluation, perhaps 10-15% were inevitably reluctant to speak in the focus group discussions. The rest, the overwhelming majority, gave impressive accounts of the positive impact of the ALP on the quality of their lives (not only on their ability to read and write) and it was clear that the programme had substantially boosted their self-confidence. Many students gave accounts of very practical ways in which their lives had changed for the better as they took on roles in their communities which they had previously not had access to.

It is important to note the unanimous view of ALP students that their attitude to education had become very positive as a result of the programme when in many cases they had previously been dismissive of its value. Students at two ALP centres (aged approximately 30 to 45) attended the public primary school in addition to their ALP classes.

Certain findings of the 2013 evaluation of the ALP differ radically from those of this evaluation. For example, the 2013 assessment of the language of the learning materials (Brazilian Portuguese) was that it was largely incomprehensible for the students and even for one of the facilitators. This is in contrast to the quantitative finding of the current evaluation, which shows that students are not only comfortable with the language of the materials (sometimes mediated in Creolo by the facilitator) but appreciate the opportunity to improve their Portuguese, which (as is typically the case with colonial languages) is perceived to open more socioeconomic doors than Creolo or indigenous African languages.




Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2016

Country:
Guinea Bissau

Region:
WCAR

Theme:
Education - Adult Education/Literacy

Type:
Evaluation

Partners:
Ministry of Education

Language:
English

Sequence #:
2016/001

New enhanced search