We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 Republic of Mozambique: Mid-line Evaluation of the UNICEF Mozambique Accelerated School Readiness Pilot Programme



Author: Juan Bonilla |Elizabeth Spier | Kaitlin Carson | Hannah Ring | Paul Sirma

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part ’ of the report, and the executive feedback summary labelled as ‘Part ’.

Background:

Despite a twofold increase in primary school enrolment in Mozambique over the past 15 years, educational outcomes remain largely disappointing (Martinez, Naudeau, & Pereira, 2012). In 2014, the average primary school dropout rate in Mozambique reached almost 7%, and only 6% of students achieved basic reading competency by the third grade (UNICEF, 2016). School readiness programmes aim to address this continuing problem by increasing students’ emotional, social, and mental preparation to succeed in school (UNICEF, 2012).

Early childhood education interventions have increasingly become a priority of the Mozambican government in order to support all children to reach their full potential (UNICEF, 2016). However, despite efforts to improve access to pre-primary education, many challenges remain in the education system overall and, in particular, in pre-primary education sub-sector.

School readiness programmes have the potential to provide additional support to students who otherwise lack opportunities to acquire the skills and knowledge needed to make a positive transition to primary school. Such programs may also increase the likelihood that children will actually enroll in primary school. As with many other countries, Mozambique is seeking effective, affordable, and scalable ways to provide school readiness supports to its children. In the absence of universal public preprimary education, Accelerated School Readiness (ASR) programmes can potentially provide an affordable and scalable model that increases on-time enrolment and early-grade achievement for children in Sub-Saharan Africa. However, the limited implementation of such programmes to date necessitates further testing to validate their effectiveness.

Purpose/Objective:

The main purpose of this evaluation was to test the effects of the ASR programme on children’s school readiness and successful transition to primary school in the Zambézia province of Mozambique.

The objectives for this evaluation were to (a) determine the extent to which provision of the ASR pilot programme improves children’s school readiness, on-time enrolment, and academic achievement in Grade 1, relative to the readiness, on-time enrolment, and academic achievement of comparable children in communities with no pre-primary education; (b) calculate the community- and child-level costs of providing the ASR pilot programme; and (c ) identify which aspects of community context and implementation seem to facilitate or inhibit the success of the ASR pilot programme.

The evaluation is intended to answer the following five research questions linked to the theory of change:
A. To what extent does provision of the ASR programme improve children’s school readiness, relative to that of comparable children in communities with no preprimary education?
B. To what extent does provision of the ASR programme improve children’s on-time enrolment in Grade 1, relative to that of comparable children in communities with no preprimary education?
C. To what extent does provision of the ASR programme improve children’s academic achievement and teachers’ perceptions of their performance at the end of Grade 1, relative to the comparable children’s achievement and teachers’ perceptions in communities with no preprimary education?
D. What are the community- and child-level costs of providing the ASR programme?
E. Which aspects of community context and implementation seemed to facilitate or inhibit the success of the ASR programme?

Methodology:

The evaluation used a cluster randomized controlled trial  (c-RCT), which selected 30 implementation villages from Morrumbala and 30 villages from Derre. Sixty communities from Zambézia took part in the study, and none of these communities had preprimary education available. Of those 60 communities, 30 were randomly assigned to get the ASR programme (treatment group) and 30 randomly assigned to a no-treatment control group. Within the treatment group communities, all children who were expected to start Grade 1 at the beginning of 2018 were offered the ASR program. In all 60 communities, children's school readiness was assessed at baseline, and again just prior to entry into Grade 1. Children’s learning and engagement in school (attendance) at the end of Grade 1 were also assessed. All 60 communities participated in a family questionnaire, school record review, and the IDELA child-level assessment to measure school readiness.

8 key informant interviews with the parent leader of parent-to-parent sessions and volunteer teachers for the summer school.

12 focus groups with three different groups of informants: school professionals, program participants, and parents from control areas.

Findings and conclusions:

On the child level impact, the findings of the mid-line evaluation of the ASR pilot demonstrated positive impacts on school readiness outcomes and school enrolment. The program caused a 14 percentage-point increase on first-grade enrolment, whereas the control group has an average enrolment rate of 70%, the rate for the treatment group was 84%. Findings also showed students were more prepared to enter the first grade than their peers who did not participate. They demonstrated new knowledge and skills, including greeting others, knowledge of numbers and letters, singing, telling stories, drawing, and painting, among others. For a program that was implemented for a short period of time (3 months), these results are promising.

On the parent-to-parent the evaluation showed increase in caregiver’s educational aspirations for their children and it also revealed that there is significant increase in the likelihood that parents - told stories to their children; sang songs to or with their child; took children outside the home; showed or taught their children new things; showed affection to the child. The results also showed increases in caregivers’ educational aspirations for their children.

The end-line evaluation of the Accelerated School Readiness programme which is ongoing currently will provide robust evidence of impact, cost effectiveness and up-scaling feasibility.

Recommendations:

The Accelerated School Readiness program demonstrated positive, statistically significant impacts on school readiness outcomes and school enrolment. For a program that was implemented for a short period of time (3 months), these results are promising. The overall recommendation is to keep the main components of the ASR program with some adjustments within the components including:

- by slightly adjusting the timing of the program to start in early January to improve student attendance.
- By providing small incentives (e.g., snacks) during the parent-to-parent sessions to improve caregiver's attendance in the parenting sessions.
- By providing more support for behavior change exercises on a regular basis to increase support to parent leads.
- By providing more information on the recruitment process to improve transparency and accountability.

Lessons Learned:

The evaluation shows that only 37% of parents in treatment areas took part in the programme parental activities. Providing snacks to children and parents who participate in programme activities can be a nudge –in the sense of the behavioural economics literature (Thaler and Sunstein, 2008)— that may alter child and parent attendance behaviour in a predictable way at a relatively low cost.

The parental sessions were led by women mainly and participants in the parental sessions were mostly mothers. Involving trusted fathers in the community to lead the parental sessions for fathers may improve not only attendance to sessions, but also increase the receptiveness to the information disseminated in sessions.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year: 2018

Office/Country: Republic of Mozambique

Region: ESARO

Type: Evaluation

Theme: Education; Education surveys and status; data systems; SitAns

Language: English

Sequence #: 2018/002 

New enhanced search