We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2015 Zambia: Evaluation of the Early Childhood Education (ECE) Interactive Radio Instruction (IRI) pilot programme in Eastern Province of Zambia



Author: Beatrice Matafwali, PhD

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

This is the report of the evaluation of the Early Childhood Education (ECE) Interactive Radio Instruction (IRI) pilot project that was conducted in the four districts of the Eastern Province of Zambia namely: Chipata; Chadiza; Katete; and Petauke. The main objective of the ECE IRI pilot project was to find out whether IRI could be used as an alternative mode of providing ECE services especially in the hard-to-reach rural communities of Zambia. In this regard, the goal was to increase access to ECE in these remote areas, where there are limited services. ECE service provision is inequitably distributed with access higher along the more urban line of rail. In addition, the project aimed at improving the quality of ECE by using the IRI mode of delivery. The target beneficiary groups in the IRI initiative were children in the age bracket 3-6 years.
In its implementation process, the IRI project constituted the delivery of early childhood education through the use of radio and this is what defines the Interactive Radio Instruction methodology.  Solar powered radios were provided to various ECE centres in   target communities, and these were managed by an IRI mentor. The IRI mentors underwent training on how to effectively facilitate the IRI programmes. The objective of the IRI intervention was to model an approach which if  found to be effective, would then be  roll out strategy for increasing access and improving the quality of ECE. The Education Broadcasting Services (EBS), under the Directorate of Open and Distance Education, is the department of the Ministry of Education with the responsibility to produce the IRI materials and manage the delivery of ECE through local radio stations in selected areas.

Purpose/Objective:

The main purpose of the evaluation was to:  evaluate the ECE IRI pilot project that was conducted in the four districts of the eastern province of Zambia namely; Chipata, Chadiza, Katete, and Petauke with the following specific objectives:

  1. To determine to what extent the objectives and outcomes of the project have been achieved.
  2. To assess to what extent the IRI methodologies constitute an effective and alternative approach to delivering early childhood education services.
  3. To identify lessons learned and provide recommendations for future programming for early childhood education.

The evaluation approach focused on assessing the project from the following perspectives:

  1. Relevance: based on the analysis of the extent to which the initiative fits into the local, and national priorities and international commitments, including its contributions to the achievements and in addressing challenges in the ECE sub-sector in Zambia.
  2. Effectiveness: consisted of assessing the extent to which the results obtained from the implementation of IRI activities, have contributed to the attainment of the planned objectives. The evaluation further highlights the factors that contributed or hampered the achievement of results with regards to: increased access to ECE; capacity building through training of mentors;  and community mobilisation. The extent to which synergies were developed between activities on national and local levels was also analysed.
  3. Efficiency:  based on the assessment of outputs/activities in relation to project inputs, costs and planned timelines. This includes an assessment of the cost-effectiveness of  IRI  in comparison to the conventional ECE service provision
  4. Sustainability: in terms of project design, implementation process, and extent of fit with national priorities. Sustainability was analysed from various perspectives: policy, institutional, capacity building and financial.

Methodology:

The evaluation methodology comprised the following three phases: desk review; field data collection; and synthesis phase.
Desk review: the evaluation team reviewed various national and international documents on the provision of ECE through the IRI mode. The desk review was done to familiarise the evaluation team with the local and international trends on ECE provision through the IRI mode.
The field phase used a mixed methods approach as follows: quasi-experimental design and qualitative design. The quasi- experimental design was used to assess children from IRI centres, non IRI centres, and those not enrolled in any ECE centre in language skills, cognitive skills, as well as attention skills. While not being exhaustive, testing children in the above mentioned developmental domains, was the most reliable way of determining whether or not IRI could be used as an alternative mode of providing ECE by comparing the means in performance learning outcomes between children receiving ECE through the IRI mode and those who are not. The qualitative data were collected through semi-structured interviews with participants to get the views from teachers/caregivers and parents on the IRI methodology as a mode of delivery of ECE services. Qualitative data were sought in this evaluation in order to hear the voices of the participants, regarding what they felt about the IRI project. In addition, observations were made of the availability of the IRI facilities in the centres that were visited.
Synthesis phase: findings that emerged from the quasi-experimental design, semi-structured interviews, as well as data from the observations that were made were integrated. These findings were then related to the literature review that was done during the desk review phase. The synthesis of the findings helped determine the relevance, efficiency, effectiveness, impact, and sustainability of the project. Meaningful conclusions, lessons, as well as recommendations were finally drawn.

Findings and Conclusions:

Effectiveness: The findings of the evaluation indicate that , the pilot project was effective in terms of achievement of the planned objectives The effectiveness was observed through: an increase in the number of established ECE centres in the sampled districts; an increase in ECE enrolments in target areas, and improved human resource capacity through training of IRI mentors.By 2014, a total of 2,711 children had been enrolled in established ECE centres across the four districts. The enrolment increase can be attributed to the increase in the number of ECE IRI centres from the initial 21 to 42 in the stated period. A total of 31 caregivers and 19 ECE teachers were trained in the IRI methodology.
Efficiency: financial documents from EBS revealed that implementation of the project was timely and in compliance with the budgetary allocation guidelines. The budget line covered programmatic, management and operational support.
Sustainability: the evaluation revealed that the project might have been sustainable if: there was continuous training of IRI mentors to run centres, community involvement was well coordinated, using community radio stations in broadcasting lessons made sustainability attainable with constant power outages leading to disruptions in broadcasting, easy-to-operate, easily serviceable radios was not difficult, and IRI mentors were paid, making it easy for them to remain committed.
The project made a major contribution to the attainment of the rights of children through creating an opportunity where education, could be accessible to all children. Secondly, the project fulfilled the children’s right to play in an organized environment under the guidance of a mentor.  Most importantly IRI  contributed to positive learning outcomes in areas such as language, fine motor skills and problem solving.   Finally, the project helped children appreciate their culture more through the use of the local language and local stories in the lessons

Lessons Learned:

There was need to empower District Education Offices with adequate funding to ensure effective implantation of the IRI programme.

Recording programmes in advance on CDs was another suggestion that teachers/caregivers felt could improve the delivery of IRI lessons. This would also address the issue of inconsistency airing of the lessons resulting from both power outages and poor radio reception.

Most of the teachers/caregivers who participated in the study suggested that IRI lessons needed to allocated specific days so that children could exclusively benefit from IRI methodology.

A teacher at Muchini centre observed that there was need to conduct one IRI lesson per day. Further, the teacher observed that there was need to construct classrooms specifically for IRI programmes so that the teacher would just go there and teach.

Teachers/caregivers in all the centres indicated that they needed adequate training in Early Childhood Education and in the skills to run IRI programmes. Continuous Professional Development would help address the high staff turnover currently being experienced by a number of IRI centres.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2015

Country:
Zambia

Region:
ESARO

Theme:
Education - Emergency

Type:
Evaluation

Partners:
Ministry of General Education

Institution:
University of Zambia

Language:
English

New enhanced search