We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 EO: Impact Evaluation of Play-Based Early Learning and Development Through UNICEF’s Early Childhood Development Kit



Author: Elizabeth Spier, Mitchell Morey, Kevin Kamto

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

UNICEF’s ECD Kits have reached an estimated 4 million children globally across 86 countries, with an initial emphasis on emergency and post-conflict (fragile) settings. Over time, the kits have increasingly been introduced into ECD programming in development contexts, as well, where children often lack access to play materials and to play-based teaching and learning experiences. UNICEF designed the ECD Kits to provide children with access to toys, games, art supplies, and activities to give them opportunities for play, as well as a variety of learning experiences.

In the “foundational” intervention—the typical design of the intervention to date—ECD Kits are distributed with activity guides intended to provide information on the way to use the materials in the kit according to the ages and interests of the children. The kits also include guides for coordinators and facilitators, and caregivers or other users of the kits typically receive a few hours of training based on the guidebooks. UNICEF designed the kits so that each can serve as many as 50 children. The kits include secure storage supplies (metal box and padlocks), a jerrycan, duct tape, pencils, paper, art supplies, and toys.

For the ECD Kits to be effective, the adults using the kits with children must possess knowledge and skills across a variety of areas related to child development and pedagogy. Therefore, UNICEF has introduced an “enhanced” ECD Kit intervention for testing in the current evaluation. The enhanced model includes the foundational kit and also provides additional training to caregivers that builds on (a) the science of adult learning, to improve the ability of adults to effectively promote children’s development and well-being with the ECD Kit and (b) the neuroscience of learning among young children. The enhanced model also includes a community engagement component to create toys, with the goal of increasing sustainable access to play materials.

Purpose/Objective:

The purpose of the evaluation was twofold: to evaluate the impact of the ECD kit interventions on children’s psychosocial well-being, early learning and development; and to evaluate the effectiveness of different approaches for improving caregiver interactions with children and the use of the ECD kits for early learning and development.  

Methodology:

The evaluation employed, a quasi-experimental impact evaluation design which used a multiarmed cluster randomised design with schools assigned to either one of two treatment arms or a control arm. Using a lottery process,
42 community preschools in the Sédhiou area of Casamance, Senegal, were randomly assigned so that 14 received the foundational ECD Kit model, 14 received the enhanced ECD Kit model, and 14 served as a no-treatment control group.

The final sample consisted of 42 schools, which included 64 preschool classrooms and their teachers, and 671 children and their parents. The randomisation was successful in creating three groups that were equivalent in outcomes of interest prior to the beginning of the intervention. There was no attrition at the school level and just 6% at the child level. Baseline data collected April/May 2017,  end line data collected June/July 2018

- Directly assessed children’s learning in language and literacy, cognitive development, fine motor skills, and behavior
- Asked teachers to rate individual children’s social and behavioral functioning
- Asked parents/guardians about their child’s background (such as health and nutrition) and opportunities for play at home
-Directly observed teaching in preschool classrooms
- Completed a checklist of toys and learning materials in the preschool classroom
- Asked teachers about their pedagogical practices and attitudes toward play
- Directly observed conditions on school grounds (such as water and sanitation)
- Asked parents and teachers about family engagement in making toys for preschool classrooms (end line only)
- Asked parents about community cohesion (end line only) 

Findings and conclusions:

Findings and conclusions are organized around:

Teacher training and Kit Distribution

Teacher belief in Play

Promotion of play and play-based learning

Teacher use of effective pedagogy

ECD Kit Effects on Psychosocial Wellbeing and Learning

Community cohesion 

Recommendations:

UNICEF should find ways to deepen teacher understanding of play-based teaching and learning; and to build teacher capacity to manage the implementation of play-based teaching and learning in their classrooms so that children gain a diversity of supportive play experiences.

Teachers also need explicit training in how to actively build children’s social and behavioural skills so they can play well with others.

UNICEF should consider government capacity for monitoring and determine ways for teachers to get the oversight and coaching needed for real change to occur in classrooms.

This recommendation applies to play and play-based teaching and learning initiatives in general, as well as to the ECD Kit in particular.

As UNICEF seeks to improve training and other aspects of the model, monitoring and evaluation should happen in a rapid-cycle manner. Timely information is important so UNICEF can quickly learn whether, for example, teachers apply what they learned in training, where training should be strengthened or changed, and in what areas teachers need (more) ongoing coaching.

Because of the lack of Enhanced Kit intervention effects on family and community engagement in making toys, UNICEF should reconsider whether that component of the intervention is feasible or needs adjustment to become feasible. If that approach is discontinued, UNICEF will need to identify other means for schools to be able to replace ECD Kit materials as they become worn out over time.

Please find the report labelled as follows:

  • Full report - Report
  • GEROS assessment - Part 2
  • GEROS summary - Part 3
  • Evaluation synthesis - Part 4
  • Policy brief [English] - Part 5
  • Policy brief [French] - Part 6


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year: 2018

Office/Country: Evaluation Office

Region: HQ

Type: Evaluation

Theme: Early Childhood Development

Language: English

Sequence #: 2018/012

New enhanced search