Evaluation report
2011 Haiti: The Use and Impact of ECD Kits - Post Earthquake Haiti 2010
Author: Lisa Deters
Executive summary
“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”
Introduction
On 12 January 2010 an earthquake measuring 7.0 on the Richter scale struck the West Province of
Haiti, the poorest country in the Western hemisphere with a legacy of political instability and
natural disasters, with the epicenter less than twenty kilometers southwest of Port au Prince, Haiti’s
capital, triggering a large scale international relief operation in response to the tragic and massive
destruction. A relief to development operation that is currently ongoing beyond the one year postearthquake
in 2011.
This report aims to provide a better understanding of a specific niche response, the ECD response,
in post-earthquake Haiti 2010. It concentrates on the UNICEF ECD Kit, a dominating ECD
response, examining the efficiency, effectiveness, impact, relevance or appropriateness,
sustainability and the overall quality of the kit and its use. A primary objective of this report is to
identify the capacities and capacity gaps in the usage of the kit while an overarching focus examines
the overall potential of the ECD Kit via such a mass distribution of the kits during the first phase of
an emergency to assist the Ministry of Education (MoE), education authorities and relevant
stakeholders to provide and include ECD during the first six months to a year of an initial response.
During emergency situations young children are considered among the most vulnerable. Early
childhood experts, as cited in the world’s leading medical journal The Lancet 2007, advocate for
appropriate support for young children founded on evidence-based knowledge about the impact of
early experiences on long-term cognitive, psychosocial and neurological development. Therefore,
early childhood care, education and overall development are emerging as essentials in emergencies
reflected in the increasing support of early childhood programs and services by leaders in
emergency response as evidenced by UNICEF’s response with an unprecedented distribution of the
UNICEF ECD Kit in Haiti.
UNICEF distributed the ECD Kits through their partners through two time periods: first from 25
January 2010 through 4 April 2010 1,566 ECD Kits were distributed followed by a distribution of
1,051 kits from 16 June through 28 October, 2010. In total, there are 2,617 ECD Kits that are
accounted for as having been delivered to partners. This distribution of a large quantity of ECD
Kits, demonstrated the commitment spearheaded by UNICEF to include ECD on the emergency
agenda following the earthquake disaster and it presented a unique opportunity to investigate the
issues related to ECD investments and interventions in emergency situations.
The report evidences many positive and increasingly dynamic developments within ECD in
emergencies. There has been a growing recognition by a range of stakeholders of the importance of
ECD in a humanitarian response, as evidenced through the formation of the ECD Working Group
(ECDWG). The report also identifies an important base of existing capacity and expertise across the
agencies interviewed for the study.
An emerging finding from the study is a lack of a shared understanding of the common elements of
an ECD in emergencies (ECDiE) response. At the country level, the overall low staff capacity was a
challenge. This was reinforced by three factors: that most agencies involved in the education were
focused on primary education provision; that there was often insufficient coordination and
programme planning across all ages and stages and phases of an education response; and that
agencies have tended to focus on immediate and short-term results. Though in a context with
limited resources, developing capacity cannot just be about individual agencies bringing their own
capacities to scale, rather, focus must be on leverage between stakeholders, including with the
relevant sections of the ministry of education (MoE).
Notwithstanding, major capacity constraint with regards to capacity building and accountability are
identified. Due to various methodological constraints it has not been possible to quantify these gaps.
Many of the constraints are associated with larger global challenges for ECDiE. The report seeks to have
relevance beyond ECD advocates and provide practitioners with practical feedback and support.
Brief research aims
The research intended to explore the emergency relief responses for young children, in particular
the heavily distributed UNICEF ECD Kit, while maintaining an overall focus on the implications of
ECD in emergencies recovery practices and any emerging policies.
1. Evaluate the use of the ECD Kits (in example: efficiency, effectiveness, impact, relevance
and appropriateness, sustainability).
2. Explore the overall implications of focusing on young children in the early phases of an
emergency.
3. Investigate the international and national responses towards young children post-earthquake.
The research study used the UNICEF ECD Kit as an entry point to support two areas of interest, a
focus on the kits as well as an investigation regarding the ECD investments and interventions in
emergencies in general.
The study was grounded in a holistic perspective, acknowledging the value and interplay of
institutions or governments, communities and families in child development, to address the use and
impact of ECD Kits and concomitant programs, services and supports for young children and
families.
The study’s design ensured the principles of ethical research by building local capacity through the
training and use of local data collectors. Through the use of the local data collectors, the goal
remained for 200 sites, the majority with the ECD Kit, to be assessed by the open-ended
questionnaire. Also included was the use of stakeholder interviews to gain insights into what other
programs, services and supports were supporting young children. As proposed in the study
proposal, thirty stakeholder interviews were desired; however, this number was halved with sixteen
interviews formally occurring.
Data collection
The field research was completed in two phases. The first period was for three months in Haiti from
13 October 2010 through 15 January 2011 followed by a shorter field visit for two weeks at the end
of July to beginning of August 2011. The data collection focused on three methods:
1. Open-ended questionnaires on the ECD Kits (goal – evaluation of 200)
2. Semi-structured interviews (with local and international representation)
3. Focus group discussions (with local communities)
Despite the challenges of conducting research in a fragile context, with the logistical support of
UNICEF HCO, the data collection period resulted in the completion of 204 open-ended
questionnaires, 16 semi-structured interviews with stakeholders in ECD and three focus groups
comprised of the facilitators and guardians of young children with exposure to the ECD Kit.
The open-ended questionnaire targeting the UNICEF ECD Kit had been designed in collaboration
with the Consultative Group on ECCD (CGECCD), UNICEF HQ NYC ECD in Emergency Unit
and the UNICEF Copenhagen Supply Division input. Additionally, it was reviewed by some
UNICEF HCO sectors involved in the ECD Kit distribution to the partners, predominantly nutrition,
child protection and education as well as the M&E sector of UNICEF HCO. The stakeholder semistructured
interview questioning route was also reviewed in a similar fashion.
Data via the open-ended questionnaire was collected in the greater Port au Prince (PaP) area within
the twenty communes that were affected in the West Department: Cabaret, Cite Soleil, Croix des
Bouquets, Tabarre, Carrefour, Grand-Goave, Petit-Goave, Leogane, Gressier, Delmas and Petion-
Ville. The greater Port au Prince area was chosen predominantly for a pragmatic reason; although,
ten communes were heavily affected in the Southeast Department with the worst damage in Jacmel,
due to time constraints with the data collectors by the time that they were identified, hired and their
short-term contracts were settled upon, they were employed for five working days to collect the data
and given their allotted transport budget of 750 gourdes per day by UNICEF HCO, they had neither
the time nor the funding to cover the Southeast Department inclusive of Jacmel as it would have
required overnight stays incurring extra costs and time. Therefore, the UNICEF partners who had
received ECD Kits and were working within the greater Port au Prince area were included and
dominate the study.
In an effort to build local capacity and to complete the scope of the work, one research assistant and
ten data collectors were hired locally. UNICEF HCO identified and recruited the research assistant
and all ten data collectors, of which the ten data collectors were sourced through the snowball
effect. As the research assistant, Mr. Rhodner J. Orisma, is himself a doctoral candidate at a
university in Florida as well as a professor at a Haitian university instructing in methodology
courses, Mr. Orisma assisted UNICEF Human Resources (HR) with identifying potential data
collectors supplementing their snowball effect in identifying qualified and experienced data
collectors. Through his contacts, two of the data collectors were identified. The ten data collector
roster includes four females and six males: Samuel Revolus, Renan Anglad Charles-Cius, Guersley
Chery, Jacqueline Baptiste, Mirlene L. Percinthe, Rodady Gustave, Martine Delisca, Patrick
Laurenceau, Marie Sonide Dorilma and Reginald Hellande Jeune. All ten data collectors claimed to
have experience in data collection either from university coursework or previous positions with
international non-government organizations (INGOs) working in Haiti. The new learning from this
study was regarding the briefing on the ECD Kit and ECD in general. The data collectors were each
given a specific area within greater PaP for which to be responsible for covering; although, due to
their difficulties in accessing and locating sites due to movements, transitions and closures of
certain programs since almost a year post-earthquake, the collectors predominantly targeted their
goal of completing twenty open-ended questionnaires and in doing so some moved around into
departments with a great concentration of partner programming for the kit usage and easier to move
around such as Carrefour, Delmas and Petion-ville.
This study heavily relied upon the collaboration with the locally hired research assistant for
translation and interpretation needs. The research assistant served to translate the data collection
instruments for this research and training materials into French and Haitian Creole. The assistant
acted as an interpreter during the full day training on qualitative research theory and practice as well
as on the implementation of the questionnaire for the ten data collectors.
Once the data collectors were hired, a full day training was what the timeframe permitted to ensure
sufficient time for them to complete the data collection before the lead researcher’s date of
departure. It was led by the lead researcher at the United Nations Stabilization Mission in Haiti’s
(MINUSTAH) Log Base where the UNICEF HCO offices are currently located, provided the ten
data collectors with a comprehensive briefing on the UNICEF ECD Kit and an overview on
qualitative research inclusive of the validity and reliability of open-ended questionnaires,
appropriate documentation practices in terms of photography and personal observations, use of key
informant interviews and the ethics of research stressing consent, confidentiality, reciprocity, doing
no harm and dignity in documentation. The training also provided the collectors an opportunity to
familiarize themselves with the main data collection tool, the open-ended questionnaire, and an
opportunity for practice and role-play.
After which, the data collectors each were each assigned twenty open-ended questionnaires to be
responsible to complete within their terms of reference over the course of five working days
scheduled over 17, 20-23 December 2010, some completed them in five consecutive work days and
others due to the challenging circumstances of informal and formal preschools and other
educational centers and programs for young children being closed for seasonal holidays in
December 2010, spread the five days over a few weeks to gain access to certain centers and
preschools once opened again in January 2011.
On the 27 of December 2010 the first data was submitted and the research assistant, employing the
assistance of two individuals who were privately sub-contracted by him, began the work on the
English translation of all of the 202 questionnaires from Haitian Creole and French. By 7 January
2011 all of the data had been submitted and all of the collectors had been paid for the completion of
their work.
During the time span of the fieldwork, the lead researcher continually was in contact with any
organization known to be supporting or working with young children to access a spokesperson
comfortable to discuss their work in a semi-structured interview. A total of sixteen formal
stakeholder interviews were completed including: UNICEF, Concern Worldwide, PLAN, World
Vision, ACF, Tipa Tipa, UJDHRD (Union des Jeunes pour le Developpement Human Reel et
Durable), GJARE (Youth Action for Reform), OCCAARH (Organization des Cityoyens
Conséquents pour l’Avancement d’Haiti), CAFT (Centre d’Apprentissage et de Formation Pour La
Transformation), Finn Church Aid/ACT, Digicel Fondation, Kindernothilife, Save the Children,
BUGEP (Bureau de Gestion du Préscolaire) and Compassion International. The interviews were
recorded when permission was granted and the transcriptions of the interviews are in the process of
being completed for further validation and analysis.
ECD Kits: Haiti study
For the purpose of this study, ECDiE is broadly defined as prioritizing the holistic needs of young
children ranging from age 0-6 years. UNICEF (2010) has drafted an Integrated Quality Framework
for ECDiE which provides a globally recognized detailed and comprehensive definition of ECD:
“Early childhood development refers to the processes through which a young child under
eight years develops his/her optimal physical health, mental alertness, emotional confidence,
social competence and readiness to learn. Through the process of development, the young
child gains the abilities to move from ‘less to more’ and acquires mastery of complex
skills...Holistic early childhood development programming integrates essential early
childhood interventions within the health, nutrition, water, sanitation and hygiene, education
and child protection sectors. This enables the young child to claim his/her rights to survival,
growth, development, protection and participation and ensures that the duty bearers, namely,
parents, caregivers, communities, sub-national and national authorities respect, protect,
promote and fulfil those rights.” (p. 6)
Early childhood development and education remains at the forefront of the education in emergency
agenda. During fragile situations whether by an emergency or disaster, young children are
considered the most vulnerable and must be provided with essential and adequate care and
education. Beyond the initial phase of instability, young children must also be provided with
appropriate care and education to address their holistic developmental needs. Early childhood
experts continually advocate for appropriate supports for young children in any fragile situation
well founded on evidence-based knowledge on cognitive child development. Regardless of the
situation or context, education is established as a fundamental human right; furthermore,
development is recognized universally to be a fundamental right for the young child. Quality early
childhood development, care and education are essentials in the field of education in emergencies as
early childhood is an effective entry point from which to address many social issues.
What remains a challenge is to identify best practices supporting the needs of young children, their
caregivers and their communities through rigorous monitoring and evaluation. Moreover, what
remains to be considered is how best to create a sense of agency and scale-up the involvement of
young children, families and caregivers and opportunities for social change to inform the
interventions and supports of the relief community. With ever increasing fragile situations in the
world, there is a push to identify effective tactics and responses to address the needs of the young in
a crisis. Amongst the international relief community, UNICEF was the first to launch an ECD kit in
July 2009 after piloting the kit in seven countries – Chad, Liberia, the Republic of Congo, Jamaica,
Guyana, Maldives and Iraq. Since then, the kit has been heavily used in the most recent
emergencies such as after the tragic January 2010 earthquake in Haiti.
At the official launch of the ECD Kit in Geneva in July 2009 UNICEF’s current Executive Director,
Ann M. Veneman, expounded on how “the Early Childhood Development Kit is a tool for young
children displaced or affected by war and natural disasters. It is the first of its kind within the
humanitarian community.”2 Furthermore, subsequently she has stated that, “it is essential to ensure
resumption of normal childhood activities without delay, even in an acute phase of an emergency.”3
In considering this surge to pre-position early childhood development material, “early childhood
development play and recreation material (e.g. UNICEF ECD kits) which is responsive to the needs
of young children should be pre-positioned for use in communities, baby clinics, nutrition centres,
child friendly spaces etc. When it is not possible to pre-position material, local procurement
arrangements should be made. This may be done either by identifying local suppliers of similar
products or by involving communities in preparing culturally relevant toys and games for young
children” (UNICEF, 2010).
Establishing the research context
On February 18-19, 2010 the EEWG Research Task Team4 met at UNICEF headquarters (New
York) and developed specific goals for the early childhood in emergencies agenda. At that time
UNICEF identified some key areas for investigation regarding the ECD response in Haiti, and
specifically the use of the ECD kits. This study was a result of such meetings to address such
concerns.
Thus, the catastrophic earthquake that struck Haiti on 12 January 2010 killing over 220,000 people,
injuring 300,000 and leaving over one million people homeless, provided an opportunity to
investigate the responses in consideration of a positive emergency.5 Apart from the loss of lives and
properties, infrastructures and complete governing bodies were also damaged or destroyed, with
recovery and repair hampered by the following hurricane season and the outbreak of cholera.
Presently, the cost of reconstruction is estimated at $11.5 billion USD. Prior to the earthquake, Haiti
was already the poorest country in the western hemisphere ranking 149 out of 182 countries on the
2 https://www.unicef.org/
3 https://www.unicef.org/
4 This proposal was developed by the Research Team at Macquarie University following a meeting of the EEWG
Research Task Team in February 2010, and with input from UNCEF personnel. The proposal incorporates research
questions contributed by the Consultative Group and Dr. Fabienne Doucet as indicated.
5 This term has been used in various personal communications amongst the humanitarian community in light of
emergencies providing an opportunity to essentially build back better.
Human Development Index 2009 with ineffective social services, a weak civil society and political
unrest (Report for the Disasters Emergency Committee, 2011). The disaster has impacted Haiti on
an unprecedented scale and its after-effects will impact Haiti’s development for decades to come.
In response to the earthquake in Haiti 2010, in the acute phase of the emergency, UNICEF had
distributed over 1,500 ECD Emergency Kits. As of April 2010, 1,546 kits have been recorded as
distributed, an additional 3,341 kits were to come from UNICEF Supply Division, based in
Copenhagen, from which 1,104 kits were to be sent to Haiti with the remaining 2,237 kits scheduled
to arrive during the end of May-June, 2010 (UNICEF Haiti Situation Report, April 2010).
Additionally, the UNICEF Latin American and Caribbean Regional Office has provided these
numbers: 200 ECD Emergency Kits have been locally procured in Panama as a first step to assess
regional procurement and the printing of 600 copies of the ECD guide in French for the UNICEF
Haiti office (personal correspondence). This extensive and unprecedented dissemination of the ECD
kit during the acute emergency response coupled with a long term commitment by UNICEF and
others for post-earthquake recovery provides an unparalleled opportunity to research the issues,
capacity, support, development and impact of services and programs which target young children
within and beyond emergency situations.
Purpose of study and methodology
This report is based on a compilation of information from secondary data sources, observations and
informal discussions during two field visits to Haiti. The process began with a desk review of
existing ECDiE literature and continued with a situational assessment of ECD in Haiti. Efforts were
made at all stages in the study to triangulate information through the varied methodology,
compiling both quantitative and qualitative data.
During the first visit, several stakeholder interviews were completed, site visits and observations
and a focus group. The following visit resulted in two additional focus groups. It is important to
note that all attempts were made to avoid assessment fatigue amongst the communities and adults
engaged in this study; a widespread occurrence due to the large scale international intervention
post-earthquake. Both field visits were completed in an effort to gain a better understanding of:
- the overall situation and the impact of the earthquake
- the response of the relief and humanitarian organizations’ efforts towards ECD
- the community perspective on children’s immediate needs, focusing on the very young child
(aged 0-6)
- gaps in provision
- document lessons learned and good practice
The agencies interviewed for this report were drawn from a compilation of logistics lists from
UNICEF HCO in addition to sectoral input from the involved sectors with the kit such as nutrition,
child protection and education predominantly. While organisations that comprise the evidence base
for this report by no means represent all those with ECDiE capacity at international level, they do
make up the majority within Haiti. Therefore, research for this report also incorporated other
stakeholders, internal to Haiti but not engaged with the ECD Kits and externally active advocates
and leaders within ECD, in order to develop a more comprehensive sense of the global ECDiE
capacity and direction.
Finally, the overall study has benefited from the expert advice of a panel of four peer reviewers.6
6 Professor Jacqueline Hayden, Dr. Emma Pearson and Dr. Kathy Cologon of Macquarie University provided oversight
and input to the design of the research study, the analysis and final report. In addition, Dr. Fabienne Doucet of New
York University also provided expertise and guidance.
Study constraints
The research process encountered two main constraints. First, in terms of information collection,
efforts were made to design survey questionnaires that could be used across different stakeholder
groups. Partly due to this, and partly due to the burden of information collection requirements,
survey responses differed considerably in terms of the questions answered and the amount of
information provided. As a result, parts of this report are based on quite limited responses. This led
to further difficulties in attempting to quantify gaps.
Secondly, the difficulty of conducting the study against the backdrop of a highly fragile and volatile
environment presented extreme weather conditions, a cholera outbreak and political and social
unrest that impeded the research to be conducted in a prompt and planned fashion.
There were a host of other challenges in the various stages of the data collection. Many of these
were in concern to the loss of institutional knowledge from the frequency of emergency staff
turnovers at both UNICEF HCO and their partner organizations. Therefore, simple mapping tasks
of who had received a certain number of ECD Kits and the specific sites for where they had been or
were being used were complicated due to the loss or movement of emergency personnel,
international and national, and their knowledge of the responses in the acute phase of the
emergency, making the initial task of securing and then maintaining updated records of who was
responsible for or working in and on ECD amongst the partners a challenge.
New ECD management and staff months post-earthquake when emergency personnel phased out
for new permanent staff posts, typically resulted with individuals new to Haiti and some to
emergency programming. This exemplified an issue with organizations’ staff continuity, which
potentially reflects in program discontinuity. It is duly noted that with emergency contexts, beyond
the staff movements, the responses in terms of distribution, coordination and monitoring are a
daunting task, particularly post-earthquake in Haiti, with a weakened government and destroyed
buildings and infrastructure, making the mapping and processing of reactive responses a continued
challenge.
When reflecting upon the ECD Kit response, it initially was a mass distribution to UNICEF HCO
partners through the five UNICEF sectors: nutrition, education, child protection, health and water
and sanitation (WASH). UNICEF HCO has some records, though not comprehensive, from their
supply section as to who received kits and how many kits were received. From there, efforts were
focused on where those kits were sent and used by the partners. For large partner organizations that
received over a hundred kits, keeping in mind emergency personnel shifts as well as movements
amongst internally displaced people and adaptive programming in shifting tent cities, there was not
always a specific address to be given or concrete knowledge to where the kit was being used or kept
in the field. Therefore, for the data collectors it was extremely difficult to first locate a tent city or
camp and then to arrive only to find that the programming for young children had never existed, had
closed or had moved despite the information provided by an organization’s central office in PaP.
Structure and contents
This report takes on an evaluative conceptual framework to understand how the ECD Kit measures
up in terms of efficiency, effectiveness, impact, relevance and appropriateness, sustainability and
quality and accountability.7 The following sections consider the relative strengths, weaknesses,
opportunities and constraints of the ECD Kit and its usage within the Haitian context.
7 The study strives to review the data in light of an adapted adherence to the Organisation for Economic Cooperation
and Development (OECD) Development Assistance Committee (DAC) Principles for Evaluation of Development
Assistance criteria (ALNAP, 2006).
The report ends by drawing together a series of recommendations with regard to the future potential
of using the ECD Kits and to the focus on ECD during the first phases of an emergency.
Full report in PDF
PDF files require Acrobat Reader.
