Evaluation database

Evaluation report

2009 South Africa: Tree/UNICEF Kusaselihle Integrated Early Childhood Development Intervention (2004-2008)



Executive summary

 

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”

Background:

UNICEF has commissioned in 2001/2002 a situation analysis of children in Nkandla,
a municipality within UThungulu district in northern KZN, with specific focus on issues
around young children’s health and early childhood development. The analysis has
revealed serious threats to young children in the rural communities. After intensive
discussions between UNICEF, Nkandla stakeholders – representatives of the
municipality, community-based organizations (CBOs), etc – in 2004 communitybased
Integrated Early Childhood Development Intervention (IECDI) project has
been initiated, commissioned to Training and Resources in Early Education (TREE)
and executed jointly by TREE and Nkandla local municipality.

The project has been performed in two wards of Nkandla municipality – Ngono and
Ekukhanyeni. It has been implemented in several phases: October 2004 – March
2005; January 2006 – October 2006; November 2006 – March 2007; April 2007 –
March 2008. The main purpose has been to establish and sustain community-based
mechanisms to support the development of orphans and other vulnerable young
children with specific focus shifting over time from establishment of mechanisms for
improving the access to health services, especially for young children, towards
building capacity and development of structures and mechanisms for support of
orphans and vulnerable children.

Purpose/Objective:

The purpose of this evaluation is to assess the results achieved through the duration
of this pilot project, the mechanisms through which it has operated and the long
lasting effect it may have on children from the project area of intervention in the
longer term. The results from the evaluation will serve the need for accountability;
they may also serve as an information source when similar projects are designed and
implemented in the future within South African context.

Methodology:

Project documentation – including Project Cooperation Agreements; project reports;
Early Childhood Development literature; situational analysis of Nkandla Local
Municipality; and relevant budget documentation, was reviewed. Semi-structured
interviews were conducted with key stakeholders in KIECDI including KIECDI project
staff; Nkandla Local Municipality; UNICEF project staff; and community members
from both project wards who had participated in the intervention. Three focus groups
were held with caregivers and Family Facilitators who have been part of the KIECDI,
and caregivers in a contrast (“control”) ward that did not benefit from the intervention.
A sample household survey was conducted in the two project wards – Ngono and
Ekukhanyeni – and the control ward Amaphuthu.

Findings and Conclusions:

The KIECDI project has been operated for about four years by TREE, with financial
support from UNICEF. It has set one example of locally driven capacity development
for ensuring better implementation of children’s rights in highly disadvantaged
economic and social environment.

An important aspect of the project has been local capacity building. The project has
achieved the relevant objectives reasonably well. Capacity of critical staff has been
developed; that in turn resulted in enhancement the capacity of Nkandla municipality,
which now has the implementation of children’s rights on its agenda. Local capacity
for running of various components of the project has also been built. Further, parents
have been taught to better care for their children.

The exact outreach to the community and the quality of provided support remains
dubious. Different information sources suggest different results. Important aspect,
which has not been managed well, is the establishment and maintenance of good
recording and reporting practices, and establishment of a clean database, which
might be equally useful for supporting of management decisions as well as for
accountability purposes.

The evaluation could not identify strong evidence of the project’s outcomes for the
target population. The sustainability of certain aspects is under question. The more
detailed results, classified according to the evaluation criteria are as follows.

---Relevance---
The project was established and developed in response to the dire situation in
Nkandla municipality where 61% of children lived in poverty. Specific challenges that
faced children included: hunger and malnutrition; a lack of access to and caregivers’
knowledge about documentation that prevented households from accessing support
such as grants; absence of and distance to critical services especially health
services; limited understanding of the rights of children and, therefore, an absence of
support and care for children; and, a limited understanding of the benefits of ECD for
children. Child-headed households and households with elderly caregivers faced
additional challenges and difficulty in caring for the young ones. ECD sites were
identified as ill equipped and under-funded and young children’s developmental
needs were not being met.

---Effectiveness---
The project’s objectives evolved over time. They included establishment of structures
and networks for OVC; capacity development of project coordinators, family
facilitators, other project staff including in the identified ECD sites; enhancement of
caregivers’ capacity to provide quality care for children; outreach of the community
with information and support for child caring. Building the project stakeholders’
capacity for record keeping on outreach and support provided to households, and
ensuring sustainability were set as stand alone objectives during the last phase of the
project.

The achievement of the set objectives varies. Undoubtedly, the most significant
achievement is the capacity development of the project coordinator, who more
recently has become ECD/HIV and AIDS coordinator in Nkandla municipality. This
has translated into pushing the children’s agenda at municipality level. Similarly,
other project stakeholders have been exposed to capacity development through
various trainings organised by TREE. Sufficient evidence of implementation of the
obtained knowledge is lacking however.

The capacity of the ECD sites included in the project to become nodes for care and
support has been improved through various training activities. As a result out of the
18 ECD sites, ten sites have been registered with DSD and further seven have
submitted registration forms. The sites have developed HIV and AIDS policy, which
still remains to be implemented, have been equipped with toys and have been used
by the community to accommodate immunization campaigns and other initiatives.
Parenting workshops have been attended, according to TREE reports, by 546
parents in 2006 and 2007. It remains unclear if some of the parents have in fact
attended more than one workshop, which would mean that the beneficiaries were
less than this number. The survey conducted within the framework of this evaluation
found that workshops helped parents to take care of their children by providing
information on the child’s needs and how to treat them, as well as providing them
with a forum to discuss difficult issues. It, however, could not prove that all
workshops have been organised by the evaluated project.
The outreach of the project remains unclear. The set objective at the beginning of
supporting 200 families has definitely been met. However the reported two thirds of
households outreach through family facilitators’ visits differs significantly from the
estimated on the basis of the survey for this evaluation around 10% involved in the
project. Part of the discrepancy may be explained at least by two reasons. One, there
may have been double or multiple counting of reached households and children. The
project does not have a clean data set on this, it is possible that FF’s have counted
reached people during follow-up visits as newly reached people. Two, it is possible
that respondents did not recognise the project exactly by name and answered that
did not know it. Further, some of them may have exited the project at earlier stage
and thus reporting non-participation.

---Efficiency---
The operational structure of the project involved numerous stakeholders and multilayer
structure. There has been duplication of activities and reporting roles. The distance
between TREE and Nkandla, and the KIECDI Project Coordinator and the wards in which
the project was implemented, resulted in a lack of support for various role players. There
is a reported absence of coordination and support in the physical location of the project
wards.

Despite the reported existence of registers of OVC at ward level, which has been
used to inform different role players about vulnerable children and households in both
project wards, a clean database that would inform of numbers of reached households
and children, and services provided to them, is non-existent. It was recognised by
evaluation informants that capturing of the data has been done in a disorganised
manner. Development of indicators, training FFs on data collection and data
capturing is one of the weakest components of the project. Development of clear
records of outreach and support to families has been introduced only in the third final
stage in 2007 as a stand alone objective of the project. TREE had little experience in
this area and needed further support, which has not been made available.

The largest proportion of the budget (39%) has been allocated to the implementation
phase of the project, for training and mentoring reflecting the focus by TREE on
capacity building. The second large proportion (31%) has been used for project
coordination. In the budget records, the various components of KIECDI are reported
as community feedback/consultation; training and mentoring; project co-ordination;
project management/networking/advocacy; and, admin and support costs. It
becomes clear, however, that around 18% of the budget has been directed to the
target community through resources (toys, etc.) and cash disbursements (to family
facilitators).

---Outcomes---
An important outcome of KIECDI is the institutionalisation of children’s issues at
municipal level. The project has raised awareness on the plight of orphaned and
vulnerable children in the two wards. The mayor has established a child desk within
the municipality thus making issues relating to children and their development one of
its priorities. The established child desk has tasks to monitor the situation of children
in the wards falling within Nkandla municipality. Children thus have become a
standard item on the agenda of the municipality in all its meetings. Other role players,
such as DSD, have become involved in the project, assisting in different ways within
their mandates, though at a later stage.

The survey conducted for this evaluation did not provide unanimous evidence of
significant changes in the life of the community, to which the project has contributed.
While community members recognised that in the recent years access to government
grants, to birth certificates’ issuing, to more information has increased, these
improvements are not necessarily associated with the KIECDI project. Other sources
of information and support seem have had stronger role for these improvements.
Another argument in support to such statement is that access to health services, e.g.
immunization, in the project wards did not differ significantly from that in the
comparison site.

---Sustainability---
Training of ECD practitioners and FFs has been valuable for these individuals. If
individuals continue working in ECD sites and within the community, their skills will
continue to be valuable for the community. The skills that community members have
acquired from the training sessions also remain an asset. The project population
expressed feelings of being empowered and confident that it has the ability to carry
on with commenced activities. There are indications that other components such as
the Child Care Forums and other stakeholder meetings could be sustained.

At managerial level, however, there has been a lack of a vision on financial
sustainability of the project - the issue has been set on the agenda only towards the
project ending. Nor has been ensured a timely buy-in form government structures. A
clear strategy to access funding for the project has not been developed. Part of such
strategy would be a decision whether KIECDI should be registered as an NPO or
continue as a multi-stakeholder intervention. The absence of resources to cover the
running costs and payment of stipends for FFs makes it difficult for the project to
continue in the designed format.

Recommendations:

The model has been developed and designed by TREE and UNICEF with input from
those for whom the intervention had been intended. Informants suggested that
KIECDI worked with the traditional systems in Nkandla which is why it was
successful in incorporating a child rights culture into the area. According to the
Councillor with the portfolio of Gender and Disability (2008), the model has been
home grown and supported by the members of the community including churches
and traditional leadership: “We did not bring a so called expert to design the model,
we sat with our people and asked them what they wanted” (Director of Community
Services, Nkandla Municipality, 2009). Implementation of any similar project in other
areas needs to be cognisant of the cultural and traditional norms for each area in
which it would be implemented.

The buy-in into the project by the community and traditional leadership has facilitated
the implementation phase of the project. According to the Councillor in the Portfolio
of Gender and Disability (Chief advisor of Inkosi, 2008): “There were no instances of
resistance by the community or its leadership of the project implementation”. The
information gathered from interviews with stakeholders during the evaluation process
confirms the smooth implementation of the project.

The value of the model lies within its ability to use local human resources. Yet, some
informants suggested the youth, which the model in Nkandla used, is the most
mobile group of people, whereas older community members are attached to their
communities more. Some of the youth working as family facilitators had left for cities
such Johannesburg and Durban in search of other job opportunities (IDP Manager in
the Municipality, 2009). Further interventions should consider involvement of “more
established members of communities” or should ensure incentives to keep young
people within their communities.

On the management side, establishment of a rigorous monitoring and reporting
system, building the local capacity to implement and actual implementation from the
inception of a similar project is of utmost importance. The KIECDI project had not
succeeded to develop a database, which would have had a dual role: 1/ to inform
management decisions along the project life span, and 2/ to serve as a strong basis
for accountability purposes.

Finally, development of resource mobilisation strategy or a strategy for sustainable
development is a must at early stage if similar projects are to be initiated.
Cooperation with stakeholders at government departments from the project’s
inception and ensuring their buy-in can be a component of such strategy. While
communities and project implementing partners are best positioned to interact with
the local government, UNICEF may play stronger facilitating role at provincial and
national level.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search