Evaluation database

Evaluation report

2006 Kenya: Final report on the evaluation of Sida support to the UNICEF Country Programme in Kenya



Author: South Consulting

Executive summary

Background:
Sida and UNICEF entered into a three year agreement effective January 2004, under which Sida is providing thematic support to UNICEF’s Child Protection and HIV/AIDS Programme, and certain activities in the Strategic Planning Monitoring and Evaluation Programme and the Communication, Participation and Partnership Programme, both of which are cross-sectoral programmes. The Child Protection Programme has six projects, four of which are substantially supported using Sida funding: Youth and HIV/AIDS; Gender Violence, Discrimination, Exploitation and Female Genital Mutilation; Children orphaned by AIDS; and Juvenile Justice. The thematic funding also covers three civil society Organisations (CSOs) active in the area of children rights and advocacy. These CSOs: The Child Rights, Advisory, Documentation & Legal Centre (CRADLE), Chldren Legal Action Network(CLAN) and Kenya Alliance for the Advancement of Children’s Rights (KAACR), previously received direct support from Sida.

The objective of this evaluation of the Sida funded components was to look at planned activities and actual achievements, outputs and progress towards outcomes and to report on the findings. To achieve the objectives of this assignment, the evaluation team:
• Held briefing sessions with UNICEF and Sida as well as with the Steering Committee of the programme.
• Consulted with the Steering Committee on the design of the evaluation and received comments on the evaluation instruments.
• Shadowed UNICEF during the Medium Term Review meetings held in Nairobi.
• Held interviews with stakeholders in Nairobi, at the districts and at the community level. A structured questionnaire guided the interview sessions.
• Conducted field visits and held Focus Group Discussions with implementing partners, community members and children in Nairobi, Kwale, Garissa, Isiolo, Mombasa, Kisumu, Homabay, Suba, and Turkana Districts.
• Undertook case studies of the three CSOs.
• Reviewed the literature on the programme.

The evaluation team experienced several limitations in carrying out this assignment. Notably, most of the Sida supported programmes are in the early stages of implementation. Secondly, implementation of the programme is based on annual work-plans which are not directly linked to a logical framework. Thirdly, the wide programmatic and geographical coverage of the Sida-supported components was a challenge given the short period of the evaluation. Finally, the current implementation arrangement in which different individuals from the same institution are responsible for implementation of different components of the programme turned out to be a logistical problem.
Notwithstanding the fact that very few of the planned activities had been completed at the time of conducting this evaluation, there are several observations to make on the relevance and effectiveness of the activities and immediate results.

These observations are made as against the following standards:
• The key evaluation questions in the terms of reference
• The requirements in results based management and human rights based approach to programming, which are the key focus of the Sida funded components of the UNICEF/GOK CPAP.
• The criteria of relevance, efficiency, effectiveness and sustainability

The following is a summary of the key findings with specific recommendations to address the findings. The recommendations are classified into immediate and long-term, with immediate recommendations targeting the medium term review of the GOK/UNICEF CPAP for 2004-2008, while long term recommendations may be considered during the development of subsequent CPAPs and future support by Sida.

Purpose/Objective:
The objective of this evaluation of the Sida funded components was to look at planned activities and actual achievements, outputs and progress towards outcomes and to report on the findings.

Methodology:
To achieve the objectives of this assignment the evaluation team:
• Held briefing sessions with UNICEF and Sida as well as with the Steering Committee of the programme.
• Consulted with the Steering Committee on the design of the evaluation and received comments on the evaluation instruments.
• Shadowed UNICEF during the Medium Term Review meetings held in Nairobi.
• Held interviews with stakeholders in Nairobi, at the districts and at the community level. A structured questionnaire guided the interview sessions.
• Conducted field visits and held Focus Group Discussions with implementing partners, community members and children in Nairobi, Kwale, Garissa, Isiolo, Mombasa, Kisumu, Homabay, Suba, and Turkana Districts.
• Undertook case studies of the three CSOs.
• Reviewed the literature on the programme. The various documents provided crucial information which complemented information from the primary sources.

Findings and Conclusions:
Notwithstanding the fact that very few of the planned activities had been completed at the time of conducting this evaluation, there are several observations to make on the relevance and effectiveness of the activities and immediate results. These observations are made as against the following standards:
• The key evaluation questions in the terms of reference (attached as Annexure VI)
• The requirements of results based management and human rights based approach to programming, which are the key focus of the Sida funded components of the UNICEF/GOK CPAP.
• The criteria of relevance, efficiency, effectiveness and sustainability

The team has summarised the key findings in each of the sections that follow, and made specific recommendations to address the findings. The recommendations are classified into immediate and long-term, with immediate recommendations targeting the medium term review of the GOK/UNICEF CPAP for 2004-2008, while long term recommendations may be considered during the development of subsequent CPAPs

Conclusions
Our overall assessment is that taking into account the significant delays in the programme start-up, there has been some progress made towards the implementation of the stated activities. Future activities need to be reviewed and refocused and we have made some recommendations on the process and content of such review and refocusing. There are however a number of significant constraints that must be addressed immediately to ensure successful completion of the programme activities, notably, sustained political will, frequent and more structured consultations, communication on financial procedures, the systematic transfer of required technical skills to implementing institutions, and strengthening the participation of the CSOs.

More important though in terms of realization of results, the implementing partners now need to focus on the intended results rather than outputs, and provide specific indicators to assess the achievement of the results. In addition the monitoring and evaluation of these results needs to be mainstreamed among all the implementing partners.

Drawing from the successes and outputs and outcomes achieved in the programme so far, the prioritization of the remaining activities, and future Sida support should be guided by the following:

(a) Programme interventions that seek to achieve three main outputs:
• Strengthening of the national systems for the protection of children and women
• Empowering communities to address social practices affecting children and women
• Protecting and empowering children especially those vulnerable to abuse.
(b) The achievement of the above outputs and results for children at multiple levels.
(c) Considerations of need, the long-term impacts and sustainability.

Finally, the evaluation team underscores the need for high-profile visibility and championing of the programme as one of the most effective ways of achieving results, and that the necessary actions by UNICEF and the lead agency be undertaken in this regard

Recommendations:
(a) Conceptual Framework and Design
The programme is relevant to the country’s development objectives and is in line with the strategic plans of the implementing CSOs. A major challenge, nonetheless, is how to ensure systematic institutionalization of programme objectives through the new contract performance management approach introduced in the public sector. Linking the programme objectives and activities to this results based management system will have the effect of ensuring sustainability of the programme – specifically the components by the public institutions – and at the same time facilitate adequate integration of the programme in GoK systems and process.

An assessment of the Results Framework reveals that while it is useful for assessing the delivery process of the programme to determine if the programme has remained on the right track, it does not provide a logical framework to guide the implementation process, facilitate monitoring of implementation and provide a solid base for performance evaluation. The annual work-plans are a useful tool in this respect, but their present format does not provide any indicators. The presentation format is also not clear on implementing institutions and their outputs. It is generally difficult to related activities to outputs or even to identify gaps that require future intervention.

Recommendations
Immediate:

1. Political leadership of the programme needs to be re-emphasized by stakeholders, and strengthened to spearhead and champion the programme and ensure the inclusion of its key outcomes in future work-plans and performance targets.
2. The Gok lead agency should ensure that priority is given to the programme and intensify specific activities around information, education and advocacy for the rights of children and women and key messages on the programme outcomes among the implementing partners.
3. The programme’s work-plans need to be critically reviewed and a results schedule developed for each component work-plan that clearly provides for each of the outputs, the intended results and indicators to assess achievement of the result.
Long-term:
1. Initial work-plans should be developed by implementing partners in consultation with the respective UNICEF programme manager, and activities to be implemented under the CPAP across identified programme components including a clear statement of the CPAP outcome to be achieved and indicators for measuring results.
2. The work-plans from the different implementing partners should be harmonized at a review meeting called specifically for that purpose to avoid duplication and confirm the strategic and practical importance of proposed activities and results.
3. A comprehensive work-plan which is institution based, (rather than programme based) be prepared and approved for signing off and
operationalisation in a stakeholders meeting.

(b) Implementation Progress
Progress in implementation of the various components of the project is largely in its nascent and/or pilot stages. These delays are attributed to several factors:
• internal bureaucratic delays in both Sida (Stockholm) and UNICEF (New York) during the negotiations on the modalities of Sida’s support to
UNICEF;
• delays in the signing off of work-plans by some of the component’s partners;
• delays in the release of funds by Sida.
As a result most activities were set back by almost a year in 2004/2005.

The process of implementation is uneven among the different programme areas. Part of this unevenness could be attributed to the late start of the programme in general. The nature of some of the activities and development approaches being engaged in under various programme components are also largely innovative and experimental and with long term policy implications, and needed more consultations, buy-in and planning before implementation could begin. The activities implemented were in the areas of policy development, surveys, capacity building, awareness-raising and service delivery. The factors often cited as contributing to the successful implementation of activities were the convergence of opportunity and priorities, the eventual availing of the funds, and the participation of stakeholders.

There are still a number of significant constraints to programme implementation, including; late funding, capacity of partners, lack of a common understanding on roles and responsibilities, inadequate consultations, lack of monitoring and evaluation and instances of resistance to the programme.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search