Evaluation database

Evaluation report

2002 SOM: Evaluation of UNICEF's Country Programme in Somalia



Author: Biswajit Sen, B.; Hadfield, H.; Callahan, C.

Executive summary

Background

Until recent years, UNICEF Somalia was exclusively focused on responding to the emergency situation through provision of essential services in health, nutrition, basic education and rehabilitation of water sources. However, over the last five years, it has moved rapidly to not only cover more program areas, but also to redefine its role and strategies towards sustainable development of Somalia. This has thrown up new challenges and opportunities for UNICEF Somalia, and a need for a more systematic Country Programme Evaluation was felt.

Purpose / Objective

The objective of the evaluation was not to do an inspection of the programs and activities of UNICEF Somalia, but to provide a holistic picture of what UNICEF Somalia has been able to achieve or not achieve, given the environment in Somalia. Hence, the evaluation exercise attempts to delineate the strengths and weaknesses of UNICEF's country program in Somalia, given the difficult situation, and ask the question, "What more and in which way can UNICEF's country program bring about changes within itself to enhance its impact, especially on the lives of the vulnerable groups of children and women in Somalia?"

Towards this broader purpose, the Terms of Reference of the Country Programme Evaluation as laid out by UNICEF Somalia defines certain objectives for the evaluation, which are the following:
- To assess the relevance of UNICEF's core strategy of assistance to children and women through its program in Somalia
- To undertake an in-depth review of progress in order to determine the extent to which UNICEF interventions have met planned objectives, strategies and coverage as identified in the last two Master Plans of Operations (MPOs)
- To examine the experience of country program implementation, and draw out lessons that can be used to improve program planning and implementation in the future
- To identify and analyze constraints to implementation, and make concrete recommendations for UNICEF's future role and interventions in the next country program
- To assess the effectiveness, efficiency and impact of supported projects and programs, and analyze to what extent results and activities are sustainable and replicable
- To review the linkages, relationships and integration between different sectoral programs, cross-sectoral projects, and between programs and operations (including finance, supply, human resources, and logistics) towards strengthening current organizational structures and processes

Methodology

This report is a distillation of the evaluation exercise, which included a sectoral review exercise; desk reviews of the extensive documentation carried out by UNICEF Somalia; field visits to the three zones by different members of the evaluation team; and an extensive round of interviews with a wide body of existing and potential stakeholders (staff, partners, SACB members, beneficiaries, local authorities). The exercise also included a dissemination workshop held on October 14-15, 2002, with participation from NGO partners, UN agencies, donor representatives, external resource persons, and UNICEF staff.

Key Findings and Conclusions

During this period, the content, spread and implementing strategies that have characterized UNICEF's programs in Somalia have undergone substantial shifts and additions. Its programming approach has systematically moved from purely emergency responsiveness to greater sustainable development. In addition to its core sectoral programs of health, nutrition, basic education, and water, it has added a variety of cross-sectoral projects and functions. Amongst projects, it has added HIV/AIDS awareness towards maintaining the low prevalence rates in Somalia; eradication of female genital mutilation (FGM) towards realizing the rights of women; a youth project towards providing an alternative vision and opportunities to a generation losing out to war; and mainstreaming of gender rights in an essentially patriarchal society. Amongst integrating functions, it has enhanced the role of monitoring and evaluation, and of program communication and social mobilization. Underlying all programs is also an element of emergency preparedness, given the still unstable situation within Somalia.

Over the years, UNICEF Somalia has also spread its operations and programs of support to the majority of regions within Somalia. To maximize coverage, it has zonal offices in each of the three zones within Somalia as well as four sub-zonal offices in the Central and Southern zone. The country office, currently based in Nairobi, provides support, guidance, and overall coordination for the country program.

As windows of opportunity have opened up in different regions within Somalia through relative peace, emergence of regional government structures and entry of other development agencies, UNICEF Somalia has also started implementing its program of support through a variety of "partnership" based strategies. These strategies have ranged from directly working with communities; implementing programs through local and international NGOs; contracting activities through private enterprises and working in collaboration with different kinds of local authorities and zonal government structures. This, in turn, has led UNICEF Somalia to work on different levels of programming. While UNICEF Somalia's emphasis continues to be the delivery of services at the community level, it has increasingly looked at more systemic issues of policy formulation and system rehabilitation. UNICEF Somalia's role within the unique and active Somalia Aid Coordination Body (SACB) in developing all these processes has also been fairly significant.

The major strengths of UNICEF Somalia's program planning, implementation and management has been the following:
- UNICEF Somalia is the only international agency that has been able to actually operationalize and implement a variety of program components in an extremely complex and security adverse environment like Somalia. This continual and widespread presence of UNICEF Somalia on the "ground" within Somalia is widely acknowledged and is a unique base for future operations.
- It has been successfully adapting implementation strategies, through both new types of partnerships and program innovations, to suit the different contexts within and across the three zones. In the four core program sectors of health, nutrition, education, and water supply, it has continuously deepened program development concepts to cover systemic dimensions and, at the same time, maintained a direct link with delivering services at the community level.
- UNICEF Somalia's program mix, although restricted to its global mandates, have been relevant in the Somali context where water is scarce and a priority felt need, malnutrition is widespread, and health and education status extremely low.
- UNICEF Somalia's focus on program sustainability issues has been both unique and effective, in a semi-emergency situation, and has covered areas such as training, implementation through multiple partners, system rehabilitation processes such as building up human resources, revenue generation and cost sharing, design of information systems, and setting quality standards in various kinds of services.
- In these processes, UNICEF Somalia has been effective in marshalling the best available resources right across Somalia in a manner that is both strategic and which has created a base for further development, as stability returns to Somalia. Even though the cost of implementation of programs are high in Somalia because of long distances, complex logistics and the adverse security environment, UNICEF Somalia has been efficient in ensuring both the reach and spread of its supply chains, as well as limited human resources.

However, UNICEF Somalia has been limited in its capacity to adequately capitalize on these strengths, so as to enhance the impact of its programs, because of certain weaknesses. It has often introduced program components and activities without adequately adjusting program strategies and concepts to suit the Somali context, especially in themes dependent around awareness generation, social mobilization and community partnership. Program structuring often has a narrow sectoral orientation that is program-output based. This has resulted in a loss of opportunity of one program piggy-backing on another, once the complex operationalization and reach to a community is achieved. This, in turn, has resulted in limited impact, as a single program does not have the ability to change the lives of children and women in its totality. A side effect of this is that while UNICEF Somalia has achieved a wide spread, all programs together do not address the needs of the same cohort of children and women, thereby reducing actual coverage. Exclusive focus on the implementation and monitoring of program outputs, at the expense of monitoring outcome and impact, has compounded this problem.

There are certain other trends which arise from the inability to match the changing contextual realities of Somalia with UNICEF Somalia's current program strategies. Given the rising inequalities in Somali society, an explicit focus on targeting the most vulnerable groups within Somalia is a future program policy that needs to be emphasized. The other dimension of this is the need for UNICEF Somalia to build a more sustained link with a particular cohort of vulnerable population groups. Narrowly defining a partnership, to mean only contracting of an activity or task of UNICEF Somalia's plan to an agency, rather than a wider collaboration, has meant partners following their own agendas that distort program results. Many of the above weaknesses arise because of the absence of an organic link that should exist, in a flexible manner, between planning and implementation processes.

While the program mix and prioritization of UNICEF's programs have been relevant for Somalia, the effectiveness of different program components has varied, both because of the implementation of partial program visions and difficulties in operationalizing planned program strategies. At the community level, where the situation in the three zones is similar, while the over-structuring of community management structures is not implementable, there is a need for a mechanism or medium of community facilitators who can continually link communities to all the programs of UNICEF Somalia in an inter-linked manner. At the wider systemic level, there is a need to balance cost sharing strategies with access to vulnerable groups on the one hand, and the need to invest in more comprehensive training and human resource development frameworks on the other.

Recommendations

The key issues and recommendations that flow from the above evaluative analysis cover the three dimensions of program development, program strategies and program implementation. Underlying the specific recommendations are four areas of concern within the Somali social sector, issues on which UNICEF Somalia has worked on but needs further prioritization and fine tuning. These are: developing, at the micro-level, appropriate models for the management of different social services; the need for building up potential institutional structures and human resource pools for delivering such services and strengthening their quality on a continual basis; the need to reach the more vulnerable groups and communities within the broader Somali society on a continuing basis; and the need for gender mainstreaming and realizing the rights of women in the Somali context.

There are also program needs and development possibilities in the areas of child protection and early childhood development, particularly in the urban areas. The transfer of program vision to implementation needs greater emphasis in the areas of hygiene education, women's rights, HIV/AIDS awareness, and alternative education for out-of-school children. Above all, there is a need for inter-linked implementation of all UNICEF's programs of support to the same group of vulnerable children and women in a specific location (settlement or village) within Somalia.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Date:
2002

Region:
ESARO

Country:
Somalia

Type:
Evaluation

Theme:
Program Review

Partners:

PIDB:

Follow Up:

Language:
English

Sequence Number:
2002/013

New enhanced search