Evaluation database

Evaluation report

2000 KOS: UNICEF Preparedness and Response in the 1999 Kosovo Refugee Emergency



Author: Greene, Stevens, Madi, Telford

Executive summary

Background

UNICEF was a key partner for DFID in the humanitarian response to the Kosovo crisis. It is important for DFID to now evaluate UNICEF programmes. This includes examining how UNICEF adapted to the rapidly changing situation, the overall quality and appropriateness of the response and what lessons can be learnt for DFID and UNICEF.

Purpose / Objective

The overarching purpose of this evaluation is: to assess the extent to which UNICEF's programmes met their objectives and to draw lessons from this assessment for future improvements; and to assess the extent to which UNICEF's programme objectives fed into DFID's overall strategy for the region.

Secondary purpose is to identify areas of UNICEF management and delivery capacity requiring strengthening, thereby helping UNICEF further define their work plan for recent funding that DFID is providing to strengthen the capacity of the orgainsation to respond to crisis globally. Also, to provide a benchmark of performance upon which any future improvements can be measured.

Methodology

A preparatory generic literature and documentation search and review were conducted. Interviewing was mainly semi-structured and included programme beneficiaries. Email and telephone conversations were held with key people not available for interviewing. Short participatory workshops and group consultations were held with UNICEF partners and staff. Evaluators attended programme coordination meetings and visited regions of significant existing or previous emergency programme relevance in Albania, Macedonia and Kosovo (including Serb enclaves).

Key Findings and Conclusions

Strengths:
UNICEF displayed a number of actual or potential strengths in relation to this crisis. Most are rooted in the organization's 'value-added' as a 'developmental' (as opposed to 'emergency') agency:

Catastrophe was avoided and major epidemics did not occur. The emergency response was largely provided by the refugees themselves and the host families. UNICEF played its important part in this achievement. Participation of, and support for, national and local levels through UNICEF-sponsored programmes has been high relative to many, even most relief agencies. Governmental (i.e. line ministries), national NGOs and local communities and groups were more palpably, effectively and more spontaneously supported and guided through UNICEF programmes than by those of major UN sister agencies, for example.

The organization's work can be categorised as people-focused (children and women, but also families and communities). Its experience in dealing with human vulnerability is a welcome balance to the life-saving sectors, generally the main priorities of major relief agencies. Advocacy and actual protection for child rights (under the CRC) are evident in particular in the fact that, by and large, education was provided to refugees and gradually to returnees, in this emergency.

The quality and commitment of UNICEF has largely been exceptional. The predominance of national staff in middle management positions had a direct impact in that it facilitated a deeper understanding of needs and realities in the three countries visited than is the case for international agencies managed almost uniquely by internationals, most of whom do not speak relevant languages. Though all senior management are male (Heads of office -- Area level, and the three main offices visited), the predominance of females at a middle management level is a heartening trend.

UNICEF's presence before and after the crisis has provided it with a level of recognition, trust and knowledge that facilitated its programmes over and above that of certain sister agencies. This longer-term continuity is a major and much needed strength in emergencies. UNICEF support to the UN educational structures in Kosovo, through the secondment of an adviser, is an example of this longer-term vision. (This strength was not exploited systematically and fully, either before or during the crisis, however. Such a comparative advantage can be developed to greater benefit in emergencies particular in emergency and disaster mitigation, preparedness and response. Specifically, this would imply detailed contingency planning with targeted counterparts, including both line ministries and non-governmental 'grass-roots' associations. It would also imply regular, as opposed to one-off, training and preparedness measures. During a crisis, it would imply a closer and more supportive relationship with these counter-parts, including material, human and technical support to ministries to coordinate and implement emergency activities.

UNICEF's renowned skills at public information and communication made it a potentially effective vehicle for channeling guidance to the public and agencies alike. (When this was not accompanied by adequate physical presence, it did, however, lead to a degree of skepticism on the part of international agencies in particular.)

Coordination on key aspects (e.g. education) and in certain locations was most effective.

The existence of a wealth of UNICEF emergency experience and expertise is a potential strength. This is manifested by the existence of the UNICEF emergency handbook, a variety of guidelines and mechanisms (e.g. 'Rapid Educational Response'), and by EMOPS.

Finally, UNICEF's presence and activities in asylum countries including Montenegro, Kosovo and other areas of Serbia/FRY, underline the importance of adhering to basic humanitarian principles as established under international humanitarian, refugee and human rights law (e.g. that victims shall be assisted and protected irrespective of any political consideration). While its activities may have been modest in non-Kosovar areas of Serbia, activities such as psychosocial attention for children under bombing are clearly a humanitarian activity and deserve recognition.

Weaknesses:
A number of important UNICEF weaknesses were evident in the preparedness for and response to this crisis. These center on UNICEF's capacity and perhaps vocation for emergency preparedness and response. They should be analysed in the light of constraints and opportunities outlined in the full report. It should be noted that UNICEF has been taking steps to address many of these issues:

Emergency contingency planning and preparedness, as an integral part of UNICEF programmes, have not been systematic in these countries. Despite preparedness activities and meetings, one might expect it to be a central component of its developmental approach. (While the level of this crisis may not have been foretold, Skopje, for instance can expect an earthquake to follow the disaster of 1963. Albania, regrettably, needs to prepare for possible disorder of the nature of that experienced in 1997. Thus, greater practical preparedness measures could have been expected.)

Though highly effective individuals were deployed, some rapidly, human resources management in this emergency was slow and somewhat erratic in getting the right person to the right place at the right time. Constraints did exist, as explained in this report. Permanent field presence outside capital cities in particular, and also programme and operations capacity was frequently found to be weak. This was in terms of number, seniority, support, preparation and duration of presence. Rotation of internationals was high despite inter-agency arrangements whereby certain functions or responsibilities were covered by other UN agencies. UNICEF needed to have sufficient staff to cover key functions. Liaison with the military is an example. Liaison staff might well have helped ensure that 'child rights' and 'child-friendly' standards were better respected by military camp planners.

The emergency supply and logistics chain was neither sufficiently rapid nor reliable for the scale of this emergency.

While emergency financing was not a major difficulty in the crisis, administrative procedures and perhaps culture, resulted in significant delays.

Uniform tracking, monitoring and evaluation systems and approaches were not applied in the emergency, nor was adequate capacity available for these functions. While not the case in every country visited, tracking of the distribution of supplies was dramatically inadequate in a significant number of cases.

Implementation of core activities was inadequate in certain geographical areas.

Water and Sanitation was not conducted or coordinated systematically as a core activity throughout the emergency by UNICEF. This role was expected of UNICEF by some international agencies.

The impact of UNICEF's policy advocacy on international agencies was inadequate on important occasions (e.g. on breast milk substitute. Also, Child Friendly Spaces, which wasn't understood sufficiently as a shared holistic endeavor, but rather than as a set of guidelines to be followed rigorously to address all issues). In some cases, this is a matter of inadequate communication of those policies, including within UNICEF. In others, it reflects inconsistency and perhaps a lack of clarity and even mistaken approaches.

Operations planning was not a major strength in the operation. Similarly, monitoring, evaluation and reporting requires a more consistent and systematic approach, staff training, and numbers and tools.

Based on the experience of this crisis, interviews with UNICEF staff experienced in emergencies worldwide, and UNICEF reports, emergency response capacities remain to be mainstreamed into UNICEF. This is despite the wealth of experience gained in emergencies worldwide and lessons learned repeatedly. While the Martigny process is welcome and timely, there is still a lack of clarity institutionally about how this should be addressed.

Recommendations

Based on its strengths, UNICEF has had, and should continue to have, an important role in emergencies. Donors should continue to support fully the development of this role. Greater clarity is required on the desired scale and specific expertise of that emergency capacity.

UNICEF's strengths in emergencies need to be exploited and developed more systematically. To do this, first the issue of whether and how emergency preparedness and response can be mainstreamed into what are essentially development programmes can be achieved. This debate appears to remain to be resolved. If such mainstreaming is to take place, every country office should be capable of, and required to, engage in comprehensive preparedness measures as an integral part of its country programme. Similarly, the quality of UNICEF's guidance on core UNICEF emergency issues (mine awareness, breast milk substitute), and the depth and systematic professionalism of its preparedness activities (e.g. capacity building, emergency vulnerability analysis and contingency planning and actual preparedness measures) need to be improved.

Despite UNICEF's declared objective to mainstream emergency preparedness and response into its institutional capacities and general programming, this is far from having been achieved. DFID funding to strengthen UNICEF emergency preparedness and response is timely. There is no doubt that the organization can benefit from such long-term investment.

A comprehensive review of all the major components of emergency preparedness and response is required. This should lead to an action plan with targets and deadlines for improving significant weaknesses that exist at the institutional level. To be comprehensive, it ought to cover policy, resources (financial, material and, above all, human) and systems, procedures and tools. According to experienced UNICEF staff, the broad conclusions contained in this report are not new to UNICEF. The process should build on experience acquired throughout years of emergency response. Lessons learned during the development of EMOPS and other relevant capacities, structures and systems (e.g. supplies in Copenhagen) need to be retained and applied.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Date:
2000

Region:
CEE/CIS

Country:
Serbia and Montenegro

Type:
Evaluation

Theme:
Emergency

Partners:
DFID

PIDB:

Follow Up:

Language:
English

Sequence Number:
2001/800

New enhanced search