Base de données d'évaluation

Evaluation report

2011 Global: Independent Review of UNICEF's Operational Response to the January 2010 Earthquake in Haiti



Author: Abhijit Bhattacharjee, Alex Wynter, Jock Baker, Mathew Varghese, Catherine Lowery

Executive summary

Background:
This ‘Independent Review of UNICEF’s Operational Response to the January 2010 Earthquake in Haiti’ was commissioned by the Office of the Executive Director (OED) and conducted from November 2010 to February 2011. While following an evaluative approach, it did not constitute a full-scale evaluation aimed at exhaustively documenting the results UNICEF achieved or did not achieve for children and women in Haiti or the many factors that have affected its response since this extraordinary disaster. Rather, as a review, it focused more narrowly on identifying key internal systemic factors that helped or hindered UNICEF’s collective organizational response in the first three months after the earthquake. Its recommendations thus concentrate on the operational performance of UNICEF's internal system for emergency response.

Overall, the review team found UNICEF’s early response to be marked by rapid reaction in the earthquake’s immediate aftermath, followed by inconsistent performance soon thereafter. UNICEF-led clusters were activated immediately but, with the exception of the water, sanitation and hygiene (WASH) cluster, their leadership remained weak and unclear. Programmatically, some sectors delivered critical interventions quickly – if not entirely to scale – even in areas where UNICEF’s work had been negligible beforehand. Notable examples include its timely mobilization in water delivery and its key role in the reopening of schools and ensuring that these were well stocked with supplies. In contrast, sanitation solutions were expensive and ineffective in what was a largely urban disaster – something most humanitarian agencies had little previous experience of.

In addition, UNICEF’s stance on child protection in Haiti, nuanced and outspoken before the quake, became muddled in the immediate aftermath and the organization was unable to break through early on and seize the debate. Effective action on gender-based violence was stymied by the fact that this issue is not yet sufficiently integrated or mainstreamed into UNICEF’s work in all programme sectors. Nutritional interventions were undertaken, but without a clear sense of what the highest-priority needs were, and its achievements in health have been difficult to systematically document.

In seeking explanations for UNICEF’s mixed performance in delivering results in Haiti, the review highlights a number of factors related to both its systems and culture. Key drivers underpinning the organization’s positive accomplishments include systematic action by its Supply Division, supported by clear procedures, to pre-position and rapidly deliver supplies. In addition, the establishment of Life-Line Haiti (LLH) in neighbouring Dominican Republic provided a vital means of ensuring UNICEF’s response in light of unprecedented physical destruction in the country itself. The review also takes note of an organizational culture in which appropriate risk-taking, though by no means the rule, is sometimes undertaken in order to achieve results. Finally, UNICEF is credited for actively engaging the Government of Haiti in select clusters, an overture that proved vital for a coordinated response in the sectors in which it was undertaken.

Explanations for why and where UNICEF failed to consistently act nimbly or effectively in Haiti point to a range of larger impediments in its internal systems and culture. Curiously, many of these weaknesses appear to be some of UNICEF’s key strengths in smaller-scale emergencies and non-emergency settings. These include the following:

1. Within UNICEF’s highly decentralized structure there was a fundamental lack of clarity about who was in charge, and formal accountabilities were hard to pinpoint. Despite OED’s effort to push for a faster response, the organizational culture led to a default insistence on supporting the country office (CO) in taking the lead (regardless of its actual capacity to fulfil this responsibility), rather than exploring more viable alternatives, and on decisions being subjected to a consensus-based ‘chain of consultation’ rather than the type of chain of command arguably required in these circumstances.
2. UNICEF’s surge staffing infrastructure had been disbanded. This led to a flurry of activity to quickly deploy staff – mostly internal from across the organization – for too short periods of time and with objectives and reporting lines too unclear to be fully effective.
3. UNICEF’s extensive rules and regulations – vital to a well-functioning, accountable organization during normal circumstances – proved insufficiently streamlined or flexible to enable an effective response on the scale required. Thus, with a handful of exceptions, staff were by and large loath to take risks, fearing more sanction for disregarding procedures than for taking well-intended risks that might fail to yield results.
4. Across almost all sectors and clusters, planning in the first three months was undertaken in a vacuum, with virtually no systematic needs assessment that might have informed subsequent cluster coordination and programme design, helped management monitor UNICEF’s progress over time or kept key stakeholders abreast of developments with accurate, reliable information about the situation of women and children on the ground. (The report does note, however, that the accuracy and reliability of information did improve after the immediate period of the response under review.)
All told, UNICEF’s challenges in Haiti depict an organization eager to tackle its responsibilities in emergencies but lacking some of the essential elements to do so effectively.


Purpose/ Objectives:
The purpose of the present review was to generate critical recommendations to help UNICEF ensure the most timely, predictable, effective and efficient response in future emergencies. In this vein, the review found that many of the findings and recommendations emerging here (at least those related to performance) echo those of previous reviews, evaluations and assessments of UNICEF’s action in other emergencies. It appears that the organization has repeatedly identified the same or similar lessons but has failed to absorb them.


Methodology:
The review’s methodology was based on both inductive and deductive approaches, using quantitative and qualitative data gathered from a carefully selected range of sources.


Findings:
1. Achieving results in the first three months
UNICEF’s contributions in key areas of immediate relief assistance – namely, water supply to affected communities in Port-au-Prince and provision of non-food items – are acknowledged by external stakeholders and in the inter-agency real time evaluation. UNICEF was able to engage with and mobilize government institutions and provided reasonably effective leadership to the WASH Cluster from early on. However, its leadership capacity in other clusters remained weak in the first three months, and programme results in areas such as nutrition, child protection and gender-based violence were sub-optimal due to understaffing at both programme and cluster levels.
2. Organizational factors affecting results
UNICEF mobilized an organization-wide response following the Haiti earthquake. However, weaknesses in surge deployment and operations support, time-consuming business processes, lack of a clear chain of command and weak performance tracking stymied the effectiveness of the response in the first three months.
3. Performance improvement – organizational culture and learning
A deeply embedded culture of risk avoidance, rather than risk management, both in individual action and in the systems to support it, hampered the response in Haiti. Lessons from prior emergencies that resurfaced in this one continually fail to be absorbed into the organization, indicating an institutional reluctance to undertake the radical changes necessary if UNICEF is serious about developing itself as a leading humanitarian player globally.


Recommendations:
There are 20 recommendations made in this report, of which the 9 clustered under ‘A. Humanitarian leadership and strategic management’ are the most critical. The other 11 recommendations relate to operational aspects and are important, but these by themselves will have little lasting impact unless the organization takes action on the strategic recommendations.

A. Humanitarian leadership and strategic management

Cluster leadership
R1: Particular challenges posed by sanitation illustrate the need for the WASH Cluster to develop approaches better suited to urban contexts (as recommended by the inter-agency real-time evaluation ), revise their assessment methodologies to understand local contexts and identify and deploy more cost-effective solutions to problems. (See Section 2.1.)
R2: Global cluster coordinators and country representatives should ensure that cluster coordinators are at the right level of seniority and experience, especially early on, and they should be treated on par with UNICEF programme heads. (See Section 2.9.)
R3: As part of ongoing training programmes, the Office of the Executive Director (OED)/Office of Emergency Programmes (EMOPS)/Programme Division (PD)/Division of Human Resources (DHR) need to ensure that all programme and operations managers, including country representatives and their deputies, are fully oriented on the concept of cluster lead agency and interagency processes in relation to cluster accountabilities. (See Section 2.9.)
R4: UNICEF needs to develop a cadre of highly trained information managers who can be deployed rapidly in any emergency and are able to support either its programmes or the clusters it leads. There is an opportunity to develop this as a career path for competent managers, as those trained and experienced in dealing with complex information management needs in large emergencies will be in high demand even in normal times in all countries. (See Sections 2.1-3, 2.1.5 and 2.1.9.)

Humanitarian accountability
R15: UNICEF defines three levels  of emergency response, corresponding to the scale of the emergency and CO capacity for managing it. The OED needs to clarify that while all COs need to have capacity to deal with Level 1 on their own as part of the country planning process, the organization does not expect Level 2 and 3 emergencies to be dealt with as part of normal country programme plans, and special emergency procedures will apply to these emergencies. (See Sections 3.4.1-3.4.2 and 3.4.4.)
R16: For Level 3 emergencies, regardless of the capacity of the CO or RO, the OED will take the overall leadership and provide strategic direction in order to facilitate an organization-wide response. This arrangement, called Corporate Emergency Procedure (CEP), will be declared within 2-12 hours after a major rapid-onset disaster. OED will immediately authorize the Director of EMOPS to act as Corporate Emergency Coordinator (CEC) for the response. However, operational decisions will still need to be taken at the country level. For this the CEC will designate an Operations Director (at least a senior D1 with humanitarian experience) for the country. The declaration of CEP will involve the following:
• The arrangement will initially be for a period of at least three months, to be reviewed at the end of this period. The CEP will only apply to emergency operations of a short-term nature. In any emergency response that involves long-term response (as in recovery and reconstruction, chronic emergencies, etc.), the default will be to handle these through established country programme planning processes, and it will be up to the CO to seek any assistance from HQ/RO.
• The Operations Director will report to CEC for operational aspects of the response and to the Country Representative for (a) all matters relating to representation to the national government; (b) any plans or commitments that go beyond the first six months; and (c) public communication and advocacy.
• The arrangement will have a built-in working mechanism to ensure that the CO can gradually take over all responsibilities for managing the programme from the fourth month onwards (subject to periodic review). All recovery and long-term programmes that arise from the emergency will be managed by the Country Representative; the Operations Director will have no authority to take decisions in this regard unless authorized by her/him. If after a periodic review of this arrangement it is felt that the CO is able to take over full responsibility for the operations, the Operations Director will no longer report to the CEC, and the Country Representative will determine whether the services of the Operations Director are needed. For this transition to happen, the Operations Director will need to work closely with the CO from the start.
• For all Level 3 emergencies, an early visit to the country by the CEC or someone designated by him/her will be mandatory. This will help the OED and the CEC get a real-time assessment of the situation and ensure that they can make appropriate decisions about resource mobilization and allocations and provide necessary strategic direction to the CO, Operations Director and Regional Director.
• The role of the RO in Level 3 response will be determined by the OED, in consultation with the Regional Director and CEC, depending on its mobilization capacity. The reporting line between the CR and Regional Director will remain as normal and the RO will play its normal oversight role vis-à-vis any recovery or long-term programme and planning. It needs to be clear that while the CEP is in operation, the CEC provides leadership for the response; the Regional Director remains responsible for ensuring that the CO is able to integrate the response into its own planning process and take over leadership and operational responsibility for the response as soon as possible.
(See Sections 3.4.1–3.4.2 and 3.4.4.)
R17: For Level 2 emergencies, the desired state will be for the RO to play the lead role, but given the limited capacity of ROs (and the fact that individual Regional Directors have varied experience and competencies in emergency response, and that emergency experience is not a key competency during their recruitment), the CEC will have an oversight as sometimes Level 2 emergencies (the Pakistan ‘superflood’, for example) can escalate to Level 3. For this reason, a joint assessment led by a senior emergency response team/designate of the CEC with the Regional Director (or his or her designate) must be undertaken as soon as possible. The purpose of the joint assessment mission will be to agree the modalities of the response. If it is felt by the CEC that RO capacity is weak for managing the overall response, and that the emergency could evolve into a Level 3, s/he will consult with OED and determine the respective role of the RO, HQ and CO. (See Sections 3.4.1–3.4.2 and 3.4.4.)

Culture
R19: UNICEF leadership needs to clearly communicate that a Level 3 emergency is not ‘business as usual’, but rather it is essential to change gear and resist the urge to revert to a culture of consensus in decision-making. The process needs to be kept lean. While attempts should be made to consult with whoever needs to be consulted, ultimate responsibility for determining the strategic direction for Level 3 (i.e., the shape, size and nature of a response) should rest solely with the OED assisted by the CEC as time is of essence in emergencies. Responsibility of various entities (Country Representative, Regional Director, Operations Director) for operations will be determined by the CEC (with guidance from the OED) at different points in time and depending on the capacity of each entity. (See Sections 3.2–3.4 and 4.1.1.)
R20: The OED and Regional Directors should ensure that performance appraisals of country representatives incorporate an assessment of their competencies for effectively engaging with corporate priorities, including the CEP. (See Sections 3.4, 4.1.2, 4.1.3 and 4.2.)

B. Operational management

Deployment and human resources capacity
R7:  In future deployments, recruiting managers need to make it mandatory for all surge deployments to be for at least nine weeks, especially for staff who will play supervisory, managerial or decision-making roles in the operation. (See Section 3.2.1.)
R8: EMOPS and DHR need to seriously invest in the development and management of a surge roster and in the creation of self-contained multi-disciplinary rapid response teams.  Taking lessons from the Supply Division on how it has developed deployment and human resources administration capacity, UNICEF needs to replicate similar arrangements for all emergency deployments, with adequate staffing located in EMOPS and working closely with DHR. (See Section 3.2.1.)
R9: The use of special service agreements and temporary assignment contracts from pre-screened external rosters should be fast-tracked by DHR for surge, especially for all recruitments for a duration of under 90 days and for one year.  (See Section 3.2.1.)
R10: DHR and Regional Directors should make sure in future emergencies that, instead of an elaborate programme budget review (PBR), a list of core staff is agreed for one year within four weeks of response with a detailed PBR to follow after three months for additional recruitments. (See Section 3.2.1.)
R14: To demonstrate that UNICEF values humanitarian work and expertise within the organization, recruiting managers need to ensure that humanitarian leadership competencies are taken into account in recruitment to senior positions, especially country representatives and their deputies.  (See Section 3.4.3.)

Operations support
R11: The Supply Division should develop dedicated capacity (through long-term agreements with suppliers or standby partners) to quickly set up accommodation and office facilities.  This must be in place for future emergencies, considering the difficulty and time needed to organize prefabs. The Haiti emergency also highlighted that there needs to be a team from Supply Division trained to set up accommodation. (See Section 3.2.1–3.2.2.)
R12: Working with the Supply Division, EMOPS needs to ensure that UNICEF has rapid response teams specializing in operations support that can be deployed immediately after a major disaster. (See Section 3.3.)
R13:  In large emergencies that require rapid scaling up of IT systems, Regional Directors and IT managers must ensure that highly experienced senior staff from the RO and HQ are deployed in the first eight weeks. (See Section 3.3.)

Administrative and financial procedures
R18: Working with the Division of Finance and Administration (DFAM), the Office of Internal Audit and ROs, EMOPS needs to further simplify key business processes in emergencies and synchronize current manuals on administrative and financial procedures in emergencies. (See Sections 3.3 and 4.1.1.)

Monitoring and reporting
R5: In an emergency response UNICEF will to a large extent be in the hands of its implementing partners, both pre-existing and prospective, despite the revised programme cooperation agreement (PCA) guidelines that incorporate tighter reporting requirements. For last-mile distribution data, UNICEF needs to simplify reporting formats and develop mechanisms for data gathering by and from partners. (See Section 3.1.2.)
R6: The use of the CCC benchmarks for reporting needs to be prioritized by CO management, and senior managers need to ensure that reports are based on outcomes rather than inputs. (See Section 3.1.3.)



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Recherche