2009 Global: EVALUATION OF DFID-UNICEF PROGRAMME OF COOPERATION: Investing in Humanitarian Action Phase III (2006–2009)
Author: Abhijit Bhattacharjee (Team Leader), Lewis Sida and Moira Reddick
“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”
The UK Department for International Development (DFID) has supported UNICEF through the DFID-UNICEF Programme of Cooperation to strengthen UNICEF’s capacity for humanitarian response and preparedness since the year 2000. This is the second and final evaluation of the Programme, which has been implemented in three main phases: Phases I and II covered the period 2000–2005, while Phase III ran from July 2006 From January 2006, UNICEF received GB₤1.9 million (US$3.4 million) as ‘bridge funding support’ to carry out ongoing and essential activities until there was greater clarity regarding Phase III of the project. The ‘Bridge Funding Phase’ was oriented as the ‘start-up phase’ of the larger 3½ year Phase III Programme of Cooperation. to the end of 2009. The first evaluation, assessed Phase I and Phase II of the Programme of Cooperation and was carried out at the end of Phase II in 2006. This independent evaluation was commissioned by UNICEF’s Evaluation Office to assess the implementation of Phase III of the Programme of Cooperation.
The purpose of the evaluation is to measure progress against the aims and results outlined in the Phase III Global Proposal of the DFID-UNICEF Programme of Cooperation and to provide direction to further UNICEF efforts to strengthen its humanitarian preparedness, response capacity and role in the inter-agency humanitarian reform process. The evaluation will be used to shape future efforts to strengthen humanitarian action and will support UNICEF’s future capacity-building efforts to enhance national capacity development. It will provide lessons to guide organizational capacity-building partnerships at both the government and humanitarian actor levels, particularly focusing on UNICEF’s contributions to inter-agency humanitarian response.
The objectives of the evaluation were to:
- make an objective and independent assessment of the current status of UNICEF's capacity in humanitarian action, highlighting systemic issues and gaps to provide recommendations on priorities and strategies for future capacity building;
- provide an overall assessment of the relevance, effectiveness, efficiency, impact and sustainability of Phase III of the DFID-UNICEF Programme of Cooperation; and
- draw lessons on policy and programming for building the capacity for humanitarian action.
As its main analytical framework, the evaluation has used a ‘non-strategy’ paper developed by the Office of Emergency Programmes in late 2005. This paper formed the basis for UNICEF’s priorities in building its capacity for humanitarian action. The Phase III proposal and subsequent agreement between DFID and UNICEF drew on this non-strategy paper and its priorities, although in the end it concentrated on a narrower base. Drawing on the first evaluation of the Programme of Cooperation, and the Humanitarian Response Review, this non-strategy paper set out five key areas to be strengthened to improve overall response. These five key areas and six outcomes from the DFID-UNICEF Programme of Cooperation formed the basis of the work done during the period under evaluation.
The evaluation employed the normal range of social science research methods common to humanitarian evaluation, namely:
· A briefing and scoping visits; comprehensive document review; five country visits and observations; online surveys: two sets of questionnaires - one for external key informants and another for internal (UNICEF) key informants; key informant interviews; semi-structured and structured interviews with a range of stakeholders including beneficiaries, UN agencies, NGOs, partners, donors, international organizations and governments; and focus group discussions and semi-structured group discussions with external stakeholders during country visits.
Findings and Conclusions
Detailed findings and conclusions from the evaluation can be found in the body of the report. To highlight a few:
… the overall conclusion of this evaluation is that UNICEF has improved its emergency capacity on many fronts during Phase III of the programme. This is particularly the case in its cluster responsibilities, despite having taken on three clusters and two sub-clusters – the largest by any single organization. Whereas the 2006 Phase I-II evaluation found UNICEF weak on reliably delivering humanitarian response, this evaluation concludes that UNICEF’s response is mostly reliable.
… UNICEF has made good progress in implementing its cluster commitments, although there remains a need for a more coherent vision of what cluster lead means for the organization at a corporate level.
… there are still indications that humanitarian action (defined as preparedness, response and early recovery) is not viewed as a core activity within the wider organization. Responses remain largely dependent on the skills and experience of the country representatives and other key staff, and humanitarian experience is still not an essential requirement for these positions.
…whilst there has been good progress in streamlining administrative and financial systems and there are encouraging signs of change to come – where it counts, on the ground in day-to-day work – there are still bureaucratic delays at the expense of timely response. There is a need for greater clarity in accountability for UNICEF’s humanitarian action, and the new risk management framework being put in place should give more room for creative decision-making.
Organizational vision and accountability for humanitarian role
R1. The humanitarian programme needs to be integrated into the MTSP as a core area of UNICEF’s work.
R2. UNICEF urgently needs policy guidance for the whole organization on the implications of the cluster approach and in particular the provider of last resort clause. This should ideally take the form of a directive from the Executive Director. While doing so, it needs to also clarify:
· the links between UNICEF’s emergency response/programme and cluster approach;
· the links between cluster lead staff and other emergency staff and programme sections; and
· the relationship between emergencies and programme divisions and the role of each in UNICEF’s emergency response and cluster leadership.
R3. Senior management need to endorse a results framework for UNICEF’s humanitarian objectives as set out in the new CCCs and make clear how all relevant departments, Country Offices and Regional Offices will be held accountable for achieving these. (Note: Evaluators have noted that work is now underway to develop such a system across the organization).
Training, learning and evaluation
R4. UNICEF needs to develop a simplified system for information management and monitoring in emergencies, based on the new CCC benchmarks, and ensure that trained information management and M&E staff are deployed in all emergency operations.
R5. Clusters and gender integration need to be part of induction and training programmes for all staff members, particularly senior country management staff.
R6. Humanitarian experience should be made a mandatory requirement in all country representative recruitments.
Human resource issues
R7. UNICEF needs to continue to strengthen the global web roster and standby roster and invest in human resource capacity in Regional Offices with emergency focal points.
R8. Systematic induction and regional and country-based training should be put in place for cluster coordinators.
Administrative and procurements issues for emergencies
R9. Countries with regular emergencies ought to have pre-positioned supplies for rapid response. Additionally, Country Offices must have framework agreements with suppliers for local and regional procurement.
R10. A best practice review should be undertaken on administrative and associated contract management systems in the context of emergency response in different countries, and lessons drawn from this for replication.
R11. UNICEF needs to make better use of partners to lead clusters at sub-national level and provide more of a focus at the strategic level – e.g., preparedness, resources, contingency stocks, etc. It also needs to make greater efforts to establish cluster coordination closest to the affected area rather than simply in the capital.
R12. UNICEF needs to have in place the capacity for directly managing programmes in the case of ‘last resort’ operations. This would mean having enhanced surge capacity and logistics and supplies system.
R13. UNICEF and other cluster lead agencies need to have a joint dialogue with IASC to clarify for agencies the responsibilities, accountabilities and limits of the ‘provider of last resort’ principle.
R14. UNICEF needs to document the actual cost of clusters so that it can have a sensible dialogue with donors about how to resource them. A short study outlining the costing implications should be prepared by UNICEF as a first step to engaging in larger inter-agency dialogue with donors.
R15. As UNICEF completes the Project Coordination Agreement revision process currently underway, systems for disbursing and administering funds to NGOs also need to be streamlined. A follow-up process to the global consultation in 2008 should be initiated, and NGOs need to be more consistently involved in strategic partnership dialogue.
R16. UNICEF needs to engage in an internal debate involving country managers on how to deal with the issue of delicate balancing a constructive relationship with governments with the organization’s ability to provide impartial and neutral humanitarian response when children are affected by natural disasters and conflict.
Advocacy and monitoring mechanism
R17. UNICEF needs to play a greater advocacy role with donors (like DFID and others), who need to recognize that well-functioning clusters and UN technical agencies alone will not take care of humanitarian needs, and that the donors have a strong role to play in engaging with governments when it comes to humanitarian access in situations where the government itself is unable to meet the humanitarian needs.
R18. The tools to undertake the monitoring and reporting mechanism need further development. Examples like the Sudan database and the Colombia baseline study are encouraging, but a suite of tools is needed that countries such as the Central African Republic can offer partners.
R19. UNICEF and donors need to commit to resourcing UNICEF’s role in the monitoring and reporting mechanism on a long-term basis.
R20. UNICEF needs to do more to inform partners about its advocacy successes such as the release of the Sudanese child combatants.
Lessons Learned (Optional)
Numerous lessons in emergency preparedness and response are given in the report. Based on the experience in implementing the cluster approach in the past two and a half years, several critical issues and lessons are emerging that need addressing:
· There are weak linkages between global cluster initiatives and country realities.
· Whilst clusters often exist at a national (capital-based) level, they are often not as strong at a sub-national level where the emergency operation is then taking place.
· There is still ongoing work in ensuring that the position of cluster coordinator should be a separate function of internal technical staff and that it stresses the skills needed for cluster leadership.
· There are unresolved resource issues, in terms of both how cluster leadership is funded and how cluster members access funding through UNICEF.
· Accountabilities for Cluster Coordinators are not consistent across responses.
· UNICEF has dutifully implemented clusters, but appears to be lacking a coherent vision for what the new approach might mean.
· There is no shared understanding of the ‘provider of last resort’ provision.
· Information and knowledge management for UNICEF-led clusters remains a challenge.
Full report in PDF
PDF files require Acrobat Reader.