Evaluation database

Evaluation report

2012 Global: The Global Evaluation of Emergency Response Funds (ERFs)



Author: Dale E. Thompson, John Horekens, Nick Mander, Yvan Conoir, Mariane Arsenault, Tracy Wallis, Nadia Cherkaoui

Executive summary

"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is "Outstanding, Best Practice", "Highly Satisfactory", "Mostly Satisfactory" or "Unsatisfactory" before using it. You will find the link to the quality rating below, labeled as 'Part 5' of the report."

Background:

This is the first global evaluation of the ERF mechanism. Prior evaluations and audits were of individual ERFs and were country level evaluations or compilations of country cases. In response to OIOS audit of OCHA's management of emergency response funds in 2010, it was determined that a comprehensive evaluation of the ERF mechanism would be conducted triennially. This evaluation was also occasioned by requests from donors to look more broadly at the ERF mechanism in general as opposed to looking at it from a disaggregated country approach. The evaluation covers all ERFs currently in operation with the exception of the Syria ERF, which was established in early 2012 and deemed to be beyond the scope of this evaluation.

Purpose/Objective:

The results of the evaluation will at the global level inform the review of ERF Guidelines in 2013 and development of policy in relevant areas. At the country level, the evaluation is expected to lead to improvements in ERF management, processes and operations. The evaluation recommendations will be addressed through the Management Response Plan as per OCHA Evaluation Policy.

The Objectives of the evaluation are to:
• Provide an independent assessment of the contribution of ERFs to the humanitarian community's ability to address critical unforeseen humanitarian needs in a timely and effective manner;
• Examine the contribution of ERFs to strengthening the leadership of Humanitarian Coordinators, coordination role of the cluster system, and building partnerships (in particular with national and local NGOs);
• Examine the role of the elements of OCHA that plan, administer and report on ERFs, map the progress made since 2009 in improving the effectiveness and efficiency of ERFs, and identify areas of strengths and weaknesses.

Methodology:

The evaluation utilized a wide range of methods, including document review, key stakeholder interviews and small group meetings. Two electronic surveys were mounted. The first was directed at OCHA Headquarters managers and the staff and management of OCHA country offices where an ERF is in operation. The second was an external survey of nearly 1,000 NGO representatives, primarily at the country level, who had received an ERF grant over the last three years, combined with key global NGO and UN partner stakeholders. The evaluation was based on the existing ERF Results Framework, augmented by a provisional Theory of Change. Five country case studies were conducted: Afghanistan, Colombia, Ethiopia, Indonesia and the Occupied Palestinian Territory. The evaluation process was supported by an advisory body consisting of the key external and internal stakeholders of the ERF, which has reviewed and approved all tools and methods utilized.

There were some limitations to the process. First, it was not well understood by field level staff that the case studies were not in-depth reviews. An additional challenge lay in a dichotomy related to the focus of the evaluation. As part of the Inception process, it became evident that there were tugs and pulls between those stakeholders who were seeking a global assessment of procedures and operational practices, and those who were seeking a more strategic assessment of the overall value of the ERF mechanism. This ambiguity was one between a process evaluation versus a strategic one that would be concentrating on the higher level objectives for ERF - such as how they fill gaps, how they build synergies and how they improve the effectiveness of the humanitarian architecture as a whole. In the end, the evaluation tends to be more focused at the strategic level.

Findings and Conclusions:

Before laying the Findings of this evaluation, it is very important to stress that notwithstanding some of the issues that are raised with respect to specific shortfalls, the ERF mechanism has, and is, making valuable albeit limited contributions to the attainment of its strategic objectives.

It is important to underscore this positive Overarching Conclusion:

The ERF mechanism works; however, like any process, is in need of continual adaptation and evolution so that it can remain relevant. 

The above general conclusion points to the overall positive value of the ERF mechanism. In addition, it is beneficial to preface the specific findings that follow with a list of some of the highlights of Best Practices that were observed. More can be found in the Final Report itself.

Recommendations:

The Report presents 15 Recommendations. They are presented in three broad categories: Strategic, Operational and Performance Management. All these recommendations have a relatively short timeframe, a maximum of two years, so as to ensure implementation before the next global evaluation, which is anticipated in three years. They are described in full in the Final Report.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search