Evaluation database

Evaluation report

2013 Somalia: Final Evaluation of the Unconditional Cash and Voucher Response to the 2011–12 Crisis in Southern and Central Somalia



Author: Kerren Hedlund, Nisar Majid, Dan Maxwell, and Nigel Nicholson

Executive summary

"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report."

Background:

The Evaluation of the Unconditional Cash Transfer and Voucher Programmes in Southern and Central Somalia includes all unconditional cash and voucher programmes implemented by 15 NGOs between August 2011 and December 2012 in response to the crisis. The evaluation serves to determine the appropriateness, efficiency and effectiveness of these cash transfer programmes (CTPs), their coordination and independent monitoring, the latter outsourced to the Overseas Development Institute (ODI).

The subject of the evaluation is one of the largest, most significant cash and voucher distributions ever implemented in such an extreme and complex humanitarian crisis. As such, donors and humanitarian agencies implementing cash and voucher programmes since August 2011 requested an evaluation of their work, to serve the dual objectives of accountability and learning:
• assess and report on the performance (appropriateness, efficiency, effectiveness, connectedness) and results (impact) of unconditional cash transfer programmes in Somalia; and
• determine the reasons for observed success or failure and draw lessons from experience to produce evidence-based findings, allowing the humanitarian community to make informed decisions on appropriate humanitarian response in Somalia and elsewhere, and improve programme design and implementation wherever possible.

Purpose/Objective:

The objective of the evaluation is to assess the efficiency, effectiveness, accountability and impact of the emergency cash transfer/vouchers intervention in Southern Somalia, including its monitoring system.

Methodology:

a. Design of the evaluation

Only qualitative data will be collected by the evaluation team during focus group and interviews.  The, already collected, ODI quantitative data will be available to the evaluation team. All regions, where activities have been implemented will be targeted by this evaluation, as well as all partners. Due to security reason one or more regions might not be accessible at the time of the evaluation. The sampling per region will be developed by the evaluation team in collaboration with UNICEF. The evaluation subjects will be vulnerable households, with a specific attention to under-five children. Each implementing partner will be evaluated individually; in addition a collective impact analysis will be done by the evaluation team.  The evaluation team is expected to go the field to meet with field teams, hawalas and if feasible conduct focus group with beneficiaries, local elders… if and where security conditions will allow it. In this regard, the evaluation team is expected to take the households sample from the overall set of targeted households and not only those that are easily accessible.

b. Learning objectives of the evaluation

1. Assessing the efficiency of the cash/vouchers distribution system
2. Assessing the effectiveness of the cash/vouchers assistance as an emergency intervention at scale
3. Assessing the accountability (to beneficiaries and donors) of the emergency unconditional cash/vouchers program
4. Assessing the impact of the cash/vouchers distribution
5. Assessing the efficiency of the coordination mechanisms of the emergency unconditional and cash/vouchers program
6. Assessing the joint monitoring system

Findings and Conclusions:

The unconditional cash and voucher response, though largely implemented after the peak of the crisis, quickly achieved an impressive scale, building principally on international and Somali NGO field capacity.  The evidence marshalled in this evaluation suggests that cash and vouchers made a quantifiable difference in reducing hunger and improving food security, enabling a more rapid recovery than would have been possible without assistance. This was achieved within an extraordinarily difficult operating environment that required significant risk-taking by organisations and individual staff members.

Attempting a large-scale cash and voucher intervention was therefore appropriate, based on the analysis available at the time and the consequences of inaction. Contrary to initial concerns, cash transfers at scale did not result in food price inflation to the detriment of the most vulnerable. Rather they ensured access to critical food and non-food items and services. Both cash and vouchers were largely appropriate to the context where they were applied, with some caveats regarding unsubstantiated assumptions about beneficiary spending that inclined agencies to implement voucher programmes.

Given the Somali aid environment, corruption and diversion were an acknowledged (and accepted) risk.  Unsurprisingly, the evaluation raises issues of misuse of funds. Evidence suggests that these were less serious than comparable in-kind interventions, but still could have been countered through better risk analysis and preparedness and were not sufficiently identified by the monitoring systems.  Future emphasis should be on prevention; sharing lessons learned about diversion, effective M&E, and how best to conduct the rigorous investigation that must follow allegations of abuse.

Recommendations:

This Evaluation of the Unconditional Cash and Voucher Response in southern and central Somalia provides an independent analysis of the appropriateness, effectiveness, efficiency and impact of the cash response, with a strong emphasis on learning for future humanitarian interventions using cash globally, and in Somalia specifically. The findings should be considered in the context of one of the most difficult humanitarian operating environments in the world. As with any humanitarian response, particularly one implemented at scale and under duress, there were many aspects that could have been improved. The following recommendations found in chapter XV attempt to take into account the difficult operating environment while renewing or reinforcing the humanitarian communities’ commitment to humanitarian principles in Somalia and ‘doing no harm’.

Lessons Learned:

One of the lessons learned is that given the particular challenges of targeting the most vulnerable, organisations should have initially focused on reaching a greater number of people over a shorter period of time, had the donors been able to support this. The evidence tends to suggest that, at least from the time the UCTP began until the deyr harvest at the end of 2011, household targeting was a waste of time and energy – irrelevant to both the humanitarian emergency of the moment and a socio-political context that effectively isolates and exploits the more vulnerable communities in Somalia. The time necessary to negotiate with those local actors who manage both access to so-called minority populations and the distribution of aid was not adequate. As a result, in many areas populations that historically – and again in 2011–12 – constituted a significant portion of the population affected by famine, did not receive aid proportional to their need. Targeting errors were further amplified given rapidly changing food security conditions. Retargeting was essential. However, in most cases retargeting was not sufficiently prioritized by the organisation concerned because of the considerable operational challenges involved.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2013

Country:
Somalia

Region:
ESARO

Theme:
Emergency

Type:
Evaluation

Institution:
UNICEF and Humanitarian Outcomes

Language:
English

New enhanced search