Evaluation database

Evaluation report

2012 Ethiopia: End Term Evaluation of Capacity Building Strategy in remote zones in SNNPR with teams of NUNV experts



Author: BDS Center for Development Research

Executive summary

"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is "Outstanding, Best Practice", "Highly Satisfactory", "Mostly Satisfactory" or "Unsatisfactory" before using it. You will find the link to the quality rating below, labeled as 'Part 2' of the report."

Background:

Although the SNNPR has made a considerable progress towards attaining its national development objectives, it still faces a number of significant challenges in the implementation of several development policies and plans. This is particularly perceptible in the remote zones and woredas of the region.  To alleviate these challenges, the region, in collaboration with UNICEF has implemented a Capacity Building Strategy by employing NUNVs since 2007 in three remote zones and one special woreda.  

Purpose/Objective:

With the closing stages of the strategy in December 2011, it was crucial to have the end of strategy evaluation to determine whether this intervention should be terminated or continued and thus this study was conducted with the aim of evaluating to see if the deployment of UNVs in the three zones had brought positive change in terms of building the capacity of program implementers.

Methodology:

1) Secondary data
Document Review: The evaluation team has reviewed key documents to identify key issues in the implementation of UNICEF’s capacity building programme. This activity includes reviewing related  existing strategy related documents, initial proposal, 2008 capacity building gap assessments,  2008 and 2010 assessment reports, revision of the modalities in 2011, meeting minutes for UNICEF ECO and UNDP/NUNV departments, relevant concept notes, field monitoring reports, annual review reports and woreda planning documents. Based on useful insights obtained from these documents, appropriate evaluation questions and interview guides were prepared and used to collect data through Key Informant interview, Focus Group Discussion as well as structured observation. 

2) Primary data
I) Interview with stakeholders- Key informant interviews were undertaken with stakeholders that are found at Federal (MoFED, UNICEF and UNDP/NUNV), regional, zonal and woreda levels. The informants were requested to discuss their expectation from the strategy and the result to date, challenges, constraints and the way forward. Efforts have also been made to ask informants for how long they have been working at their current positions as well as the major challenges (such as turn over) they have encountered. More specifically, the key informants interviews conducted  include: 
o At the national level, four key informant interviews were conducted from both government (MoFED) and UN offices (UNICEF, UNDP/NUNV). 
o At the regional level, two separate interviews were conducted; one with regional experts from the five sector offices and the other with stakeholders/BOFED officials/others. In total, eight experts were interviewed from the five sector offices.
o Interviews with the three zones and Konta special woreda /SW/government officials.  in each sector and Administration Council.  At the zonal level, three checklists were prepared - one for zonal officials from the five sector officials; the second for stakeholders/ZOFED officials and the third checklist for NUNVs teams. The informants were asked about the integration of the NUNV teams and their implication in the programme management i.e. in the planning, implementation, and monitoring and evaluation processes. For Konta SW, we had two interviews; one for woreda officials from the five sector offices and the other for stakeholders/WoFED officials. A total of 22 experts were contacted for the in-depth interview.
o Interview with a sample of 2 woreda offices per zone in each sector. Six woredas were selected in consultation with UNICEF technical team from the 3 zones. A total of 43 experts from different sectors were interviewed. The woreda officials explained about the overall contribution of the strategy in the planning, monitoring and reporting of the woreda activities and other related technical issues.
o Interview with NUNVs- A total of 11 NUNVs from the three zones were interviewed on condition of work (transport, equipment), adequacy of payment (allowance, DSA), modalities of field work. 

II) Structured Observation: Field observation is useful to obtain timely information by observing the general environment which will help to generate insights and findings that can serve as a base for further analysis of the collected data through other means. On top of that this method is critical to complement collected data is used to understand the context in which information is collected. Despite the shortness of the field visiting time, such observation enabled the evaluation team to critically assess and understand the NUNV’s overall working environment in the targeted three zones, and the special woreda as well as sampled woredas in the three zones.

Findings and Conclusions:

The Evaluation results clearly indicated that the NUNV programme has been providing valuable service in the areas of HIV/AIDS, water and sanitation, education, health and nutrition. Zonal and woreda officials stated that the support provided by NUNV members was consistent with their plan as NUNVs provide technical assistance based on the plan of the woredas and on the plans of the zones since the introduction of the new modality. With supports from the NUNVs, zonal and woreda officials enjoyed improved participatory approach in planning, technical support, resource sharing (vehicles), integrated supervision activity and sharing of feedbacks. The NUNV team supported the different sectors during field visits for monitoring, report preparation and liquidation of the strategy resources and in these regards the strategy was implemented effectively as expected. They also assisted the sector offices in developing proposal to finance the gaps.

In addition to this, the NUNV team provided supports in terms of strengthening the institutional capacity of the zone and woreda for service delivery, implementation and monitoring of the programmes on the ground. This means, moving from a remote control management to the one that is more robust and takes into account the concrete reality and actual situation in the remote zones and woredas. In conclusion, the NUNV programme appears to be working very well especially in strengthening zone and woreda capacity for expedited services delivery on the ground. However, there is still an immense capacity gap in the remote zones and woredas that requires the support of programmes like the NUNV scheme.

Recommendations:

The NUNV programme has contributed immensely to bridging critical capacity gaps in support of the development process in the SNNPR. Working to improve capacity building is not the responsibility of donors, local government, or capacity building providers alone. Therefore, a concerted effort by all involved parties is needed to ensure that capacity building contributes mean-ingfully to a sustainable development. Based on the findings of the evaluation result, one of the following major options of NUNV has been identified to be implemented for the future: 

1. Maintaining the Current Modality: Almost all interviewed sector official at the regional, zonal and woreda levels and the NUNV team members recommended the continuation of the current modality which has been in place since April, 2011. Furthermore, the center for the NUNVs being at the zone level rather than at the woreda level, as mentioned by all actors, enables the NUNV team to work in all woredas in the zone in coordination with zone level sector officials. This modality is the most fevered by all actors and the evaluation team also recommend this modality as the most effective among the other modalities for it enables the UNV team and other Zone level officials to make concentrated efforts by deploying the UNV team and other resources form the zone to the woredas where and when capacity building is at most needed.   

2. Concentrating Efforts in Limited Areas: To enable the NUNV team render concentrated efforts, the strategy can be planned to work in the areas of health and education instead of continuing working in all the five areas. It was mentioned that experts in some areas in those selected zones and woredas were missing and thus it is advisable to concentrate efforts in priority areas. 

3. Making Zones Cost Centers: To enhance the implementation of the strategy it is recommended that this strategy should be transformed into a project with a project manager and project fund, and the location should shift to the proposed zones. 

Lessons Learned:

Contribution: The NUNV programme has contributed immensely in bridging critical capacity gaps in support of the development process of the region, especially in the remote zones and woredas.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search