2014 ESARO, Kenya: Africa's Nutrition Security Partnership (ANSP) Mid-term Evaluation
Author: Joanne Harnmeijer (team leader), Huibert Johan Lof, Joanne Philpott, Albertine van der Veen
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report.
UNICEF is has a long-term agreement with ETC Nederland BV to support the mid-term and summative evaluations of two programmes: the four-year Maternal and Young Child Nutrition Security Initiative (MYCNSIA) in the regions of East Asia and Pacific, and South Asia, and the Africa’s Nutrition Security Partnership (ANSP) in Eastern and Southern Africa and West and Central Africa regions. This Terms of Reference (TOR) is for the second of four evaluations that will be executed: (i) mid-term evaluation for Asia (MYCNSIA), (ii) mid-term evaluation for Africa (ANSP), (iii) final evaluation for MYCNSIA, and (iv) final evaluation for ANSP.
The Africa’s Nutrition Security Partnership (ANSP, 2012-15) are designed around four interrelated Result Areas: (1) Up-stream policy work regarding nutrition security, (2) Capacity building of decision-makers, service delivery personnel and communities, (3) Data analysis and knowledge sharing, and (4) Scale up of key proven interventions. The ANSP is targeted in four countries including Burkina Faso, Ethiopia, Mali and Uganda. Through the ANSP, UNICEF works to improve child growth and development in Africa by improving nutrition security using intersectoral approaches.
The overall purpose is to undertake independent evaluations, the results of which (at mid-term)will improve the implementation during the remainder of the programme, and (at final) will generate knowledge and identify best practices and lessons learned which can be transferred to other programmes and inform global policy on nutrition security. The evaluations (mid-term and final) will assess the relevance, effectiveness, impact, efficiency and sustainability of the MYCNSIA in five targeted countries in Asia and the ANSP in four targeted countries in Africa as well as regional efforts (in Asia and in Africa) at mid-term and after the four years of implementation.
This external evaluation is being commissioned to (1) obtain an unbiased assessment of whether or not the planned inputs have led and/or contributed to the achievement of the anticipated results (outputs, outcomes, impact), (2) identify the reasons why or why not the achievements have been made, and (3) examine any unintended positive or negative results of the programs. Further, through the evaluation at mid-term, it is expected that the nine target countries and the Regional Offices will gain information and advice necessary to make any adjustments to their plans for the remainder of the implementation period, which can help intensify achievement of results through the evaluation process.
The objectives stated above have informed the selection of the 6 evaluation criteria: impact, relevance, effectiveness, efficiency, sustainability, and equity.
Phase 1: Inception Phase
Review to what extent the approaches and methods of the mid-term evaluation of MYCNSIA (in Asia) are applicable, or need to be modified and improved based on the experience of the previous year’s MYCNSIA mid-term, as well as the anticipated specific needs of the ANSP mid-term.
Phase 2: Data Collection Phase
Review of available reports and relevant documents. Interviews with key partners and stakeholders; review of baseline data from the four focus countries; review programme monitoring data; field visits and interviews with national and local implementing partners (government and/or non-governmental, as relevant); semi-structured interviews with programme beneficiaries. Triangulation of various data collection methods will help to inform the full picture as per the evaluation objectives and criteria.
Phase 3: Analysis
Analysis, triangulation, and synthesis of the information collected in the field visits and through correspondence. Preparation of a first draft report of the ANSP mid-term evaluation. Findings and recommendations relating to the 6 evaluation criteria (impact, relevance, effectiveness, efficiency, sustainability, and equity) which can improve the implementation for the remaining period of the ANSP programme should be clearly and explicitly demarcated from recommendations beyond the scope and timeline of the ANSP programme.
Phase 4: Reporting and Dissemination
UNICEF and the Reference group shall provide timely review and feedback on the draft report, and this shall be followed by systematic and responsive revisions by the contracted agency.
Findings and Conclusions:
1. The EC/UNICEF partnership on nutrition security addresses two continents; the current report concerns the Africa Nutrition Security Partnership (ANSP)
2. MYCNSIA and ANSP have a similar design; both have four result areas
3. The objectives, scope and methodology of the mid term evaluations (MTEs) have been set to ensure comparability and also to inform policy-making and programming, both for UNICEF and for the EC
4. The ANSP programme is deemed highly relevant in terms of its objective
5. ANSP/UNICEF operates in a rapidly changing nutrition environment and context, which demands responsiveness and ability to adjust.
6. ANSP programming is both a continuation of previous UNICEF activities and routines and a start of new and strategic activities and programmes
7. ANSP/UNICEF has in the various write-ups of the programme lost some of the original gist, and has also added new activities and outputs and dropped original ones. The basic concept, of a multi-level, multi-pillar programme that complements other actors is sound and could be distinctive.
8. Although the expected results for the two regional programmes have been identical, in reality the interpretation, or rather: the priority setting, has been different. This is especially so for pillars 2 and 3.
9. The regional offices have thus far not given sufficient systematic attention to the ANSP countries.
10. There is a rather unique opportunity to address prevention of malnutrition in ways that respect equity and that signal deviations (“bottlenecks” ); as shows the case of ANSP Burkina Faso
1. (relevance/ design): ANSP/UNICEF should review its outputs and activities in view of the rapidly changing context in the 2 regions and 4 countries. In particular alignment with SUN, REACH and with the 2013 EC Nutrition Policy should be clarified and emphasized to reflect that ANSP/UNICEF is in fact a main contributor at regional and national levels.
2. (equity) In order to effectively make a difference, ANSP/UNICEF should become much more ambitious in its pillar 4 programming: it should aim to both reduce chronic malnutrition and reduce the chronic malnutrition gap between the bottom and the top wealth quintiles. It should in addition seek to demonstrate how this can be done.
3. (leverage external resources) ANSP/UNICEF should seek to profile and report on the programme as a catalyst of essential, strategic interventions. Additional funds will need to be leveraged on the strength of proven (early) results.
4. (effectiveness/ operational efficiency) ANSP/UNICEF should explore and document where in its ANSP programmes mutual reinforcement, and acceleration, has taken place both between “pillars” and between national and sub-national levels within countries; and possibly also between the ANSP levels (continental; regional; country and the other way around).
5. (internal mainstreaming) ANSP/UNICEF should seek and profile cases of mutual reinforcement between ANSP/UNICEF nutrition and other UNICEF interventions, notably in health, education, WASH and child protection. Here there are no lessons learned as the MTE has not found a single example in the ANSP countries.
6. (relevance / coherence) At all levels, but particularly at continental and regional level, ANSP/UNICEF should profile itself more as a “programme with a long term vision” and use the lessons learned in the ensemble of countries and regions.
The ANSP has continuously followed the original concept laid out in the 2011 Project Document.
However, we agree that reporting could be improved to better bring out where ANSP has done more than was envisaged. The actions planned will thus address this by 1) preparing a more explicit global logframe for the ANSP; 2) preparing a monitoring matrix that can better show progress towards results; and 3) revising the evaluation framework for the final external evaluation to highlight where the ANSP has promoted and supported collaborative planning, learning and adaptation to accommodate the dynamic opportunities and challenges in the context at continental, regional and national levels.
Full report in PDF
PDF files require Acrobat Reader.