We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2013 Albania: Reducing Child Malnutrition in Albania: Final Evaluation of the Joint UN Programme (2010-2013) implemented by UNICEF/WHO/FAO



Author: Richard M. Chiwara

Executive summary

"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report."

Background:

In December 2006, the United Nations Development Programme (UNDP) and the Government of Spain signed a major partnership agreement for the amount of €528 million with the aim of contributing to progress on the Millennium Development Goals (MDGs) and other development goals through the United Nations system. Under the Millennium Development Goals Achievement Fund (MDG-F) M&E Strategy and Programme Implementation Guidelines, each programme team was responsible for designing an M&E system, establishing baselines for (quantitative and qualitative) indicators and conducting a final evaluation with a summative focus. In accordance with this guideline, the United Nations joint programme partners commissioned the final evaluation of the Joint Programme - "Reducing child malnutrition in Albania, (MDG-F 2035)". The evaluation was undertaken from May 24 to July 31 by a two-member team of independent evaluators with an international team leader and national team member.

The unit of analysis was the JP MDG-F 2035, which in this context included the set of outcomes, outputs, activities and inputs that were detailed in the JP document and in associated modifications made during implementation.

Purpose/Objective:

The overall purpose of this evaluation was to (a) Measure the extent to which the JP delivered its intended outputs and contribution to outcomes, and (b) Generate substantive evidence based knowledge, by identifying good practices and lessons learned that could be useful to other development interventions at national (scale up) and international level (replicability). The primary users of the evaluation include the JP partner UN agencies, national and local government partners, civil society organizations and beneficiary communities, the MDG Fund Secretariat as well as the wider UN development system organisations.

Methodology:

Overall approach

An initial desk review of official background documents, JP files and reports was conducted culminating with drafting of an Inception Report outlining the scope of work and evaluation design. The Evaluation Reference Group (ERG) and JP partners reviewed the Inception Report and provided comments resulting in the revised Inception Report. Based on the agreed plan and design, active data collection in Albania was carried out from June 10 to June 28, 2013.

The in-country data collection mission included individual interviews with the UN JP Partners and national government partners in Tirana, as well as individual interviews and focus group discussions (FGD) in the targeted prefectures of Kukes and Shkoder and the peri-urban Tirana municipalities of Kamez and Paskuqan. Additional documents were also made available and reviewed during the in-country mission. The list of documents reviewed is at Annex 1 to this report. At the end of the country mission, a presentation of preliminary findings, conclusions and recommendations was made to the ERG and Programme Management Committee (PMC), and their comments were incorporated in the draft report.

Data Collection and Analysis

Main sources of data included both secondary (document review) and primary (interviews and focus group discussions). Individual interviews were conducted in Tirana with partner UN agency staff and senior management; officials of participating national Government departments, and civil society organisations. In the prefectures of Kukes and Shkoder, and the municipalities of Kamez and Paskuqan, individual interviews were conducted with officials of participating local government departments and civil society partners. FGDs were conducted with the frontline health workers, agriculture and education officers; as well as target beneficiaries, including mothers and farmers. The list of individuals interviewed is provided at Annex 2.

Quantitative analysis techniques were applied to assess JP performance related to quantitative targets and indicators; for example, decrease in malnutrition. However, mostly qualitative analysis was used to determine the JP’s contribution to outcomes (Box 2, Page 18).

Findings and Conclusions:

Relevance

The major focus of the JP to strengthen policy and build national institutional and local level capacities for household food security and nutrition was well aligned with the policies of the Government of Albania. In a critical review of the national Food and Nutrition Action Plan (FNAP 2003-08), stakeholders noted that the policy environment placed more emphasis on food safety compared to nutrition, and its implementation was generally regarded as not very successful due to its lack of specific strategies and resources for addressing nutrition related outcomes.

Management and Implementation

The JP was approved in December 2009, and implementation started in January 2010 with an estimated end date of January 2013. A Programme Management Committee (PMC) co-chaired by the lead UN and Government partners (UNICEF and MOH) was established and met quarterly to provide oversight and guidance for the joint programme. The PMC, through the national coordinator (MOH) established Technical Working Groups (TWGs) charged with planning, implementing, monitoring and reporting of specific JP interventions. A mid-term evaluation (MTE) was undertaken in November 2011. The JP addressed most of the recommendations of the MTE through an improvement plan, which was developed and shared with the MDG-F Secretariat. The improvement plan included a revision of the JP outcomes to better reflect principles of results-based management, as well as strengthening the output and outcome indicators.

In June 2010, five Ministries formally agreed to take joint inter-sectoral actions to improve nutrition in Albania.1 As a direct consequence of this inter-sectoral collaboration; (a) the MOH appointed a national coordinator who was located at the Ministry facilities, (b) the line Ministries appointed focal points for coordination, (c) all formal JP meetings were convened by line Ministries, with official letters issued and signed by the MOH and MOAF&CP, and (d) the Government and UN partners undertook joint field trips for activity monitoring and meetings.

Effectiveness

All JP interventions reflected the key strategic areas that were globally recognised as best practices for addressing food security and nutrition at national level. The JP undertook a two-pronged strategy; the first was to facilitate a multi-sectoral approach to address food insecurity and malnutrition through establishment of a national coordination structure for food and nutrition at high government level, and the second strategy was to develop and implement an effective communication and advocacy strategy that would pave the way for the development and implementation of a national Food and Nutrition Action Plan.

Efficiency

The programme efficiency as measured by UN agency delivery rates (including direct costs for administration and cost sharing) was 98.4%. The interventions that were implemented were appropriate to achieve the overall objective to address the problems associated with mother and child malnutrition. However, the evaluators were of the opinion that the agricultural interventions were small and did not have sufficient time to effectively demonstrate change in improved access to food and nutrition. In addition, the individual farmers and households that were selected to demonstrate the models were not sufficiently representative of the most vulnerable groups and food security challenged households.

The reporting frameworks for direct beneficiaries changed during the JP lifecycle, thus making it difficult to objectively assess the implementation efficiency on the basis of value for money in the context of direct beneficiaries.

Sustainability

Some of the JP outputs indicated that they were sustainable in the long run due to their very nature. For example, reform of education curricula was approved and the government had allocated funds for activities in the mid-term budget framework. However, there were other interventions and outputs that required further institutional support in order to be sustainable. These outputs included, for example, the flour fortification, which was not yet agreed between the Ministries of Health and Agriculture, the roll-out and implementation of the National FNAP, which was not yet signed (although the evaluation team was informed that it had been endorsed by the Ministry of Finance, which is an expression of commitment that the FNAP will receive budget support).

Recommendations:

Recommendation 1: The government should continue to prioritise food security and nutrition for children by supporting the implementation of the FNAP 2013-2020; and UN agencies should continue to engage central and local government authorities, including through participation in technical working groups in order to complete the roll-out and implementation of the FNAP.

Recommendation 2: UN agencies, including FAO and WHO should continue to engage the sector ministries of health and agriculture, by supporting further studies to provide additional scientific evidence to inform national policy on flour fortification.

Recommendation 3: The government should implement the recommendations of the pilot model to combine economic aid with nutrition services and food packages; and UNICEF should continue to engage the MOLSA&EO to support the roll-out and implementation of the model.

Recommendation 4: The government should develop models to ensure that the most vulnerable and disadvantaged communities and groups have adequate participation and representation in initiatives to improve access to food and nutrition.

Recommendation 5: UN agencies should support pro-poor government policies in order to ensure that the poor have equitable access to services and can also benefit directly from government programmes.

Recommendation 6: UN agencies should continue to engage and support civil society capacity for nutrition advocacy and participation in local development planning.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2013

Country:
Albania

Region:
CEE/CIS

Theme:
Nutrition sectoral or partnership

Type:
Evaluation

Language:
English

New enhanced search