2016 Jordan: Evaluation of Infant and Young Child Feeding Programme
Author: Dolly Bassil et al
“Infant and Young Child Feeding (IYCF) Programme in Syrian Refugee Camps and Host Communities in Jordan” Evaluation
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report, and the executive feedback summary labelled as ‘Part 4’.
A report by the United Nations High Commissioner for Refugees (UNHCR) published in August 2016 states that while the political and civil unrest in Syria completed its fifth year in March 2016 , the Government of Jordan estimates hosting up to 1.3 million Syrians , including around 656,198 registered refugees. While 79% of refugees had settled in urban host communities within Amman and the northern governorates, the remaining poured into refugee camps; mostly in Zaatari and Azraq camps. Of the total refugee population, children between 0 – 4 years of age attribute to 8.3% and 7.8% of males and females, respectively. The conflict in Syria has severely strained economic and social systems and institutions in Jordan. In view of the prevailing situation in Jordan as an impact of the Syrian crisis, the IYCF Programme, supported by UNICEF, was initiated and implemented by Save the Children Jordan (SC-J). The Programme started in November 2012 and continues to date, aiming at maintaining and improving infant and young child feeding among populations in Jordan that are affected by the Syrian crisis. Since its launch in November 2012, the Programme has witnessed an expansion in scope and area of implementation; including camps and host communities. UNICEF’s Programme requirements include performing an external evaluation while the Programme implementation is ongoing, in order to determine whether or not the Programme is achieving its stated objectives and continues to serve the evolving needs of the targeted communities.
The purpose of this evaluation is to independently assess the IYCF-P, supported by UNICEF and implemented by Save the Children Jordan in the target communities (camps and host families). In particular, the evaluation aims at conducting a systematic and impartial examination of the Programme’s performance, including the relevance, effectiveness, efficiency, and sustainability of the Programme. This evaluation is primarily intended to support the decision-making process of the UNICEF Jordan Country Office and the Ministry of Health to examine Programme’s performance and inform future health and nutrition programmes in Jordan. The secondary audience includes relevant institutions working in the nutrition sector in Jordan, including SC-J, nutrition and health partners, as well as donors.
The specific objectives include:
- To assess the relevance of the IYCF-P (with integrated IYCF+ services for emergency response and resilience components) in the current context in Jordan and its national priorities, including those in the context of the health sector reform.
- To assess the effectiveness of the IYCF-P and to measure to what extent the Programme has achieved its stated objectives and any intended and unintended effects.
- To assess the efficiency of the IYCF-P and to measure to what extent the Programme has used resources (human, financial, and others) in an efficient manner, and if the achieved results justify the resource.
- To assess the sustainability of the IYCF-P and its results, considering the likelihood of Programme sustenance in the absence or reduction of ongoing UNICEF and donor support.
Given the lack of baseline data and further to related discussions with UNICEF officials during the inception period, the current evaluation did not attempt to measure the Programme impact but rather examine the likelihood of impact. This was achieved through assessing the enablers and contributing factors associated with the Programme’s success.
The overall design of this evaluation is based on the use of the Programme’s emerging Theory of Change (ToC), as articulated in the previous section. The evaluation uses the Programme theory to assess: i) how the different contexts where the IYCF-P operates affect the Programme’s performance; ii) whether or not the proposed logic of results holds; and iii) whether the assumptions made in terms of external factors and conditions needed to enable and sustain change are valid, and if not, how such discrepancies affect performance. Given the nature of the Programme, objectives, scope, roll out schedule and reach; the evaluation has adopted a general qualitative methodology. The research approach is the case study methodology, where the IYCF-P is evaluated using the multiple case study design with embedded units of analysis. Since the Programme has witnessed expansion both at the geographic and the activity level, and in an attempt to overcome the challenge of capturing information that is representative of the different geographic areas and different stages of the Programme, the multiple cases encompassed different units of analysis to capture different geographic areas, different duration of implementation, and different contexts. Data and information was collected using a combination of the following methods targeting the different project counterparts/stakeholders: (1) records and documents reviewed through a desk review; (2) semi-structured interviews with key informants; (3) focus groups discussions with service providers; (4) focus group discussions with end-users benefiting from the Programme at different stages; and (5) e-questionnaire targeting beneficiaries of capacity building activities.
Findings and Conclusions:
The evaluation found that the IYCF Programme is relevant while it can be further enhanced through a stronger alignment to national priorities and systems. The Programme is partially effective in a sense that it has achieved results at activity and output levels, yet fell short of achieving outcome level results. The programme design must explicitly consider the roles of duty bearers, such as men and fathers, and strengthen approaches for capacity development of service providers, influencing policies and addressing social norms.
The evaluation also found that the Programme is efficient. However, integrated approaches and engaging volunteers can further enhance not only efficiency but also the effectiveness. Likelihood of impact was noted in term of changes at individual level. Changes at institutional level or nutritional status of children could not be determined while assessment against the Theory of Change indicated potential of contribution towards impact. Sustainability of the programme in its current modality and approach is questionable, and there must be an articulated and phased plan for integration and institutionalization of the programme, capitalizing on the achievements and strengths.
- Review the Programme scope for an integrated and comprehensive Primary Health Care Programme covering IYCF, Early Childhood Development, newborn and child care, antenatal and postnatal care, and mental health.
- Enhance the Programme design to influence outcome-level results such as: 1) providing mother-to-mother support that contributes to sustainability of the Programme; 2) Incorporating a “men’s component” into the Programme where men will be targeted separately; 3) conducting regular capacity building and sensitizing activities targeting camp service providers including NGOs.
- Address gaps in monitoring, data and financial management. The current system does not capture all the necessary information to track achievements against the stated results. The Theory of Change developed for this evaluation can be used to refine the results framework and monitoring system.
- Develop an exit strategy with concrete and phased actions, including capacity building, coaching and mentoring of Syrian volunteers and partner staff. SC-J can conduct capacity assessment to identify existing centres with the necessary and adequate capacity. The exit strategy should be accompanied with the recommended sustainability measures.
- Conduct a thorough assessment of bottlenecks in existing structures and health systems within MoH to improve effective IYCF service coverage. Accordingly devise a plan to strengthen IYCF within existing Primary Health Care system platforms.
- Integrate Infant and Young Child Feeding into the National Health Strategy and support the implementation. This evaluation provides knowledge and learning for strengthening the national strategy and its implementation.
- Develop a plan for Infant and Young Child Feeding activities to be part of an integrated package of Primary Health Care services for children aged 0-2 years of age, encompassing newborn, child health, early childhood development and nutrition.
- A baseline assessment needs to be incorporated at the initiation phase of any similar future Programmes.
- Longitudinal tracking of change in breastfeeding rates among PLW can give an indication about the impact of the IYCF-P. The establishment of a monitoring and tracking system as well as implementing representative longitudinal surveys contribute to assessing the impact and capturing the effects of the Programme.
- Sufficient quantitative indicators need to be reported throughout the Programme to ensure reporting and measuring impact of the Programme. Although the Programme did not have a baseline assessment, still, throughout the three years of implementation, there could have been some modifications in the indicators to include some form of evaluation of the Programme impact including rate of breastfeeding mothers or other IYCF indicators.
- Stronger capacity building and mentoring components needed to be incorporated from the beginning. It was apparent that from its beginning the Programme was designed as an emergency intervention.
- Stronger engagement of the MOH from the beginning is needed to ensure sustainability. As the Programme progressed to target host communities, no attempt was made to provide a sustainability plan or at least documentation for the gaps needed to be filled to provide appropriate IYCF services through a sustainable approach.
- Better financial quality assurance was needed. The evaluation team noted gaps in financial management from the part of the implementing partner (SC-J). Attempts are being made by SC-J to improve financial quality, and these need to be taken forward.
- Further communication and guidance from UNICEF needs to be provided throughout the project cycle. UNICEF consistently provided feedback and guidance to SC-J during the PCA development stage; however, this guidance appeared to be less during the implementation period.
Publications below labelled as:
- Final Report - Report
- Evaluation Addendum - Part 2
- GEROS quality rating - Part 3
- GEROS executive feedback summary - Part 4
Should you be interested in the Annexes, kindly contact the Evaluation Office at firstname.lastname@example.org.
Full report in PDF
PDF files require Acrobat Reader.