2016 Pakistan: Evaluation of Humanitarian Actions - Nutrition Emergency Response in District Tharparkar, Sindh, Pakistan
Author: E&Y Cnsultants
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.
The Nutrition Emergency Response was implemented by UNICEF Pakistan in partnership with two national Non-Governmental Organizations, Shifa Foundation and HANDS, between January 2014 and March 31, 2016. The overarching purpose of the response was to address emergency nutrition needs of children in the age group of six months to two years (6-59 months) and Pregnant and Lactating Women (PLWs).
The overall goal of UNICEF’s response was: “strengthened capacity for reducing risks, as well as planning, preparing, and effectively responding to disasters in accordance with UNICEF Core Commitments to Children in Humanitarian action.”
The purpose of the evaluation was to help improve the effectiveness and quality of UNICEF’s nutrition response and learn lessons for similar future emergencies in Tharparkar or in other districts of Pakistan. The evaluation analyzed both the processes and outcomes of the intervention.
The main users of the evaluation are UNICEF’s Pakistan Country Office, the Regional Office for South Asia (ROSA) and the Global Emergency Unit. Intended users also include key government agencies, including the National Disaster Management Authority, the District Disaster Management Authority, the Provincial Disaster Management Authority, Sindh, the Provincial Department of Health, Sindh, and the Provincial Nutrition Cell and Planning and Development Department, Government of Sindh. The implementing partners of UNICEF and the implementing partners of the Department of Health, Sindh, could also benefit from the findings and recommendations for implementation of the nutrition strategy.
The objective of the evaluation was to assess UNICEF’s humanitarian response in Tharparkar, using standard criteria for evaluation of humanitarian action, such as relevance, efficiency, effectiveness, coverage, connectedness, coordination and coherence. The evaluation was also expected to assess crosscutting issues such as coordination, management and the inclusion of gender equity and human rights and to identify challenges and lessons for implementation of multi-sectoral nutrition interventions.
The methodology adopted was primarily qualitative and the bulk of the data was collected using qualitative methods. A mini-survey with structured, open-ended questions was administered to 60 households. An observation checklist complemented the data. Both secondary and primary data were collected. Documents related to design, implementation and monitoring were reviewed. Data on coverage from the Nutrition Information System (NIS) were also analyzed. Primary data was collected in Focus Group Discussions, Key informant interviews, the observation checklist and the mini-survey. During the field investigation the evaluation team visited 10 out of 44 static sites established by the implementing partners in four Talukas of Tharparkar. At least two sites from each of four Taluka and five OTP sites for each partner, Shifa and HANDs, were visited by the evaluation team.
Findings and Conclusions:
Both of UNICEF’s implementing partners successfully set up static Outpatient Therapeutic Programme sites in each of the 44 target Union Councils to provide nutrition services. Each partner also established two mobile teams to deliver services to remote villages. In addition, both partners used all the teams deployed in the static OTP sites as static-cum-mobile teams, with the exception of teams deployed at static sites in District Headquarter (DHQ) and Tehsil Headquarter (THQ) hospitals. The evaluation found that CMAM services remain functional in static and mobile sites, although OTP sites closer to the center were better equipped and more organized than the ones located farther from the center.
The partners were notably effective in increasing awareness among women community members, particularly among PLWs, about the causes of malnutrition, ways to prevent, and increasing women’s knowledge of good feeding practices. However, they were far less effective in reaching out to male community members, in general, and in particular to fathers of children 6 to 59 months old. Widespread behavioural changes will take some time to become evident although there are some reports of this already.
The targets for all of the project services were exceeded: the over-achievements were: 214% in screening children aged 0-59 months; (294,232 vs 93,600); 51.6% in screening PLWs (97,582 vs 62,400); 63.77% in registering SAM children (18,508 vs 7,207), 238% in MAM children registered and treated (40,407 vs 11,957) and 100% in PLWs registered and treated in TSFP program (21,833 vs 10,920). In addition 28,538 children were de-wormed (no target had been set) and there was a 76% over-achievement in reaching women for promotion of IYCF practices (109,779 vs 62,400).
- Strategies must be generated to ensure consistent and long-term implementation: this requires, inter alia, that projects and services are not shut down and re-opened, that staff members are assured of long-term employment and that community members know that staff are engaged with them in the long term.
- Future projects should develop and use project-specific HRGE analyses, strategies and implementation plans to inform context-specific actions. These should be integrated into monitoring and evaluation plans and into reporting requirements. Reaching men, especially fathers, and also men who may serve as ‘champions’, such as school teachers and religious leaders, must be part of the HRGE. An HRGE Specialist should be recruited to ensure that this process is institutionalized and mainstreamed. Sufficient funds must be allocated to ensure that this happens.
- The implementing partners’ approach to community outreach focused almost exclusively on verbal communication and written messages placed on the walls of OTP sites. The partners should ensure that messages are delivered ‘deeply’ into communities, and that the reasons for present ideas and practices on childbirth and infant and child nutrition are thoroughly understood, so that messages on new practices can address those. UNICEF and the implementing partners should use more creative media with a strong emphasis on visual media (especially for non-literate community members and older children) such as puppet shows, street theatre and videos, where possible. FM radio should also be used for developing and conveying nutrition-related and ‘healthy child’ messages.
- Access to high quality MNCH services, particularly EMOC services, should be increased. Comprehensive EMOC service is only available at the DHQ hospital in Mithi. Because of long distance and high travel costs, people tend to avoid using these services, often to the detriment of women’s and children’s health.
UNICEF’ organizational systems have yet to catch-up with changed institutional priorities. As a result, as shown by the experience of the project shared by UNICEF representatives, integrated programming did not receive sufficient attention and projects were formulated in department-specific silos. Unless concerted efforts are made to harmonize organizational systems with changing institutional priorities, it takes more time for systems to catch up with institutional priorities.
- Terrain, the quality of roads and the quality of communications technologies are critical factors in successfully implementing programmes of this nature. In areas where the population is scattered and access to service delivery points is limited by long distances, lack of transport and poor road infrastructure, mobiles team work best, provided they are well equipped and have appropriate vehicles and communications technologies.
- For this project, the relevance of static sites was strong, but it also shows that in areas where the population is scattered and access to OTP sites is limited by long distances, lack of transport and poor road infrastructure, mobiles team work best, provided they are well equipped and have appropriate vehicles and communications technologies.
- Partners have found through experience that they can use their visits to communities to support EPI coverage. They have frequently encouraged EPI extension workers to accompany them for this purpose. Where possible, this sharing of local resources and knowledge should be integrated into implementation plans.
- Government records including items as basic as a list of villages do not always catch-up with the changing reality (e.g. shifts in population, demographic trends, the state of local water supplies) on the ground, underlining the need to augment government records with local knowledge.
Full report in PDF
PDF files require Acrobat Reader.