We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2014 Pakistan: End of Project Evaluation for Norway-Pakistan Partnership Initiative



Author: Population Council, Pakistan

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

The Norwegian Pakistan Partnership Initiative (NPPI) project was implemented for six years—from 2009 through 2014—and aimed to reduce maternal, neonatal, and child mortality through increased coverage of quality maternal, newborn and child health (MNCH) and family planning (FP) services, along with improved MNCH and FP self-care as well as care seeking behavior within families and communities. After deciding to support Pakistan in its efforts to improve its MNCH indicators, the Royal Norwegian Government decided to focus its funding on Sindh province because of its weak MNCH and FP health indicators. It was decided that Norway would support the concept of ONE UN, and project implementation would be through multiple UN agencies in closely collaboration. Norwegian experts were involved from the beginning, designing the project.

The project was implemented in 10 rural districts of Sindh, namely Kambar Shahdadkot, Ghotki, Kashmore, Badin, Jamshoro, UmerKot, Tharparkar, Larkana, Shaheed Benazirabad and Shikarpur, for reaching the most vulnerable populations. The UN collaborating agencies in this initiative were UNICEF, as convener and implementer, with WHO and UNFPA as the other major implementers. UNDP, as administrative agent, was responsible for funds disbursement. As ONE UN, they implemented a range of health interventions with their respective implementing partners, through several innovative approaches to improve MNCH in Sindh.

NPPI assisted with key challenges to health care access in rural Sindh including financial barriers, women’s low status, abject poverty, a sub-optimal health system, and poor quality of care, which affect both health care seeking and non-existent or extremely weak service delivery in most areas.

Each UN agency undertook responsibility for a distinct set of interventions, based on their expertise, which, for the most part, were implemented in distinct geographic areas. The project experienced setbacks due to natural disaster, specifically devastating floods in 2010 and 2011, which affected project momentum and early project gains due to diversions of health and development resources, management, and oversight to emergency relief and rehabilitation. After the mid-term review (MTR), the project focused on phase out planning and a road map for its exit strategy ending in December 2014.

Purpose/Objective:

The End of Project Evaluation's key objectives are:

- Documenting and disseminating NPPI's results and achievements;

- Generating knowledge of evidence-based best practices for program approaches, cost effectiveness, and sustainability.

Methodology:

This End of Project Evaluation (EPE) is based on a desk review, qualitative interviews with key informants, field visits verifying activities, as well as a review of District Health Information System (DHIS) statistics and other survey findings. This information was triangulated and utilized for analysis.

For the four principles of relevance, effectiveness, efficiency, and sustainability, the EPE’s focus is on reviewing individual interventions though a general assessment of the project is also included. This EPE assesses how the interventions addressed gender equality and equity, while recognizing that it was too close to activities’ end to measure impact. While there was a baseline evaluation, an in-depth end line was not conducted; consequently, we analyzed the outcomes of various interventions using statistics from DHIS and other large surveys such as the Pakistan Social and Living Standards Measurement Survey (PSLMS). Furthermore, there were limitations to assigning improvement and impact attributions due to the interventions’ uneven geographical spread in addition to data limitations.

The EPE team attempted to document the entire project period of 2009 to 2014, with particular attention to progress after the MTR, as it comprehensively evaluated the project’s earlier implementation and made clear recommendations for phase out. One of the MTR’s key recommendations was for redesigning the project, which was not totally feasible. Hence, the Norwegian Government decided to curtail funding from 245 million kroners to 145 million kroners. Post MTR, an exit strategy and road map was developed jointly by three implementing agencies (UNICEF, UNFPA, WHO),the donor and the Sindh government. The focus on the exit strategy and improved coordination did increase the momentum of NPPI project implementation.

Findings and Conclusions:

Relevance:

Project relevance is demonstrated, in part, by its alignment with MNCH program objectives and the draft national health policy of 2009. The project clearly set out to address the most important impediments to achieving MDGs 4 and 5 by 2015 in Sindh province.

Effectiveness:

In the absence of an end line survey enabling comparisons, the EPE took advantage of the availability of district data from the PSLMS surveys of 2008-2009 and 2012-2013 to gauge changes in NPPI MNCH indicators for skilled birth attendance, institutional delivery, antenatal care visits, and immunization rates for children under the age of two.

Efficiency:

Overall utilization of the grants by the three UN agencies and UNDP’S costs of disbursement have been analyzed; they are available through UNDP. The burn rate of the grant was very slow in the first and second years (2009–2011), as can be expected in the life of any project, and picked up in 2012 and 2013 when it was at its highest level. The effect of the 2010 floods led to a diversion of efforts. Among the UN agencies, UNFPA had the most efficient burn rate followed by UNICEF, with WHO trailing substantively. The expectation is that the project will expend all funds by December 2014.

Sustainability:

There is a huge opportunity for scaling up the NPPI interventions, as they are so closely aligned with the Government of Sindh’s objectives for reducing maternal and infant mortality and raising contraceptive prevalence in a short period of time. While there is general awareness of NPPI and a clear desire by project managers to sustain and scale up components, the process could have been taken further.

Recommendations:

In conclusion, had NPPI interventions been implemented simultaneously and allowed a greater implementation timeframe the project would have produced more tangible results. It is recommended that innovative schemes such as the following pro-poor initiatives be implemented fully in the poorest ten districts of Sind in line with the health sector strategy:

- Maintaining sick newborn care units and expanding them throughout Sindh;

- Pre-medical training sustained and expanded to other universities in Sindh;

- Replication and expansion of 24/7 EmOC services province wide;

- Upscale of the voucher scheme;

- Scale up of behavior change strategies such as the development of focal families.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2014

Country:
Pakistan

Region:
ROSA

Theme:
Health

Type:
Evaluation

Language:
English

New enhanced search