We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 Pakistan: Evaluation of Mother and Child Weeks Programme, Pakistan



Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

 

 

Background:

 

Pakistan has the third highest burden of maternal and child mortality in the world. Hence, improving maternal and child health is an integral part of the country’s national development plan, Vision 2025, and its National Health Vision 2016-2025. To bring about positive change in the health of mothers and children in Pakistan, several international organizations, including UNICEF, are playing a significant role – especially in the areas that lack basic health services. To support the efforts to translate Pakistan’s National Health Vision into action, UNICEF developed a Mother and Child Health Week (MCW) Strategy which was launched as a pilot project in 2007 in three districts of Punjab by the National Programme for Family Planning (FP) & Primary Health Care (PHC) and the Expanded Programme on Immunization (EPI), with UNICEF’s technical and financial support. In April 2009, the MCW was expanded to 29 districts in Punjab, Sindh, Balochistan, KP and in Azad Jammu and Kashmir (AJK) and Gilgit Baltistan (GB). Based on the pre- and post-MCW assessment of 2008, the Ministry of Health decided to scale up the programme to the national level. As of April 2012, ten rounds of Mother and Child Weeks have been completed. The MCW package includes 1) Immunization of children between 0-2 years of age; ii) Tetanus Toxoid (TT) vaccinations for all pregnant women; iii) Deworming of all children between 2-5 years old; and iv) Raising awareness of health.

 

Purpose/Objective:

This evaluation aimed to mainly assess the extent to which the MCWs’ implementation followed its intended plan (i.e. to strengthen the existing MNCH services and enhance access to these services for potential beneficiaries); and i) to assess the extent to which the MCW met the basic health needs and rights of women, girls and boys, especially belonging to marginalized, excluded, crisis and disaster-affected populations – the intended primary beneficiaries of the initiative – and whether there were any unintended outcomes and its sustainability issues; ii) to provide a descriptive cost analysis of the MCW initiative; iii) and to identify lessons learned, exploring what has worked well and what has not worked. The intended users of this evaluation include government health institutions and officials, international organizations, donors and other partners working in the field of maternal and child health. Although, UNICEF could have limited to scope only to the bits it supported, however, to make a useful and comprehensive analysis for the government’s national level initiative, the evalaution covered the entire country and all key queries that could inform the government effectively.

Methodology: 

To address the complex, wide-ranging set of research questions for this evaluation, a systematic and scientifically sound methodology was adopted. The evaluation followed the widely used criteria developed by the Development Assistance Committee (DAC) of the Organization for Economic Cooperation and Development (OECD). A household survey of programme beneficiaries was conducted to identify the extent of the MCW initiative’s coverage and efficiency. To explore the initiative’s relevance, efficiency, effectiveness, outcome and sustainability, qualitative research methods were employed with different stakeholders. The evaluation followed the United Nations Evaluation Group’s (UNEG) standards for evaluation (2005), including professional integrity, absence of bias, behavioural and organizational independence, honesty, anonymity and confidentiality of individual information, impartiality and the participation of stakeholders throughout the evaluation process. An analytical approach to contribution analysis was adopted to provide a credible assessment of cause and effect. Contribution analysis helped the evaluation team to verify the theory of change that was developed in retrospect for the MCW initiative and to examine whether the MCW contributed to achieving its intended outcomes.
The evaluation design comprised both primary and secondary data. The primary quantitative data was based on a household survey of 1,540 mothers of reproductive age and 220 Lady Health Workers. For quantitative data collection, two union councils in each district were selected randomly. Qualitative data was based on 22 case studies of mothers, 172 Key Informants Interviews with federal, provincial, district and union council level health managers and 66 Focus Group Discussions (FGDs) with the community health workers (LHWs, LHS and vaccinators). Both data were analysed separately at the initial stage of the evaluation.

 

Findings and conclusions: 

• The qualitative and quantitative data affirms that the initiative was very good at achieving its intended goals. Stakeholders at the federal, provincial and district levels acknowledged that the MCW raised knowledge of health among mothers.

• The qualitative data indicates that available human, financial resources and supplies were used quite efficiently. The MCW was considered cost-efficient as it utilized most of the existing programme resources. However, the data from Balochistan, FATA and GB show that the MCW initiative’s overall efficiency was undermined as resources did not reach to the marginalized or poorest groups. The qualitative data also reveals that health workers lacked motivation in the absence of monetary compensation for additional work. This had a negative impact on their efficiency and job productivity.
• The qualitative findings reveal that MCWs were an effective approach for improving the MNCH indicators, strengthening the service delivery system and raising awareness among the communities by using Pakistan’s existing network of outreach workers.
• The qualitative findings showed that the MCW is beneficial and effective initiative in achieving vaccination targets, providing deworming tablets and reaching the ‘unreached’ children.

The evaluation concludes that the MCW initiative effectively provided MNCH services to the targeted populations. However, variations are apparent across regions. Some regions experienced better results, such as Punjab, Sindh, AJK and KP while, others performed relatively poorly with regard to the efficiency of the MCW, such as GB, FATA and Balochistan. However, the MCW initiative has the potential to increase health coverage by enhancing community participation through community mobilization. The initiative can be further improved through effective participation of civil society and communities, adequate and timely provision of logistical supplies, and by enhancing financial incentives for field teams.

Recommendations:

UNICEF LEVEL
• UNICEF should continue its technical support of the MCW initiative.
• The establishment of MCW Secretariats at the federal and provincial levels would support the effective planning and implementation of MCW activities.
• The monitoring and evaluation system needs to be strengthened. Information and communications technology (ICT) may be extensively used to make the monitoring and evaluation system more efficient, cost-effective, and user-friendly.
• An operational guide should be developed, outlining the explicit responsibilities of each stakeholder at district, provincial and federal levels.
POLICY LEVEL
• To address provincial and regional disparities in socio-economic and health profiles, each province’s/region’s health care legislation should be in line with its specific socio-economic realities.
• There is a need for an approach grounded in learning from best practices. For example, the MCW initiative’s planning and implementation could be improved by learning from best practices in other countries, where similar interventions are working more effectively and efficiently.
• The MCW initiative’s implementation should be embedded in the health sector’s action-planning documents.
• To maintain health equity and address issues of accessibility and affordability, the MCW initiative should be scaled up in areas that are not covered by LHWs.
• As maternal and child health is intimately linked with gender equality and gender mainstreaming, there is a need to include a gender component in the MCW activities by involving both women and men.
OPERATIONAL LEVEL
• Using mobile phone technology, specific applications and software to monitor MCW activities can increase the effectiveness of the initiative.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year: 2018

Office/Country: Pakistan

Region: ROSA

Type: Evaluation

Theme: Health; Health systems development/ management; Primary Health Care; ACSD

Language: English

Sequence #: 2018/003

New enhanced search