We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2014 Malawi: Evaluation of the Primary Health Care Project



Author: CHAI Evaluation Team

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

The Primary Health Care (PHC) Essential Medicines project was launched in January 2012 to address the essential medicine and health commodity stock out and under supply emergency in Malawi. A project of national reach, the project supplied standardized kits of essential medicines and supplies on a monthly basis to over 620 Ministry of Health (MOH) and Christian Health Association of Malawi (CHAM) facilities in all of Malawi’s 28 districts through a parallel supply chain. The project received a total of USD38M in funding from the governments of the United Kingdom, through DFID, Norway, through the Royal Norwegian Embassy in Malawi, and Germany, through KfW.
The project was coordinated by UNICEF under a multi-partner project approach. PHC Project partners included the MOH, the Pharmacy, Medicines and Poisons Board (PMPB), the Central Medical Stores Trust (CMST), international donors, and John Snow Inc. (JSI), who oversaw the warehousing and distribution planning for the kits. The project was designed to be implemented from January 2012 to June 2013. It was then extended to the fourth quarter of 2013 in order to distribute the remaining kits and supplies. The last full distribution during this phase of the project took place in September 2013, with partial deliveries continuing into October 2013.

Purpose/Objective:

The overall goal of the PHC project was to contribute towards keeping mortality and morbidity among children under 5 and pregnant women on a constant declining rate in Malawi. It was intended to meet the 2015 Millennium Development Goals (MDG) 4 &5 by averting drug shortages over the implementation of the project. Operational objectives included: the implementation of an efficient and accountable supply chain system; national level project oversight; a functioning independent monitoring mechanism; health workers sensitized and knowledgeable about the kit items; and an effective communication campaign that sensitized communities about the kit project.
In addition, in order to address longer term national supply system concerns, the Central Medical Stores Trust (CMST) was to receive 55% of the kit costs from recipient MOH and CHAM facilities to support their recapitalization initiative.

Methodology:

The Evaluation was done in two distinct areas: (A) Operational Evaluation (B) Health Impact Evaluation;
A) Operational Evaluation
This assessed (1) Project Management and (2) Product Availability
1) Project Management
Assessment was done in four areas which included (i) Project coordination and planning; (ii) Supply Chain effectiveness and efficiency; (iii) Product Selection; (iv) Project Communication

Project management was seen as core in ensuring product availability

2) Product Availability
Not only was the availability assessed, but also product quality

B) Health Impact Evaluation
Under this evaluation, two areas were looked at critically and included: (1) Health Services Outputs and (2) Public Health Impact

1) Health Service Outputs
Three areas were further assessed under this and included (i) Disease Management; (ii) Health Service Utilization; (iii) Commodity Management Capacity

2) Public Health Impact
The evaluation looked at the impact of the project on mortality and morbidity among children under 5 and pregnant women

Tracer items selection
In order to understand recordkeeping practices for kit items and assess potential leakage levels, 10 kit items were selected as tracer items. They were designated to be reflective of priority primary health disease burdens as well as representative of each kit box (1-7, and briefly 1-8), as well as the loose items that arrived under a different schedule than the regular boxes.

Sample Size
A total of 59 health facilities were visited during this evaluation due to time and team capacity constraints.

Findings and Conclusions:

The PHC project was a timely and relevant project and while it did not eliminate essential medicine and supply stock outs in Malawi, it had a significant impact in reducing their occurrence and duration.

• 15,844 kits in total were distributed to over 620 MOH and CHAM facilities in 28 districts
• The total number of medicines and supplies distributed by the PHC project potentially served up to 9,487,697 patients.
• The kits program reduced product stock outs from 31% to as low as 7%.
• An estimated MKW 3,194,875,917 (US$8,670,690) in total went to the recapitalization of the CMST from the PHC project.
• Under its multi-partner model, the project was rapidly set up and exhibited a high degree of supply chain visibility and controls and all PHC kit contents passed PMPB quality testing.

Recommendations:

• Design kits specific to health centers with maternity wards, facilities with surgical wards, and central hospitals for future kit project implementation.
• Revive and implement the 2012 Joint Strategy on Supply Chain Integration Plan.
• Strengthen CMST’s invoicing and account reconciliation capacity.
• Standardize and streamline product redistribution and relocation policies and procedures.
• Implement more targeted supply management improvements at the DHO level.
• Strengthen the inventory management capacity and performance of CHAM facilities.

Lessons Learned:

• Standardized kit distributions are effective in emergency situations and in environments characterized by chronic undersupply and limited inventory management capacity. In the Malawi context, they are most appropriate for rural, public health facilities.
• Increased product availability at the PHC level reduces patient burdens at higher levels in the health system and leads to overall improved health service delivery.
• Due to the over and under supplies that result from the implementation of a kit distribution model, product redistribution is a critical mechanism for averting product stock outs, overstocks and expiry.
The DHOs play a critical role in the effective planning, oversight and management of medicine and health commodity stocks at the health center level.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2014

Country:
Malawi

Region:
WCARO

Theme:
CHAI; Ministry of Health; John Snow Inc,; Central Medical Stores Trust;Christian Health Association of Malawi; UNICEF; Norway, KfW, DFID

Type:
Evaluation

Language:
English

New enhanced search