We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2014 Mali: Summative External Evaluation of the Catalytic Initiative (CI)/ Integrated Health Systems Strengthening (IHSS) Programme in Mali



Author: Donela Besada, Sarah Rohde, Emmanuelle Daviaud, Kate Kerber and Tanya Doherty

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report.

Background:

Mali is a landlocked country in West Africa, bordered by Algeria in the north, Niger in the east, Senegal and Mauritania in the West, and Burkina Faso and Cote d’Ivoire in the South. Over 90% of the country’s 14.5 million reside in the country's southern half, with access to the Niger and Senegal rivers for their livelihoods.

Mali experienced years of peace and stability following the election of Amadou Toumani Toure, in 2002. The peace lasted for 10 years, before an armed conflict erupted in the north of Mali, where Tuareg rebels took control and declared an independent state of Azawad. A military coup followed in March 2012. Tuareg control of the north was short-lived, with Islamist groups. In response to land gains by these Islamic groups, France launched a military intervention in January 2013, and within a month’s time, recaptured most of the northern territory. Presidential elections were held in July 2013, and the second round in August, seeing Ibrahim Boubacar Keita voted in as the current president of Mali.

Mali’s health system is divided into three levels: a central level with five National Hospitals, an intermediate level with eight Regional Hospitals and a district level with sixty Centres de Santé de Référence (CSREF, Referral Health Centres). CSREFs are financed by the State (infrastructure, equipment and top management), by local government (middle level clinical staff), and by the CSREF’s own funds through cost recovery (support staff and running costs). CSRefs are akin to health centres, however better resourced ones perform the role of district hospitals. The CSREF’s represent the link between the Centre de Santé Communautaire (CSCOM, Community Health Centres) and the Hospitals; each CSREF is responsible for a few CSCOMs (Figure 1). Primary health care is also provided by a range of para-public, religious, and private facilities.

Purpose/Objective:

The purpose of the external evaluation was two-fold:
1. To evaluate the effect of the IHSS programme on coverage of a limited package of proven, high impact, and low cost maternal and child health interventions in Mali.
2. To inform programme and policy decisions in Mali and regionally.

Objectives
To assess the effect of the IHSS programme on the following:

Relevance:
 - Contribution to an enhanced policy environment for child survival
 - Alignment with national priorities and plans
 - Strengthened multi-sectoral collaboration
 - A health systems strengthening approach, a focus on women’s participation and a gender equality approach.

Effectiveness:
 - Strengthening the health system, including all six WHO health system building blocks namely health workforce, service delivery, information, supplies, financing and leadership/governance;
 - The capacity of government and/or civil society organizations to train, equip, deploy, and supervise front-line health workers to deliver a limited package of proven, high impact and low cost health interventions.

Impact:
 - Coverage of selected maternal, newborn and child health and nutrition interventions (promotion of breastfeeding and vitamin A supplementation), particularly integrated Community Case Management (iCCM) of diarrhoea, malaria and pneumonia, which were
supported by the IHSS programme.
 - Number of additional lives saved by the IHSS programme calculated using the Lives Saved Tool (LiST) disaggregated by groups of interventions, e.g., iCCM, and by individual interventions according to the phases of the programme.

Sustainability:
 - Costs of implementing iCCM.
 - Financial sustainability of this programme.

Methodology:

A mixed method approach to this evaluation was used7,8 in that quantitative, qualitative and economic evaluation methods were utilised. For analysis of coverage, trend analysis was performed using a nonparametric test of trend across years and wealth quintiles for all available surveys (DHS 2001, 2006, 2012 and MICS 2010). Raw data was not available for the 2012 DHS survey, and consequently, confidence intervals around the point estimates could not be calculated. The coverage and trend analysis was restricted to the country’s five Southern Regions (Kayes, Koulikoro, Mopti, Sikasso, and Ségou). Data to assess implementation strength, utilisation and quality of care were taken from the Lot Quality Assurance Sampling (LQAS) Survey conducted in 2013.

Findings and Conclusions:

Relevance
 - The health systems strengthening approach of the IHSS programme was well aligned with the health policies of the Malian government
 - MoH leadership and co-ordination was an important factor in the success of the IHSS programme
 - Women’s participation and gender equality remains a problem in Mali

Effectiveness
 - THE IHSS programme contributed to the expansion of community-based health-care for pregnant women and children through deployment of ASCs but their availability needs to be improved
 - The IHSS programme strengthened the health system through training of ASCs and facility-based staff
 - The IHSS programme strengthened the health system through strengthening supervision of ASCs
 - The IHSS programme enabled the procurement, supply and distribution of medicines and commodities, with increased support during a financial and political crisis, but more systems strengthening is required
 - Assessment of the contribution of the IHSS programme to coverage changes found large improvements in both ORS coverage and the proportion of children under five sleeping under an ITN during the IHHS programme period

Impact
 - Mali is on track for Millennium Development Goal 4 for child survival, with an under-five mortality rate (U5MR) of 128 per 1000 live births in 2012 and a target of 84 per 1000 live births by 2015.

Sustainability
 - Qualitative findings show that there are still concerns with the future sustainability of the programme, although discussions are underway to develop long-term solutions.

Recommendations:

Relevance
 - While the MOH has been successful in ensuring coordination of funding and programme implementation among partners, issues of various models of supervision linked to iCCM
implementation and health care subsidies by partners despite an overall health system that implements user-fees represents areas in which further coordination and management of partners is required.

Effectiveness
 - Increase investment in IMCI at the CSCOM level is necessary to ensure a strong continuum of care, starting at the community level, and to ensure that the CSCOMS are fully equipped to handle referrals and complicated cases. Furthermore, as supervisory roles are the responsibility of the CSCOMS, IMCI training is necessary to ensure high level of competency of supervisors.
 - Supervision for ASCs should be integrated along with other outreach activities to reduce the burden on the health system, which is currently understaffed. Furthermore, extending supervisory roles to other health cadres, including the nurses at the CSCOM level, rather than leaving the responsibility solely on the head doctor in charge, should be considered.
 - It is important to ensure that once utilisation increases at the ASC level that there is an effective system to ensure that their stock is replenished and not held and used at the healthfacility. However, in the interim, these large volumes of drugs should be used at the health facility until demand is sufficiently generated at community level to prevent wastage.
 - Mali has not yet achieved elimination of maternal and neonatal tetanus9 and greater efforts need to be made to achieve this important milestone.
 - The lack of further progress in coverage of breastfeeding practices across the IHSS programme period highlights a need for further investment and emphasis to ensure that trained relais and traditional birth attendants are effective at promoting these key child survival practices.
 - Though some gains were made from 2001 in antenatal attendance, these rates are still very low and efforts to increase access to antenatal care are imperative.
 - While user fees and drug sales have ensured replenishment of drug supply and health care worker salaries, they represent a major barrier to health care utilisation and result in ongoing use of informal health care providers. The Government of Mali will need to assess its capacity to reduce user-fees for health care in an effort to promote increased utilisation. A further assessment of the Ministry of Health’s capacity to sustain its free malaria care in the absence of donor funding will be required.

Impact
 - Plausible contribution of the IHSS programme to lives saved has been assessed through modelling .It is acknowledged that other interventions not included in the IHSS programme package, such as skilled birth attendants, have also played a role in child survival and that the modelled estimates reflect changes in coverage at all levels of the health system, beyond the community level. It is recommended that future evaluations strive to collect data at other levels of the health system.
 - The underlying reasons for decline in coverage in important preventive interventions such as vitamin A need to be further scrutinized.
 - It is recommended that programmes are allowed sufficient time for roll out and full scale implementation to ensure that evaluations are able to fully capture their impact.

Sustainability
 - Currently 85% of net treatment costs reflect the share of ASC fixed costs. It may be desirable in areas where villages are not too distant to increase the catchment area per ASC, as this could decrease the number of ASCs required and increase utilisation per ASC, thereby decreasing the cost of the programme. This in turn would require that ASCs are given an allowance for cell phones and that bicycles are properly maintained.
 - Recognising that care-seeking patterns take time to change significantly, a new sustainability study should be undertaken when iCCM implementation has reached higher maturity, and possibly covers other conditions and older children. Patterns of utilisation at health centres and community level would have stabilized, and the additional costs of the program could then be better put in the perspective of other savings in the health system.
 - With such stabilization, health impact could be quantified, and new costing should calculate the cost per life saved.

2015 Health Section: CI IHSS Summative Evaluation Reports and Appendices

Find below:

"Report" - Evaluation Report
"Part 2" - Annexes
"Part 3" - GEROS



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2014

Country:
Mali

Region:
WCARO

Theme:
Health

Type:
Evaluation

Language:
English

New enhanced search