We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2016 Zimbabwe: Independent Evaluation of the Health Transition Fund in Zimbabwe



Author: Prof. Nynke van den Broek; Dr. Thidar Pyone; Dr. Helen Owolabi; Prof. Stephen Munjanja; Mrs. Margaret Caffrey; Mr. Michael Lijdsman; Dr. Sarah White; Ms. Siv Steffen Nygaard

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report, and the executive feedback summary labelled as 'Part 4'.

Background:

In the 1980s and 1990s, Zimbabwe was a model for many African countries, for the quality and availability of health and social services to the population. Afterwards, the health system was severely hit by the crisis that affected Zimbabwe between 2000-2008, and the impact of this crisis was further exacerbated by the severe HIV/AIDS epidemic that hit the country during the same period. Life expectancy dropped and maternal and child mortality increased as a consequence of reduced access to goods and services. The infant mortality rate increased from 49 per 1,000 live births in 1998 to 60 in 2005-2006 and remained stable at 57 in 2010-2011 (ZDHS 1998; 2005-6; 2010-11). Similarly, the under-five mortality rate was estimated at 71 per 1,000 live births in 1998; 102 in 1999 and 84 in 2010-2011.  Maternal health indicators worsened even more dramatically due to the HIV/AIDS epidemic: the maternal mortality ratio increased from 364 per 100,000 live births in 1994 to 960 in 2010-11 (Source: ZDHS 1994; 2010-11).

In such social, economic and political context, the Health Transition Fund (HTF) was designed to contribute to reduced maternal mortality and under-5 mortality and eliminate user fees for children under 5 and pregnant and lactating women in Zimbabwe by 2015. The programme also aimed to contribute to reducing the prevalence of underweight children under 5 by half and to combat, halt and reverse trends in HIV/AIDS, malaria and other diseases.
 
The HTF was a pooled fund mechanism and during the period 2012-2015, various donors including DFID, EC, Governments of Ireland, Sweden, Norway, Canada, UK invested approximatively 210 Million USD in the health sector via HTF.  The HTF was governed by a Steering Committee, responsible for the fund governance and for decision making. UNICEF, as fund manager, was tasked to ensure the implementation at scale of all the initiatives approved by the Steering Committee, as well as to provide technical assistance.


Purpose/Objective:

To assess to what extent HTF strategies, approaches and overall intervention logic have contributed to changing the health situation of the population with a special focus on maternal, newborn and child health and at the same time to determine whether the resources have been used in the most efficient way to achieve those changes.

The purpose of the final evaluation was two-fold: a) To assess the achievement of HTF intended results by the end of its implementation in 2015; b) To document lessons learnt and identify success factors and areas to be improved after the end of the program.

In line with the OECD-DAC criteria for international development evaluations, the evaluation sought to provide an assessment of the achievements of the HTF against the following criteria:

Relevance   (Are we doing the right things?)
Relevance is defined by OECD-DAC as “the extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor”.

Effectiveness/Efficiency (Are we doing things right?)
Effectiveness is defined by OECD-DAC as “A measure of the extent to which an aid activity attains its objectives”. Efficiency measures the outputs in relations to inputs.

Impact (Did we contribute to change?)
Impact is defined by OECD-DAC as “the positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended”.

Sustainability (Will change last?)
According to OECD-DAC, “Sustainability is concerned with measuring whether the benefits of an activity are likely to continue after donor funding has been withdrawn”.

Methodology:

In line with the OECD-DAC criteria for international development evaluations, the evaluation was designed to assess the plausible contribution of the HTF to national level progress in improving maternal and child health outcomes in Zimbabwe, according to the following criteria: relevance; effectiveness; efficiency; impact; sustainability. 

A set of evaluation questions were defined and agreed with the HTF Steering Committee to assess each of the evaluation criteria; the HTF Logframe formed the cornerstone against which progress made during implementation was analysed.

A mixed methods approach utilising quantitative and qualitative evaluation methods was used for the final evaluation. 
In particular, methods of data collection and analysis have included:

  • A nationwide health services assessment, conducted in 2014, 2015 and 2016 to assess the availability of health services at district level and at primary care level through a modular survey administered at three levels of the health system: District Health Management, District Level Hospitals and Level 1 Facilities.
  • Key informant interviews conducted in 2015 and 2016 and used to gather views and perspectives of stakeholders at three levels: national, district in health facilities, and community in health centre committees.
  • Focus group discussions with members of district health offices, village health workers and beneficiaries (women of reproductive age, caregivers of children under-5) conducted in 2016.
  • Secondary data and reports available in Zimbabwe and internationally

Findings and Conclusions:

Relevance
The HTF was consistent with global and national health policies following the structure of the MoHCC. The evaluation observed a good degree of relevance of the HTF to the needs and priorities of target groups.
Effectiveness
The HTF has demonstrated consistent progress in achieving intended results. The assessment of effectiveness focused on output level indicators as defined by the HTF log frame, which identifies 54 indicators as the key measures to monitor progress of the program at output level. Of them, 29 indicators present baseline and endline data, which allow to measure progress between 2012 and 2015: 28/29 indicators (96%) showed progress. This suggests that the HTF strategies have been effective in achieving intended results. 
Efficiency
The HTF was efficient in disbursing funds.  Overall, the HTF absorbed nearly 90% of allocated resources throughout its life, with a marked improvement in the expenditure rate during the FY 2014 and 2015, where 98% of resources were utilized. This suggests that solid planning mechanisms were in place; that donors disbursements were efficiently and timely managed; and that resources were absorbed by the system as per plans. The management costs of the fund manager (UNICEF) appear to be competitive vis a vis with standard practice in the sector.
Impact
The HTF contributed to reducing mortality rates. Modelling shows that 4.000 to 6.000 per year additional children’s lives were saved during the period 2010/11-2015. This is due to significant improvements in coverage of essential MNCH interventions across Zimbabwe.
Sustainability
The ownership of the HTF results and of its strategies is high at all levels of the system. However ownership may be endangered by current inefficiencies in the system, by information asymmetry and by uncertainty, especially at lower levels of the system.

Recommendations:

  1. A leaner and strategic steering committee, possibly supported by dedicated technical working groups on operational matters, should focus on a communication and fundraising strategy aimed at broadening the donors’ base in support of the HDF, nationally and internationally.
  2. A simplified and more focused monitoring and evaluation mechanism can be used to assess progress at national and local level.
  3. Harmonization and simplification of various financing mechanisms that are currently in place in Zimbabwe to sustain facilities and health care managers and workers should be a priority for the HDF since its early stages. 
  4. Staffing shortages and heavy workloads are contributing to health worker demotivation; a revised and more realistic establishment is needed to ensure adequate numbers of skilled health workers are available, distributed equitably and accessible across the health sector. 
  5. Continued support to retention and critical post allowances should be framed within a broader strategic approach to human resource planning and management.
  6. Continued support to procurement and supply of a minimum package of health products for Level 1 facilities is conditional to the well-functioning of the health sector and should be sustained in full through external assistance in the short term. 
  7. At lower levels of the system, the investment in governance structures and mechanisms should be sustained, to further reinforce district, facility and community level planning, monitoring and accountability mechanisms.
  8. The investment in village health workers, with clearly defined roles and responsibilities, needs acceleration, possibly under a broader, revised strategy for community health looking at a mix of approaches to address demand side bottlenecks to access, care seeking practices and behaviour.


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2016

Country:
Zimbabwe

Region:
WCAR

Theme:
Health

Partners:
Government of Zimbabwe (Ministry of Health and Child Care), The Governments of the United Kingdom of Great Britain and Northern Ireland ("the United Kingdom") represented by the Department for International Development (DFID); The Government of Ireland represented by Irish Aid; The Government of Sweden; The Government of Norway; The European Commission represented by the Delegation of the European Union to Zimbabwe    

Type:
Evaluation

Language:
English

Sequence #:
2016/001

New enhanced search