We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2017 Guinea Bissau: Final Evaluation of the Community Health Component of the “Programme for reducing Maternal and Infant Mortality (PIMI) in Guinea-Bissau



Author: Juan-Luiz Dominguez

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’

Background:

The whole PIMI I purpose and goals were to accelerate reduction of child and maternal mortality in Guinea Bissau through a three-pronged action: effective community-based basic healthcare, improved quality of healthcare services, and the strengthening of the coordination of peripheral structures and management of resources of the health facilities. The community-based component was implemented under the leadership of UNICEF by three international NGO – VIDA, ADPP and AIFO – in 5 different regions (Cacheu, Biombo, Oio, Farim and Gabú)

Purpose/Objective:

The overall objective of this evaluation was to conduct a summative evaluation of the PIMI intervention by meeting specific objectives, those being:
- To systematically assess the programme logic, rationale and relevance;
- To describe and explain the programme results including its success factors, effectiveness and efficiency;
- To assess the sustainability of the programme, including lessons learnt;
- To make appropriate recommendations for improvements and further scale-up.

This evaluation was aiming at the assessment of the PIMI I community-based health care component from its start in 2013 through 2016 until its phasing out in early 2017 and was conducted for accountability and learning purposes so as to provide stakeholders with an assessment of the relevance, performance, quality and sustainability of PIMI and learnt lessons.

Methodology:

A mixed methods approach was used to collect evidence for this evaluation in an attempt to elicit the following:
- Perceptions and opinions of main stakeholders and implementing partners: meetings and consultations with UNICEF and other UN agencies (WHO, UNFPA, UNDP), MOH staff.
- Community health workers (ASC), supervisors and leaders’ perceptions and opinions on quality of health promotion and sensibilization delivered (training, supervision, equipment, drug provision, etc.): key informant interviews and Focus Group Discussions.
- Beneficiaries’ perceptions and degree of satisfaction over the improvements brought about by the interventions: key informant interviews and Focus Group Discussions.
- Statistical changes on healthcare facilities’ frequentation and trends to show more permanent changes in health-seeking behaviour of women in fertile age and under-5 children: quantitative and qualitative data analysis, and data triangulation

Findings and Conclusions:

Relevance:
• Intervention extraordinarily well designed and deeply focused on maternal and child health needs.
• The community health component was superbly tailored to respond to the objectively identified needs of the most vulnerable population. PIMI as a whole was a thoroughly comprehensive approach to tackle severe weaknesses in the Guinea-Bissau healthcare system.
Effectiveness:
• Lack of a reliable baseline data and a KAP survey at the beginning of the action.
• Extraordinary effort made to strengthen health area staff’s capacities, considered as the main pillar of PHC management.
• The proportion of under-5 consultation decreased over the three years. The reason might be an early detection and treatment of common diseases by the ASC .
• Important increase in the proportion of women having received 4 ACN consultations in VIDA regions (Cacheu and Biombo). • The rate of assisted deliveries over the 2013-2016 period does not show a consistent impact, even accounting for a demographic increase in the number of expected pregnancies
• Fair increase in the percentage of assisted deliveries by qualified health staff.
• Poor coordination of the different PIMI partners in the field
Efficiency:
• Not significant expenditure deviations from what could be expected from a project like this.
• The percentage allocated to Human Resources (slightly higher than 25%) looks less expensive than other similar development projects. PIMI intervention’s staff costs appeared to be fairly efficient against the outcome obtained.
Impact:
• Very positive changes brought about by the PIMI community-based healthcare programme
Sustainability:
• No implementing partner appeared to having put in motion an exit strategy.
• Collapse of many of the activities that had become standard practice in both ASC and health workers alike once the project phased out

•Some policy measures taken by the government may keep up with some the gains obtained by the intervention

Recommendations:

• Adjust the duration of any action accordingly.
• An effort must be done to provide a health intervention with a reliable point of departure from which to assess performance indicators.
• An encompassing effort to reshape the decaying national health information system (SIS) has to be attempted. A short-term consultancy to evaluate SIS present condition and propose options for revamping it together with an estimate budget should be proposed.
• Any intervention as intently wide-ranging as PIMI, must build operational spaces into-the-field for coordination of planning and management actions, at all levels of the healthcare ladder:
• There are a few basic strategies to ensure a certain degree of sustainability:
- Intervention length.
- Consensual exit strategies in place.
- Binding commitment by the MOH to ensure that there is:
• An enforced MOH policy of keeping health staff turnover – DRS and health centres’ – to a minimum.
• MOH staff salaries and bonuses timely paid.
- MOH progressive appropriation of the financial and operational inputs brought about by the intervention.

Lessons Learned:

• The lack of either a reliable baseline data and/or a KAP survey at the beginning of the action made the intervention progress difficult to appraise.
• There simply was not enough time to consolidate gains in a subject as frail as behavioural change.
• The coordination and interaction between PIMI partners (UNICEF partners, IMVF, EMI) were so weak that a complete lack of knowledge about what the other partners were doing in the same background was the norm, missing opportunities for profiting from complementary actions.
• Many of the pre-established performance targets established were unachievable considering the abysmal situation of the country ‘s healthcare system.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2017

Country:
Guinea Bissau

Region:
WCAR

Type:
Evaluation

Theme:
Health

Partners:
European Union

Language:
English

Sequence:
2017/003

New enhanced search