We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2014 Kyrgyzstan: Evaluation of Perinatal Care Programme



Author: Mrs Tamar Gotsadze, MD, PHD and Mrs.Chiara Zanetti MD, MPH

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 4’ of the report.

Background:

Perinatal Care Programme was  conducted in the framework of the UNICEF DaO Project Ensuring Access to Affordable Health Services for Women of Reproductive Age and Newborns in the Affected Areas of the Kyrgyzstan Republic. The external evaluation was commissioned by UNICEF Kyrgyzstan CO and was a joint and participatory process involving key stakeholders. The evaluation team consisted of international consultants Mrs. Tamar Gotsadze, MD, PHD and Mrs. Chiara Zanetti, MD, MPH and experts from the MoH Mrs. Batma Dolbaeva, Docent in Obstetrics-Gynecology at Kyrgyz-Russian Slavic University and Mrs. Janar Botbaeva, Assistant at the Pediatrics Department at Kyrgyz Medical University for Post-Graduate Studies.

Overall objective of the project is to reduce perinatal mortality in the Kyrgyz Republic. Specific objectives are to enhance equitable access to health care services of the poor and vulnerable groups by improving the quality of maternal and newborn care and monitoring data in selected maternity hospitals. Project Expected results are: i) Continuous Quality Improvement process demonstrated, institutionalized in maternity hospitals; ii) Enhanced capacity of medical experts on the national level; iii) Improved quality of antenatal and perinatal care through critical lifesaving equipment, infrastructure; iv) Improved practical skills of medical workers for antenatal and neonatal care in target primary health care clinics and 20 selected maternity hospitals (introduction of Making Pregnancy Safer, Effective Perinatal Care, neonatal resuscitation, Baby Friendly Hospital Initiative); v) Effective registration and monitoring system in place including analysis of critical cases (i.e. pregnancy registration, Newborn Register, The Near Miss Cases Review); vi)Adequate referral and remote consultation system in place; and vii) Improved accessibility and utilization of maternal and newborn services especially in rural and remote areas through increased awareness in population.

Purpose/Objective:

The main purpose of the evaluation was to document and increase the knowledge of results, good practices and lessons learnt in perinatal care with specific recommendations. It also specifies UNICEF’s contribution to enhancing maternal and child health care system, quality of care and maternal and child survival in the country as well as determines whether UNICEF pilot projects have been effective and should be scaled up countrywide. In addition to UNICEF Country Office, other UN Agencies and development partners as well as the Ministry of Health will benefit from the evaluation in planning, implementing and coordinating perinatal care. 

Methodology:

The evaluation focused on assessing the project’s current and potential contribution to the improvement of MCH situation. Each evaluation criterion was analyzed from the perspective of assessing the implications of project activities’ on: i) final beneficiaries - women and children; ii) service providers - health care professionals whose capacity has been built (including doctors, midwives, and health facility managers); iii) sub-national decision-making level - Regional health authorities and (local governments; and iv) national decision-making level - national health authorities and key stakeholders, International Development and Implementing Partners, etc.

The evaluation examined the relevance, effectiveness, efficiency and sustainability of UNICEF’s contribution for which the OECD DAC evaluation approach has been applied. To translate the questions for the evaluation and the contextual issues, the Evaluation Framework (EF) was developed which structures the issues and questions as indicators that enabled to measure or assess them during the evaluation.

The evaluation methodology comprised of site visits and observations, face-to-face semi structured interviews, focus group discussions, criterion-based audit of medical records, desk review of existing documents/data and considered ethical issues. The six major sources of data were used during the Evaluation:  Individuals were consulted through semi-structured interviews and focus groups; Data collected during the visits to sampled project supported sites; Project and thematic area related documents (primary and secondary data sources) were reviewed; Qualitative data collected during observation of service delivery at the visited facilities were carried out; Data was collected through clinical audit of the patient medical cards. The ET utilized quantitative analysis to examine changes in selected but comparable indicators from available data.

Findings and Conclusions:

High relevance was pointed out due to clear alignment with national perinatal care policies and strategies as well as with the CPAP and has a grate potential for improving health of mother’s, children and vulnerable groups with a particular focus on maternal and child mortality decrease.

Quality of the project design requires improvement. The project approach proved relevant in practice, though the rationale and workings of it were not made sufficiently explicit in the project document. The project was not built around an explicit and shared theory of change on the whole.

In terms of efficiency the governance and management mechanisms set up proved to work well. So did the coordination system set at a national level.  Efficiency was ensured by adequate resource allocation, selection of most efficient funding modalities. 

The project has been able to reach mostly outcome level changes: a decrease in postpartum hemorrhage, complications and increase in newborn survival in the intervention provinces. In particular improvements in quality of perinatal care occurred due to the training of health personnel, improved infection control due to the important investments in infrastructure and better access to the vitally important life saving equipment. 

Effectiveness was largely undermined by system related challenges. While it is well understood that UNICEF operates within it’s organizational competence, issues beyond UNICEF’s competence could have been addressed either by advocacy or by mobilization and involvement of technical expertise from other developing partners.

The project was instrumental in building national and sub-national ownership and influencing national policies. Nonetheless, the capacity built in the system and the benefits achieved so far are at risk to maintain, if system wide shortcomings are not adequately addressed.

Recommendations:

Recommendation #1: Continued support to GOK’s Perinatal Care Strategy
In MCH sector, UNICEF is recognized as one of the leading agencies. This confirms its legitimacy and the capacity to continue work in MCH area and EPC in particular. Ensure continuous support to the GoK’s perinatal care programme implementation in line of recommendations provided in the section 6.2 below.

Recommendation #2: Strengthen inter-agency collaboration
Further improvement of inter-agency collaboration where all partners will have a role to play in supporting the MoH in the implementation and update of programmes in the area of perinatal health, including further investment in strengthening the main health system components, and policies to reduce inequities in access to care, as well as in the quality of services provided, all of which will further improve the health status of mothers and children in Kyrgyzstan.

Recommendation #3:  Enhance advocacy
The new challenges identified, in the section below, will require promotion of greater linkage and partnership through strengthening of the UNICEF Country Office (CO) technical capacity in the health policy advice. When selecting final set of interventions for the new project phase, attention has to be paid to CO capacity. Some recommended actions might demand specific technical expertise, which require additional resourcing. Moreover supporting research and analysis of the MCH sector performance will be instrumental for effective advocacy. Building on “what’s already working” will help to influence the government policy decisions.
 
Recommendation #4: Improve project design
Comprehensive Project design, addressing all health system blocks, with clear distribution of functions (activities and geographical and or thematic areas between key UN agencies), coordination mechanism between UN and other developing partners, as well as well formulated Results framework with annual targets, should be given a priority.

Find below:

"Report" - Evaluation Report Part 1
"Part 2" - Evaluation Report Part 2
"Part 3" - Annexes
"Part 4" - GEROS



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2014

Country:
Kyrgyzstan

Region:
CEE/CIS

Theme:
Health

Type:
Evaluation

Language:
English

New enhanced search