We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2014 Ukraine: Prevention of Mother-to-Child Transmission and Improving Neonatal Outcomes Among Drug-Dependent Pregnant Women and Children Born to them in Ukraine



Author: Tamar Gotsadze, MD., PhD

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

To remove inequity in care, treatment and support for drug-addicted pregnant women, overcoming barriers to the provision and utilization of services by them, UNICEF initiated a pilot project ‘Prevention of Mother-to-Child Transmission and Improving Neonatal Outcomes among Drug-Dependent Pregnant Women and Children Born to Them in Three Cities in Ukraine’. 

The main objective of the project is: to establish, maintain, and improve gender responsive, comprehensive, and integrated services that address the needs of drug-dependent pregnant women and children born to them. Expected impact of the project is the reduction of mother-to-child transmission of HIV among vulnerable pregnant drug addicted women in selected pilot cities through provision of support to the government and civil society organizations to develop and implement effective HIV prevention, treatment and care services for drug addicted pregnant women and their children.


Over the life span of the project (2011-2014), the pilot was implemented in three phases. The initial phase (June - December 2011) aimed at assessment of situation and design of the most appropriate and feasible service delivery model.  As a result, the needs in establishment of two types of model of services for drug-dependent pregnant women and their children in pilot districts have been identified. In the 1st phase – pilot phase of the project (2012 – 2013) the pilot was launched in three cities of Ukraine – Kiev City, Dnipropetrovsk and Poltava and ensured provision essential equipment, drugs and medical supplies to new sites as well as building capacity of service providers. The 2nd phase of the project (mid 2013 – 2014), the project has been expanded to Kryvyi Righ, the city most affected by HIV and drug abuse and with high needs in gender sensitive services provision.

Purpose/Objective:

EVALUATION PURPOSE - The purpose of evaluation is to produce relevant information on the design and effectiveness of the pilot project and identify lessons learned and provide strategic policy and implementation recommendations. These recommendations provide guidance on how to strengthen the on-going pilot, how to ensure that expected outcomes are achieved and how to ensure that relevant policies and support is provided to ensure that adequate models of service provision for drug-dependent women are adopted and sustained in the future.

OBJECTIVES - Present formative evaluation analyses of whether outputs and activities within the project are lead to expected outcomes and goal of the project; analyses of the bottlenecks and barriers, including policies, practices and other structural barriers in medical and social areas; document lessons learned and good practices of the pilot project activities, along with evidence of outcomes; based on evidence to demonstrate whether or not a nation-wide scale up of the pilot approach and practice is possible and whether a scale up will effectively lead to closing of equity gaps in the area of work; and provide recommendations of how the on-going pilot can be effective and sustained in future, thus informing policy development and framework of the national scale-up of the pilot.

SCOPE OF EVALUATION - The evaluation covers all period of the pilot project implementation from June 2011 when the project was launched in three cities of Ukraine – Kiev City, Dnipropetrovsk and Poltava until 2014 when the project was expanded to one more city of Krivui Rig, Dnipropetrovsk oblast.

Methodology:

EVALUATION METHODOLOGY - The evaluation examines relevance, effectiveness, efficiency, impact and sustainability of the project and utilizes OECD DAC evaluation  criteria as defined in the UNICEF guidance on equity-focused evaluations, 2011. For achieving evaluation objectives, the evaluation framework (EF) has been developed. The evaluation focus areas were assessed against OECD evaluation criteria and the report answers questions stipulated in the TOR. Furthermore, the evaluation also integrated Human Rights (HRB) and equity (EQ) based dimensions. The evaluation questions were informed by i) the UNEG guidance on how to integrate Human Rights HRB & EQ considerations in evaluations  and ii) UNICEF’s equity based evaluation . It examines to what extent the PMTCT project benefited right-holders, including a wide range of program beneficiaries and strengthened the capacities of duty bearers and other key players to fulfill their obligations and responsibilities. In order to assess attainment of stated targets for programs outcomes and outputs, the RF was reconstructed as the current one lacked measurable outcome and output indicators. Where possible, these data has been used to establish causal relationship between results and interventions. Present report documents lessons learned, draw conclusions and formulate recommendations.

EVALUATION METHODS: The methodology comprised a mix of site visits and observations, face-to-face in-depth interviews, focus group discussions, desk-based research and review of existing reports, documents and available secondary data. 

Findings and Conclusions:

Project demonstrates high relevance - the evaluation discloses project’s high relevance. It is fully aligned with national context, national HIV policies and strategies as well as with the country cooperation programme of UNICEF and its mandate. Project remains to be relevant for coming years. Those who used services at integrated care facilities had high uptake of core medical and social services. The capacity building activities implemented within the project together with international knowledge sharing resulted in improved staff attitudes towards IDU pregnant women as well as improvement of quality of services. Project clients express satisfaction with the quality of services received and staff attitudes at Centers of Integrated Care (CIC) compared to non-project clients. Wide range of partners involved in project implementation leveraged financial and expert resources together with sound coordination contributed to the effectiveness of the project.  The project could have benefited more from results based management and more strait forward and operational monitoring and evaluation framework and practices. Nevertheless, the project was moderately effective in its contribution towards removing key bottlenecks in project areas. So far, none of the pilots have yet been able to completely fulfill the goal of truly bridging different parts of the health care and social systems, particularly public social services, in order to improve the continuity of care and outcomes for drug dependent women and children born to them. At present the project demonstrates integration of both services, though the role of social services and support is mainly played by NGOs, whereas the role of state social services is less evident. More operational project performance monitoring could have resulted in enhanced effectiveness of the project. 

Recommendations:

Recommendation #1: Elaborate most effective outreach interventions for increased targeting of drug dependent pregnant women - The evaluation revealed low coverage of target population by the project. If the government is committed to attainment of stated objective, effective outreach approaches have to be elaborated to ensure that those on the margins of society are aware and able to demand services. For this purpose it is highly recommended to develop the most effective outreach interventions. 
Recommendation #2:  Ensure continuity of care and strengthen family and government support, particularly for most disadvantaged - A certain degree of continuity in service provision to drug dependent pregnant women and children born to them are confirmed by evaluation, though some needs of target population remain either partially met or unmet. The ICM takes relatively good care of women during pregnancy and delivery by provision of antenatal care, access to VCT, ART and OST, but less focuses on the provision of postnatal care. Evaluation findings show a great need in strengthening postnatal care for women and children with particular emphasis on early childhood development.  The Government of Ukraine as well as all partners involved is strongly advised to expand the basic package of health services for drug dependent women and children born to them by inclusion of postnatal care and early childhood development for target beneficiaries.
Recommendation #3: Explore other opportunities for social service provision to most disadvantaged
Recommendation #4: Enhance regular data collection and analysis for monitoring project impact and outcomes

Lessons Learned:

Lesson 1:  Bridging health and social systems has potential to close equity gap
The pilot project helped to bridge the health care and social systems in order to improve the continuity of care and outcomes for vulnerable to HIV pregnant women and children born to them.  Albeit none of the pilots have yet been able to completely fulfil the goal of truly bridging different parts of the health care and social systems in order to improve the continuity of care and outcomes for patients.

Lesson 2:  Shaping staff attitudes and tackling stigma among service providers
The pilot project exhibited other positive, but less tangible, results as well. The project had clear impact on shaping staff attitudes and tackling stigma among service providers, and confirms recognition by medical personnel that the ICM helps better to serve their clients.  Many staff cited feelings that they are now better able to serve their clients, and consequently more fulfilled in their work, under the IC model. “We got used to our patients, learned to trust each other under this system, and now we see them, I’d say, almost like relatives,” said the head of maternity department at Dnepropetrovsk. Lesson 3:  Recognition as a global model for IDU care
The Ukrainian approach to the issue was recognized as the best practice in the CEECIS region that should be replicated to other countries.

Lesson 3: Recognition as a global model for IDU care
Experience with innovative ICM model piloted in Ukraine has been shared within the CEECIS region. The project’s experience was presented at the International Harm Reduction Conference in Vilnius on 11th of June 2013 in Lithuania. The Ukrainian approach to the issue was recognized as the best practice in the CEECIS region that should be replicated to other countries.

Lesson 4:  Setting a scene … before national scale up
Considering highly vertical public health and social systems design, establishing an integrated treatment model for IDUs in Ukraine that has gained recognition as a global model for IDU care faces many challenges along the road.  Uncertainty with on-going political and economic crisis and reforms in health and social sectors, funding issues and staffing shortages, lack of correspondence between the legal and regulatory structure of Ukraine’s medical and social systems created obstacles for national scale up. 



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2014

Country:
Ukraine

Region:
CEE/CIS

Theme:
HIV-AIDS - Prevention Mother-to-Child Transmission

Type:
Evaluation

Language:
English

 

New enhanced search