Evaluation database

Evaluation report

2011 Ukraine: Repeat Evaluation of Youth Friendly Clinics in Ukraine



Author: David Rivett, Lydia Amjadeen, and others

Executive summary

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”

Background:

Development of the YFCs is aligned with the Government priorities in Ukraine. The State Reproductive Health Programme sets up a target of 90 per cent of children’s polyclinics having the YFC embedded in the existing facilities by 2015. For 2010 – the statistics of the M&E Centre under the MoH provides for 24 per cent coverage (76 YFCs set up at the children’s polyclinics). The State Targeted Social Programme ‘Youth of Ukraine’ for 2009-2015 recognizes two main problematic areas regarding the young people in the country, such as: widespread drug use & HIV epidemic and employment. One of the stated Programme’s aims is to promote healthy lifestyles among youth. The MoH Decree N382 (02.06.2009) established ten quality standards and certification procedure for the YFCs in Ukraine.

The YFCs are drop-in centres or clinics that aim to create a safe and supportive environment for youth, and provide them with information, counselling and health services with the main focus on prevention of HIV and sexually transmitted infections (STIs). Such services should be accessible, acceptable and appropriate for adolescents as per WHO characteristics .

Since 2004 UNICEF has provided assistance to the Ministry of Health in the re-organisation of the national health care system for children by facilitating introduction of new standards of health care for children up to 18 years of age through capacity building of health care professionals, establishment of YFCs and development of new standards of new health care standards including job norms etc. UNICEF piloted an establishment of the YFCs, supported strengthening the quality assurance system of the YFCs and building capacities of the service providers in HIV VCT and reproductive health counseling.

The development of the YFCs in Ukraine is led by the Ministry of Health with technical assistance provided by UNICEF and through the Advisory Board, the coordination mechanism, established under the Mother and Child Health Department of the MoH for participation of key stakeholders, such as social services, UNFPA, WHO, etc.

A Monitoring and Methodological Centre for YFCs established under the Ukrainian Specialized Hospital ‘Okhmatdyt’ provides support for quality assessment, certification of the YFCs and coordinates the capacity building of YFCs’ personnel nationwide (http://kdm-ldd.org.ua/).

In 2008 UNICEF and MoH commissioned the first evaluation of the YFCs in the country. The Contractors are required to review the evaluation report (findings and recommendations) that might be found at http://www.unicef.org/ukraine/ukr/Evalution_Report_full.pdf. The need for the repeated assessment of the YFCs and produced recommendations in the end of the programme cycle has been identified by UNICEF and key national stakeholders during the MTR review in 2009. It is important to evaluate what has been achieved and what needs to be done to reach the national target of 90 per cent as well as to suggest a development strategy for YFCs in view of the lacking substantial progress in the health care reform and social and health sectors remained underfunded and understaffed.

Purpose/Objective:

The Research Team planned and implemented a research evaluation focused on client views and perceptions and provided a comparative analysis with similar research conducted in 2008. This was not a formative evaluation of a project funded by UNICEF but rather evaluation of effectiveness of YFC service itself. The Research Team consisted of the national institutional consultancy (local research team) and international consultant, both contracted separately by UNICEF but acting together as the ‘Research Team’.

The main purpose of the evaluation was to look whether the YFCs are accessible, acceptable and appropriate for adolescents and to assess the YFCs from the point of view of their relevance, efficiency, effectiveness, potential sustainability, and scaling up.
Evaluation criteria:
Accessibility
Acceptability
Appropriateness
Relevance
Efficiency
Effectiveness
Potential sustainability and scaling up

Beneficiaries’ involvement
From a young person’s perspective, there is a number of barriers—broadly understood in terms of access and quality— to seeking reproductive health and HIV prevention services. Issues of access and service quality often determine whether or not young people are willing or able to effectively utilize reproductive health services. Therefore the views of young people are key to this evaluation and their perception and satisfaction with the services are to be assessed and analysed. The evaluation team has to ensure the voice of the YFCs’ clients is given due attention in the report. In addition to interviews with the young people, the Research Team should plan for a number of meetings with the reference group from Youth Organizations who would be able to provide insights into the young people’s views.
The evaluation team evaluated the YFCs against the following characteristics:
Health Facility Characteristics:
• Convenient hours
• Convenient location
• Adequate space and sufficient privacy
• Comfortable surroundings
Provider and Staff Characteristics:
• Specially trained staff
• Respect for young people
• Privacy and confidentiality honored
• Adequate time for client and provider interaction
• Peer counselors available
Administrative/Process Characteristics:
• Youth involvement
• Boys and young men welcomed and served
• Group discussions available
• Necessary referrals available
• Affordable fees/No fees
• Wide range of services available
• Drop-in clients welcomed and appointments arranged rapidly
• Educational material available on site and to take
• Publicity/recruitment that informs and reassures youth
Young People’s Psychosocial Characteristics:
• Perception of privacy at a facility
• Perception that clients are welcome
• Perception that confidentiality is honored
• Perception that youth are welcomed regardless of their status i.e. economic, social, marital, gender, sexuality, etc.
• Perception that surroundings are comfortable
• Perception that providers are attentive to youth needs

The Research Team to accomplish its objectives was guided by ten quality standards as per MoH Decree N382 (02.06.2009).
The Research Team was guided by the UNICEF Evaluation policy and standards (see http://www.unicef.org/evaluation/index_13477.html).

Methodology:

- desk review (documentation of the Monitoring and Methodological Centre for YFCs, annual reports, regulations and legislation, reports and guidance available etc.)
- focus group discussions
- individual semi-structured in-depth interviews
- inventory of facilities and services
- mystery clients
- client exit interviews
- client satisfaction forms

Findings and Conclusions:

This report contains the main results of the repeat evaluation of the Youth-Friendly Clinics (YFC) national programme.in Ukraine. There is no doubt that Ukraine has made considerable progress in establishing a youth friendly health services programme within its health system. By deciding to set up youth friendly services as part of the sector’s responsibility, it has opened up an area of activity that addresses some of the critical health issues facing the country. The first network of the Youth Friendly Clinics programme was established in 2005 as a collaborative initiative between the Ministry of Health (MoH) and the UNICEF Country Office. The network started with youth friendly clinics YFCs in nine regions of the country. As the programme progressed additional clinics were opened and the Methodological, Organizational and Monitoring Centre of NCSH ‘OKHMATDYT’ was given the responsibility of providing support and coordination to the programme. Further developments in 2009 included the official adoption of national standards for youth friendly services which are now used in clinic certification, monitoring and evaluation, To date there are now 91 YFCs in Ukraine, 31 of them certified. OKHMATDYT in planning to de-centralise the national certification, support and training functions it currently performs to its network of Regional Breast Feeding Support Centres which heralds a scaling up of the programme as certification and other support needs are delivered at a much more local level. This repeat evaluation provides insight into the functioning of the YFC programme and should provide a great deal of information to inform its expansion plans over the coming years.
The repeat evaluation spreads the scope of its investigation wide, from the experiences of young clients using clinic services to the planners and decision makers responsible for guiding the programme’s activities.

Some of the key observations coming from the assessment indicate that many aspects of the programme are functioning satisfactorily:

• The legislation, although complicated and bureaucratic, provides a suitable base upon which to build the youth friendly clinic programme.

• In general, young clients are satisfied with the scope of services provided by the clinics and the competence of service providers in delivering their services in youth friendly ways.

• The network of clinics operates from within existing health and social services facilities, mainly based in paediatric poly clinics and centres for social services for families, children and youth. For the level of services the network provides now, this infrastructure appears to work.

• The standards that were developed, approved, adopted and are now being applied in clinic certification and monitoring have been widely welcomed, especially by service providers who see them as an effective tool to guide their work.

• Training of service providers, using adapted WHO resources, is in place both at pre service and in service level.

• The coordination centre based in OKHMATDYT is an efficient model for the current level of activity in the provision of training, data collection and certification. There are plans for its expansion.

• Cooperation and collaboration with other services such as Social Services for Youth and the Ministry of Education is practiced, especially at the local level, and there is encouragement for young people to be involved in clinic activities

Recommendations:

From these observations, the work of establishing the programme has been accomplished. However there are elements within all the areas mentioned above that will need strengthening if the programme is able to reach all young people and operate successfully as demand increases and the range of clients widens. These include:

• There is an urgent need for the programme to more effectively reach most at risk adolescents and young people. This most vulnerable group needs much greater focus from the YFC national programme. Good practice in youth friendly approaches is already evident within the NGO sector and this should be replicated within the national programme. .

• Costing of the programme at national, regional and local level needs to be clearly identified. Lack of transparency in this area of the programme could to lead to financial insecurity. In addition, clarity in matching costs to services is essential in understanding coverage and impact.

• The YFC financing arrangements need to be clarified and made more flexible, allowing YFC managers more autonomy in staffing, procurement and operations.

• The monitoring and evaluation responsibilities of OKHMATDYT should be strengthened to ensure the level of services is maintained, especially in the areas of coverage, quality and cost.

• Training of service providers should be expanded, especially in view of the potential expansion of the programme. Training for YFC managers is particularly important.

These and further recommendations are part of this assessment’s findings. There is much to praise in the development of the YFC national programme, but at the same time, the programme should still be seen as in its early stages of development. Much learning is emerging on how the programme can respond effectively to the health and development needs of the generation of youth in the country. With commitment from all stakeholders, the findings of this report will contribute much to the next steps the YFC programme should make in expanding its reach, and building and strengthening quality into the services it provides.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search