Facing COVID-19 with resilience: boosting evidence-based learning and adjusting evaluation in times

UNICEF completed the youth-accompanied Kosovo Programme Evaluation to inform programme implementation for the upcoming five years

Fiona Kelmendi
UNICEF/2020/N. Maxhuni
18 October 2020

The UNICEF Kosovo Office completed its Programme Evaluation for the 2016-2020 cycle Commissioned and managed by the UNICEF Regional Office for Europe and Central Asia (ECARO) in collaboration with the Kosovo Office, its implementation was advised and supported by the Evaluation Reference Group (ERG), co-chaired by UNICEF and the Office of Good Governance/Prime Minister Office. The ERG included representatives of Kosovo Institutions, CSOs, donors and other stakeholders. 

In response to the unprecedented situation created with COVID-19, the evaluation methodology was adjusted, shifting to a fully remote approach, and utilizing alternative means for data collection, such as Skype, Viber, Zoom, Microsoft Teams, phone calls and Survey Monkey. The high degree of internet penetration among targeted stakeholders and the nature of the evaluation questions enabled the success of this fully virtual approach. The evaluation team interviewed 121 stakeholders, conducted 32 focus group discussions, and collected information from 486 young people for the tracer study questionnaire.

The evaluation specifically highlights that the virtual data collection process might have contributed to greater quality of data collected compared to a face-to-face approach. The findings suggest that the increased feeling of anonymity, talking on Zoom or Viber or by phone, might have contributed for participants to feel more comfortable in expressing themselves. It was observed that during the interviews, respondents showed greater willingness to provide direct and honest information. “COVID-19 taught us that as evaluators, we can be responsive and we should consider innovative, non-traditional approaches to data collection now and in the future” – says Ashley Wax, ECARO Evaluation Specialist.     
In line with the firm commitment of the UNICEF Kosovo Office to amplify the voices, participation and empowerment of youth, five university students from the University of Prishtina were engaged as volunteers to support the implementation of the Kosovo Programme Evaluation.

The students were mentored by a university professor to ensure continuous follow up and sharing of their experiences with other students and their contribution was acknowledged with internship credits. The youth were engaged in every step of the evaluation process, benefiting from shadowing evaluation team during the data collection. They specifically contributed with data analysis, sharing their observations and insights on the evaluation findings, and writing their youth-led and youth-focused brief evaluation report, included as an Annex in the Evaluation Report. “Engaging youth in the evaluation process, not only empowers them and prepares them for the labour market, but more importantly, it brings valuable insights on keeping UNICEF programme fit-for-purpose and responsive to youth needs” – says Murat Sahin, Head of UNICEF Kosovo Office.    

The five students highlighted that their participation in the evaluation served as a capacity-building exercise, during which they acquired skills and knowledge needed for their fields of study and professional development. In their own words: “Believing in youth means also giving them the chance to grow themselves by ensuring opportunities for development, beyond words and promises”.

The evaluation assessed the totality of the Kosovo Programme, focusing on the relevance of UNICEF’s programme portfolio, its positioning and approaches chosen in response to emerging needs and Kosovo Institution priorities. A specific focus of the evaluation was placed on youth programming. The findings and recommendations of this evaluation will strategically inform the implementation of the 2021–2025 Kosovo Programme.
The key findings highlight that the UNICEF Kosovo Office is fit-for-purpose, and its programmes and projects are considered successful, achieving the desired outputs, in addition to being responsive and timely. The UNICEF Kosovo Office is found to have created, cultivated, and maintained good relationships across multiple stakeholders and is trusted and respected at multiple levels with a strong role on convening partners around child rights issues. In overall, the evaluation highlights that UNICEF has a strong reputation as the “evidence agency”, and on its work around youth participation and empowerment. UNICEF is seen as to have timely responded to unexpected emergent themes. The evaluation emphasizes that during COVID-19 pandemic, UNICEF was the fastest responder to mobilize support to affected populations.

The key evaluation recommendations are structured around supporting the linkages in the cascade from policy to the implementation of services to affected populations, increasing shared understandings regarding sustainability and improving frameworks for identifying and monitoring the contribution to impact. The UNICEF Kosovo Office, in close collaboration with Kosovo Institutions and young people will develop a management response to follow up on the implementation of recommendations.

Read the full Kosovo Programme Evaluation Report here: