Author: Sam Clark, with Partners Ilir Hoxha
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labeled as ‘Part 2’ of the report.
Improving Health of Women and Children in Kosovo 2007-2010
The Joint Programme (JP) for Improving the Health of Women and Children of Kosovo was funded by the Luxembourg government to form a unique collaboration by three Kosovo-based UN agencies (UNFPA, UNICEF and WHO) to support government initiatives to strengthen the health care system, improve the quality of mother and child health care services and achieve the United Nations Millennium Development Goals 4 (reduce child mortality) and 5 (improve maternal health). The JP budget of 2,311,200 euros was to be implemented over a period of 3 years beginning in 2007 and was given a one-year no-cost extension. The project was developed to address five major MCH and RH related outcomes to be achieved through a series of outputs and project activities. Now that many of the components have been implemented, this end-of-project evaluation assesses the JP impact on mothers’ and children’s health in Kosovo. It attempts to document lessons learned for decision making on the continuation of the intervention and its approaches.
After four years since the beginning of the JP project, now that many of the components of the JP have been implemented, it is essential to measure, monitor and evaluate the impact that the project has had on mothers’ and children’s health in Kosovo. It is important to document the lessons learned for future programs for decision making on the continuation of the intervention and its approaches. The evaluation will be designed to inform the stakeholders concerning the progress that has been achieved as well as the areas where progress has not been achieved. The results from the evaluation will guide future stakeholder decisions and interventions and to ensure that they are appropriate for Kosovo.
The evaluation will assess the extent to which the outputs and outcomes of the project have been met. The final evaluation report, including the evaluators’ recommendations, will aim to be a practical tool to identify which best practices could be used and recommend improvements in the overall approach for the improvement of the health of women and children in the future. The final report will be designed to be relevant for a wide range of stakeholders, including Kosovo Ministries, civil society, public- and private-sector health care providers, local non-government agencies (NGOs), UN agencies and donors, to make informed choices toward interventions in the Kosovo health sector. The report will recommend which components of the JP should be removed, maintained, modified or added in the upcoming projects.
Per the inception report for the evaluation, the methodology was developed based on the evaluation terms of reference (TOR) and initial meetings with relevant staff and JP participants within the Ministry of Health (MoH), implementing partners and civil society. A desk review was conducted to address all outcome areas with an assessment of the respective outputs and activities within each outcome guided by seven core criteria: relevance, effectiveness, efficiency, sustainability, long-term impact, gender, and joint program management. Interviews were conducted in the local language with more than 42 stakeholders in Dragash, Fushë Kosovë, Mitrovicë, Prishtinë, Prizren and Skenderaj. Semi-structured questionnaires were used for training follow-up interviews with a total of 16 respondents (doctors, nurses and peer educators); 15 client exit interviews were conducted with antenatal, family planning (FP), and post-partum patients at regional hospital and municipal level maternal and child health facilities in the six locations.
The evaluation was limited in time and resources and it was not feasible to employ quantitative research methods (i.e. to collect statistically representative samples). Therefore the evaluation is inherently qualitative in nature, with the goal of using purposive, non-random samples to ensure a wide coverage of project activities. Due to the short time permitted to plan the evaluation and limitations due to the summer holidays, the response rates for certain interview categories was lower than desired.
The analytical framework for the evaluation follows a logic model approach. The evaluation employs a simple but plausible model where project activities are performed to achieve outputs that result in the desired outcomes.
One of the JP characteristics is its complexity; it attempts to cover a wide range of areas pertinent to the improvement of the health status of women and children in Kosovo. Each of the five main outcomes has from one to four outputs. For each of 13 outputs, plans were made for up to eight specific activities, a total of 49 planned activities in all. Based on objective assessment of available evidence, this evaluation has concluded that four out of the 13 proposed output objectives were fully achieved; while there was very substantial progress within most of the remaining nine outputs, they did not fully achieve the stated output objectives. A large majority of the planned activities were carried out; 42 of the 49 planned activities were fully implemented. Most planned activities were highly relevant and several activities were highly effective. Based on a triangulation of findings from the desk review, stakeholder interviews, exit interviews and training follow-up interviews, all but six of the proposed activities were considered highly relevant to the initial and current context of MCH and RH in Kosovo. For many of the activities, however, even those that were fully implemented and highly relevant, due to the lack of baseline and follow-up data, it was not feasible to determine if they were effective or not. Nonetheless, using the available data more than a quarter of the activities was considered highly effective.
PDF files require Acrobat Reader.