2013 Republic of Montenegro: Mid-Term Evaluation of the "Social Welfare and Child Care System Reform: Enhancing Social Inclusion"
Author: Arkadi Toritsyn, Ph.D.
"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labeled as ‘Part 2’ of the report."
The Ministry of Labour and Social Welfare– MoLSW and the Ministry of Education and Sport - MoES) are implementing the “Social Welfare and Child Care System Reform – Enhancing Social Inclusion” Project with the support of UNICEF and UNDP, and with the financial assistance of the European Union. The total budget is 3,335,000.00 EUR of which EU contribution amounts to 3 million EUR while the remaining amount is provided through national contributions.
The project has three main components:
- Project component 1: Inclusive education is coordinated by the Ministry of Education and Sport and implemented by “Dominus”. The implementation of Component 1 began in December 2011 and will be completed by March 2013. Component 1 was not evaluated.
- Project components 2 and 3: Social welfare and child protection. These components are coordinated by the Ministry of Labour and Social Welfare and implemented by UNDP and UNICEF, respectively. The implementation of these components began in January 2011, and is due for completion in July 2013.
This evaluation took place during the project implementation to inform the second part of the project. It was a formative exercise, the results of which would be used by key project stakeholders such as UNICEF, UNDP, MoLSW, Ministry of Education and Sports, other relevant line ministries and central agencies as well as the EU Delegation in Montenegro to inform its future implementation.
The period covered by the evaluation was between the beginning of the project implementation (mid-January 2011) and the beginning of the evaluation (mid-November 2012). The geographic coverage was national.
The evaluation report captures good practices, lessons learned and gaps in the approach to the reform and includes recommendations for potential improvement of the project until its finalization as well as recommendations concerning the reform processes beyond this project.
Considering that this is a mid-term Evaluation, the focus of the evaluation is on relevance, effectiveness and efficiency and expected impact and sustainability. Sustainability and impact will be more fully assessed during the Final Project Evaluation to be carried out close to the finalization of the project.
The Mid-Term Evaluation of Components 2 and 3 of the project was conducted during November/December 2012 with the following objectives:
- to provide recommendations for the most effective continuation of the project and the reform process based on an assessment of the implementation of the two components to date, and
- to explore relevance of potential revisions, requests for top-up (costed) or no-cost extensions of the current project and elaboration of future support of the ongoing reform process beyond the current project.
A large set of different and complementary evidence was collected and analyzed by utilizing both qualitative and quantitative methods that included:
- Desk review of relevant documents.
- Consultations with UNICEF and UNDP management and teams.
- A one-day Mid-Term Review workshop/meeting gathering project partners, stakeholders and experts was facilitated on November 27, 2012.
- Four brief informal group discussions with project beneficiaries.
- Semi-structured interviews with pre-determined sets of questions with stakeholders and project beneficiaries;
- Informal conversational interviews with questions emerging from the immediate context were conducted on ad hoc basis with UNICEF/UNDP project management and staff;
- Validation of preliminary findings and recommendations with UNICEF/UNDP.
The main limitation of this mid-term evaluation is that it has been carried out in a very compressed timeframe with limited resources applied across a very wide and complex scope of inquiry. In reality, the consultant had to evaluate two projects implemented by UNICEF and UNDP, respectively and had in total 20 days to complete this assignment.
Findings and Conclusions:
The project is highly relevant to realities of Montenegro. The project addresses the underlying causes of social exclusion, responds to the needs of the most vulnerable target groups and is well aligned with the national policies and strategies. In November-December 2012, when this mid-term evaluation is conducted, both UNICEF and UNDP components remain highly relevant to realities of social welfare and child protection reforms in Montenegro that was confirmed in particular by 70 participants of the one-day Mid-Term Review workshop/meeting gathering project partners, stakeholders and experts as well as all interviewees. See Conclusions in chapter 6.
As the analysis above demonstrates, the project would not be able to meet all its delivery targets by June 2013 due to the parliamentary elections resulted in delays in passing the Law on Social Welfare and Child Protection that is still in the Parliament, limited Government commitment to social sector and child protection reforms and its decisions not to establish a Social innovation fund and the Institute for Social Welfare but create a new sector in the MoLSW instead, limited capacities of a respective sector of the MoLSW as well as too ambitious project targets. It makes it necessary for both UNICEF and UNDP to seek a project extension. Time extension is needed mostly to support effective implementation of the Law on Social Welfare and Child Protection as well as to achieve other project objectives.
The following recommendations regarding key activities for project extension should be seen as an agenda of possible topics for discussions between UNDP, UNICEF, MoLSW and other relevant Government institutions, partners and donors. The recommendations are listed in descending priority order and identify key areas that can be addressed by the project till the end of 2014. They are based on the following assumptions:
- the Law on Social Welfare and Child Protection is adopted by the Parliament without major changes by early 2013;
- a new Government does not embark on a radical decentralization reform and does not delegate all the responsibilities for social welfare and child protection to local authorities till the end of 2014; and
- the country economic performance and budget revenues improve or at least do not deteriorate.
Read more in chapter 7.
Full report in PDF
PDF files require Acrobat Reader.