We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2016 Serbia: Formative evaluation of implementation of inclusive practices in the Education System in Serbia (2009 – 2014)



Author: Mr Péter Radó, Mr János Setényi, Ms Danijela Petrovic, Mr Dejan Stankovic, Mr Gábor Markovits

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

The inclusive education initiative of the Serbian Government is part of a coordinated effort to overcome the serious equity problems of the education system. According to the data at the time of introduction of the new Law on the Foundations of the Education System, 85% of children with disabilities were not covered by any systematic education; primary education was attended by only 75% of rural children and no more than 37% Roma children. As many as 68% of Roma children were leaving elementary school early and were representing a majority in special schools (up to 80% of all students).
The problem of the prevailing separated education of children with Special Education Need (hereafter: SEN) was very much connected to the exclusion of Roma students; as in other South-East European and Central-Eastern European countries, special schools became one of the channels for the segregation of Roma students. The inclusion policy also connected to the efforts of the government to ensure full school enrolment for disadvantaged students in general. Also, SEN inclusion has been widely considered in Serbia as one of the drivers of the modernization of mainstream education in order to improve learning outcomes, as well as of ensuring the rights of children and parents.
This report summarizes the results of the formative evaluation of the inclusive education policy of the Government of Serbia initiated in 2009. The main objective of the new policies and strategies has been to improve the quality and coverage of preschool and primary education, to enhance the educational achievement of all students in general, and children from vulnerable social groups in particular. Inclusion is the process of addressing and responding to the diversity of needs of all children, youth and adults through increasing participation in learning, and reducing and eliminating exclusion within and from education

Purpose/Objective:

This evaluation project serves formative purposes. Therefore, the primary purpose of the evaluation is supporting the most important actors who are involved in the design, implementation and monitoring of the inclusive education policy. Especially, the evaluation intends to support reflection on the implementation process, and to identify the necessary interventions in order to ensure that the original goals of the policy are effectively served.
The formative evaluation serves multiple goals:

  • It should inform policy makers – especially the Ministry of Education, Science and Technological Development (hereafter: MoESTD) – and the major stakeholder groups about the extent to which the original goals of the inclusion policy have been met in the course of implementation so far;
  • It should identify the risks, challenges, obstacles and possible problems emerged in the course of implementation that may call for interventions or corrections;
  • It should identify further development needs at all levels and in all relevant institutions involved in the implementation of the inclusion policy in order to inform donor coordination and the planning of donor activities;
  • It should identify those activities that have the potential of improving the work of all actors if shared and scaled-up;
  • It should promote the professional discourse on educational inclusion, as well as the exchange and discussion among various stakeholders.

The Terms of Reference sets five evaluation objectives: (1) assessing relevance, (2) assessing effectiveness, (3) assessing sustainability, (4) assessing impact and (5) assessing efficiency. In addition to these main objectives the TOR determines three key cross-cutting issues. These are: the contribution of the measures to the promotion of child rights, the extent to which an equity focus is ensured and the reflection on gender mainstreaming issues.

Methodology:

The applied methodology is based on gathering evidences from multiple sources along the 30 evaluation questions of the ToR and 16 thematic clusters set by the approved Inception Report. The applied methodology includes document analysis, legal analysis, statistical analysis, focus group interviews, individual interviews and classroom observation. The use of multiple evaluation instruments provided information for answering each evaluation questions.

The fieldwork evaluation has comprised the following sources: a sample of 6 settlements and 7 educational institutions, the list of resource organisations and persons, a selection of documents for analysis (strategies, policy documents, reports, research publications and others), a selection of various level regulations for legal analysis, and data sources for statistical analysis. The sample has been determined in cooperation with UNICEF and the Inclusion Team of the MoESTD on the basis of predetermined criteria.

For fieldwork evaluation, four types of instrument have been developed and applied: (1) semi-structured guides for individual interviews, (2) semi-structure guides for focus group interviews, (3) assessment criteria, and (4) a classroom observation protocol. The evaluation process was based on the involvement of all major stakeholder groups and contained ethical safeguards in order to ensure the information of respondents, confidentiality and independence of the evaluators.

Findings and Conclusions:

Due to a clear shift towards a mainstreaming policy approach, the relevance of the policies and implementation measures against the challenges stemming from the equity related problems of the Serbian education system is very high. The inclusion policy is highly comprehensive; it addresses almost all relevant dimensions of societal disadvantages: low social status and poverty as well as ethnicity (Roma affiliation) and impaired individual capacities. The only dimension that was neglected by policy is the problem of gender inequities.
The 2009 inclusion policy, to some extent, was driven by information. The relevance of the policy was greatly improved by the 2013 amendment to the law that deployed the mandate of incorporating dropout prevention measures to their school development plans.
The various developments have created a large number of good practices – mainly in those schools that had the overall institutional and professional absorption capacity to apply the know-how. For the time being, however, the systemic impact of these good practices is limited. The inclusion policy has created institutions with stable regulation backgrounds, such as the Intersectorial Committees or the inclusion teams in schools. These institutionalized elements proved to be sustainable changes. ne of the most important impacts of the inclusion strategy is its contribution to the gradual acceptance of the general goals and principles of inclusive education. Also, as a result of the inclusion policy, the overall proportion of children enrolled to special schools or to the special classes of regular schools has declined.The inclusion policy has not achieved a significant move towards the elimination of the participation gap between the Roma and non-Roma children. 25. The various projects serving implementation seem to have been very efficient, since some of them created sustainable elements of inclusive education with very limited funds and management capacities.

Recommendations:

1) Short-term recommendations :

  • Renew the work of the Intergovernmental Committee with the most important ministries; 
  • Establish national small grant scheme for schools for the implementation of the inclusive education related components of school development plans;
  • Create conditions in the Regional School Authorities for improved professional support services;
  • Develop an operational scheme of the support functions of special schools.

2) Long-term investments :

  • Initiate a new wave of capacity building programs for teachers on the application of the methods of differentiated instruction both by in-service training and the initial training of teachers;
  • Develop support manuals for each sub-domains and indicators of the Standards for Work Quality of Educational Institutions that are closely connected to inclusive education;
  • The development of underlying data classifications, online reporting system and information management platform for individual student level data collection;
  • The development of a comprehensive policy strategy for the systemic scaling up of isolated good practices on the basis of international experience applied to the specific Serbian context.

3) Recommendations for UNICEF Serbia :

  • Preserve the current balance among grass root developments, piloting for the development of know-how, knowledge management and policy advocacy with a stronger focus on educational outcome gaps between students with different backgrounds;
  • Initiate a series of events that provide the platform for intensive and open dialogue between educationalists working in any roles on the promotion of integration and inclusion and experts of special education on the required role of special education professionals and institutions.
  • Initiate a larger scale research project in cooperation with other organizations on the magnitude of segregation

Lessons Learned:

The evaluation of the Serbian inclusion policy provides for certain generalized lessons to be learned. For example, the Serbian case demonstrates the importance of the involvement of non-governmental stakeholders in ensuring the sustainability of policy initiatives. Also, the Serbian policy is a good example of policy initiatives that are tailored according to the specific context of the country. Another lesson is that if implementation is largely driven by the very different absorption capacities of schools, that may result in rather isolated islands of good practices with limited systemic impact. Other important lessons can be learned in relation to the effects of the lack of financial incentives, of too strong central government control and of the narrow capacity building approach to the professionalization of teachers



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2016

Country:
Serbia

Region:
CEE/CIS

Type:
Evaluation

Theme:
Education

Partners:
Ministry of Education, Science and Technological Development

Language:
English

Sequence #:
2016/004

New enhanced search