We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 Romania: “School attendance initiative” - School participation campaign implemented by UNICEF Romania. Summative Evaluation-Final Report



Author: Evaluation team (in alphabetical order): Veronica Gabriela Chirea, Mariana Dogaru, Olivia Maria Jidveian, Roxana Mihail - Team Leader, Marian Popa

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

The Romanian population has decreased significantly over the last two decades and is rapidly aging. Since 2002, the population has dropped by 1.6 million people (7.2%), mostly because of low birth rates and emigration.  If these trends continue, the number of pupils will decrease by 40% by 2025, showing an urgent need for educational reforms that focus on quality, efficiency, equity, and relevance. In Romania, approximately 400,000 children don't go to school every day, approximately 12,000 primary education pupils and over 28,000 pupils from the lower secondary education leave the system each year, the PISA  international study shows that 40% of Romanian 15-year-olds are semi-literate. The early school leaving rate was 18.1% in 2014, the same as in 2011 and 7% higher than the 2014 European mean and 6.8% higher than the national target for 2020.
The School Attendance Initiative was carried out by UNICEF, in partnership with the Ministry of Education and Scientific Research, with the overall goal of testing a complex intervention programme for reducing absenteeism, school dropout and early school leaving, in disadvantaged communities, selected from all over the country. The Initiative was implemented in successive stages, between 2010 and 2015, and included a variable number of schools.  In the final stage, the Initiative targeted 32 schools in 19 counties. The intervention had the following components:
(1) Improving school management;
(2) Improving teachers’ educational and methodological skills;
(3) Strengthening the parent-school relationship;
(4) Parent education;
(5) Offering positive and successful role models, especially for Roma children;
(6) Developing a network of community actors to prevent school dropout;
(7) Training school mediators and Romani language teachers.

Purpose/Objective:

The general purpose of the evaluation was to assess the specific intervention programmes of the implementing partners and the impact of the Initiative on absenteeism and school dropout, from the perspective of target schools and groups (pupils, teaching staff, parents).
Evaluation had the following general objectives:
• Identify the impact of the School Attendance Initiative, carried out between 2011 and 2015, regarding school dropout and absenteeism, on the 32 schools included in the evaluation.
• Identify the efficiency and effectiveness of the Initiative from the perspective of all intervention components.
• Identify transferable examples of good practice and provide lessons learned for the implementation of the new integrated model of social services in Bacău County.
• Identify certain limitations and constraints for the implementation of SAI components in schools, families, communities.
• Generate relevant information for decision-makers, supporting local, regional and national policy development to prevent and diminish school dropout and absenteeism.
The evaluation focused on finding answers to the following questions:
• How effective has SAI been in reducing the risk of dropping out and dropout rates in the schools involved?
• Have SAI interventions produced management changes in the schools involved, regarding the implementation of strategies for school dropout prevention/control and the development of school-community partnerships?
• Has SAI produced changes in the instructional strategies used by teachers in their day-to-day activities?
• Has SAI produced changes in the teacher-pupil relationship and the teacher-parent relationship?
• Has SAI produced changes in parents’ attitude towards education?
• Has SAI produced changes regarding parents’ involvement in school life?
• Has SAI produced changes in the community to contribute to the reduction of school absenteeism and dropout?
• Are SAI interventions sustainable in the schools involved?

Methodology:

• Evaluation paradigms included both a quantitative approach (school information, standardised questionnaire results) and a qualitative approach (field observations, interviews, focus groups, case studies).
• The evaluation model was based on longitudinal analyses, conducted during implementation years, and comparative ones: between categories of subjects (pupils-parents-principals-teachers), between SAI schools and those in the control group, which was set up after SAI completion.  Whenever possible, results were verified using the triangulation method. 
• Information sources were highly diverse, including: people (pupils, teaching staff, parents, school mediators, community members, representatives of intervention partners; schools (summarised and individual school data, management documents, administrative information, etc.); a variety of documents (partner reports, public statistical sources, etc.).
• Evaluation techniques consisted of: questionnaires, interview guides, focus group guides, observation sheets, analysis of school records.

Findings and conclusions:

• Absence and dropout rates have not decreased, nor have they increased during SAI implementation, as was the general trend in the Romanian education system.  In half of the schools participating in the Initiative, a decrease in absence and dropout rates has been reported.  This trend is present mainly in the schools with a greater proportion of Roma children.  The fact that the SAI impact on absences and dropout has not reached the estimated level shows that there are root (social, economic) causes to this phenomenon that SAI interventions could not mitigate.      
• SAI interventions have produced positive management changes in the schools involved, regarding the implementation of strategies for school dropout prevention/control and for the development of school-community partnerships.
• SAI has produced positive changes regarding the educational strategies used by teachers in their day-to-day activities.
• SAI has produced positive changes in the teacher-pupil relationship and the teacher-parent relationship.
• SAI has improved parents’ attitude towards education and their involvement in school life.
• The positive changes observed constitute a convincing premise as to the sustainability of SAI impact.  
• The positive results and the examples of good practice observed indicate the fact that the success of the educational intervention is determined by the following factors: the timely identification of absenteeism situations which may lead to dropout; the systematic and continuous monitoring of the “individual cases”; a prompt and appropriate intervention through social and/or medical/health care services; individualization by necessity and specificity, doubled by the development of a personalized “future trajectory” for each case of “recovered pupil”, using counselling services and family involvement.  

Recommendations:

• Carrying out such an intervention programme requires a complete four-year intervention cycle.  For each community, the intervention cycle needs to be superimposed on an educational stage (for example, primary/lower secondary/upper secondary education).
• Continuing micro-grants and supplementing them with study microcredits for pupils selected based on transparent procedures. 
• In schools with a high number of pupils, it is recommended, on the one hand, to increase absence monitoring and control measures and, on the other hand, to allocate additional human and material resources in order to constantly support the said measures. 
• A better coordination of interventions, actions, projects, and other endeavors that involve the school ethos is necessary.
• Updating training modules by adding new examples and lessons learned, coupled with facilitating open access to the www.ise.training.ro platform to all interested teachers, as well as to principals and school mediators, by type of interest and training.
• Applying and constantly checking school attendance monitoring mechanisms, coupled with concrete measures addressing the cases concerned.  Measures must be flexible and address specific cases, adapting to each situation since the “one size fits all” principle has been proven not to work in this context.
• A complex diagnosis of the needs/necessities of the school, in the context of the community it represents.  Based on this diagnosis, key focus points must be set for the intervention, depending on the vulnerabilities and specificities of each community.  The implementation programme (including components and resources employed) thus has to be adequately adapted to the needs of the learning community. 
• Dynamic adaptation of the intervention plan to the actual evolution of the situation in each school community, from one year to the next.

Lessons Learned:

• School coaching is very necessary. The next step should be to systematize continuous teacher training courses for activities with pupils at risk of school dropout.  Medium-term professional training plans are necessary, with the specific purpose of reducing absenteeism and school dropout.
• Another dimension is that of initial teacher training. The training experiences catalyzed by the activities organized for teachers in the School Attendance Initiative highlight the need for the teaching master’s programme to reflect the educational specificities of working with pupils at risk of school dropout. The need has been identified, expertise is available, educational resources have been created and successfully tested, now it is necessary to recognize it within the public policy aimed at solving the identified problem. 
• An important lesson learned is that of monitoring as a means of control and reduction of absenteeism: optimized school attendance and participation outcomes are correlated with the individual monitoring of “the cases”.  Where the principal, the school mediator, the Romani language teacher and the school teachers establish close, direct relationships with the family and the pupil at risk of dropping out, the probability of containing and decreasing the phenomenon grows significantly. 
• This finding is consistent with successful educational programmes, with strong social and societal dimensions, which work under similar conditions to those of the School Attendance Initiative.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year: 2018

Office/Country: Romania

Region: ECAR

Type: Evaluation

Theme: Education, Multi-thematic Education

Language: English

Sequence #: 2018/003
New enhanced search