Evaluation database

Evaluation report

2013 Romania: Helping the ‘invisible’ children. Second Evaluation Report



Author: Manuela Sofia Stănculescu (coord.), Simona Anton, Cătălina Iamandi-Cioinaru, Georgiana Neculau, Bogdan Corad, Andreea Trocea

Executive summary

"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report."

Background:

The project Helping the ‘invisible’ Children (HIC), prepared in April-June 2011 and implemented during June 2011-September 2014, is part of the UNICEF's Community Based Services (CBS) Programme in Romania, which focuses on the development of the preventive approach in the social protection system, particularly in rural areas.
The HIC project has targeted the 'invisible' children defined as the children who face one or more types of vulnerabilities  and have been reached by social workers through fieldwork activities. Within HIC, the ‘invisible’ children refer also to children acknowledged as being in a vulnerable situation, but with ‘invisible’ vulnerabilities (also identified through fieldwork activities).
The purpose of the project Helping the ‘invisible’ children is to increase the impact of social protection policies for poor and socially excluded (‘invisible’) children and families. Taking into account the underdevelopment of the social assistance services at community level, the HIC programme theory has considered that children’s welfare in Romania will improve only if and when the children, especially the worst-off (‘invisible’ ) children, will have enhanced access to social services (education, health, and social assistance services). In 2012, the project coverage was limited to 64 communes. A basic package of community preventive social assistance services was piloted: the social workers have carried out outreach activities including needs-assessment, monitoring, informing and counseling, and have provided appropriate social assistance services to the worst-off children and families. This model is supposed to be implementable at the national level (with a total of 2,858 communes in the country) and to impact progressively the main gaps of the social protection system at four layers: i) legislative provisions, ii) institutional building, iii) resources allocation, and iv) social control mechanisms, including monitoring and evaluation.

Purpose/Objective:

The purpose of this evaluation is to determine the relevance, effectiveness, efficiency and sustainability of the HIC project. This is the Second Evaluation Report of the HIC Project in Romania. The report presents the results of an extended field research carried out in December 2012- January 2013.

Methodology:

The evaluation has covered all key stakeholders, all counties (8), all communities (95)  and all activities carried out in the project in 2012 (in 64 rural communities that remained in the project).
The evaluation has included a counterfactual comprising the ‘invisible’ children identified in the 32 communities which were not anymore part of the project in 2012. These children were supposed to benefit of the services existing in these communes (e.g. SPAS regular services, Roma mediator, community health nurses, etc.) but not of the services provided within HIC project. Also, in these 32 communities, the outreaching activities of identification of the most vulnerable children and their families were ceased in 2012. In contrast, in the 64 communities remained in the project, the identification of the most vulnerable has been kept as a component of on the ground activities and the ‘invisible’ children have been supposed to benefit of the services provided under the minimum package approach. Thus, the evaluation compares:
• ‘invisible’  children in both types of communes and how their situation evolved in 2012;
• the services in both types of communes, including support provided from county level to communes in the project versus virtually no support in communes not included in the project;
• the results for children in both types of communes.
The evaluation used a mixture of primary and secondary sources of information so that to ensure high quality and credibility. However, most data were collected with the special purpose of this evaluation. Primary data were collected both at the micro (individual, household) and meso (community) levels, through quantitative and qualitative research techniques. The field research was carried out by CERME, in December 2012 - January 2013.

Findings and Conclusions:

The project has produced proofs that the issue of 'invisible' children is highly relevant for the rural communities and it represents a serious problem that needs an urgent policy response. At the same time, the project has demonstrated that: (i) the development of preventive community services is possible in spite of the limited human resources at local level and of the insufficient local budgets; (ii) outreach activities are possible and essential for ensuring the right to social security for children (and other vulnerable groups). Moreover, it presents clear evidences that the preventive community services are more effective and much cheaper in real life and not only in theory.
In one year and a half (2011-2012) the project identified 5,758 'invisible' children who face a complex cumulus of vulnerabilities. Based on the minimum package of community services approach, over 3,400 children and their families, from 64 rural communities, received a variety of services from diagnostic to information, counselling, accompaniment and support, referral, as well as monitoring and evaluation. Thus, access to health, education, social protection, and opportunity to develop into the natural family has been enhanced for many children at risk.
In addition, people's attitudes toward children and their rights have changed for the better, the preventive community services have improved, the no. of cases of violence and abuse within communities has declined. The community activation and participation has improved compared to 2011. Results indicate that project objectives may be achieved in the next 2-3 years. Hence, the project is on the right track.
Yet many areas need adjustments. The project is very relevant and highly efficient. Problematic aspects relate to effectiveness and sustainability. For improving effectiveness and sustainability, the evaluation points out two priority areas for action: diagnostic of vulnerabilities; and, monitoring and evaluation activities.

Recommendations:

The general recommendation is to continue the intervention and to enhance the advocacy efforts so that the model of community preventive service piloted within HIC to be undertaken and scaled up at national and county levels.
Availability of human resources at the community and county levels is critical both for project sustainability and effectiveness. The project has proven that the implementation of minimum package is possible even based on a network of social workers with less experience in social assistance. Nonetheless, in order to improve performances human resources need better training, increased methodological support from the county specialists, and, better tools (database, guides, methodologies and procedures).
For improving project effectiveness, a consolidated database of 'invisible' children is needed. A procedure for the registration/deletion in the database should be developed to ensure that every case is regularly monitored at least for 1-2 years. The database should collect systematic information about all household members. In this way the intervention plan would consider also the relevant others who decide and are responsible for the child.
A methodology for a complete diagnostic of vulnerabilities based on a grid of indicators needs to be applied unitary for all children and across all communities. Given the complexity of vulnerabilities faced by the 'invisible' children the approach of the project need to be integrated and not to treat each vulnerability separately. A methodology for linking the diagnostic of vulnerabilities with the individual plan of intervention is definitely necessary.
To help and support the supervisors and social workers, the new methodologies, procedures, formats and indicators should be integrated in an intelligent online application. Vertical and horizontal collaboration among key institutions, professionals and stakeholders is vital and should be better regulated. 

Lessons Learned:

1/The accuracy of the social diagnostic is of utter importance as ~40% of 'invisible' children  face multiple vulnerabilities. 'Invisible' children live in atypical households and cumulate vulnerabilities. Still, only some vulnerabilities are generally assessed and recorded, while others are overlooked.
2/Segmentation is a risk: organizing the intervention by types of vulnerabilities and services may lead to some children without ID papers for instance being deleted from the database after getting the ID but in the absence of a complete assessment of their situation. Given the complexity of vulnerabilities faced by the 'invisible' children, integrating vulnerabilities is essential.
3/Enhancing quality in recording the data is important. Discrepancies between the provided services declared (and acknowledged) by families and those reported by social workers (SW) are frequent.
3/Case identification and management has to be strengthened: in households with many children, some SW record 1 or 2 children, while the others remain 'invisible'.
4/Monitoring is occasionally difficult and time consuming as some children live with relatives and/or change their residence often alternating separation from the family with homelessness. 



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2013

Country:
Romania

Region:
CEE/CIS

Theme:
Child protection sectoral or partnership

Type:
Evaluation

Language:
English

New enhanced search