2015 Turkey: Evaluation of UNICEF’s response to the Syrian refugee crisis in Turkey
Author: James Darcy (Team Leader);Susan Durston; Francesca Ballarin; Jeff Duncalf; Burcak Basbug; Hasan Buker
An independent evaluation of UNICEF’s response to the Syrian refugee crisis in Turkey, 2012-2015
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report.
Now in its fifth year and showing no signs of abating, the conflict in Syria has to date caused some four million Syrians to flee their country and seek refuge in neighbouring countries. Of these, Turkey hosts the largest number of any of the host countries: currently over 1.7 million, including over 900,000 children. The generosity of the Turkish response is remarkable; but it is being put under strain as the numbers grow and the situation becomes ever more protracted. Although Turkey is a high middle-income country, the financial burden of hosting the refugees is high and growing; while the immediate burden of hosting refugees has fallen disproportionately on the poorer South-Eastern provinces of the country. Here tensions are rising locally over the growing pressure on overstretched services and competition for jobs. In some areas, the ratio of refugees to host communities is close to 50:50.
The extent of the Turkish Government’s ownership of and control over the response to the refugee crisis is one of the defining features of the context. UNICEF’s response and that of the wider international community have to be understood in the light of this. The wider geopolitical situation, including hostile relations between Turkey and the Syrian government, the unresolved political conflict with the Kurdish PKK, and Turkish and international concern about immigration and the potential for radicalised elements coming into Turkey from Syria and Iraq, are all part of the context within which the refugee situation and the response to it need to be understood.
The evaluation is intended to serve both an accountability function (historical / summative) and a learning function (forward-looking / formative). As described in the ToR, the purpose is to provide a comprehensive assessment of UNICEF’s overall response to the Syria refugee crisis in Turkey against its own mandate and standards, its stated objectives, and OECD/DAC evaluation criteria. This is intended to generate lessons and recommendations for UNICEF’s future humanitarian responses both in the sub-region and elsewhere. It is also intended that it should help inform the new Turkey Country Programme strategy for 2016-20.
The main focus of the evaluation is on the period mid-2012 to March 2015, though it includes some analysis of future scenarios and future strategic options for UNICEF. Reflecting the balance of the programme, the sectoral focus is on education and child protection, though some analysis is also made of other areas of intervention, including health and nutrition.
This evaluation was commissioned by the UNICEF CEE-CIS Regional Office together with the Turkey Country Office. It has been conducted in parallel with a wider evaluation of the overall UNICEF response to the Syria crisis and it shares the same basic logic as that evaluation – for which it effectively constitutes the Turkey component. However, the current evaluation is different in kind from the wider evaluation, providing a more in-depth consideration of a specific country context and UNICEF’s response to it.
After scoping visits to the TCO in September 2014 and January 2015, followed by a period of documentary review and preliminary interviews, the in-country component of the evaluation was undertaken from 15th March to 3rd April 2015. Key informant interviews were held with UNICEF staff, with Government officials and with other agencies and informants in each location. The team visited refugee camps and host community refugee facilities in Gaziantep town, Nizip, Kilis, Suruc, Sanliurfa, Mardin, Nusaybin, Islahiye, Osmaniya, Adana and Cevdetiye. Informal focus group discussions were held with refugees at the location of UNICEF-supported facilities, particularly Child Friendly Spaces. Discussions were also held with (mainly Syrian) teachers, who were themselves refugees and parents.
The site visits were chosen to give a reasonably representative sample of UNICEF-supported work in both camps and host communities. The geographical spread allowed visits to refugee communities of different ethnicities (including Arab and Kurdish) and different places of origin within Syria.
The evaluation considers the response under four main headings:
- UNICEF role and strategy
- Programme performance
- Working with others
- Internal management and process.
A modified version of the OECD DAC evaluation criteria is used to assess UNICEF’s response against relevance and appropriateness, timeliness, effectiveness, efficiency, coverage, coherence and sustainability (or ‘connectedness’).32 It was not possible to evaluate the wider impact of UNICEF’s interventions, although consideration of impact is included in the analysis of sustainability. In parallel with use of these criteria, the evaluation considers the question of the quality of UNICEF’s response, judged largely against best practice standards, internal and external. Beyond these generic criteria, the UNICEF Core Commitments for Children in Humanitarian Action (‘the CCCs’) are used as key benchmarks.
Findings and Conclusions:
Space for operational engagement by UNICEF was limited in the early stages of the crisis, and was initially confined to the camps.
In addition to collaboration with AFAD and the Ministries, UNICEF forged an important partnership with the Turkish Red Crescent Society, notably in the establishment of Child Friendly Spaces (CFS) in camps and in some urban locations. This partnership should continue and be expanded. UNICEF has limited partnerships with the few international NGOs present; but some of these (e.g. with IMC) hold potential for more extensive collaboration in partnership with Turkish NGOs.
With regard to the relevance and appropriateness of UNICEF’s response, the evaluation concludes that UNICEF established essentially the right role for itself in this context, with a strategic focus on education and child protection. Basic needs including WASH were well met by others in the Government camps. UNICEF played an important role in the polio vaccination campaigns conducted by the government in the border areas, in collaboration with WHO, following the polio outbreaks in Syria in 2013/14.
In the education sphere, UNICEF supported school construction and supplied educational equipment. Crucially, it also helped to shape policy concerning refugee education (e.g. on Syrian curriculum) and established a teacher compensation payment scheme in collaboration with MoNE and the Turkish Post Office. The establishment with MoNE of YOBIS – a data system for foreign students and teachers in Turkey – has been slow but is an important innovation that will help fill a vital data gap.
UNICEF now faces an immediate challenge concerning the sustainability of the teacher compensation scheme. This has allowed Syrian teachers to practice, but if they are to be retained it is crucial that they are able to secure a proper salary and job security – something that constitutes an advocacy priority for UNICEF.
In summary, the report makes the following recommendations:
- UNICEF should articulate, within a single overarching strategic framework, its strategies on education and child protection in relation to the on-going refugee crisis in Turkey. This should build on the No Lost Generation agenda and identify areas of synergy between the two sectors. It should establish short, medium and longer-term objectives and provide a clear basis for programming, monitoring and fundraising, while being flexible enough to accommodate changes in priorities over time and new opportunities as they arise.
- UNICEF should include in its overall strategy components relating to refugee health, nutrition and WASH, based on a re-assessment of needs and of UNICEF’s comparative advantage in these areas.
- An evolving advocacy strategy should be an integral part of both the overall strategy and its sector-specific components.
- The design of the strategy should be informed by a process of consultation with others, including in particular the Government of Turkey, other agencies working in the same field and refugee communities themselves. It should be designed in collaboration with UNHCR, underpinned by a renewed strategic partnership between the two agencies and a country-level LoU.
- Specific programme elements should be designed, in collaboration between Education and Child Protection, to address the growing social cohesion agenda in refugee hosting communities.
- UNICEF should seek to increase the priority given to the refugee child protection agenda and should itself take a more leading role on that agenda.
- UNICEF should continue to give more emphasis to collaboration at the provincial and local levels, and to the scaling up of both programmatic and operational support staffing in the Gaziantep/South Eastern Turkey intervention area.
- UNICEF should endeavour to strengthen its role as the link between I/NGOs and the Government of Turkey.
- The report sets out what the evaluators believe to be areas of best practice in the UNICEF response to the refugee crisis in Turkey. These should be properly documented for sharing with others.
- UNICEF should adopt to a more rigorous and systematic approach to programme monitoring, with direct feedback to programme implementation. This should include monitoring of quality and continued relevance of programme outputs, as well as target achievement.
- Building on the “No Lost Generation” initiative, a global awareness campaign highlighting the plight of Syrian refugee children in Turkish host communities should be undertaken, as part of a wider campaign on the situation of child refugees in the region.
- UNICEF should use the experience of the Turkey refugee crisis response to review the guidance provided on applying the CCCs and SSOPs, particularly with regard to contexts where the humanitarian role played by UNICEF is non-operational or only semi-operational.
- Based on a review of the experience from Turkey and elsewhere in the Syria sub-region, UNICEF should review the adequacy of its advance financing mechanisms, and specifically the Emergency Programme Fund, to meet response demands in gradually emergent crises of this kind.
LL in Education- (i) The Code of Conduct for teachers that is signed by all Syrian teachers at the end of their training, sets out standards for behaviour. This has given the authorities an instrument with which to address corporal punishment, and other behaviours, while training is strengthened in these areas.
(ii) Incentive payments under the Compensation Scheme for Syrian teachers have been introduced with an innovative modality, through the Post Office (PTT) that requires only one account that all teachers can access, and which is also said to be of potential use for payment of personnel in other sectors.169 While the level of remuneration was criticised by some Syrian teachers during field visits, it was in general appreciated. The eventual success of this will be proved by its sustainability; the programme has perhaps one more year of funding to support the payments. However, there is a possibility opening up through dialogue on the granting of a specified number of work permits to Syrian teachers, leading to salaried remuneration on the Turkish system scales.
LL in CP- Both the Code of Conduct and the Compensation scheme are part of the Management Strategy for Education Personnel and have use beyond the programmes implemented with UNICEF support.
Child Protection: The Adolescent Empowerment and Participation methodology, which has the potential to become a standard operational methodology for CFSs in the Region. Based on what the evaluators saw, it is recommended to extend the age group of youth leaders up to twenty-four as suggested in the way forward of the One Year review of the No Lost Generation Initiative.
(ii) The online monitoring platform has also the potential for being replicated in similar context. (iii) Particularly since it is a new model of response for emergency actors in Turkey, the CFS experience in the camps needs to be evaluated in its own right. (iv) The Family Parenting Training Programme is a further example of best practice.
Other Evaluation Resources on the Syrian Crisis.
Full report in PDF
PDF files require Acrobat Reader.