Evaluation database

Evaluation report

2011 Afghanistan: Evaluation of the Social Work Coaching



Author: Brigitte Krogh-Poulsen

Executive summary

 

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”

Background:

With support from the UNICEF ACO, four international NGOs: Children in Crisis (CIC), Child Fund Afghanistan (CFA), Save the Children (SC) and War Child UK (WCUK) began implementation of the Social Work Coaching Projects in 2007/2008. By 2010 two phases had been implemented in 12 provinces of Afghanistan. Each phase lasted 12 months and was guided by an overall concept note. A concept note for a third phase has been developed and one partner (CIC) is currently implementing a phase III project.

The projects aim to strengthen child protection through capacity development for social work. There is not yet a formal social work training programme in Afghanistan, though efforts are currently underway towards establishing such a programme. When the projects started in 2007 social work was neither widely understood nor recognised as a key component in child protection. The term “social worker” in Afghanistan is therefore not used to define a person with formal social work qualifications; rather it is used to describe anyone who works on “social issues”, including support to children. In other contexts these “social workers” would be referred to as community workers, care workers etc.

Initially, the projects focussed on children in orphanages. This focus was chosen because institutional care is widely used and perceived as beneficial in Afghanistan, in spite of the substantial international body of knowledge, documenting the detrimental effects of institutional care. Subsequently, the projects widened their focus to also include children in conflict with the law and, to a lesser degree, early marriage and child labour.

The projects all had two major components:
• Capacity development for social work, which included training and coaching of “social workers” and improvement of case management at institutional level. The projects developed training and case management tools and worked with CPANs, Orphanages and JRCs as well as community structures with child protection responsibilities to strengthen capacity.
• Direct support to children and families for repatriation and reintegration of children held in orphanages or JRCs. This included social workers providing psycho-social support and facilitating access to justice for children in conflict with the law.

The projects built upon the experiences and outcome of earlier support to MOLSAMD to strengthen basic social work capacity. The objectives of the projects are defined as follows:
1) To develop a regional social work coaching system in at least four regions
2) To develop a system of peer-support among social workers at provincial level
3) To develop capacity of para-social workers working in government and NGO child protection organizations and agencies

Purpose/Objective:

To evaluate the interventions under social work coaching projects supported by UNICEF to assess their relevance, impact, effectiveness, efficiency, sustainability and appropriateness in the context including:
a) Extent of skill enhancement among social and care workers who were part of this project;
b) Number of children and families who benefitted from the services provided by the social workers and quality of services provided by social workers;
c) Quality of services provided by the social work agencies to Government in JRCs, Orphanages and CPANs;
d) And Recommend a cost effective strategy to sustain the skills of the workers trained under these projects and promote protection of children within communities and institutions.

To assess the activities implemented under the social work coaching projects by the four agencies since 2007 and having reached 12 provinces by the end of 2010.These activities although aimed towards ensuring family reintegration of children in institutional care or children in conflict with law, essentially focuses on building the skills of the social /care workers in knowledge and processes to facilitate towards this end.

Methodology:

The evaluation mainly used participatory and qualitative methods, though simple quantitative methods were also employed. A number of detailed evaluation questions were developed for each of the standard evaluation criteria (relevance, appropriateness, impact, effectiveness, efficiency and sustainability) used in the evaluation.

Key questions to be answered by the evaluation as follows:
• Were stated outcomes or outputs achieved?
• What factors have contributed to achieving or not achieving intended outcomes and outputs?
• What strategies have been used, what are working, and what are not working?
• What factors contributed to not working, what would be alternative?
• Has the project been appropriate and effective
• What factors contributed to effectiveness or ineffectiveness?
• Has the project been efficient
• What factors contributed to efficiency or inefficiency
• Can the project be sustainable, and replicate to other places in the country?
• Were the project cost effective to be measured against impact on skill development as well as benefits to children, i.e. were the project’s cost efficient and cost effective?

Findings and Conclusions:

Findings:
The evaluation found that the projects were relevant and appropriate. The all address real needs both in terms of child protection and capacity development. The training and coaching methodologies used are appropriate to the context and to the capacity development needs identified. The focus on children in institutions and children in conflict with the law is in line with national priorities as they are spelled out in the National Strategy for the Protection of Children at Risk and the National Social Protection Strategy.

The projects have been very effective in strengthening capacity for quality social work in Afghanistan. At the beginning of the projects, social work was a largely unrecognised field and the projects have contributed to placing social work and child protection on the national agenda and preparing the ground for more systematic, large scale interventions. This is due primarily to the quality and appropriateness of the training and coaching.

The tools developed by the projects, both the training and the case management tools, were found to be good quality and were generally appreciated by stakeholders interviewed. Due to minor inefficiencies in the development process for the tools, there is need to streamline and replicate tools across the four projects, however. In particular, consolidated handbooks on social work could increase the value of the projects’ tools. Moreover, only the SIR appears to have been fully institutionalised. The institutionalisation of the SIRs was realised through a successful combination of micro-level support to developing the tool, and training social workers and other relevant stakeholders to use It, and macro-level policy dialogue that resulted in a national level LOA, which stipulates the role and use of the SIR in juvenile justice.

In terms of direct services to children and families the picture is less clear. Interventions to support reintegration have been less effective and the impact is not yet clearly visible, partly due to the short time during which children and families were supported. However, the evaluation also found, that in order to effectively create sustainable impact in children’s lives, the projects most likely need to refocus their strategies.

Firstly, it may be necessary to change the protection approach from one of reaction, once a child is a victim of abuse and exploitation, to one that focuses more on prevention of exploitation and abuse to improve effectiveness. Moreover, the evaluation found that direct support to children is not sufficiently holistic to create sustainable impact. In particular, more attention to access to education and training for children and more attention to economic empowerment for families is required. In addition, the current interventions are strongly biased in favour of boys. This is probably a reflection of the focus on institutions that house many more boys than girls. Hence, broadening the scope to include other protection issues and more preventive approaches is needed to improve the gender balance in the projects.

The projects have increasingly mobilised communities for first line protection and social work. This strategy was found to be very effective in reaching out to more children, in creating ownership and acceptance among families and key community stakeholders, in sustaining activities (including during implementation breaks between phases and in situations of deteriorating security) and in changing attitudes and behaviour locally.

Overall, the projects have registered substantial impact on attitudes and behaviour among professional groups and community members alike. Staff in orphanages and officials in the juvenile justice system reported changing attitudes and behaviour more in line with the provisions of the UNCRC, after receiving coaching and training. The coaching seems to have played a key role in facilitating the translation of improved knowledge and understanding into behaviour change.

The strengthened capacity also extends to the CPANs in the project areas. Perhaps the most important contribution from the projects to strengthening the CPANs has been to actually place social workers on the ground, who can follow-up cases overseen by the CPANs.

Real and large scale change is challenged, however, by limited government capacity and commitment to changing practises and to allocating sufficient resources for social work towards child protection. Positive steps have been taken, for example towards establishing a formal social work education, but at the same time detrimental practises, such as admitting children seeking education into orphanages, continue.

The limited government capacity means that the current child protection system in the project areas relies very heavily on the social workers employed or otherwise supported by the implementing partners. CPANs appear to rely heavily on social work services provided by the projects for case management and support to children and families. Therefore, actual case follow-up appears to decrease (both in terms of frequency and quality) when the projects are not implementing full scale. This is essentially a case of NGOs gap filling and making up for limited government capacity. While this is necessary in the short to medium term to ensure that children can be protected it is not sustainable in the long run.

Conclusions:
Based on the findings summarised above, the evaluation’s main conclusions can be summarised as follows:
• The projects have successfully and effectively strengthened capacity for social work and child protection in the project areas. The methodology used for capacity development, i.e. the combination of training and coaching, has been very effective. Moreover, the tools developed under the project were useful and appropriate overall.
• The projects have registered fewer results in direct support to children and families, primarily because more time, and a more holistic approach with a stronger prevention focus, is required to create sustainable impact.
• The main challenge for the projects at this point is sustainability, both in terms of institutionalising social work in the government system and in terms of sustaining change within communities, where more time is needed to consolidate the results.

Recommendations:

Based on the findings and conclusions, the evaluation recommends that the projects continue operations for a further three to five years, but with a substantial change in strategies and approaches to adapt to the changed circumstances, resulting from the project results in capacity development, and address the impact and sustainability issues identified in the evaluation.

Towards this change, the evaluation provides recommendations for concrete short-term actions as well as longer term changes to the strategies and approaches in the projects.

Key short term recommendations:
• The first and foremost recommendation is to carry out thorough strategic planning for any future phases of the projects. The strategic planning should involve all relevant stakeholders and result in a comprehensive and coherent results framework for a more holistic and programmatic approach. The planning must also include detailed planning around time frames to avoid breaks in implementation in the future.
• Clear exit strategies must be included from the onset and partners must, in addition, develop resource mobilisation plans to ensure that interventions do not rely solely on UNICEF funding.
• Baseline research should be carried out as a matter of priority to inform the strategic planning and to feed into longer term evidence based advocacy and policy dialogue.
• Partnerships, both amongst the implementing partners and with other stakeholders, should be strengthened and nurtured as a matter of priority to prepare the ground for implementation of more holistic interventions in the future.
• Tools, both training and case management tools, could be streamlined and distributed more widely. The replication of a handbook for social workers and referral directory, developed under one project, is recommended in particular.

The above recommendations could be implemented through a short bridging project. A bridging project could also include provisions for keeping social workers on the ground while a new, more programmatic intervention is being prepared.

Key long term recommendations:
• The projects must capitalise on their results in capacity development and move towards facilitating sustainable support to children and families.
• The most important step towards sustainable impact on children’s lives is to systematically identify and provide holistic services that tackle the root causes of exploitation and abuse. This includes promoting access to education, economic empowerment for families, and continuing to facilitate changes in attitudes and behaviour in communities and families towards creating a protective environment.
• It also entails changing the focus from reactive services only to more interventions that will prevent exploitation and abuse. It is important to ensure that social workers and other key change agents have capacity for prevention, while maintaining preparedness to support children who become victims of abuse, exploitation and neglect.
• A more holistic and preventative approach would also entail widening the projects’ focus to include other priority protection issues (for example early marriage). This is already happening, but must be strengthened and done in a systematic manner, not least to ensure that the services provided through the projects reach girls and children in remote areas.
• Continued and increased community mobilisation is essential in this regard and will be a key contribution to sustainability in the long run, especially in areas with high insecurity.
• Increased focus on evidence based advocacy and policy dialogue is equally important in order to promote government ownership. This must be combined with support for government capacity development.
• None of these strategies can be effectively and successfully implemented in isolation or by one partner alone. Therefore, the projects need to focus on partnerships for service delivery and advocacy. This will entail building and nurturing partnerships with external partners as well as working for closer integration within the projects themselves among the implementing partners.
• Though the recommendations focus on scaling up and improving the quality of direct protection support and prevention, the evaluation stresses that social work is not a static field and continued capacity development is essential. On the job coaching has proven effective and could be continued, but other capacity development initiatives may also be included to improve efficiency and effectiveness. Mainstreaming of social work components into pre- and in-service training for other professional groups, such as teachers and police officers, could be one such intervention.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search