Evaluation database

Evaluation report

2012 Pakistan: Evaluation of Social Reintegration of Street Children Project



Author: H&H Consulting, Pakistan

Executive summary

"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is "Outstanding, Best Practice", "Highly Satisfactory", "Mostly Satisfactory" or "Unsatisfactory" before using it. You will find the link to the quality rating below, labeled as 'Part 2' of the report."

Background/Purpose:

UNICEF initiated a pilot project, titled, ‘Social Reintegration of Street Children’, through two implementing partners, Azad Foundation (2005) in Karachi and Mehran Welfare Trust ( 2007) in Larkana. UNICEF commissioned an external evaluation of the project to draw best practices and lessons that can guide the Government of Sindh in implementation of the project for children living and working on the streets as per PC-1. The pilot project was evaluated for relevance, effectiveness, efficiency, sustainability and impact. Human Rights based approach, Gender and Child participation issues were explored and analyzed as cross cutting. The evaluation covers the time frame of 2005 to June 2011. A participatory qualitative methodology was used which included 15 in-depth interviews; 5 focus group discussions; 2 meetings with key stakeholders and beneficiaries. Observations were made throughout the field visits and desk review of the important project documents was also undertaken. Ethical principles of informed consent, privacy and confidentiality were followed throughout the evaluation process. The evaluation was limited in its scope as much time had lapsed between the project end and the time when the evaluation was undertaken. It was also limited due to insufficient data and documents for desk review and secondary data analysis.

Findings and Conclusions:

Relevance: The activities undertaken to address the rights and needs of children living and working on the streets was relevant as these were based on the findings of mapping studies1 undertaken in Karachi and Larkana. The work with children living and working on the streets was also in line with UNICEF’s Country Programme2 priorities set for Pakistan for the time period of 2004-2008 and 2008-2010. The inclusion of the provincial social welfare department (SWD) in all stages of project design and implementation was a relevant strategy as it was envisaged that the findings from pilot project would help bring the issue on the mandate of provincial government.

Effectiveness and Efficiency: While most of the services of the drop in centres (DICs) were found useful by children, an important contributing factor towards their effectiveness was the positive and friendly attitude of the service providers at the DICs. One issue that may have affected the efficiency of the DICs is with regards to its location, which did not always suit all the children reached out in the identified ‘hotspots’. Another area for improvement is with regards to information about HIV/AIDs. While children were regularly given information about STDs including HIV/AIDs also verified by children met during the evaluation, it appears that they may not have necessarily been told about condom use (safe sex) which would have been useful considering the data showing a high ratio of sexual activity. The discussion with project staff as well as field visits suggests that the work with girls was less effective than that with boys. This was due to the reason that accessing girls is difficult because the community disapproves attempts of engaging with them. As a result, there is denial either of them working on the streets or the issues and dangers that they could potentially face. The denial of the problems faced by girls working on the streets reflected in the guarded discussion by the girls benefitting from the project services. This five step rehabilitation process was deemed essential based on an understanding that the willingness of children to go back after undergoing a process of counseling and rehabilitation will ensure more effective reintegration as opposed to forcing or/and narrowly focusing on the reunification alone. Limited outreach and/or scope of the existing government run financial support/micro credit loan schemes was especially seen as a hindrance in the efforts to reunify street living children with their families. The detoxification facilities did not necessarily have ‘right based standards’ and thus treatment often involved keeping children in a locked facility. As a result, many children dropped out of the rehabilitation process on return from the facilities, making project objective of reunification and rehabilitation more complex. Information about the project beneficiaries has been gathered throughout the project period by both Mehran Welfare Trust and Azad Foundation. However, gaps exist in presentation of the data which complicates the interpretation, analysis and utilization of the available output level data. While project output level data is available, there is little information or/and quantifiable data that can help track the outcome of the services. Linkages with shopkeepers, street vendors and ‘group leaders’ proved to be an effective outreach strategy as it helped the project team deepen its understanding about the dynamics and issues experienced by the children. They also helped identify children new on the streets who would then be motivated to return home. Within the outreach strategy, the work with parents and communities from where children come on the streets appears less effective as compared to other activities. Activities included one-off theatre performances, awareness sessions etc. The effectiveness of one-off sparsely spaced awareness activities, with limited follow-ups may not be able to bring about the change in mindset required to address the root cause of children living and working on the streets.

Sustainability: Despite the efforts towards capacity building, presence of PC-1, the recently approved Child Protection Authority Act, contempt of court notice3 to SWD by the Sindh High Court, there appears skepticism among stakeholders about sustainability of the efforts. The delays in implementation of PC-1 (which was to start in 2010) and approval process of SOPs (developed in 2008); frequent shifting of SWD staff and police whose capacity is built to address the issue; lack of priority and mechanisms to address the issue within the relevant government bodies (Batul Maal, Zakaat); coordination issues between the provincial SWD and city district government; lack of trust among children on the streets towards government bodies (who are often viewed by them as aggressors) are some of the reasons pointed out for this skepticism. Most of the people were of the view that government alone may not be able to address the multilayered dynamics of children living and working on the streets and would thus require support from civil society organizations and NGOs with experience of working with children living and working on the streets.

Impact: The project was able to make significant impact at two levels, firstly on the lives of the children who benefitted from the services and secondly in the public policy arena. Some of the changes observed in children include increased ability to recognize and manage feelings of anger, increased understanding of personal hygiene, improved ability to communicate with respect, decrease in self-harm behaviour, increased responsibility towards their life and increased ability to dream and aspire for a better life.

Human Rights Based Approach: The human rights based approach was an intrinsic part of the project design reflected in the work with children. The level of children’s involvement in actual design of the project, DIC and outreach activities could not be ascertained. However, it was noted that children were actively engaged in decorating and upkeep of the drop in centre and rehabilitation centre, deciding themselves to come up with consensus about programs to watch on the television, games to play etc.

Recommendations:

The findings and learnings of the project provide useful recommendations for the relevant public and private sector organizations, services and institutes.

Collective efforts through a collective vision: It is recommended that a long term and holistic approach to address the issue of ‘children on the streets’ be developed by defining a clear vision for this work at the provincial level. The vision will then guide relevant provincial departments such as the planning division, social welfare, police, health, education, city district government, etc. to clarify their role, expectations from organizations working on the issue and the kind of funds needed to accomplish the vision.

Tackling the root causes: Once children are on the streets the type of effort, coordination and funds required for their reintegration and reunification are much more than efforts that one can make to ensure that they do not leave their homes. Thus, tackling the root causes by developing long term holistic programs & interventions for identified communities would be essential.

Multipronged approach towards working with children: It is recommended that a variety of services and interventions for children living and working on the streets be made part of the design. These should range from outreach services (motivational sessions on the streets, mobile services, linkages with street vendors, group leaders); services at the drop in centre (washing/bathing, educational, recreational, skills based) to more long term rehabilitative services (counseling, job placements, reunification efforts with families etc.). It is recognized that providing all these services may be a challenge for one institute alone and thus public-private partnership and linkages must be explored.

Unpacking Concepts for more effective interventions: While there is overlap in dynamics and challenges faced by children ‘living’ and children ‘working’ on the streets, it is recommended that the distinction between the two be identified while designing interventions, so that these distinct needs could be addressed. It is also recommended that the distinction between the term ‘Reintegration’, ‘Rehabilitation’ and ‘Reunification’ is made after consultation with relevant organizations so that some minimum indicators and activities that can help services gauge the effectiveness of these efforts are ascertained.

Review and endorsement of the existing SOPs: A minimum Standards Of Procedures (SOPs) for the Prevention, Protection, Participation & Rehabilitation of Children were developed in 2008. These SOPs provide a detailed account of the types of services to be provided by the DICs and rehabilitation centers, outreach work, referrals, important definitions etc. It is recommended that the SOPs be reviewed to incorporate lessons learnt from this evaluation as well as by other relevant organizations between the time period of 2008 and 2012.

Human Rights Based Approach: The project was based on a human rights approach and helped inculcate in children, feelings of respect and trust. The approach helped children avail the services and internalize the information given to them at the DICs. It is thus recommended that the human rights based approach must form the basis of programming. Participation of children and gender issues must also be addressed as part of the project design, implementation, monitoring and evaluation.

Counseling and Life Skills as core intervention: Counseling provided by trained psychologists appeared as project strength based on the insight developed among children who had utilized the project services. It is thus recommended that counseling be made part of the core activities with children in the DICs and the rehabilitation centre. It is also recommended that life skills based sessions should be conducted with children, tailored to the needs of the children and issues picked up while working with them.

Programming for the girl child: There were limited interventions with girls in this pilot project and thus the interventions provide a replicable model primarily for boys. Owing to the vulnerabilities experienced by them as girls and the fact that they might not be as visible a group as boys (maybe working but not living on the streets), programming for these girls would be essential. It is thus recommended that small scale interventions undertaken by project partners as well as by other organizations be further assessed through a joint consultative process to come up with a holistic model of working with girls.

Maintaining output and outcome level data: It is recommended that both output and outcome level indicators be set at the time of designing the interventions. A database tracking progress of the indicators will help ensure better record keeping and continued service/program improvement.

Referrals through formal mechanisms: Owing to the complex nature of issues faced by children living and working on the streets, it is recommended that formal mechanism be developed for referrals to medical, psychological, legal, financial, drug rehabilitation and shelter services. Sensitivity and capacity of the referrals to meet the children’s needs must also be regularly assessed and made part of this formal referral mechanism.

Capacity building and debriefing support for service providers: It is recommended that mechanisms are developed within the services and programs that provide the service providers an ongoing opportunity to build their capacity to have a better understanding of the issues faced by children living and working on streets and ways of dealing with their behavior. Ongoing debriefing mechanisms must also be developed for the service providers to allow them the space to discuss their personal challenges and stressors while working with children.  



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search