2012 Global: Protecting Children from Violence: A Synthesis of Evaluation Findings
Author: Zosa De Sas Kropiwnicki
"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is "Outstanding, Best Practice", "Highly Satisfactory", "Mostly Satisfactory" or "Unsatisfactory" before using it. You will find the link to the quality rating below, labeled as 'Part 3' of the report."
This meta-synthesis provides an evidence-base for guiding effective advocacy and programming for protecting children against violence, as outlined in the Terms of Reference. This report is based on a review of findings of 52 evaluations commissioned by various UNICEF Offices from 2005 to 2010. These evaluations are focused on the issue of ‘violence’, which herein is broadly defined as sexual abuse and exploitation, child trafficking, child labour, children working and/or living on the street, children in residential care, children in conflict with the law, harmful practices and injuries to children. The evaluations focus on developing and transitional contexts, including countries transitioning from an emergency situation (e.g. armed conflict or natural disaster) to a recovery and early development phase.
The purpose of the report is to facilitate the exchange of knowledge among UNICEF personnel and its partners. The content of this report does not necessarily reflect UNICEF’s official position, policies or views.
A qualitative approach was used to synthesize findings from the evaluation reports, addressing the key criteria of programme relevance, effectiveness, efficiency, and sustainability. The UNICEF Child Protection Strategy (2008) was the basis for the initial synthesis framework that focused on the broad strategic themes of ‘child protection systems strengthening’ and ‘social change’, specific programme areas (e.g. child-trafficking prevention, mine risk education, and reintegration of child soldiers), and cross-cutting issues (e.g. reaching the most vulnerable children and communities, expansion and scaling up, sustainability, and ethical considerations). Because the strategic themes or action areas are intertwined and complementary in practice, it was difficult to classify any child protection intervention under a single strategic category.
Findings and Conclusions:
a) Brief summary of programmes
‘Violence’ is a broad issue with many overlapping features and characteristics, as defined by Article 19 of the UN CRC (1989) and related articles. The scope of this meta-synthesis was to look at child protection issues not strictly considered to be ‘violence’ but that cause harm to children. Accordingly, the evaluations that UNICEF included in the sample (52 in total) look at a wide variety of interventions from the seven UNICEF programming regions. It is important to note, however, that many other aspects of UNICEF Child Protection work, were not included in this review, such as efforts to increase birth registration, capacity building of social welfare workers and legal reform initiatives that do not pertain to violence.
b) Overall relevance and effectiveness
The overall strategic goal of all UNICEF child protection interventions is to protect children from harm; this includes both prevention and response components. However, each programme should be driven by an objective that is relevant to the beneficiary (child, family, and community) requirements, country needs, global priorities, and partner’s and donors’ policies. In the 43 reports that explicitly discussed relevance, 23 programmes were found to be completely relevant, 19 partly relevant, and 1 not relevant.
In terms of overall effectiveness, the programmes were rated on the basis of whether they were not effective, partly effective or effective in meeting their programme objectives. Thirteen percent were deemed not effective, 67% partly effective, and 19% effective. It is important to note that these ratings were, however, tempered by the variable quality of the evaluations as discussed above. Further, some evaluation analyses were overly or under-critical toward the programmes, thereby skewing the results. Detailed analysis of overall relevance and effectiveness can be found in the main report.
c) Findings on appropriateness and effectiveness of programme strategies and interventions used
• Multi-sectoral approaches to capacity building and system strengthening that also target harmful social norms are effective.
• Integrating child protection into intersectoral programmes that combine longer term social change with short term tangible ‘entry points’ is a viable strategy.
• Addressing both prevention and response in the continuum of services is an effective strategy in protecting children.
• Isolated and vertical programmatic responses are ineffective and inefficient as compared to more holistic interventions.
• Understanding and considering underlying socio-economic, cultural and political determinants is critical in designing child protection programmes.
• It is important to move beyond advocacy and technical assistance to monitoring and oversight.
• Systematic capacity strengthening and coordination mechanism are essential for improving effectiveness of partnership and community mobilization efforts.
• Meaningful participation of children, families and communities need to be planned and implemented more systematically.
• Weaknesses in applying results-based planning and management tools are common in child protection programming.
• Lack of comprehensive monitoring and evaluation frameworks is another common problem which requires capacity strengthening of child protection and monitoring and evaluation staff.
• Lack of exit strategies and sustainability issues in programme planning is a key concern.
• Major effort is needed to strengthen equity-based programming to address the gaps noted by past evaluations.
• Child safe-guarding and associated ethical policies need to be reviewed and improved.
a) Overall child protection programme design / strategies
• Programmes should adopt a two tiered strategy in relation to child protection, targeting both systems strengthening and social change, while recognizing the overlapping nature of these two strategic areas.
• The positioning of programmes within the continuum of care should be considered to ensure that children have access to comprehensive services and that essential components such as prevention and family reintegration are not overlooked in favour of ‘band aid approaches’ centred exclusively on interim care.
• Programmes should be informed by an ecological framework that considers a child within the context of the family, community and broader country. Such holistic programmes are more effective in addressing the range of causes of violence and, in turn, the impact of this violence on children, families and communities.
• The child protection sector should better incorporate integrated, multi-disciplinary and intersectoral programming principles; doing so will require forging functional partnerships between different sectors within UNICEF and with other development partners.
• Holistic as opposed to vertical approaches to child protection should be encouraged in order to move beyond the silos of issue-specific programming to more comprehensive and mainstreamed systems approaches.
• Child protection systems-strengthening should be factored into emergency programming to ensure that UNICEF is able to transition efficiently and effectively into a recovery and early development phases of programming.
b) Cross-cutting issues
• Evidence-based planning and meaningful participatory programming should be prioritized in programme design to ensure greater relevance, appropriateness and flexibility to changing circumstances.
• Comprehensive monitoring and evaluation (M&E) frameworks should be developed at the outset of programmes based on clear indicators that do not simply focus on processes but on outcomes and impact. This should be accompanied by comprehensive information management systems for the documentation, compilation, analysis, and storing of disaggregated quantitative and qualitative data.
• To ensure that programmes are appropriately designed, implemented and monitored in relation to their objectives and intended results (outputs and impacts), logical frameworks or related tools should also be utilized across the programming cycle based on evidence, realistic objectives, logical coherence, and a sound, holistic strategy.
• UNICEF should facilitate internal communication (for intersectoral programming) and information sharing between partners, stakeholders and communities, rather than support vertical reporting structures alone. • Beyond tokenistic gestures of consultation, meaningful participatory programming should be encouraged to improve levels of ownership and, thereby, effectiveness and sustainability.
• To improve efficiency in coordination, partners should be selected using transparent and rigorous procedures; clearly delineated roles and responsibilities should be formalized; and training, mentoring and support should be improved, particularly in relation to programme management.
• Sustainability should be included in programme design, with a focus on exit strategies, long term fundraising strategies, community commitment, government buy-in, and institutional positioning.
• An inclusive approach to equity should be used that emphasizes ‘vulnerable children’ in the context of their families and wider community, in order to address the root causes of marginalization and moderate any unintended effects of equity-focused programming. From a practical perspective, this requires in-depth qualitative and ethnographic research, disaggregated data, multi-partner guidance, and capacity strengthening of UNICEF and partners.
• It is essential that UNICEF and partners develop and adhere to a strict child safeguarding policy that includes background checks for prospective partners, employees and volunteers; a code of conduct; initial and refresher training; the identification of emergency focal points; internal referral protocols; whistle blowing policies; and guidance on the collection, use and storing of sensitive data. Funding for partners should be contingent on the development of these policies and mechanisms. The safety and security of beneficiaries (adults and children), staff, and even evaluators should be prioritized at all times. It is also important to note that this recommendation applies broadly to UNICEF programmes and is not confined to Child Protection.
c) Headquarters / Regional Office guidance and support to Country Offices
• Headquarters and Regional Offices have started to shift the scope of their work; however, more work is needed on providing Country Offices with conceptual clarity about specific child protection issues and broader strategic areas including the continuum of care, child protection systems, ecological systems approaches, multi-disciplinary and intersectoral programming, mainstreaming, and equity.
• At a strategic level, UNICEF’s Child Protection Division at Headquarter and Regional Office levels should engage with other sectors in UNICEF to forge partnerships, develop principles, and define the parameters of intersectoral programming.
• In addition to guidance on specific child protection issues, Headquarters and Regional Offices should support internal capacity development exercises in relation to the cross-cutting issues identified in the report (evidence-based planning, results-based programming, participatory programming, knowledge management, monitoring and evaluation, equity and ethics). UNICEF Country Offices should then take responsibility for extending this capacity building to partners. It is critical that programmes incorporate attention to base-lines and monitoring impact in order to assess progress and build the evidence for what works.
• The staffing of the child protection sector at Country Office level should be carefully considered to ensure that UNICEF has the human resource capacity to adequately support partners and effectively implement programmes.
• At Headquarter, Regional Office, and Country Office levels, procedures (administrative, financial management, funding disbursement, procurement and contractual) should be streamlined.
• Country Offices should assume a greater role in managing and overseeing the implementation of government and Nongovernmental Organisation (NGO) partner commitments at national and decentralized levels.
d) Future evaluations
• The profile and selection of evaluators should be carefully considered so that evaluations reflect a balanced perspective of local and international knowledge and expertise.
• Evaluations should be designed to focus more on outcomes and impact. This may require greater knowledge of what are often unexplored methodologies such as mixed methods, ethnography, quasi- experimental approaches, and longitudinal studies; funding for such evaluations should be allocated accordingly.
• Beyond a desk review of key programmatic documents, evaluators should be required to (a) undertake research, or consult with UNICEF, on the key concepts underlying child protection programmes and (b) conduct research on the context in which child protection programmes are implemented. This will allow for more comprehensive and relevant analyses.
• Although evaluators should be encouraged to capture highly contextual information, efforts should be made to standardize the structure and content of evaluation reports to allow for country, programmatic, or temporal comparisons. Similarly, the criteria used by evaluators should be consistent across child protection issues and include both quantitative scores and qualitative descriptions. Model evaluation TORs and technical support to field offices from experienced monitoring and evaluation specialists are key inputs towards building capacity of UNICEF staff to design and carry out high-quality evaluations.
• Given the dearth of quality evaluations in the area of violence prevention and response, in the coming years it will be important to conduct evaluations on children and violence programming to track progress in efforts to address violence against children. Furthermore, efforts need to be made to integrate children and violence themes into related large-scale evaluations, thus increasing data on violence prevention/response efforts and gaining cost and programme efficiencies by linking these efforts to on-going sectoral evaluations.
Full report in PDF
PDF files require Acrobat Reader.