We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2017 Jordan: Evaluation of the Ma’an (Together) towards a Safe School Environment Programme 2009-2016 - Jordan



Author: AAN Associates

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report, and the executive feedback summary labelled as ‘Part 4’. 

Background:

The Jordan Ministry of Education banned corporal punishment in 1981. Despite the ban, children still experience high levels of emotional and physical abuse at school and at home.  The study also revealed wide acceptance of corporal punishment among families as they found it an effective tool for changing children’s behaviour.

To reduce this violence, in 2009 MoE, UNRWA and Military Education Schools started the implementation of Ma’An (Together) Towards a Safe School campaign with UNICEF support, in close collaboration with the Office of Queen Rania Al Abdullah.  This nationwide campaign aims to reduce violence against children by educators in all schools, promotes new disciplinary methods in schools, advocates to end societal tolerance of violence in schools and supports media coverage to spread the message nationwide.

In 2011, an assessment of the campaign was conducted revealing that teachers felt they were incapable of managing their classrooms without resorting to violence. Based on these results, UNICEF implemented a ‘Tarbiyah’ programme for behaviour management and introduction of classroom management tools in 6 pilot schools across the Kingdom. Based on positive outcomes of the pilot phase, the programme was scaled up to 50 schools. Online MoE school survey results reflect significant reductions in verbal and physical violence rates as well as in the percentage of positive disciplinary alternatives.

In 2016, UNICEF implemented Tarbiyah in six schools in Za’atari camp to test the methodology of Tarbiyah and its successes in reducing violence in schools in a camp settings.

Given the scale-up of the different components (Tarbiyah, Advocacy Groups, survey, school-level activities, etc.) of the Ma’An campaign from 2010 and the resources dedicated, UNICEF is conducting an evaluation of the scale-up of the key components of the Ma’An campaign to generate evidence based knowledge of the impact of the campaign on the reduction of violence in schools.

Purpose/Objective:

This is an ex-post evaluation with the purpose ‘to determine the relevance, efficiency, effectiveness and impact of Ma’An Programme interventions on reducing violence against children in schools in Jordan’.

Methodology:

The evaluation employed a Mixed-Methods Approach. Both quantitative and qualitative data was collected and used. For primary data collection, a range of qualitative data collection methods such as key informant interviews, focus group discussions, and field observations (non-checklist based) were used. Other data or information, which was necessary to answer evaluation questions, was gathered from review of secondary sources, programme documents, reports, or records available with MoE and UNICEF.

The entire evaluation was undertaken in a participatory manner, which is demonstrated through active involvement of key stakeholders (Evaluation Reference Group comprising representatives from public organizations, UNICEF and UNRWA) for their inputs at critical stages such as evaluation design, tools development and application.

The evaluation made use of “Child Friendly Methods/Tools” to create a friendly and enabling environment for children to feel safe and free, and share their experiences and suggestions. A series of measures were put in place for quality assurance at all levels throughout the evaluation (details in the report).

The evaluation design and conceptual framework, is drawn from and guided by, the evaluation purpose, objectives, and the key evaluation questions. The Evaluators have approached this as ‘Summative-Formative Evaluation; summative to inform what has been achieved, and formative to inform scalability. The evaluation is guided by or uses the OECD-DAC evaluation criteria i.e. relevance, effectiveness, impact, sustainability and efficiency.

The evaluation used ‘Quasi-Experimental’ research design. This suited most to assess the Programme effectiveness. The design was used to assess and analyse the extent of change for key outcome indicators over the Programme period from 2009 to 2016 on key intervals e.g. at baseline (2009), mid-point (2012) i.e. start of the online monthly survey system, and on yearly basis from 2012 to 2016.

Findings and Conclusions:

The Evaluators endorse the view of most stakeholders met during the evaluation that the Ma’An Programme is an undertaking of national importance to address the deep-rooted problem of VAC in Jordan. The Programme has indeed created a national acceptance and a momentum against VAC in schools, with a balanced, inclusive, gender sensitive and HRBA approach. The achievements realised at astoundingly low costs is commendable. The context in which the Programme was launched featured high incidences of VAC in schools and as such required a strong Programme like Ma’An to make schools a place of learning for students. However, the approaches and strategies adopted for implementation were seriously challenged due to unforeseen external factors and internal oversight.

It is appreciable that despite all these challenges, the UNICEF JCO and MoE remained determined in the implementation of the Programme. Equally important, is paying attention to the existing school system that is challenged with a deteriorating and overcrowded infrastructure, contributes to the prevalence of VAC in schools. Thus far, the Programme has relied on the SAG forum as the pivotal node of propagation and accountability at the school level. Relying on SAG, and MOSS in this manner without scaling up the SAG in all public schools with the same capacity development as done for the 2,000 schools, and without upgrading MOSS, will be detrimental to long term sustainability. Hence, SAG capacity development must be scaled up.

Recommendations:

1. Reactivate the steering committee and make it more proactive and empowered to resolve inter and intra departmental and organizational level issues.
2. Strengthen the role of School Advocacy Group (SAG) by; (a) Increasing the number of parents in; (b) Ensure a better ratio of counsellors with respect to number of students and schools. (c) Ensure that vacancies of the counsellor positions are filled; (d) Use the online survey system, a cost-effective tool to determine best performers and to hold the principals accountable in any school that consistently shows a higher or prevailing incidences of VAC.
3. Enable all key actors to become ambassadors, advocates and catalysts. Acknowledge, recognize and reward the high achievers by making such children ‘Peace Ambassadors’. Explore the potential of the Social Media in fast and viral transmission of social messages in particularly of the Ma’An Programme.
4. Continue the recently started capacity development and sensitisation training for the Media entities and the staff associated with Media reporting on VAC issues.
5. Introduce the tried-and-tested behavioural modification approach into the regular curriculum of all teachers training institutions and in universities imparting degrees in education.
6. Review the counterproductive question sets and eliminate any chance of confusion in solicited responses from the children.
7. Create a Ma’An Fund. Use the money for events at the school level for students, teachers and SAG. Funds can be solicited from the private sector and philanthropists.
8. Proactively invest in (a) Completing the missing documentation trails; (b) Preparing English versions for all Programme Documents; (c) Collecting valuable data aligned to the TOC and Results Framework for Ma’An, knowledge and case studies.

Lessons Learned:

1. The Programme like Ma’An which aims to bring a change in social norm requires enduring efforts of behaviour change communication at various avenues with multiple strategies. For Ma’An Programme, the avenues are the school, home, and community; and strategies are the administrative support by MoE and community mobilization support through Media and MoAIA. Undermining the potential of any of these approaches and strategies will not yield desired results, therefore Ma’An must be implemented in its entirety rather partial implementation as witnessed for the involvement of Media and religious leaders.
2. participation and strong ownership of relevant Ministries is a must for a national level programme to achieve its objectives. Continued meaningful involvement of MoAIA and NCFA is vital for the success of Ma’An Programme. The role of religious leaders faded out gradually, eventually the magnitude of success was reduced. For continued progress the relationship should have continued beyond the first year!
3. VAC in schools cannot be eliminated though standalone school-based intervention; this will require parallel work at the community level to abandon the existing social norm of using violent means to discipline children.
4. A balanced approach of sanctions and positive reinforcements is required to achieve better results. Though the role of the school principal and the counsellors emerged as a pivotal intervention, yet accountability of poor performers and recognition for high achievers remained weak.
5. Community Mobilization events at national level are a useful strategy and behavioural change must be seen in long term.
6. Frequent changes at top level positions negatively affect the progress, therefore programme level management structures must be able to withstand transitions of top level leadership.
7. Without proper exit planning, institutionalization will remain a challenge.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2017

Country:
Jordan

Region:
MENA

Type:
Evaluation

Theme:
Child Protection

Language:
English and Arabic

Sequence:
2017/004

New enhanced search