2011 MENARO: Evaluation of Adolescents: Agents of Positive Change Programme (2005-2011): Phase 1 and 2
Author: Kartini International team
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labeled as ‘Part 2’ of the report.
This report outlines and summarizes the findings of the regional evaluation “The Right to Participation: Adolescents - Agents of Positive Change”. This programme has been funded by the Swedish International Development Cooperation Agency (Sida) since 2005 and implemented by UNICEF’s Middle East and North Africa Regional Office (MENARO). Implementation of Phase I took place from 2005 to 2007, and Phase II from 2008 to 2011. The programme involved seven (7) countries from the region in Phase I from 2005 – 2007 (Egypt, Jordan, Lebanon, Morocco, occupied Palestine territory, Syria and Tunisia) and Phase II from 2008 -2011 with nine (9) countries, with Algeria joining in 2006 and Iran in 2008 during the second phase. The programme has highlighted issues concerning the rights of adolescents and youth in the region, with a particular focus on participation.
The evaluation’s purpose was to measure the extent to which the programme attained its planned and any unintended results during its implementation and improve future development and participation programmes through feedback of lessons learned. It also focused on identifying how the programme’s goal and objectives have been met. These objectives included to:
1. Establish a consistent knowledge base across the participating countries in both phases on adolescents and young people in order to influence policy and programming;
2. Strengthen the capacity of national governments, partners, service providers and young people themselves to streamline their priorities in national policies and to provide support for youth structures and to support networking;
3. Promote opportunities for adolescent participation in friendly spaces through home, school and community; and
4. Build partnerships for advocacy to promote the rights of adolescents to development and participation.
The purpose of the evaluation is to “contribute to improve future adolescent development and participation programmes through feedback of lessons learned. The primary evaluation objectives are to measure the extent to which planned and any unintended results have been attained during the programme’s implementation, and how the overall goal and four objectives stated above have been met in the nine countries participating as well as at the regional level. The evaluation team was also asked to determine how relevant the programme was for different groups of adolescent and youth, particularly from the perspective of the quality of their participation in it; how successful the programme was in identifying relevant partners and what the added value of having a regional approach to the programme for both partners and UNICEF.
From the effectiveness perspective, the team was tasked with finding out progress made towards achievement of the expected outcomes and unexpected results, the effectiveness of monitoring and evaluation mechanisms used; beneficiary satisfaction with the programme results and degree of involvement of programme participants in the planning, designing, implementation and monitoring of programme activities.
Efficiency issues to be measured included degree of maximization of resources; whether additional resources have been allocated by partners/stakeholders/counterparts to cover costs; timely achievement of objectives and outcomes; distribution of inputs and outputs between different age groups of young people (10-14, 15-19); degree of consideration of gender equality principles resource allocation; and UNICEF’s comparative advantage in designing and implementing this programme.
From a sustainability perspective the evaluation objectives were to determine the level of buy-in of national partners; assess what measures were built into the programme to sustain the programme outcomes; and replicability of the programme in other locations/countries/regions?
The evaluation team used the following data collection methods to evaluate the programme:
1. Document review of programme proposals and regional and country progress reports from Phases I and II, as well as additional relevant documentation gathered during the country case study missions.
1. Three (3) country case studies (Egypt, Morocco and occupied Palestine territory) involving key informant interviews and focus group discussions with programme beneficiaries and other stakeholders based on Most Significant Change methodology (MSC). In Egypt, the evaluation also administered an on-line survey for users of a youth-oriented website working in collaboration with UNICEF. Key informant interviews covered UNICEF staff, implementing partners and other UN agencies. UNICEF MENARO selected the countries for the field missions to represent a mix of types of programming and programme size (Egypt), programming within a conflict context (oPt) and a balance between Middle Eastern and Magreb countries (Morocco).
2. Posting of two on-line surveys for countries that did not participate in a country case study: one survey was directed towards adolescents and youth programme participants; and the second towards adults involved with the adolescents and youth in the programme.
3. Phone interviews with implementing partners and UNICEF staff in the six (6) countries that did not participate in the country case studies.
4. Phone or face-to-face interviews with UNICEF regional staff and with Sida personnel responsible for the programme.
Through these diverse processes the evaluation team either directly interviewed or received survey feedback from a total of 400 persons, including 223 adolescents and youth.
A unique feature of the evaluation methodology was the inclusion on the evaluation team of a small group of young researchers who were also programme beneficiaries. Under the supervision of the international and national consultants, they served as focus group discussion facilitators in Egypt and occupied Palestine territory (oPt). This was done following training on how to facilitate a focus group discussion (FGD) in an objective manner and on Most Significant Change methodology.
Full report in PDF
PDF files require Acrobat Reader.