2010 Global: Final Report: Evaluation of UNICEF’s Programme and Work in Relation to Adolescents and the Participation of Children and Young People
Author: Dana Peebles; Robert Brydon; Raymond Gervais; Wendy Peebles; Neena Sachdeva; Kisanet Tezare
“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”
This evaluation is taking place at a time when the world now has the largest population of young people in human history, representing up to 30 per cent of the global population, with the number of adolescents and young people estimated at 2.2 billion. The urban populations of Asia and sub-Saharan Africa will double in less than a generation and more than 50 per cent of these will be urban dwellers under the age of 25.1
The evaluation TORs noted that:
“youth represent 25 per cent of the working age population and account for 47 per cent (88 million) of the world‟s unemployed [and that] an estimated 515 million young people, nearly 45 per cent of the total, live on less than two dollars a day. In many post-crisis transition countries, over half the population is younger than 25 and many of them are teenagers. Half of the world‟s out-of-school population of 39 million children live in conflict-affected countries… Globally, over 18 million children are affected by forced displacement.”
Recognizing that UNICEF had paid inadequate attention to adolescent children, the organisation established the Adolescent Development and Participation Unit (ADAP) in the Programme Division in Headquarters in New York at the end of 2001 to provide programme support and technical guidance in the area of adolescent health and development. The 2006-2009 MTSP also includes reference to the need for the organisation to integrate children and young people‘s participation throughout UNICEF programmes. After 10+ years of UNICEF work in this area generally and eight years of strategic efforts through the ADAP Unit to assure the integration of an adolescent rights perspective into all levels of UNICEF‘s work, the organisation decided it was time to evaluate and reflect upon this work provide some guidance regarding related future directions.
The evaluation was conducted in two phases, with the first phase focusing on a desk review supplemented by interviews. The key findings of Phase I are summarized in Section 1 and copies of the related reports can be obtained from the ADAP Unit. This report represents the findings of Phase II of the evaluation that included a significant field component.
1 Refer to Evaluation TORs, Annex 6.
The overall purpose of the ADAP comprehensive evaluation was to:
1. Assess the results achieved in the promotion of the rights of adolescents in the last eight years (2001-2009) to provide UNICEF with evidence and strategies to inform a ―second generation of programmes‖ for more efficient fulfillment of adolescent rights in all the MTSP focus areas.
2. Provide UNICEF with more clarity on the organization‘s position on adolescents and young people, as a foundation for future strategic planning and programming.
3. Better define a paradigm shift towards a positive development approach to adolescents and the benefits of child and young people‘s participation.
4. Support UNICEF in identifying its comparative advantage regarding adolescents and young people for a more effective partnership with other UN and partner agencies that have been increasingly involved in the area of youth programming.
In addition, the evaluation had three very specific purposes. These were to provide:
● UNICEF and the Swedish International Development Agency (SIDA) with information on the effectiveness and impact of the programme for the mainstreaming of adolescent rights in UNICEF work, and of the gaps and uncovered needs towards achievement of the programme objectives.
● UNICEF with evidence on adolescent focused programming in the tsunami response for further articulation of humanitarian programmes and policy in relation to adolescents and young people.
● Evidence of the quality and impact of child and young people‘s participation in country programmes as a contribution to the Monitoring & Evaluation (M&E) of the Medium-Term Strategic Plan (MTSP) in relation to Key Result Area 4, Focus Area 5.
The evaluation frameworks developed were based on assessment criteria adapted from the Women‘s Equality and Empowerment Framework, the Hart Ladder of Participation, the CRC and the FreeChild Project Model (refer to Annex 7). The data collection tools included:
1. Document review of representative sample of emergency programme-related documents.
2. Review of sample of United Nations Development Assistance Framework and Country Programme level Annual Reports from ten countries.
3. Review of selected UNICEF generated background and guidance documents provided by the UNICEF Steering Committee.
4. Dissemination of an on-line survey in 33 Country Offices (COs) to the Deputy Representatives, Adolescent Focal Points and staff working in each of the five Focus Areas (where there were relevant personnel in each focus area in each CO). The surveys were provided in English, French and Spanish.
5. Dissemination of an on-line survey in English to Senior Managers and Regional Adolescent Focal Points.
6. Dissemination of an on-line survey of former participants in J8 events from 2005 to 2009.
7. Dissemination of an on-line survey to Voices of Youth participants.
Interviews and Focus Group Discussions
8. Interviews with key personnel at UNICEF Headquarters (HQ) in New York as part of the Inception Mission.
9. Follow-up interviews with selected Senior Managers and Regional Adolescent Focal Points who participated in the on-line survey.
10. Focus group discussions with participants of a global meeting of staff from the HIV/AIDS sector in New York and in a Regional Managers Team (RMT) for the East Asia and Pacific region held in Hanoi, Vietnam in March.
11. Telephone interviews with selected international partners of UNICEF.
12. Telephone interviews with a representative sample of Regional Youth and Adolescent Specialists and Adolescent Focal Points.
13. Telephone interviews with Regional Emergency Specialists
14. Local partner interviews in Vietnam.
15. Interviews with CO staff, plus NGO and government partners, in three country case studies (Cambodia, Sierra Leone, and Brazil).
16. Focus group discussions with child, adolescent and young people involved in UNICEF-funded programmes in Cambodia, Sierra Leone and Brazil.
In total, the Evaluation Team interviewed 610 people (including 367 adolescents and youth) and received survey responses from 150 people. The Team also reviewed an extensive set of documents. Given the wide breadth of the evaluation‘s scope these documents still only represented a sampling of what was available. The Evaluation Team was also asked to focus on reviewing programming and documents covered by the period of the 2006 – 2009 MTSP.
The chart in page viii represents the overall framework used to anchor the evaluation and provides a summary of the Evaluation Team‘s assessment of UNICEF‘s staff in the different categories and results levels outlined.
UNICEF needs to:
1. Develop an effective process to ensure that Senior Managers at the HQ and RO levels make a policy decision in the near future (in consultation with the COs) regarding how much of a higher priority UNICEF should be giving to adolescent rights and development programming, as well as on which specific groups of adolescents it would be most strategic to focus UNICEF support. This decision would form the basis for the development of a global policy on adolescent rights and development, with adolescent and youth input. (Senior Managers).
2. Ask each sector focus area and the emergencies staff at HQ to:
Map out their specific engagement to adolescent rights and development and children and young people‘s participation and determine the priorities for action based on existing evidence compared to the needs of other target groups covered by UNICEF‘s mandate.
Review the existing guidance in each sector from the perspective of how well they explicitly integrate adolescent rights and development and children and young people‘s participation
Determine to what extent the sectoral programmes for which they are responsible are meeting their related accountabilities
Where needed, update the guidance documents to integrate adolescent rights and development and children and young people‘s participation more consistently.
Make these guidance documents more easily accessible for RO and CO staff. (Sectors and Emergencies Units – HQ).
3. Take a leadership role at a senior multilateral level and initiate a consultative process with ROs, COs, UNICEF UN and other partners and interagency groups to find effective ways the organisation can harmonize and simplify its age definitions of children, adolescents, young people and youth with those used by its UN system, and international, regional and national partners, as well as simplify and build a common understanding of the terminology it uses to describe different aspects of adolescent development and children and young people‘s participation to reduce the confusion that exists and strengthen the related planning, monitoring and evaluation processes. (ADAP & ROs).
Commitment and Accountability
4. Use the results of this evaluation and other reporting mechanisms to review to what degree UNICEF is falling short of meeting its commitments to integrate adolescent rights and development and children and young people‘s participation in Focus Areas 1 to 5 and ensure that the organisation allocates sufficient human and financial resources to meet the current MTSP commitments by the time this strategic plan expires. (Senior Managers)
5. Develop ways to institutionalise the integration of adolescent rights and development and children and young people‘s participation in programming, such as:
Ensuring that programme design is informed by prior age, gender and vulnerabilities-specific analysis, that resource allocations are made based on the results of this analysis and that children and young people have input into the programme design in any programme that affects them directly
Ensuring that each programme (where relevant) has an adolescent and/or participation specific objective and related indicators to measure progress
Holding staff accountable for these processes and reporting on related results explicitly (ROs and COs).
6. Develop a strong global advocacy campaign to lobby for the systematic collection of age-specific data related to adolescents and youth within international, regional and national data collection processes. (ADAP, ROs and ROs).
7. Negotiate with donors and internally within UNICEF to allocate sufficient resources from the core budget so that at:
At the HQ level the ADAP Unit has a permanent core of technical staff at least as large as its current staff complement currently on contract or funded through other sources
The Regional level there is at least one full time Adolescent, Youth and Participation Specialist in each region accompanied by an adequate programme budget
The CO level in those countries that determine increased adolescent and youth programming is a priority that they can hire specialist personnel on an on-going basis. (Senior Managers)
8. Provide strategic training of the key technical advisors in each sectoral unit at the HQ level, as well as of the Regional Advisors from the diverse sectors at the regional level on how to work effectively with adolescents, and youth; provide funding for regional workshops on the same theme for key personnel from the sector units at the CO level, as well as deputy representatives; and incorporate relevant modules into the PPPM for inclusion in new staff orientations. A curriculum developed recently by the Brazil CO could be adapted for these purposes at relatively low cost with some additional support allocated to the Brazil CO for this purpose. (HQ, ADAP, Regional Offices/Country Offices)
9. Convene a series of regional workshops on good practices related to children and young people‘s participation such as how to integrate children and young people‘s capacity development processes and participation in community decision-making effectively at the municipal level and how to use these processes to make consistent and strong links with decision-making at the sub-national and national levels. (ADAP and ROs)
10. Coordinate the documentation and dissemination of good practices on successful programming approaches related to adolescent rights and development and children and young people‘s participation, and work closely with the Adolescent Focal Points to both identify and distribute these good practices. (ADAP, ROs)
11. Streamline which high policy events the organisation will continue to support and establish minimum standards for consulting with children across UNICEF as EAPRO has done with the Inter-Agency Working Group for Children‘s Participation.5 This would include, at a minimum, the following core principles:
Participants are selected by their communities and their peers
Participants are selected sufficiently ahead of time that they can consult their communities regarding the main issues to be discussed at the meetings
Sufficient lead time and staff resources are available to prepare the participants so that their participation is meaningful and effective
Children and young people‘s input is considered from the beginning of the policy drafting process and not left until the end
Related meetings are set up in such a way that they are child and adolescent friendly
There is funding and technical advice available to support follow-up activities at the national, sub-national (where applicable) and community levels. (ADAP, ROs and COs)
12. Conduct a mapping of major organisations in each cluster that work with adolescents and youth before, during and after emergencies with its partners in the interagency mechanism to help identify and define UNICEF‘s own niche area of expertise, and responsibilities more clearly. (Senior Managers, ADAP, Emergency sector HQ)
13. Develop a roster of specialists with expertise in developing integrated approaches to children, adolescent and youth participation in the preparedness, response and post-emergency transition phases of emergencies to support COs with the following types of activities:
Coordinating and developing inter-agency mechanisms in emergencies and joint needs assessments
Liaising with Ministries of Youth
Identifying youth-focused organisations in emergencies or developing their capacity to support emergency work at the three different phases
Conducting related situational analyses
Conducting assessments for marginalised youth
Facilitating focal groups
Developing adolescent-sensitive preparedness plans
Conducting Vulnerability and Capacity Assessments
Conducting assessments in DRR/preparedness education
Mapping agencies working on sectors where UNICEF has a lead and defining what is missing. (Emergency sector HQ, ROs)
14. Use the evidence gathered to date in this and other related evaluations to develop a niche within the emergencies community of organisations to advocate adolescent and youth participation at all three phases of emergency (Emergencies Unit – HQ).
15. Find a viable way to ensure the ADAP Unit is able to re-establish on-going contact and provide strategic input into work being done by the Programme Division while still maintaining a strategic presence and input in the Policy Division. (Senior Managers and ADAP).
Refer to Annex 12 for recommendations regarding additional research needed related to adolescent rights and development and children and young peoples‘ participation.
Full report in PDF
PDF files require Acrobat Reader.