We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2015 CEE/CIS: Young People's Media Network Initiative Evaluation



Author: Joachim Raffelberg

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

In 2002, UNICEF set out to inspire young people in the Central and Eastern Europe and Central Asia to take an active role in the emerging civil society through media-related activities in the Young People’s Media Network (YPMN). The object of this evaluation is the YPMN.  The Initiative aims to increase young people’s ability to advocate for child rights through media activities. The key activities are: 1. Development of youth-produced short videos (OneMinutesJr) about issues of concern for them and dissemination of the videos through mass and online media. 2. Support for Youth Reporting/journalism, including how to disseminate interviews of opinion leaders through mass or online media and social media channels. 3. Support for youth advocacy through information dissemination about media training activities. The Initiative’s scope covers 21 countries and one territory. 

YPMN is intended to contribute to UNICEF’s Regional Knowledge and Leadership Agenda (RKLA) within which UNICEF articulates its aims with respect to 10 priority results for children  and defines a common theory of change (TOC) associated with these results.

Purpose/Objective:

UNICEF commissioned this evaluation to take stock and examine what skills, opportunities and other benefits young people and civil society at large have gained and how the Initiative might further develop.

The objectives are to:

  • Assess the Initiative’s performance in line with the criteria of relevance, effectiveness, efficiency, impact and sustainability. The assessment considered the changing situation, needs, opportunities for engagement at country and regional level over the 2004-2014 period (with emphasis on 2012-2014), as well as the UNICEF national committees’ work on advocacy on child rights, including the equity refocus;
  • Draw and document lessons learned on what has worked and how; and
  • Make recommendations for future communication-related youth participation activities, taking into account the RKLA, the rapidly changing media environment, and the long-term viability and potential areas for further strengthening the initiative.

Methodology:

The evaluation has applied the five OECD-DAC  evaluation criteria of relevance, efficiency, effectiveness, (emerging) impacts, and sustainability to answer the evaluation questions  as per the evaluation matrix

To ensure impartiality and triangulation of information, the methodology relied on a cross-section of information sources and used mixed methods (quantitative, qualitative, and participatory).
Ethical dimensions were considered during the evaluations.

Sampling methods: The UNICEF RO selection of the five countries visited was designed to reflect geographical sub-regions, the different stages of YPMN in the various countries, and to illustrate a variety of strategic choices made by offices within the YPMN initiative to maximize results from their youth advocacy and the NGO partners they work with.

The YPMN initiative evolved in an “organic” way and the activities that were largely needs-driven. An evaluability assessment by UNICEF RO concluded that, despite some quantitative data gaps, there were enough reliable qualitative disaggregated data available to conduct this evaluation. These sources were supplemented by information from the sample country offices from indepth interviews, focus group discussions, press clippings, Internet research and observation. Thus, this evaluation relied to a large extent on qualitative methods and data.

Findings and Conclusions:

Findings
The YPMN is embedded in the UNICEF TOC and is contributing substantial activities for and by youth advocates.  The evaluation has also found that the Initiative is relevant to the long-term strategy for the region since it aligns with the RKLA and the core roles. The RKLA, which began in 2012, identifies as one of UNICEF core roles as advocacy to include the most vulnerable children in development and humanitarian work – by being the voice for children. The YPMN, especially the OneMinutesJr initiative, is an effective tool in support of the RKLA through the thematic focus of workshops on realizing children’s rights to a caring family, access to justice, inclusive education, social protection, disaster risk reduction and equal opportunities  but this needs to be more clearly articulated in the generic regional TOC as promotion of children’s participation (giving children a voice).

Conclusions
YPMN performance in relevance, effectiveness, and impact is generally positive. UNICEF Regional and country offices and YPMN management have successfully organized workshops and training sessions and have broadly aligned the Initiative to refocus on the most vulnerable children and a changing media landscape. The Initiative has disseminated over 3,000 outputs (videos, interviews with policymakers, information on media capacity building activities) through social media platforms.

The efficiency and sustainability of YPMN require major improvement. Stakeholders see the Initiative in danger of failing if it does not become a global crosscutting programme, if it cannot establish clear objectives and strategy, and if it does not attract more support from UNICEF regional and country offices.  NGO partners must have sustained multiyear engagement from UNICEF country offices in training and follow-up. YPMN needs rebranding to emphasize OneMinutesJr as a signature activity and the nascent M&E has to be developed into a full-fledged quality control mechanism.

Recommendations:

  1. To increase relevance, YPMN shift focus to OneMinutesJr, clearly demarcating it as a UNICEF advocacy tool to be integrated into UNICEF country programmes. Distribution should emphasize the online platform options most used by youth.
  2. To improve efficiency, YPMN be upgraded to full program status and develop vision, mission, and clear objectives and corresponding strategy, operations, and budget. OneMinutesJr should adopt results-based management, including full-fledged M&E that tracks training follow-up.
  3. UNICEF facilitates the progress of OneMinutesJr by developing resources, and revisiting staffing structure and funding. New branding gives the Initiative more weight and strengthens its role within UNICEF, assisted by high-caliber internal and external marketing and promotion.
  4. While shifting focus to OneMinutesJr, YPMN is maintained as supporting basis for CO, local youth media, alumni groups and NGOs to empower young people for advocacy.
  5. OneMinutesJr training policy reform incorporates innovative teaching methods; new subjects, including multimedia, ICT and other specialist topics; plus competitive equipment enabling mobile reporting and social network integration. Trainees are to be screened to satisfy UNICEF criteria for advocacy and support of most vulnerable; regional training is to focus on cluster strategy. Workshop results must be communicated to public and officials for greater impact.
  6. UNICEF RO and CO integrate OneMinutesJr activities into all programme and fundraising efforts to improve outreach and to achieve social norms change through deeper cooperation with civil society and government. Membership scheme to be developed as a possible way to attract partnership and revenue generation through NGO and private sector funding, which could be part of an exit strategy to make the network self-sufficient.

Lessons Learned:

Resilience
YPMN has delivered key outputs and outcomes under the theory of change and has adapted to several different programming frameworks, the latest of which has been realignment under RKLA.

Youthful Mailing List
The Youthful Mailing List has been the longest running YPMN product. Much of the dialogue initiated by YPMN takes place offline, with the result that valuable issue-driven debate and opinion-sharing are lost to the wider YPMN community.

Strategy/Management/Communication
Delivering YPMN for over a decade as a mere Initiative has proved inefficient and unsustainable, and left it poorly documented. More promising local sustainability results of the Initiative may have been achieved if relevant efforts by CO staff had been tied to annual performance appraisals. 
 
Scope
The scope of the Initiative had been too ambitious and, therefore, should have been reduced from the current 21 countries and one territory. The large volume of workshops delivered should have been cut to a manageable number, especially given the prospect of limited ongoing funding.

Use of YPMN Products
YPMN was never fully utilized in UNICEF programme delivery, outreach and fundraising efforts. UNICEF could also follow the advice of government representatives to capitalize on the treasure trove of videos for education and outreach. Digital asset management should have been better.

Planning
Owing to the absence of objectives, long-term strategy and performance monitoring, planning has been identified as a weak point, particularly evident in the Youth Reporting activity. An effective M&E mechanism would have ensured proper goals and oversight of this component.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2015

Region:
CEE/CIS

Theme:
Advocacy and Communication

Type:
Evaluation

Language:
English

New enhanced search