We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2013 Ethiopia: Final project Evaluation of the Joint Programme (UNICEF & UNFPA). A Rights-Based Approach to Adolescent and Youth Development in Ethiopia



Author: Dr Terri Collin, Ms Yasmin Yusuf, Dr HaileMichael Tesfahun and Mr Sintayehu Dejene

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

The Joint Programme, “A Rights-Based Approach to Adolescents and Youth Development in Ethiopia” aims to contribute to improved development of adolescents and young people by promoting rights relating to HIV/AIDS, sexual and reproductive health, gender equality and sustainable livelihoods. The programme is funded by the Royal Norwegian Embassy (RNE), and implemented by UNFPA and UNICEF country offices. The Joint Programme has been implemented over the period 2007-June 2013 in 25 target woredas of five regions of Ethiopia (Addis Ababa, Afar, Amhara, Oromia and SNNPR). These Five project regions have 23 million adolescents and young people and the programme woredas include 12 public universities and HIV “hotspots”. The programme targets adolescents and young people (aged 10-24) as primary beneficiaries addressing HIV prevention, sexual and reproductive health and rights among young people, in particular girls and gender based violence including female genital mutilation/cutting, rape, abduction and child marriage.

The purpose of the final, end-of-programme evaluation was to determine whether the Joint Programme has achieved its objectives over the period 2007-2013. The evaluation also aimed to fill knowledge gaps, document lessons learnt and suggest ways forward for the Joint Programme.

Purpose/Objective:

The main purpose of the end review is to ascertain whether the JP objectives have been achieved, as well as fill knowledge gaps, provide lessons learnt and suggest the way forward for the Joint Programme: should it be continued, and if so, how the effectiveness of the programme could be enhanced. The findings will feed into relevant Ethiopia UNDAF (2012 to 2015) baseline development. Also, the end review focus would be on assessing the JP role and contribution to influencing key outcomes f the national HIV/AIDS as well as FGM/C responses and SRH issues in young people in Ethiopia. The review framework should build on the following key areas.

1. Ascertain results against the baseline, and assess JP contribution to these results.
2. To assess the relevance, efficiency, effectiveness, outcome and sustainability of the project in relation to the objectives (and supporting outputs) set out in the programme document.
3. To assess the added value of the rights based approach to programme implementation, in particular, whether the approach has assisted in improving youth ability to claim their rights, as well as improving equity, and what have been major bottlenecks?
4. Test the programme theory, especially the vertical logic between output and outcome. assess follow-up of recommendations made during the midterm review.
5. Assess the JP partnership strategies: how does UNICEF and UNFPA work with relevant partners, such as the government at different levels, civil society and communities, and other partners.
6. Extract lessons learnt, findings and recommendations on how to enhance the response to the SRH needs as well as other gender-based violations of young people, improve the HIV/AIDS response and youth development in general at the country, regional and community levels.

Methodology:

The methodology was based on a mixed method approach ( both qualitative and quantitative) that included: a) a desk review; b) interviews with more than 80 individuals from government, youth organizations, donor agencies, UN agencies, coordinating and implementing partners at national, regional, woreda and kebele levels; c) focus group discussions with community members and intended beneficiaries; and d) snap-shot surveys of 75 youth leaders and 32 users of youth-friendly services. We conducted field visits in a sample of 12 programme woredas and visited one ‘tracer project’ run by a faith-based organization. The main limitation of the  methodology employed was that a population level survey was needed to provide credible data on outcomes, but this was beyond the scope of work for this evaluation. Unfortunately, it was not possible to find reliable secondary data on outcome level results that covered the right timeframes, geographical areas and age sets.

Random sampling used for the Mid-Term Review is used .Field visits were also conducted in a representative sample of 12 of the 25 programme woredas.

Findings and Conclusions:

The programme design is relevant to the current policy, social and HIV/AIDS context, although consideration needs to be given to some residual challenges, as well as emerging opportunities.  The programme has been operating with reasonable efficiency, although recent efforts to improve fund flows need to be sustained and the programme needs to be fully transparent in accounting for its operational costs.  On balance, the programme has been effective in achieving expected outputs and annual targets are regularly exceeded. However, there are weaknesses in the levels of youth participation in annual planning and in target setting, as well as anomalies and gaps in data reported from the regions. There also appears to be considerable diversity in programme approaches, both between regions and within them from year to year. Although this may reflect the responsiveness of the programme to context and local priorities, these differences in approach need to be more transparently explained.
There is considerable scope for strengthening the programme’s M&E system at regional and woreda levels. -- The programmes performance on outcomes and value for money remains inconclusive until a follow-up of the baseline study has been completed. Although the Joint Programme’s work with government implementing partners inherently builds in some sustainability, attention needs to be given to timely exit strategies.  Federal, regional and woreda HAPCOs provide useful leadership for coordinated working, but there is scope for more strategic partnerships with other International Development Partners and non-governmental organization. There has been good reporting of beneficiary case studies and lessons learnt in annual and bi-annual progress reports but there is potential for improved documentation, analysis and communication of innovation and good practice.

Recommendations:

The Joint Programme, “A Rights-Based Approach to Adolescent and Youth Development” should be extended into a second phase. However, the design of the second phase must be based on lessons learnt from the first phase and engage with the numerous issues, challenges and opportunities identified in this report.  There are 23 million adolescents and young people in the five regions where the joint Programme operates. There has been an intention to scale-up successful components of the programme since its inception (RNE, 2007); however, this evaluation has found little evidence of this scaling up in action.  The Evaluation Team suggests the following steps would provide a platform for strengthening the programme design:
- Commission a follow up to the Population Council’s 2010 “baseline” survey of young adults, making sure the survey is conducted in all programme woredas and addresses all the agreed outcome indicators for the Joint Programme, Commission a review to synthesis and align data from this evaluation, the follow up survey and additional operational data to identify the most important components of the programme in terms of effectiveness and outcomes, factors in success and the components that offer the best value for money, Use the findings of the review to develop an explicit, evidence-based theory of change (or transformation). Make revisions to the logical framework/results chain and adjust the programme’s M&E framework so that it is aligned to the new results chain and draws on data that can be reliably and routinely collected (and analysed) from woreda level up, develop and implement an exit strategy for the current programme woredas and universities and/or b) identify programme woredas that will become “centres of excellence”, develop a plan for scaling-up.  Importantly, the above process needs to bring together the skills and expertise of ‘duty holders’ with the enthusiasm, creativity and experience of young people.

Lessons Learned:

There is now a rich archive of beneficiary case studies from the programme.  However, it is not clear how they are being used to promote learning and who they are intended for. There seems to have been little success in capturing more comprehensive accounts of good practice. The ‘Big Sisters’ initiative at Gonder University is an example of a potential good practice study.

The Evaluation Team has also been impressed by the systematic documentation of challenges and lessons learnt in annual progress reports. These include some references to useful innovation. Many of the challenges documented relate to delays in funding flows and liquidation and associated operational challenges.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2013

Country:
Ethiopia

Region:
ESARO

Theme:
Youth and Adolescents

Partners:
HIV/AIDS Prevention and Control Office, UNICEF, UNFPA

Type:
Evaluation

Language:
English

New enhanced search