We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2013 Global: Evaluability Assessment of the Peacebuilding, Education and Advocacy Programme (PBEA)



Author: Terrence Jantzi, Margaret Stanberry, Peter Bauman, The Konterra Group

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

The Peacebuilding, Education and Advocacy Programme (PBEA) is a four-year (2012-2015) programme funded by the Government of the Netherlands (GoN), currently being implemented in 14 countries. The strategic vision of the programme is to “strengthen resilience, social cohesion and human security in conflict-affected contexts,” with the strategic result of “strengthening policies and practices in education for peacebuilding.” The strategic result will be achieved through five outcomes:

  1. Increase inclusion of education into peacebuilding and conflict-reduction policies, analyses, and implementation
  2. Increase institutional capacities to supply conflict-sensitive education
  3. Increase capacity of children, parents, teachers and other duty-bearers to prevent, reduce and cope with conflict and promote peace
  4. Increase access for children to quality, relevant, conflict-sensitive education that contributes to peace
  5. Contribute to the generation and use of evidence and knowledge on policies and programming on linkages between education, conflict and peacebuilding

Purpose/Objective:

The UNICEF Evaluation Office commissioned an evaluability assessment of the PBEA in 2013. The evaluability assessment is a systematic process that was used to determine if the programme has a set arrangements that would make it “evaluation ready”. Hence the purpose of the evaluability assessment was to provide the evidence required to answer the following question: “To what extent does the PBEA have the technical and strategic elements in place to manage effectively towards results and to credibly demonstrate such results in future evaluations?” The evaluability assessment was organized around eight categories of questions for investigation proposed by UNICEF. Four of the eight categories addressed technical and strategic elements of the programme (programme coherence, feasibility of results, monitoring and evaluation requirements, and attribution of and/or contribution to results). The other four categories (internal understanding of goals and accountabilities, management and governance, resource allocations, and risk management), addressed elements of programme management and governance.

Methodology:

The EA team employed four methods to determine the evaluability of the PBEA. First, a comprehensive document review of key foundational documents, background research publications and country-level documentation was conducted. Second, 71 interviews were carried out with global-, regional- and country-level stakeholders. Third, field visits were carried out in three selected PBEA implementing countries (Pakistan, Somalia, and South Sudan) which included an additional 61 interviews. Fourth, after formulating the findings, a member of the EA team attended the Global Annual Review Workshop (in Istanbul) to present preliminary findings and carry out a data validation exercises with representatives of each country programme, the PBEA team, and UNICEF senior leadership. The findings of the evaluability assessment are organized according the eight categories of questions mentioned above, highlighting strengths of the programme, as well areas that required additional action to improve the programme’s evaluability.

Findings and Conclusions:

The conclusion of this evaluability assessment is that the PBEA, in its current configuration as a global programme, faces significant challenges to its evaluability. Hence one of the major conclusions and recommendations of the evaluability assessment is to view the PBEA as a programme that has a “global approach”, but being implemented from the country level where most of the decision making is situated, given the decentralized nature of UNICEF. A number of additional actions are required  to enhance evaluability, beginning from acknowledging that countries are at different places in implementation, adjusting expectations and scaling down deliverables in some programming contexts, as well as providing the necessary support to ensure that programmes use the remaining time efficiently.

When assessing evaluability at the implementation level, country programmes cluster into three groups. In the first category are five country programmes that are yet to complete their conflict analyses, or to put a credible programme on paper – for these, the EA team did not have enough information to determine their evaluability. The second category are four to six country programmes that are in the process of finalizing their conflict analyses and developing programmes based on the conflict analyses. Most of these are on “Fast Track” mode, but are far from where they need to be at the half way mark of programme implementation. Each of the countries in the third category (three or four in number) is evaluable, albeit with inputs still required for some elements of their programmes. The evaluability assessment concluded further, that there is significant variation even within each cluster of country programmes.

For Category 1 country programmes, one country needs to initiate their conflict analysis, while others needed to accelerate its completion. In both cases, country programmes are yet to commence programming based on the conflict drivers. Category 2 country programmes need to move beyond capturing what activities were completed and refine their articulation of the type of change expected as a result of their programmes. With a better understanding of the types of change that are feasible, the country team needs to ensure activities are necessary and sufficient to achieve the change that is articulated. They also need to design instruments for baselines, and initiate data collection. Another challenge for both Category 1 and 2 programmes is to ensure that countries have sufficient management and M&E resources and support to the implementing partners to safeguard the accuracy and quality of the data that is collected. Finally, all countries could improve the way they capture country-level project design. Comprehensive project design documentation that adequately describes the role of all implementing partners will provide critical information during the end of programme evaluation phase.

Recommendations:

Read the report to see the set of recommendations for initiating adjustments considered to be the most critical for programme success and to enhance evaluability:

  1. The global PBEA team should consider reframing PBEA from a “global programme” to a programme that has a “global approach” to implementation.
  2. The PMT should developing a strategy to assist programmes to narrow the focus in each country programme, as well as strengthen the evidence for demonstrating education’s contribution to peacebuilding.
  3. The Evaluation Office should consider an end-of-programme evaluation strategy that assesses the contribution of the PBEA for each country programme with a subsequent evaluation synthesis of findings for global aggregation.
  4. The PMT and the Accountability Committee should clarify roles and accountabilities between the centralized PBEA programme and country offices.
  5. The PMT and UNICEF senior leadership should negotiate with the donor to adjust deliverables for Category 1 country programmes to allow them to focus on establishing a quality foundation for peacebuilding programming.
  6. The Accountability Committee should update the resource allocation criteria to reflect where country programmes are in their implementation, and what can be realistically expected in the time that is left before the end of the programme cycle.
  7. Country PBEA programmes should negotiate exemptions from UNICEF human resource quotas with country office leadership, at least during this cycle of the PBEA which includes piloting of new approaches to peacebuilding programming.

 

 



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Date:
2013

Region:
Global

Type:
Review

Theme:
Education - Peacebuilding; Emergency

Language:
English

 

New enhanced search