We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2015 Global: GEROS - Global Meta-Evaluation Report 2014



Executive summary

Purpose/Objective:

The purpose of UNICEF’s Global Evaluation Quality Assurance System (GEROS) is to ensure that UNICEF’s evaluations uphold the high quality standards set for them. The GEROS review process and the information provided in annual meta‐evaluation reports help UNICEF to monitor its progress, identify its strengths and become aware of its areas for improvement with regard to evaluation.

The four main objectives of GEROS are to:

  1. provide senior managers with a clear and independent assessment of the quality and usefulness of evaluation reports;
  2. strengthen internal evaluation capacity, through practical feedback on how to improve future evaluations;
  3. contribute to corporate knowledge management and organizational learning, making available good quality evaluations; and
  4. report to the Executive Board on the quality of evaluation reports.

Each of UNICEF’s regional offices is responsible for submitting completed evaluations to the GEROS process. These include a diverse range of evaluations which differ in terms of thematic area, geographic coverage, management approach, and other elements.

Methodology:

Reports completed during the 2014 GEROS review cycle were reviewed over a five‐month period (mid‐January to early June 2015). In total, 69 reports were analysed using the GEROS assessment tool, which contains 58 standards for evaluation reports established by UNICEF. In the GEROS review process, each question and section is given a rating based on a colour-coded four‐point performance scorecard that goes from Outstanding/Best Practice to Unsatisfactory. For each of the section and sub‐section ratings, a narrative justification is also provided and supported by evidence from the report (e.g. examples and page or section numbers).

This meta‐analysis report presents the analysis of aggregate ratings from the 2014 review cycle. Reports with a rating of Highly Satisfactory or Outstanding/Best Practice, are considered to be “quality” reports. Data from recent GEROS cycles (i.e. from 2010 to 2013) has also been included for comparison purposes.

Findings and Conclusions:

  • The quality of UNICEF’s evaluation reports continued to increase through 2014, but only moderately.
  • The contribution of reports to the GEROS process by some of UNICEF’s regional and country offices appears to be dropping.
  • Reports continue to demonstrate similar shortcomings found in 2013.

Recommendations:

  • UNICEF should examine whether the increase in quality of evaluation reports has resulted in senior managers having greater confidence in evaluation reports.
  • Within its decentralised evaluation strategy, UNICEF should continue to build its own regional/country office evaluation capacities and national capacities to conduct relevant types of evaluations.
  • Special efforts should be made to strengthen certain aspects of evaluation reports that have been consistently weak in the past few years.
  • UNICEF should continue to update and systematically communicate its requirements for evaluation reports across its entire evaluation oversight/management system. These updates should take into account evolving standards for evaluation in the UN System.
  • As part of the periodic review of GEROS , UNICEF should consider revising the rating scale and several elements of the GEROS template in order to ensure greater precision in the messages that are provided about evaluation quality and the characteristics of evaluation reports, and to create more efficiency in applying the template.

Lessons Learned:

  • Clear and systematic communication of evaluation standards and priorities favours the effective alignment of evaluations with UNICEF standards, from the outset (i.e. TORs stage).
  • While common standards help improve evaluation quality, quality assurance systems such as GEROS should provide sufficient flexibility to account for different types of evaluations.
  • In a decentralized system, compliance with quality assurance systems such as GEROS is affected by incentives, available resources, and the perception of relevance.
  • Quality assurance systems such as GEROS need to strike a balance between consistent application over a period of time (which allows for comparison) and making major adjustments in order to improve utility and reflect changes in the environment.

 



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2015

Region:
Global

Type:
Review

Theme:
Management Excellence (Cross-cutting)

Language:
English

New enhanced search