Evaluation database

Evaluation report

2011 India: Evaluation of UNICEF Strategic Positioning in India



Executive summary

 

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report.”

 

Background:

One of the key elements of the UNICEF India Country Programme Plan of Action (CPAP) 2008-2012 is the upstream work at the national and state levels in terms of knowledge management for policy and programme influencing, partnership, replication of innovations from the Integrated District Approach, and strengthening of systems and capacity development in the on-going flagship programmes of the government leading to better results for children and women. These have been adequately articulated for each of the programmes and their results in the CPAP. The CPAP also clearly emphasizes the need to reduce social inequities based on gender, caste, ethnicity and region. This focus represents a major strategic shift – Changing Gears - from the earlier programming efforts of the UNICEF, which were largely focused on supporting actual implementation of programmes through supplies, trainings, and service delivery.

The importance of these strategies was reconfirmed during the Mid Term Review of the CPAP in 2010, and they are likely to form the main strategic pillars for the next Country Programme. While the results achieved by programme sections have been reviewed though Mid Year and Annual Reviews, there has been no systematic review on whether and how these strategies have contributed to the expected results of the CPAP.

Purpose/Objective:

The evaluation was intended to answer whether and how the key strategies have contributed to better position UNICEF in the national development agenda of India, and whether and how these key strategies may have accelerated and strengthened the achievement of higher level results beyond the sum of the sectoral results delivered by the programme sections. The focus was placed around the four overarching strategies of CPAP (knowledge management, systems strengthening / capacity development, policy advocacy, replication of innovations from the Integrated District Approach, and partnerships), as well as the strategic area of social inclusion (including gender mainstreaming), clearly outlined as a cross cutting commitment of programming in the current Country Programme.

Methodology:

Key approach was to test a thoery of change, applying the five key strategies: Results achieved or innovations from interventions should be assessed, evaluated or documented (knowledge management) to influence policy and programme of the Government of India at the Union and state levels, which leads to leveraging of government and partner resources for replication and upscaling. Social inclusion should be taken into acount throughout the process, and the government capacity is developed to ensure sustainable, inclusive results.

The evaluation covered the first three years of CP 208-2012. Evaluation questions were assessed through the following methods (1) literature review; (2) key informant interview, (3) focus group discussions, and (4) case studies. Four states (two from priority and the other two from 'transition' states) were seleted for field visit.

A Management Group was formed to ensure transparency and quality in the process. Members included representatives from CO (Delhi and state), Regional Evaluation Advisor, NY (Evaluation Office and Division of Policy and Planning), as well as external partner (International Development Research Centre).


Findings and Conclusions:

Key findings were:
- The evaluation found areas where the CPAP strategies have contributed to improved performance by government. There are important examples of the contribution of ICO’s strategies at the district and panchayat levels in the Integrated Districts. There are many other examples in various sectors where the Strategies have resulted in improved government performance as well.
- Strategies have worked better at the state level than at the national level. There is a perception of insufficient capacity in research, and policy analysis, and as an organization that is always willing to push the government on controversial issues.
- The key ICO strategies such as providing technical assistance, helping to leverage GoI resources and partnering at all levels are contributing to programme sustainability within the GoI.
- ICO’s execution of its knowledge management strategy is generally felt to be weak by both ICO staff and outside observers. Although efforts are made to document and share knowledge on new innovations that are piloted, few are adequately evaluated before being scaled-up.

Recommendations:

1. Continue to emphasize the strategy of focusing on the upstream level. If impact on policy is desired, ICO should increase senior staff capacities in order to better meet the needs of engaging in policy issues. ICO should endeavor to change the image as a technical organization to one with strong research and policy analysis capacity.
2. ICO should improve its management of the Innovation Cycle.
3. Strengthen ICO research, evaluation and knowledge management practices.
4. Encourage early government involvement in the innovation process.
5. ICO should encourage and support the government in evaluating pilot innovations. As well, ICO should support capacity development of the national and state governments in evaluation.
6. ICO should strengthen its planning and monitoring systems to ensure better coordination between the ICO and the state field offices.
7. The ICO should improve its results framework.
8. ICO should investigate the value of increased emphasis on partnering with the private sector and civil society.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search