Evaluation database

Evaluation report

2012 Fiji (Pacific Islands): Evaluation of the Convergence Approach in UNICEF supported programme in the Pacific 2008-2012



Author: Frank Noij

Executive summary

Background:

UNICEF’s support in the Pacific from 2008-2012 has focused in particular on three of the Least Developed Countries in the sub-region, i.e. Kiribati, Solomon Islands and Vanuatu. The programmes in each of these countries made use of a convergence approach, in which support from all UNICEF’s programme components was meant to be provided to the same underserved geographical areas. After four years of implementation the convergence programming is being evaluated with the aim to inform UNICEF support during its next programme cycle 2013 - 2017. The present evaluation report provides the details of the evaluation findings and conclusions, identifies lessons learned and provides a set of actionable recommendations. The TOR of the evaluation is presented in annex 1.

UNICEF’s support in the Pacific from 2008 - 2012 has focused in particular on three of the Least Developed Countries in the sub-region, i.e. Kiribati, Solomon Islands and Vanuatu. The programmes in each of these countries was underpinned by a convergence approach, in which support on sub-national level was meant to be provided to the same selected underserved areas by all UNICEF’s programme components. The approach was meant to be a way to integrate programmes to holistically address the development needs of children at the sub-national level. Results were meant to be achieved across programme silos to enhance children and mothers’ well-being throughout the lifecycle. Focusing resources on the lowest performing sub-national areas, the approach was expected to contribute towards acceleration of achieving the MDGs. After four years of implementation in the Pacific, the convergence programming approach is being evaluated with the aim to inform UNICEF Pacific support for the upcoming programme cycle 2013 - 2017.

Purpose/Objective:

The evaluation made use of a formative approach with the evaluation objectives focusing on relevance, efficiency, effectiveness and sustainability, which were further detailed for the specific requirements of the evaluation during the inception phase.

The present evaluation is timed at the end of the 2008 – 2012 programme cycle. The Evaluation is meant to inform the development and implementation of UNICEF Pacific’s next Multi-Country Programme cycle from 2013-2017. Findings and recommendations are expected to be applied to the management of existing convergent programming, as well as to the design of new initiatives and activities. Moreover, results will inform work planning with governments and are expected to be used to further strengthen programme monitoring and partnership arrangements. Expected users of evaluation results include Governments of the three priority countries, UNICEF Pacific and its field offices, service providers in the convergence areas and community based and civil society organizations supporting the interests of vulnerable and worst-off groups.

The evaluation is meant to be a formative endeavour, providing lessons and recommendations to inform the next programme cycle. Use is made in particular of four of the five OECD DAC / UNEG Evaluation Criteria, i.e. relevance, efficiency, effectiveness and sustainability. The criterion of impact is less applicable with the focus of the evaluation in particular on intermediate level results, rather than on impact level changes. Based on the guidance in the TOR, the objectives of the evaluation have been further worked out, adapted to the requirements of the evaluation and in particular to the focus of the evaluation on the convergence approach as an important strategy in UNICEF Pacific’s programme. The objectives were presented in the inception report and based on discussions with UNICEF Senior Management including the Evaluation Manager these were further adapted to meet the requirements of the evaluation.

Methodology:

A mixed methods approach was used in the methodology of the evaluation, with desk review of secondary data, semi-structured interviews and focus group discussions with key stakeholders and validation meetings at country and regional level to discuss findings. Combination of methods and participation of a wide range of stakeholders provided ample opportunity for triangulation. Country visits conducted included one selected convergence area in each of the focus countries. Ethical considerations were guided by the UNEG Evaluation Norms and Standards. Limitations to the methodology included the unavailability of some stakeholders which had moved position, some of which could be located and interviewed but some others could not be involved.

Findings and Conclusions:

RELEVANCE
Support provided to the convergence areas has addressed the needs of vulnerable children and women, with each of the selected areas scoring low on several child and women related indicators. Overall, however, the relevance of an area based approach to convergence in the Pacific, in which sub-national programming focuses on selected geographical or administrative areas, has proved limited. As the approach was not linked to the UNDAF or national development strategies it remained focused UNICEF supported programmes and related social services. The area-based characteristic of the approach was not owned by key stakeholders, including National Governments, who need to address social services country wide, as well as several UNICEF programmes, to whom an area-based approach did not fit their programming focus and requirements. The ways in which the application of the approach has been adapted from a strictly area-based principle to a more diverse means of enhancing the holistic development of children has improved the relevance and applicability in the Pacific.

EFFICIENCY
Although field offices were set up in the focus countries, implementation of programme components in convergence areas was primarily steered by the programmes based in Suva, leaving the FO with few means to actually make convergence work in the field. The centralized UNICEF financial system further limited the management responsibilities of the Field Office while on the other hand they felt the consequences of multiple financial delays which occurred in practice and which at times undermined the credibility of UNICEF. Though these management constraints were identified in the MTR in 2010, these issues largely persisted during the second part of programme implementation.
Joint monitoring, the use of Most Significant Change technique and Sentinel monitoring (with overlap in convergence areas) have each proved to have added useful elements to programme monitoring. What has been missing is the regular collection of output and outcome level data on the indicators in the results frameworks of the various programme components. Though baseline data have been gathered in several of the programme components, no similar follow up data collection has yet taken place and reporting is primarily activity oriented. This has limited the use of results based management.
So far efficiency has mainly been realized through application of an integrated approach to holistic child development as one of the applications of a holistic approach to child development. The EPI programme is a good example with the introduction of multiple services in addition to vaccination. Moreover, the use of learning of local interventions to inform national level advocacy and replication of initiatives has enhanced efficiency, and proved a useful way to make use of experience built in convergence areas. The development of Minimum Quality Standards for Primary Schools and the certification of Child and Mother Friendly hospitals are examples concerned. Such approaches did though not necessarily reduce transaction costs for parties concerned. These were in particular reduced through UNICEF’s participation in SWAps in Education with MoE and donors in each of the focus countries. However, the SWAps are nationally oriented and single sector focused and are not necessarily conducive with convergence or programming for holistic child development as such.

EFFECTIVENESS
Results have been achieved in each of the programme components in the selected convergence area, and are likely to have contributed to the reduction of disparities in the countries concerned. Results in terms of children vaccinated through the EPI programme and the number of birth registrations are examples in case. Experiences obtained in the convergence areas have been used to inform national level legislation and policy-making and there have been various examples of replication of initiatives from the convergence areas to cover wider geographic areas. A holistic perspective to child development has been used to integrate multiple sector services in sector based programmes, bringing about multiple results for the same children and women. The organization of child health days is a good example of combining multiple services at a single event. Moreover, initiatives have focused on enhancing coordination at sub-national and local level in planning and delivery of social services with the support to the development of provincial and local level child development plans. This is of particular importance with local level agencies lacking a tradition of attention to the delivery of quality social services.
Constraining factors for reaching results have included limited capacities in coordination amongst the UNICEF programme components as well as government agencies and limited support to enhance such capacities so far. These constraints add to the geographical isolation and transport constraints that characterize large parts of the region.
Strategies to enhance results include working through government systems and building capacities in the process. Attention to social and cultural aspects and recognizing the value in terms of identity for children and women can enhance programme implementation. Moreover, UNICEF’s engagement in SWAps and expansion of the WASH programmes in Kiribati and Solomon Islands can provide new fora for UNICEF to promote a more holistic approach to child development.

SUSTAINABILITY
Results so far cannot yet be considered to be sustainable with some of the capacities required not necessarily fully in place and government funding of programming costs of initiatives unlikely in the short term. Attention to disaster risk reduction and emergency preparedness is varying and needs to be enhanced to improve sustainability given the relatively high risk of natural disaster in the Pacific.

Recommendations:

  • Completing the work in the convergence areas
  • Focus ‘Convergence' on a wider set of ways in which to support holistic child development
  • Adapt monitoring systems and develop capacities concerned
  • Develop means to enhance access to quality social services on outer islands
  • Promote holistic child development in new for a that UNICEF has become part of and seek linkages with initiatives concerning holistic child development in the region


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search