2012 Bosnia and Herzegovina: Final Evaluation of MDG-F Culture for Development Programme
Author: Robert Travers
"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is "Outstanding, Best Practice", "Highly Satisfactory", "Mostly Satisfactory" or "Unsatisfactory" before using it. You will find the link to the quality rating below, labeled as 'Part 2' of the report."
The Joint Programme Improving Cultural Understanding in Bosnia and Herzegovina is an $8 million intervention financed under the MDG-F’s ‘Culture and Development’ window. The MDG-F was established in 2006 and is an international cooperation mechanism whose aim is to accelerate progress on MDGs worldwide. Substantial funding comes from the Spanish Government. MDG-F is supporting four programmes in Bosnia and Herzegovina of which the programme now being evaluated (Culture and Development) is the largest. The programme builds on existing efforts of the United Nations Country Team (UNCT) in the areas of the protecting and revaluing cultural heritage, inter-ethnic dialogue, tolerance building in education, and human rights-based approaches to programming, including social inclusion. The Joint Programme Document is the overall guiding document which sets out the strategic basis and intervention approach. It outlines the activities to be implemented by UNDP, UNICEF and UNESCO and is a binding contract. The document points out that culture was an integral part of the conflict in Bosnia and Herzegovina, and that at the same time, the re-visioning and revival of culture in the country is crucial to its future, including its social, political and economic development.
Culture is defined in the Joint Programming Document as including cultural heritage (the document recognises Bosnia and Herzegovina’s unique ‘east meets west’ history). Cultural tourism is also highlighted and seen as having potential for economic development. In addition to tourism, the cultural industries which have a potential for wealth and job creation are outlined as including publishing, music, cinema, crafts and design.
The Final Evaluation of the Improving Cultural Understanding in Bosnia and Herzegovina Joint Programme has the following objectives:
- To assess the Joint Programme’s quality and internal and external coherence with the United Nations Development Assistance Framework for Bosnia and Herzegovina, national development strategies and priorities, the Millennium Development Goals at the local and country level, the level of contribution to the objectives of the Culture for Development thematic window, and find out the degree of local ownership.
- To assess how the Joint Programme operated and what was the efficiency of its management model to guide future joint programming among United Nations agencies.
- To assess the design and relevance, effectiveness, efficiency, impact and sustainability of the programme and the level of achievement of envisaged programme results.
- To assess the quality, results and impact of local programme interventions including the evaluation of co-financing modality and implementation capacities on a local level.
- To assess the programme’s different internal and external monitoring and evaluation systems and tools developed and assess the programme’s communication strategy, outreach activities and impact.
- To identify key recommendations and lessons to be learned.
1.1 Purpose of the evaluation
Evaluation is about Are we doing the right thing? It examples the rationale, the justification of the undertaking, makes a reality check and looks at the satisfaction of intended beneficiaries. Evaluation is also about Are we doing it right? It assesses the effectiveness of achieving expected results. It examines the efficiency of the use of inputs to yield results. Finally, evaluation asks Are there better ways of achieving the results? Evaluation looks at alternative ways, good practices and lessons learned. (United Nations Evaluation Group [UNEG]).
The methodology approved for the evaluation was outlined in the Inception Report (March 2012). It sought to integrate secondary and primary research, together with observational research of the relevant stakeholders and examined the wider context in which the MDG-F programme is embedded. An extensive programme of consultations was undertaken in country (annex 2). Actual outcomes, global and specific, in the form of key performance indicators (KPIs) were compared and contrasted against the planned outcomes in the Joint Programme Document and results framework, which was updated evaluation (annex 3). The roles of these three implementing agencies, and of partner organizations, and the extent of common coherence and cooperation was reviewed. The extent to which there are commitments to continue initiatives commenced under the Joint Programme with government, the European Union (EU) or other funding was also reviewed. Consultations and site visits were important during a mission to Bosnia and Herzegovina.
Findings and Conclusions:
In terms of relevance, effectiveness, efficiency and impact, the Joint Programme has been extremely sound. In terms of sustainability, the programme appears to be sustainable, especially in terms of institutional sustainability. It was an excellent example of a ‘One UN’ approach with very strong engagement with state entity and municipal governments and the NGO sector. Close alignment with the MDG-F thematic window for Culture and Development was observed.
There is a need to continue the Joint Programme’s activities in fostering education on interculturalism and dialogue, to build of strategic cultural and educational policies put in place, and to give greater attention to peace-building through releasing the economic development potential of Bosnia and Herzegovina’s very rich and unique shared cultural heritage
Table 6: Rating of evaluation criteria
Criterion Rating (Excellent, Good, Moderate. Fair. Poor)
1 Relevance: Good
2 Effectiveness: Excellent
3. Efficiency: Good
4. Impact: Good
5 Sustainability: Good (institutional sustainability); moderate (economic, social and networking sustainability)
In order to stimulate economic development through culture, there needs to be a greater focus on fostering private enterprise, particularly in a post-socialist economy still dominated by a very large public sector . The MDG-F Culture for Development Programme in Turkey had a focus on private sector development and there may be opportunities to compare approaches and lessons learnt in future United Nations activity.
It is recommended that future interventions related to the cultural industries in Bosnia and Herzegovina give specific attention to sustainable job creation opportunities (outside of the public sector).
In future interventions it is recommended that attention is given to investigating and embedding the economic development potential of culture in economic development strategies by all relevant development agencies and government departments.
The Joint Programme’s database systems and implementation frameworks should be carried forward into other United Nations projects.
Key indicators need to be more closely related to the achievement of results rather than the monitoring of activity as it takes place and more attention needs to be paid to estimating economic impact. Using more conventional project management cycle logframes which tie activities more closely to objectives, objectively verifiable indicators and results would assist this
To further United Nations transparency and accountability, it is recommended that in future Joint Programme webpages should be hosted on all participating United Nations agency websites in addition to beneficiary webpage placement. As a knowledge-sharing initiative consultancy and other reports should be made widely available through publication and/or placement on a wide range of appropriate Internet sites (including relevant United Nations sites and research libraries).
Full report in PDF
PDF files require Acrobat Reader.