2009 Azerbaijan: Evaluation of the United Nations Development Assistance Framework 2005-2010
Author: Fuat Andic, Ph.D. and Elshad Mikayilov
“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”
- The United Nations Development Assistance Framework (UNDAF) appears to have been prepared thoroughly and with great detail. All requisite documents have been reviewed before its design.
- In its design, UNDAF has followed the logical steps of convening a prioritization workshop and liaised with the Government in order to assure concordance with Poverty Reduction Strategy Paper (PRSP) and MDG strategies. The UNDAF that emerged identified the issues to be tackled, the
correct outputs and expected outcomes, although the gender equality issue has not received sufficient attention. The baselines and outcome indicators have been expressed mostly in quantitative terms.
- In most general terms, UNDAF is a well-prepared document. Nevertheless, it is not totally free of certain shortcomings which are not grievous, but have bearings on the next UNDAF. In some cases, which are not very many, outcomes and outputs are not clearly delineated. Some indicators are left to be determined later. Listing a number of indicators for which there is no reliable source to verify became a meaningless task and deprived the consultant from verifying whether expected outcomes, let alone impacts, are in fact obtained or likely to be obtained. The document is detailed, but not excessively, which essentially provides flexibility to adjust projects according to changing conditions and priorities of the Country.
- Two basic concepts are used in judging the relevance of UNDAF: namely, a) the strategic positioning and focus of UN on key outcomes; and b) the outcomes and impacts relevant to national priorities, as well as consistent with the achievement of the Millennium Development Goals in Azerbaijan. UNDAF is relevant to the needs of Azerbaijan.
- Measured by administrative cost and by resource mobilization UNDAF appears to be efficient.
- While many project components of various agencies include capacity building, there are very few projects that specifically aim at capacity building which acquires an entirely different dimension, since Azerbaijan aims to become a net contributing country (NCC) and a donor country. It is imperative that the next cycle takes into consideration this particular dimension.
- In very general terms, overall outcomes, as delineated in the UNDAF document, are likely to be sustainable. However, the final verdict will have to wait until all projects completed and the programmes of each agency are subjected to an in-depth evaluation.
- The implementing partners appear to have a high regard for UN agencies. They frequently cited responsiveness, neutrality, administrative efficiency and flexibility, as well as UN’s understanding of the realities of Azerbaijan. The donor agencies also expressed high satisfaction with UN. Overall, UNDAF is perceived very positively.
- Regarding the fulfillment of the objectives, it appears that almost all UN agencies will succeed in reaching the expected results, thereby making UNDAF a successful document.
- Different UN agencies have different interlocutors within the government. This practice is less than desirable. Undoubtedly, different ministries will continue to be the interlocutor of the pertinent agencies for operational purposes. However, in order to maintain consistency and unified coordination, it is advisable that one single government entity should undertake coordination function of UNDAF and an active/efficient coordinating body should be established within the government.
- An important issue which is worth questioning is whether the programmes/projects of various agencies would have achieved the results they have achieved, had there been no UNDAF, and whether UNDAF is a fifth wheel or, in fact, has some added value. The answer to the questions is
that UNDAF is seen as an indicative document for inter-agency cooperation and the articulation of the national priorities within the overall policy formulation of the UN family.
- The design was correct and appropriate and relevant to the realities of Azerbaijan. Given the particular conditions of the Country, it is reasonably efficient.
- It is recommended that the next UNDAF should be a road map towards sustainable human development, rather than a process document.
- The next UNDAF will start in 2011 and end in 2015. This timeframe coincides with the expected realization of the objectives enshrined in the State Programme of Poverty Reduction and Sustainable Development (2008-2015) (SPPRSD) which is aligned with the Millennium Development
Goals (MDGs). Hence, the majority of the programmes and projects must be directed to the realization of these objectives.
- The projects contained capacity building components, some implicitly, few others explicitly, yet the indicators to verify the results of capacity building per se are lacking. The end results of these capacity building efforts, in more cases than not, remain either unknown or, at best, are simply assumed and certainly difficult to assess. The report strongly urges that, whenever appropriate, capacity-building components of projects/programmes be very clearly spelt out and worked into UNDAF. The results related to capacity building should be verified vis-à-vis the efforts and achievements attained in improved infrastructural and institutionalizing dimensions. Appropriate and measurable indicators must also be spelt out.
- UNDAF should contain a systemic approach to capacity building/training, using proper Monitoring & Evaluation (M&E) tools to track the respective progress related indicators.
- Too many indicators, both for impact and outcome assessments, will very likely end up yielding conflicting results. A few, but measurable indicators, should be selected with the assurance that they will be readily available and timely. In that vein, it is recommended that UNDP Human Development Report be considered as a major source for setting up indicators coupled with annual state statistics.
- Internal M&E process of UNDAF has been neglected until recently. It is strongly recommended that UNDAF M&E Group should be constituted, technically strengthened, and given due consideration for its mandate, which should be clearly spelled out.
- UNDAF should be viewed as a flexible instrument taking into consideration the particular mandate of different agencies. Long and detailed outcomes tend to put different agencies in a position that they first design projects and then seek the appropriate outcome category in the UNDAF document for a particular project. This goes exactly against the raison d’être of UNDAF. Such practices should be avoided at all cost.
- The aspiration of Azerbaijan to become an NCC, certainly, will make all projects/programmes demand-driven. In essence, this is a welcome transformation, but the implications on UNDAF are likely to be different. The next UNDAF should reflect the new realities.
Full report in PDF
PDF files require Acrobat Reader.