2013 Moldova: End of Project Evaluation and Future Programming entry-points in the field of statistics and evidence-based policy making
Author: Arkadii Toritsyn, International Consultant , Gheorghe Caraseni, National Consultant
"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labeled as ‘Part 2’ of the report."
The UN Joint Project “Strengthening the National Statistical System in Moldova” (hereinafter Project) was designed relying on the findings of the Common Country Assessment, conducted jointly by the Government of Moldova and the UN system in 2005. The Project was designed on the basis of the needs identified by the NBS to enable it to perform its basic role of coordinating the statistical system ensuring standards to statistical production, and providing relevant, accurate, timely, comparable, coherent and complete statistical data. The five UN agencies, namely UNDP, UN Women, UNFPA, UNICEF and ILO, have joined efforts to provide more strategic and coordinated interventions to the Government of Moldova through the Project aimed to strengthen data production, dissemination and use.
The Project specific expected results from strengthening the national statistical system were:
• Improved capacities of NBS and other line ministries, which are responsible for the production of official statistical information, to produce data of appropriate quality in a timely manner and with a necessary level of desegregation by gender, age and geographical dimensions;
• Enhanced availability of statistical data through the improvement of data dissemination systems, practices and tools (availability of high-quality statistical publications addressing needs of data users, development of "easy-to-use" thematic materials for data dissemination, informing the users about the available materials);
• Increased use of available data for participatory policy-making through enhancing statistical users’ literacy, increasing the capacities within the Government, CSOs and other stakeholders, as well as encouraging in-government and outsourced policy analyses on the basis of available statistics.
The specific objectives of the evaluation include:
• Provide a full and independent assessment of the achievements and results, failures and constraints of the Project, as well as an analysis of its performance, management, impact, relevance and sustainability of the interventions;
• Generate lessons learned and good practices from each of respective outputs established in the Project since its inception in 2007 to inform current and future programming at the country level;
• Assess whether the results achieved are relevant for the current statistics related environment in the country and provide solid basis for UNDP and other partner UN agencies for future programming;
• Provide sharp and forward-looking recommendations that can guide UNDP, partner UN agencies and other players in developing effective, realistic and evidence-based strategies.
A large set of different and complementary evidence were collected and analyzed by utilizing both qualitative and quantitative methods that included:
• Desk review of relevant documents;
• Consultations with UN management and Project team;
• Semi-structured interviews with pre-determined sets of questions with a large number of Project partners and beneficiaries;
• Four focus group of Project beneficiaries and partners;
• Informal conversational interviews;
• Validation of preliminary findings and recommendations with the UN Project partners;
• A wrap up session with the UN Resident Coordinator.
The consultants reconstructed a Project Theory of Changes and developed the Evaluation Matrix.
Findings and Conclusions:
Relevance: Project design is relevant vis-à-vis the overall Project goals. and project remains highly relevant to realities of reforms of statistics and evidence-based policy making in Moldova. The Project correctly identified and addressed fundamental limitations in the areas of statistics and evidence-based policymaking.
Effectiveness: The Project strengthened data collection, storage, compilation, dissemination and statistical analysis, and improved gender, social, labour, demographic and other statistics within the NBS. It implemented a range of activities to improve statistics use by policy-makers and other key users, as well as made data access and dissemination consistent with the international best practices. Efficiency: The Project exceeded its objectives within established timeframes and costs and has demonstrated good efficiency. The Project strived to meet its expected outputs by looking for cost-efficiencies in Project operations and utilizing prudent business practices.
Impact: The Project activities contributed to systemic changes in the field of statistics and evidence-based policy making in Moldova.
Sustainability: there is a substantive level of ownership of the Project within the NBS, at both management and staff levels who enhanced their skills and confidence to carry the reform process forward. The NBS management and staff were very clear in identifying their strengths and needs and were open to exploring new statistical tools, processes, methodologies and adopt successful international experiences. Sustainability of the Project, however, is tightly connected with the degree of political commitment to public administration reforms and recognition of the importance of statistics in policy making and monitoring.
Recommendations for UN partners: Identify one or two priority areas to sustainable human development in Moldova. The project has a long experience in statistical capacity building, but in the evaluators` opinion a more systematic and focused approach has to be adopted for the next project. The partners have to prioritize areas that would have the most significant impact on human development in Moldova and reflect key Government priorities, international commitments and mandates of respective UN Agencies. One potential area worth exploring is sustainable human development. What is the impact of environmental changes on people’s lives and livelihoods? Which social groups and communities are most vulnerable to environmental changes? How to design and implement policies that would address both poverty reduction and social exclusion and environmental changes? Another area worth exploring is social inclusion. The project may reinforce its social exclusion study and reflect the current status of socially excluded groups. It may inform development of National Action Plan (NAP) against poverty and social exclusion that could be aligned in principle with the common EU guidelines in the area of social exclusion. Such areas as income poverty, long-term unemployment, health and lifelong learning and others could be covered in depth.
• Fragmented and uncoordinated capacity building activities do not result in sustainable improvements in statistical production, dissemination and use. A holistic approach to reforms is necessary as the improvements are needed in all parts of the system, including line ministries and other producers and users of statistics.
• Reform of statistics cycle is a time consuming process that requires continuous commitment of multiple stakeholders and a strong political will. The reforms of statistics should be linked to promotion of evidence-based decision making culture and should have strong support of the central public administration.
• NBS has the necessary authority under the current legislation to strengthen capacities of line ministries in administrative data production and use, but central public administration support is necessary to ensure that this function is effectively performed as required.
• The ownership by the NBS and close coordination of all stakeholders are critical to Project success. The improvements of the national statistical system should have a clear focus on users. Users should be clearly identified and supported through diverse awareness raising and capacity building interventions to demonstrate through practical examples the benefits of quality statistics and how to use the data. Although the analytical work and supports were undertaken for the benefit of data users (e.g., how to read and use statistical data), it is insufficient to conduct seminars, roundtables and other ad-hoc activities to support them. The interventions should be systemic and comprehensive, with some practical hands-on exercises tailored to users’ data needs.
• The biggest challenge of any interventions in the area of statistics is ensuring sustainability of reform efforts. It is not sufficient to ensure a broad partners’ buy in of reforms and strengthening capacity of key partners is needed to ensure sustainability.
Full report in PDF
PDF files require Acrobat Reader.