We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2013 Costa Rica: Assessment of PANI's Institutional Strategic Plan (ISP) 2008-2012



Author: José Pablo González R., MBA; Edwin F. Vega, MBA; Dennis B. Alvarado, MBA; Mac Arthur F. Murillo, MBA; Álvaro García Mora, Ph.D.

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

The Institutional Strategic Plan of the National Child Welfare Institute (PANI) 2008-2012 marked a significant change in the work of PANI, for two reasons: (1) it is the first time the institution has a planning mechanism to guide its work, and (2) it used a participatory approach in the definition of its contents, as requested by the Ministry of Planning and Economic Policy and the Comptroller General of the Republic.  In its general structure, the plan sets the Mission, Values and Principles, defines lines of action (Directorate, Attention, Advocacy, Prevention, Defense, Warranty Protection Organization, Management and Administrative Support), areas of impact (external and internal) and levels of institutionalization (Regulatory, Organizational and Socio - Political).

By the end of the 2008-2012 Institutional Strategic Plan, it was necessary to conduct an assessment that could provide useful information on the strengths and weaknesses of the strategic plan  and its implementation process within the  Institutional Management Model; , the role that the different departments and offices of PANI played in its  implementation, taking into consideration the  expectations of the different actors of the SNPI and particularly of the target population, to proceed with the recommendations for a better institutional  performance.

Purpose/Objective:

To develop a comprehensive assessment process of the Institutional Strategic Plan (ISP), for the purpose of identifying opportunities for organizational improvement.

Methodology:

The fundamental concepts and criteria used in the evaluation process meet the commonly accepted processes of social impact assessment (relevance, effectiveness, efficiency, impact, and sustainability); these concepts are in turn, based on independent criteria and objectivity, with clear priorities and transparency as well as positive and purposeful construction.

PANI, as a priority, commissioned a high-level technical committee that worked as a counterpart to the consultancy responsible for the evaluation. The committee monitored and participated in the evaluation process.

The research process was based on four aspects:

Documentary Research: Based on the review of documents, papers, presentations, laws, regulations, documents from the institution and other sources related to the subject at hand.

Interviews and meetings with end users and experts: The research was strengthened through unstructured interviews and conversations with officials, beneficiaries (children, young families and some government agencies), subject matter experts, and officials from various institutions.

Interdisciplinary work sessions: Together with the support team, the consultants developed group work sessions in which the progress of the evaluation was discussed as well as the progress in achieving goals, activities and tasks performed. Research topics were also discussed as well as achievements and results. To achieve adequate coverage, regional workshops and specific work sessions with regional directors and local administrative offices were conducted. In the case of organizations participating in workshops, they were selected by the consulting team from lists provided by PANI. Selection was made according to three criteria: regionalization, type of activity and random selection.

Observation by consultant: This process intended to obtain the particular reflection of the consultant on the matter, using the consultant's experience extensive knowledge on related issues.

Findings and Conclusions:

The philosophical model chosen by PANI during the process of developing the ISP is valid, and is a practice that identifies with strategic planning. However, mixing strategic priorities with institutional processes can obscure strategic issues and confuse the objectives of the organizational work processes.

When this evaluation was carried out, it was clear that PANI had failed to effectively balance its use of budget, human, technological, procedural, and infrastructural resources for the purpose of improving the management of their projects and programmes that intend to address the issues surrounding the care of the children and adolescents as well as the protection of their rights.

With regard to the strategic goals, there is no way of successfully measuring what the ISP intended, as it lacks quantitative indicators to determine whether or not the intended goal was achieved. However, in each of the strategic areas, we performed an analysis of the specific objectives and actions taken to meet objectives, considering how the restrictions noted above affected them in terms of quality.

In relation to the model of client services, the PANI defined a client services model based on a comprehensive protection system for the purposes of guiding its institutional performance. The model is in the process of being implemented, and thus it cannot be determined to be producing concrete results since not all offices have implemented it. However, the model is consistent with the philosophical framework of the National Policy of NNA.

About the organizational and functional structure, beyond its rigidity, hinders the process of coordination of activities, decision-making and the interrelations of effective institutional mechanisms amongst other institutions and allies.

Judging by the way it was defined in the 2008-2012 period, there is a limited relationship between the budget, the institutional strategic plans, and the operational plans.

Recommendations:

In developing the new ISP, we must take into account the National Policy for Children and Adolescents, and we could even consider longer-term planning in line with the 2009-2021 term.

The priority actions and indicators are fully consistent with the findings made in 2008 by the STAP with regard to the vision, mission, principles, and definition. We call attention to the emphasis on the assessment in order to keep these components at the strategic level only.

 An analysis must be performed if the "principles" incorporated in the law are consistent with the current values of the institution under the definition of the ISP Annex: "VALUES: Principles, beliefs, convictions and rules of conduct that guide the actions of the organization"

Incorporating the executive area in the 2008-2012 ISP was appropriate and important to clarify the scope of what constituted the management concept of in protecting the rights of Children and establishing the future framework of coordination with other PANI institutions. However, future ISPs should consider if this is relevant since this item corresponds to strategic actions that give sustenance to other objectives.

PANI should be particularly careful to encourage adoption and implementation of this model more rapidly.

Whether PANI has the necessary resources to successfully implement the selected management model should be evaluated.

The search for structures that are adequate to current strategies is causing organizations to adopt process-based models that enable an ongoing review process. In light of this management model and the processes that sustain, it is important to redesign work processes that adjust to the organizational strategy.

Here the situation is similar to that of the operational plans and budget. In this particular case, budget estimates are defined separately without specifically reflecting which funds will be allocated to strategic priorities. The direct relationship between the two is blurred.

Lessons Learned:

In the definition of an ISP, it is essential to establish indicators closely linked to large corporate goals. Creating too many "indicators" may induce the measurement of strategic actions, without necessarily establishing the scope of the strategic objectives, which is the reason for measuring and evaluating an ISP.

The ISP design is essential with regard to establishing priorities for the strategic objectives, including the setting of priorities and the establishment of an action plan when appropriate. Not all objectives should be treated in the same manner or priority.

The ISP must provide comprehensive reviews and documentation on a regular basis (i.e.: Annually) to establish the appropriate actions towards achieving the objectives or justify the setting of new goals.

Creating a policy of accountability at all institutional levels can have a positive impact on the implementation of strategic and operational plans. Its absence causes the diffusion of efforts. The proper definition of responsibility within the scope of the indicators can facilitate, to some extent, a compromise and a clear impact on the operational levels of the organization.

The budget should be subject to the strategic and operational plans at the institutional level in order to allow an allocation of resources to each of its strategic goals. In this fashion, greater efficiency and effectiveness can be achieved when it comes to using financial resources to meet strategic objectives.

With regard to the evaluation process, we can say that it was smooth and participatory in the political, technical and institutional levels. This allowed us to achieve a comprehensive assessment that was highly objective and reflective. The approval of the Evaluation Report was handled by the Executive Board by unanimous vote. PANI is currently in the planning stage of the new ISP and is accepting the recommendations and considerations provided by the evaluation.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2013

Country:
Costa Rica

Region:
LACRO

Theme:
Child Protection

Partners:
National Child Welfare Institute of Costa Rica (PANI)

Type:
Evaluation

Language:
Spanish

New enhanced search