Evaluation database

Evaluation report

2001 UGD: Assessing the Capacity-Building Process in Uganda



Executive summary

Background

This Assessment was carried out at the end of the fifth year of the 1995-2000 Government of Uganda (GoU) - UNICEF Country Programme (CP). It focuses on capacity building (CB) as a key strategy of the CP and situates this specifically in the context of Uganda's decentralisation policy. The Uganda CP is of particular significance within UNICEF as an example of a programme with a very explicit focus on CB, targeting capacity at various levels of society and experimenting with a cross-sectoral programme as part of the CB strategy. At the time of the Mid-Term Review, the programme was viewed by evaluators and programme managers as a "radical departure from previous vertical, sectorally-based, goal-oriented programmes" (UNICEF Uganda, 1997).

The Assessment is, therefore, intended to feed into discussions around the formulation of the new CP for the next programming cycle. The Assessment is also intended to contribute to a wider UNICEF initiative to more systematically learn from UNICEF experience in capacity building strategies, as well as to develop methodologies for the monitoring and evaluation of capacity building. As such, the Assessment draws from ongoing work on the development of analytical frameworks and evaluation tools on capacity building, and has served to test and further shape these. It is hoped that the Assessment also will contribute lessons on capacity building of broader relevance for UNICEF globally.

Purpose / Objective

The Assessment strives to answer three broad questions:
- Was the approach to CB appropriate?
- Was the approach to CB effective?
- What is the baseline of capacity in broad terms for the next CP cycle?

In addition, it was agreed that a process objective should be to contribute to establishing a broad consensus around a framework that would serve the Country Programme Management Team (CPMT) in future programming, both for assessment of capacity and evaluation of capacity building. The dialogue around analytical frameworks would ideally involve a wide range of key partners at various levels. The Conceptual Framework, which is provided, presents a general understanding of the interrelated factors that influence capacity. A number of important considerations about CB follow from this.

Methodology

Two levels of analysis were used. A broad level of analysis was carried out in relation to the appropriateness of the CB strategy of the CP and its four programmes (a mix between multi-sectoral and cross-sectoral programmes.) This level of exploration was supported by a desk review, a limited number of key informant interviews at national level as well as consultative meetings. By contrast, in order to examine the effectiveness of CB strategies, which required new data collection at field level, it was necessary to narrow the scope.

Special attention was given to the one cross-cutting programme, the Communication, Coordination and Advocacy (CCA) Programme, to address concern over the linkage between broad cross-cutting CB efforts of the CCA programme and more substantive development goals vis-à-vis changes in children's and women's rights -- a concern which had been raised in the MTR.

In addition, the evaluation considered a selection of interventions -- one from each programme: Health; Water and Environmental Sanitation; and Basic Education, Child Care and Adolescent Development -- based on the UNICEF CO section chiefs' perspective on what would be critical issues or facets of programming in the next CP cycle.

Given that the focus of data collection was on understanding the hows and whys of the implementation of CB, the methodologies adopted were designed for depth of analysis as opposed to wide coverage. Three districts were selected where at least 3 programmes could be found operating together, whether or not this was at different administrative levels. Within each district, contrasting strong and weak cases were selected at sub-county level and at least one additional level, parish or village, depending on where programme strategies extended. For each District, key informants, focus group members and participants of the stakeholder consultation workshops were selected, representing a range of perspectives.

Key Findings and Conclusions

APPROPRIATENESS OF THE CB STRATEGY

Implementation:
Inherent in the CP is a very clear articulation of the causal links between the policy environment, supporting national functions, key functions at service delivery and community levels, and eventual substantive changes in children's and women's rights. The CP was designed with a fairly explicit systems view. This conceptual perspective appears to have provided the framework within which the CPMT was able to evolve a fairly coherent, multi-level CB strategy, even in the absence of an explicit definition or conceptual model for CB.

The CP proposes a very powerful orientation that responds to the question "capacity for whom?" The implicit CB strategy proposed entails a strong focus on community capacity. This means that all the CB interventions at different levels - for example, with service delivery providers, facilitators, managers, and planners - are viewed in terms of how they strengthen capacity at community level. This has introduced a conceptual shift by which families and communities can effectively be promoted as actors as opposed to "beneficiaries," and is consistent human rights-based programming.

This orientation appears to have pushed or guided the CPMT in their adjustments and improvements of their CB strategy over the course of the CP. The trend in adjustments, albeit not perfectly achieved, has been towards a greater marriage of bottom-up and top-down approaches to programming. Over the course of the last five years, the CPMT has tweaked and adjusted the CB strategy, introducing some very important positive changes in this direction:
- the training of district and sub-country planners in PRA techniques to support their skills in involving community members in district planning processes
- establishment and training of PDCs to ensure involvement of village and parish representatives in problem solving, planning, social mobilisation, management of parish level health activities
- establishment and training of village water source management committees to support village level role in management
- linking CBMIS to PDCs and decentralisation of the Village Infrastructure Inventory system to LCII and LCI, both supporting decentralised information management functions

The CB strategies in all programmes have also made an important effort to identify women and youth as important actors who must be involved in key functions at decentralised levels, and have addressed this through defining the membership of organisational structures, efforts to sensitise other actors involved and the skills training provided to the organisation members. This is significant as an effort to broaden participation in development process with attention to marginalised actors.

Across the programmes, the selection of activities does seem to address a wide range of constraints or weaknesses in relation to key functions, some of which have the potential for system-wide impact. This includes activities to develop standards, guidelines and information systems (tools for accountability systems, CBMIS, VII,), as well as system-wide development of organisational structures to support roles of newly involved actors (LCS, PDCs, village water source management committees). This, in fact, is a strength noted in three of the four key CB interventions selected for closer study -- the CCA support to resource allocation, the Health programme support to the health management committees and the WES support to MIS -- all entail activities targeting a wide range of capacity constraints.

Responsive to decentralize policy: In general, the implicit CB strategy has been supportive of, and consistent with, the GoU decentralisation policy. The CB strategy also demonstrates an effort to respond to the context of insecurity.

Design:
Complex CP Management structure - Management of the three-dimensional CP framework was complex. It did not favour the clear conceptual distinction between levels of intervention, functions or capacities, actors targeted, and the particular barriers or constraints to be addressed in relation to each function. These distinctions could be partially inferred from programme models, but were often blurred.

Demand approach vs. targeting - Following from this, the CB strategy does not appear to have clearly identified NGOs and CBOs as key actors to be involved in planning or information management functions, or to have contemplated their own capacity constraints. Similarly, apart from the WES programme, the CB strategy does not appear to have adequately contemplated harmonisation with the powerful international actors at district levels as a capacity constraint or opportunity.

Cascading approach compromises results - The strong conceptual focus of the CB strategy on eventual results for community capacity and realisation of human rights was weakened by the methodology that heavily relied on cascading. This appears to have led to a lack of clarity about entry levels and realistic expectations in terms of results at community or village level.

Weak complementary and integration of CB interventions - The CB strategy in and of itself did not make the necessary links across programmes, particularly around efforts to strengthen key complementary functions at each level of entry point. This entailed missed opportunities
- in linking resource mobilisation and planning functions coherently across sectors;
- in finding connections between management information systems such as the SCS, CBMIS and VII
- in linking newly formed or revitalised organisational structures such as village water source committees, PDCs, SCHCs, district health committees and as actors within a network

The criteria for targeting programme interventions based on district demand has, in fact, undermined the CB strategy's potential for coherent linkages across programmes.

In general, the CB strategy has inadequately taken into consideration two key aspects of the Uganda country context -- the necessary time lapse for the effective realisation of the decentralisation strategy and the significant presence of international actors active at decentralised levels.

THE EFFECTIVENESS OF THE CB STRATEGY

At a general level, the conclusions are as follows:
It appears that the CB strategy was pursued in a flexible manner allowing adjustment and refinement in-course. It is encouraging to note that the mechanisms that favoured this were part of the usual programme process put to good use -- a broad programme approach in planning with logically-linked interventions at different levels addressing complementary objectives under an overall goal; review processes, particularly the MTR, involving key stakeholders and thus validating analysis and corresponding programme adjustment

Drawing also from findings discussed in the above section, it appears that the CB strategy was pursued in a manner that served to broaden the networks around CP goals and allowed small, but positive, advances to a more inclusive human-rights based approach to the CP goals.

However, one area of general concern was raised - It seems clear that there is a need for training approaches at decentralised levels, in particular, that are much more rooted in local context and practice. This would better allow for adaptation to the stage of capacity growth in different localities, as well potentially to greater sense of involvement and ownership.

More specifically in relation to each of the CB interventions selected for study:

CCA Programme -- Resource allocation
The CB interventions around resource allocation, planning and information management systems do appear to have contributed positively towards a greater awareness of the issues affecting children's and women's rights. The interventions did support new and stronger roles for the NCC and DPUs. The attitudes and traditional roles of actors at district and central levels proved to be a constraint to the promotion of new roles for actors at more decentralised levels through the CBMIS and SCS approaches. Results in terms of the actual contribution to increased allocation of resources to social services according to need have been demonstrated. However, some examples of positive results in mobilising resources for COPE and health issues do suggest that there is promise.

Health Programme -- Health management committees
The intervention has made advances in establishing a functioning organisational structure at parish level in the PDC, which has the potential to make a significant impact on the participation of women in decision making. The same PDC structure has contributed to establishing an effective role for parish level actors in problem analysis, planning, resource management and advocacy/sensitisation in the community. There is evidence that the PDCs, HUMCs and SCHCs have contributed to better relations between community and health services, and improved management of health resources (staff incentives and user fees). By contrast: Across these structures, the fulfilment of the functions of coordination, supervision/ follow-up support and resource mobilisation needs attention; and the decreased immunization coverage and the reliance on PDCs in the new National Plan for Revitalisation of Immunization points to the importance of monitoring the development of PDCs, HUMCs and SCHCs and, above all, ensuring support for supervision mechanisms.

WES Programme -- MIS
Need to explore integration of WES MIS with other existing MIS
In the WES intervention to strengthen MIS, efforts have clearly advanced to revise and link village and national roles more effectively. However, no evidence was found of an awareness at decentralised levels about the potentially important role in decentralised information systems or their contribution to better planning and resource allocation.

BECCAD -- Awareness and sensitivity to child rights
Increased awareness of children's rights among leaders
The broader efforts to raise awareness and sensitivity to child rights do appear to have some effect on attitudes. In particular, there is greater awareness of the issue of child abuse and responsibilities to respond and protect children. There is some preliminary indication that the content of the understanding absorbed may be problematic, particularly in relation to parents and community roles in the context of the best interests of the child, the need to respect the views of children and the right of children to participate.

Recommendations

For the Country Programme Management Team:
In developing the new CP, it seems critical that UNICEF and national partners pursue the dialogue on an operational framework from which to develop a CB strategy. This would seem particularly important to pursue within sectoral or issue-focused networks with other donors or major actors to facilitate a more rational targeting of CB approaches.

Similarly, it will be necessary to pursue the generic capacity assessment within sectoral or issue-focused networks, including some level of rapid district level validation. This can effectively be combined with planning processes.

Within each sectoral or issue-focused programme, the new CP should clearly define: Which key capacities or functions it is striving to strengthen at which levels? Which actors are and must be effectively involved, and what interventions or strategies will be required to achieve this? In relation to each key function, at each level, what are the key constraints or gaps to be targeted, and what other constraints or gaps must be monitored? Which specific "activities" or interventions will be used to build linkages between functions, between actors (including other donors), and across sectors?

The new CP should be based on a much more simplified framework.

For UNICEF:
In planning the CP management strategies, the CO management team must recognise the existing incentive system around reporting and seek to encourage staff in examining CB results from a different perspective. This entails recognising the long-term nature of CB, pushing for meaningful qualitative analysis of progress.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Date:
2001

Region:
ESARO

Country:
Uganda

Type:
Evaluation

Theme:
Capacity building

Partners:
Government of Uganda

PIDB:
YP600

Follow Up:

Language:
English

Sequence Number:
2001/005

New enhanced search