Evaluation database

Evaluation report

2004 MOZ: Added Value and Long Term Viability of Community Councils in Zambézia Province, Mozambique



Author: Gaspar, F.; Robson, P.

Executive summary

Background:

UNICEF has supported from early 2001 to 2004 a malaria project in Zambezia implemented by World Vision. The focus has been community-level interventions through Community Councils. World Vision has also been implementing other community-level health projects in Zambezia through a total of 710 Community Councils. The other projects are HSDS (support to the health services in 8 Districts of Zambezia Province) and Ovata (a more recent health and food security project).   UNICEF was concerned about the high cost of the malaria intervention (measured by the cost per ITN distributed), about the lack of clarity about other benefits from the intervention, and about sustainability. UNICEF, therefore, contracted 2 consultants to examine the experience of Community Councils in Zambezia Province and similar health experiences in other parts of Mozambique. The purpose of the consultancy was to consider whether the experience of the Community Councils in Zambezia, as developed by World Vision, could be a potential sustainable community tool to be used in other projects. The consultants were asked to investigate whether community councils represent a realistic approach to CCD and a methodology in which UNICEF should invest more resources in the future. The consultancy was carried out in June 2004 though visits to Gaza and Zambezia Province, and through interviews in Maputo with UNICEF staff, and staff of other agencies and Mozambique Government officials.

Purpose/Objective:

  • To assess the added value and long-term viability/sustainability of the CC in Zambezia province related to the malaria project in particular, and as a means to supporting community capacity development strategies in general and make recommendations about future implementation.
  • Determine the added value of using community councils in terms of implementing a malaria project as the one being implemented.
  • Determine if there is a real potential to use the existing CCs to support other health-related activities e.g. improved nutrition and care, increased attendance at immunisation sessions, activities in the field of girls' education, identification of vulnerable families etc.

Methodology:

The consultants visited Zambezia Province from 7th June 2004 to 18th June 2004 to observe the operation of Community Councils that are part of World Vision health projects. The projects are the malaria project, financed by UNICEF, and HSDS and Ovata, financed by USAID.

Three Districts of Zambezia were selected with different characteristics, namely Guruè, Mocuba and Morrumbala and a total of twelve Community Councils were visited in these Districts:

  • Guruè 5
  • Mocuba 3
  • Morrumbala 4

The most important difference between the Districts is that Mocuba was part of the first phase of the malaria project, Morrumbala the second, and Guruè the third phase. The Community Councils that were visited were the following:

  • 09 June: Macuarro, Intuba and Muximua, District of Guruè
  • 10 June: Muahupo and Namiepe, District of Guruè
  • 12 June: Sassamanja and Mbuma 1, District of Mocuba
  • 13 June: Mpasso, District of Mocuba
  • 14 June: Maço and Mecange, District of Morrumbala
  • 15 June: Murda and Vungantinta, District of Morrumbala  

In-depth group interviews were carried out with these Community Councils, each one lasting over two hours. Each Council consists of members and volunteers; the two were interviewed together.

Interviews were also carried out with personnel of the Ministry of Health and World Vision in Maputo, Quelimane and in each District visited. Personnel of Project Hope were also interviewed in Maputo and in Ile District (Zambezia).
One of the consultants also visited, with UNICEF staff, the project implemented in Gaza Province by World Relief.

Findings and Conclusions:

1. Developing an effective ITN distribution and re-treatment system
ITNs have been shown to be an efficacious intervention against malaria. They are probably the most important tool for controlling malaria, given the lack of a vaccine, resistance to chloroquine and the complexities of other vector-control mechanisms.
  Achievement of the goal of the Zambézia malaria project (reduced burden of malaria in pregnant women and children under 5 in Zambézia) and its purpose (effective community-based strategies for malaria and treatment adopted) requires an effort to find effective sustainable strategies to make ITNs accessible to the population of Zambézia Province (many of whom live some distance from health units and normal commercial networks) and promoting their effective use.

The project documents of the Zambézia malaria project that are available to the consultants do not state clearly how a sustainable effective distribution strategy is to be developed through the project. They do not indicate how the experience should be used to develop a national policy on ITN distribution, nor who should take the lead in studying how the distributions systems are operating and how they could be improved. The documents do not state clearly how it is intended to use the money raised from the sale of ITNs and kits, and whether it will serve as a rolling fund for further purchases of ITNs and kits (which is the normal pre-condition for a sustainable system); they do not state how such a fund will be monitored and whether UNICEF (which has contributed to the initial funding) will be able to monitor the status and use of this fund; they do not state how the fund will be recapitalised if there are losses due to subsidies, transport costs or devaluation of the local currency.

The documents do not state what information will be collected, and by which partners: they do not make explicit how such data will be used. In practice, there seem to be gaps in information: it is unclear what data have been collected about far the geographical distribution of sales through PSI (which make up the majority of sales of ITNs), the real costs of distribution through various systems, and how that varies between different parts of the Province, nor about the costs of the different components (such as transport costs, storage, education, advertising, shopkeepers' margins).

2. Community Councils as part of an effective ITN distribution and re-treatment system
Community Councils are a potential way of improving access to ITNs and insecticide kits in areas that are distant from Health Units and normal commercial networks. They can potentially improve geographical coverage, and the sale of ITNs and kits fits in well with their other activities and their role in raising awareness of health and other development issues.

The malaria project in Zambezia could have had benefits in addition to the distribution of ITNs, if its work with Community Councils had been planned in an integrated way with the long-term plan by the DPS and World Vision for development of Community Councils in the Province. In practice, there has been little added value as many Councils developed by this programme will now be difficult to integrate into a plan of supervision of Councils by health staff or World Vision. Those Councils formed as part of HSDS and Ovata have made greater progress in consolidating their activities. If World Vision continues to accompany their activities and to link them in to other projects and State institutions, there is some chance that they will be sustainable. Those that were created for the malaria project (and not linked in to other projects) have a lower probability of being sustainable, as these tend to be weaker and there has been no strategy for the continuation and consolidation and linkage to other institutions.

Creating Councils for a limited number of activities implies poor cost effectiveness, and creates frustration among members and volunteers when they have completed these few activities. In some Districts, the Zambézia malaria project has created large numbers of Councils mainly for malaria activities, and their future is now very uncertain: most have sold only about 100 ITNs. However, other World Vision projects have created limited numbers of Councils that may only cover a small part of each District, and the tendency of the HSDS project has been to create Councils close to Health Units (so that there should be a strong link between the Council and the Unit): thus, use of existing Councils as one of the ITN distribution channels may leave extensive areas of a District unserved.

3. Community Capacity Development and the Human Rights' Approach to Programming
It is unclear why the Zambézia malaria project is highlighted as an HRAP or CCD project. The Zambézia malaria project does not have many of the features of HRAP and CCD projects. It has not, for example, developed an analysis of the duty-bearers and rights-holders for a sustainable ITN distribution system and what capacities they require. It has focused on a limited number of objectives and actions in a specific period: the documents of the project define goals and purposes in terms of malaria and HRAP and CCD would seem to be envisaged as a means to reach this end.

HRAP and CCD are long-term transformatory approaches that cannot be used as a means in a time-bound project with specific goals and purpose. It is challenging to combine specific goals and purposes with an HRAP or CCD, which imply a long-term engagement to build community capacity in stages and in parallel to build the capacity (and attitudes) of other actors to effectively work with communities. The Zambézia malaria project has used participative educational tools, which are undoubtedly effective but, in themselves, these do not make it a CCD project. There is a tendency, in any project, for the aspects of such tools that should create analytical capacity in communities to be neglected and for the emphasis to be on transfer of information.Many of those involved in the implementation of this project said that they were puzzled by the fact that UNICEF was highlighting this project as an HRAP or CCD project. They were in favour of transformational development, but were of the opinion that it had not been clear at the outset that this was UNICEF’s intention. Their impression was that most donors were only interested in tangible outputs (such as numbers of ITNs distributed) and that this project was no exception: their perception was that the main objective had always been ITN distribution to reduce malaria prevalence and that the provision of information to communities had been intended to support this objective. They were also of the opinion that the implementation design did not provide opportunities for such an HRAP or CCD approach.

On the other hand, some of those involved in the original design of the project (but no longer directly involved) expressed the opinion that this project initially had been intended primarily as a project of Community Capacity Development: malaria activities had been intended to be only an entry point for a longer-term process of strengthening of community's capacity for analysis, assessment and action, and the role of the volunteers had been to introduce community to CCD processes and participatory communication (rather than passing on information or giving advice). There would thus seem to be different perceptions about the original aims and the history of this project. The authors of this report are not in a position to make a judgement about which of these perceptions are correct. However, it would appear to the authors of this report that UNICEF is making claims about the malaria project as an HRAP or CCD project that are based on some possible original aims of the project, and not on the present reality of the project.

Recommendations:

1. Community Councils as part of an effective ITN distribution and re-treatment system.
Community Councils should be one of the channels of sale of ITNs and insecticide kits. However, Community Councils should not be created just for a limited range of activities: sale of ITNs and kits should be done through Councils that have already been created or that have a good prospect of long-term continued support and supervision, and inputs for a wide range of activities.

This may mean that it is difficult to have a satisfactory geographical coverage of a Province in the short-term. This, in turn, may mean that the extension of access to ITNs has to be gradual, through the gradual geographical extension of commercial sales and of Councils and identification of other channels (such other NGO projects, agricultural extension workers, churches).

2. Developing an effective ITN distribution and re-treatment system
The Mozambique office of UNICEF should decide how it could best support the Government of Mozambique in developing effective strategies for making ITNs available and ensuring their effective use (including re-treatment). As a UN agency with a mandate to work with the Government, but with also the possibility of working with other partners such as NGOs in the implementation of projects, UNICEF is in a position to support policy-level work based on experience and operational research from project implementation.

If the Mozambique office of UNICEF decides to take this approach, UNICEF malaria projects (in Zambézia or other Provinces) should have objectives that explicitly mention a sustained reduction in malaria burden, the sustained adoption of community-based strategies and a sustained improvement in access to affordable ITNs and insecticide kits (and their use). UNICEF malaria projects should clearly state the challenges in doing this, such as the remoteness of many people from commercial networks and Health Units, and that the projects will have an operational-research component to identify how a community-based strategy can best be developed. Consensus should be reached beforehand before the various partners (such as UNICEF, commercial companies, NGOs and the DPS) about the fact that operational research is required.

If UNICEF is to provide a start-up fund for an initial order of ITNs and insecticide, and if this fund is then to be used for future purchases of ITNs and insecticide, UNICEF should be in a position to monitor the state of this fund and its use. The project documents should state clearly how any losses through devaluation of the local currency will be made up, how transport costs will be absorbed and how losses due to subsidies will be absorbed.
The systems for data collection should be devised before the start of the project, which will indicate quickly how many ITNs and kits are being sold, where and to whom, how they are used. It should indicate any difficulties in distribution and costs of distribution. There should be regular meetings between the various partners to look at the monitoring data, and decide any changes required in order to meet the objectives.

The various partners should agree to obtain and analyse as much information as possible about the experience to date in Zambezia. They should agree to work together to understand the real costs of the various components (transport, education, supervision, management etc.) in different geographical areas and how these activities could be carried out in the most cost-effective manner.

3. Health challenges in Mozambique
The Mozambique office of UNICEF should decide how it could best support the Government of Mozambique in developing effective strategies for reaching people who live at a distance from Health Units. As a UN agency with a mandate to work with the Government, but with also the possibility of working with donors and with other partners such as NGOs in the implementation of projects, UNICEF is in a position to support policy-level work based on experience and operational research from project implementation.

If UNICEF decides to take this approach, UNICEF should advocate with donors to help develop a system at the community level for the delivery of a variety of health services, rather than continue to think in terms of vertical programmes delivering services to communities.
UNICEF should work with the Ministry of Health in analysing the experience with Community Councils in the areas where they have been introduced to understand better what they have achieved and any constraints.

Where UNICEF has the opportunity to be involved with projects that include Community Councils, UNICEF should advocate for objectives that include the development of a sustainable strategy for reaching people who live at a distance from Health Units. This would involve taking an operational-research approach, so as to identify how a community-based strategy can best be developed. Consensus should be reached beforehand before the various partners (such as UNICEF, commercial companies, NGOs and the DPS) about the fact that operational research is required.

An operational-research approach would need to explore how Community Councils (and other community health workers), Mobile Brigades and Health Units can share responsibilities and work together effectively. It would need to explore how much they cost to function effectively and any other constraints, in different contexts.

4. Community Capacity Development and the Human Rights' Approach to Programming
UNICEF should further invest in Community Councils as an approach to community development. However, if UNICEF is committed to an HRAP or CCD approach, the Mozambique office should understand that this implies a long-term engagement to build community capacity in stages and in parallel to build the capacity (and attitudes) of other actors to effectively work with communities. It should understand that short-term, vertical programmes, on their own, are difficult to reconcile with an HRAP or CCD approach as the costs of a long-term engagement with community institutions (and with the State and private structures that interact with them) can be difficult to justify within the frame of a vertical programme. The sustainability of Community Councils will require investing in them and in the government structures that will interact with them and support them.

It should understand that specialist technical staff of vertical programmes will have difficulty in implementing an HRAP or CCD approach unless they have the full-time support of staff with an HRAP or CCD background. The Mozambique office of UNICEF should consider more integrated inter-sectoral programming in particular areas in which different sectors can together support capacity building of communities and service providers. It should alternatively consider general CCD programmes in particular areas into which sectoral programmes fit. It should consider having specialist staff in CCD and HRAP who can design and manage CCD programmes and support sectoral staff ensuring, for example, that the role and capacity analysis of the various duty bearers is adequately carried out and capacity building plans devised. It should also consider having a presence of staff at a Provincial level in areas with CCD programmes, to improve coordination between the various actors involved in programmes (training centres, implementing agencies, government agencies, UNICEF) and to assist in monitoring inputs (quality, quantity and timeliness), progress, outputs (immediate results), outcomes (changes in behaviour, knowledge and attitudes of the various duty bearers) and impact.

The Mozambique office of UNICEF should invest in monitoring systems for CCD and HRAP programmes that cover, and distinguish between, inputs, progress, outputs, outcomes and impact. CCD programmes pay particular attention to outcomes, that is to the behaviour, knowledge and attitudes of the various parties (their willingness and ability to carry out their roles). The technique of Outcome Mapping has been designed for monitoring changes in behaviour, relationships, actions and activities by people, groups and organisations involved in programmes.

If UNICEF is committed to an HRAP or CCD approach, Community Councils are a possible entry point to a long-term programme. Community Councils do represent a realistic approach to CCD and a methodology in which   UNICEF should invest more resources in the future, though little of the potential has been realised as yet.

As a UN agency with a mandate to work with the Government, but with also the possibility of working with donors and with other partners such as NGOs in the implementation of projects, UNICEF is in a position to support policy-level work based on experience and operational research from project implementation. UNICEF could work with various Ministries to support the transformation of the present Community Councils (that focus on health activities) into Community Councils or Community Leaders' Councils with a wider remit. It could help in analysing the experience with Community Councils to understand better what they have achieved and any constraints, and in developing and ratifying a government Community Strategy that goes beyond the health field.

In particular, the Mozambique office of UNICEF should be able to provide assistance to the various Ministries in developing a "partnership for change" approach, in which Ministry staff work in cooperation with community organisations rather than an "information giving" approach. This should tackle the natural tendency of government institutions to see their links with community institutions as being a one-way "conveyor-belt" of information (and sometimes instructions) from government to communities. It should assist in the development of an understanding of the mutual rights and responsibilities of communities and those working with them.

There are many tools available that can support this change in attitude, such as the REFLECT literacy method developed by the NGO "Action Aid", the SARAR methodology and various tools for participative health education. However, tools on their do not guarantee the change in attitudes by those working with communities, and there will always be a risk that the tools will be used to support "information giving" rather than "partnership for change". UNICEF will need to accompany programmes closely so as to monitor how far, in practice, they help to develop relations of mutual respect, rights and responsibilities between communities and the various institutions working with them. This implies some change in the way in which UNICEF works in practice. Partnerships will be required with Ministries and NGOs and training institutions that, while being codified in contracts, will go beyond the "contract culture" and will involve a high level of interaction during the planning, implementation and evaluation of programmes.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Date:
2004

Region:
ESARO

Country:
Mozambique

Type:
Evaluation

Theme:
Health - Malaria

Partners:

PIDB:

Follow Up:

Language:
English

Sequence Number:
2004/005

New enhanced search