Evaluation database

Evaluation report

2000 ZAM: Review of UNICEF Support to Malaria Control in Zambia



Author: Hill, J.; Webster, J.; Basimike, M.; The Malaria Consortium

Executive summary

Background

The Community-based Malaria Prevention and Control Programme (CBMPCP) began as a pilot project in Samfya District, Luapula Province, in 1994 with support from UNICEF. In 1998-1999, implementation was expanded to the rest of the communities in Samfya and to communities in three more districts in Luapula Province - Mansa, Mwense and Nchelenge. The focus of the programme is on community-based sale of insecticide-treated mosquito nets (ITNs) through community health workers (CHWs), capacity building of health workers in malaria case management through Integrated Management of Childhood Illness (IMCI), training of CHWs in malaria case management and community-based information, education and communication (IEC). This review is being conducted prior to expanding the programme to 28 districts in the Eastern, Northern and North-Western provinces. It was also tasked to review UNICEF support to malaria control in Zambia in general, including the plan for expanding CBMPCP including capacity building.

Purpose / Objective

Specific objectives in the Terms of Reference were:
- To conduct a rapid review of the progress and processes of implementation of the CBMPCP in the four initial districts in Luapula province (Samfya, Mansa, Mwense and Nchelenge) and make recommendations for the way forward.
- To review UNICEF support to malaria control in Zambia in general, as well as the action plan for expansion of the CBMPCP to the 28 districts in Eastern, Western, North-Western and Northern provinces, and to make recommendations - in the light of recommendations from objective 1 above. The recommendations should also feed into the planning/strategizing process for UNICEF programming for malaria for the new country programme cycle.
- To support the detailed action planning process to operationalize the national Roll Back Malaria strategy for Zambia, in collaboration with key partners in Zambia (NMCC, the Central Board of Health (CBoH), WHO, USAID, etc.). Priority areas include nationwide scaling up of the ITN component, changing the antimalarial drug policy, capacity building (including scaling up IMCI implementation), advocacy and IEC. Review of the capacity building strategy and materials for UNICEF-supported programmes.
- To assess monitoring and evaluation systems and mechanisms for malaria programmes within the systems of the Central Board of Health/NMCC and make recommendations to strengthen this component. Conduct a review of current ITN programme/revolving fund monitoring systems, identify constraints and modify, as necessary, to develop a user-friendly and workable reporting/monitoring system for the district health management team (DHMT) and health centre staff and community workers. Support NMCC to ensure its incorporation into the mandatory reporting systems of the CBoH.
- To support NMCC and CBoH to develop detailed action plans for malaria prevention and control (with a focus on ITN access) for the following vulnerable groups: pregnant women; refugees; displaced populations; orphans and vulnerable children, including child-headed households; and the very poor.

This report covers objectives 1, 2 and 4, all of which relate to the community-based malaria prevention and control programme. The reports on operationalising RBM and on malaria in vulnerable groups are covered in separate reports.

Methodology

The tools used in the present rapid review were adapted from those used in the mid-term review of 1996. These tools consist of interview guides for DHMT, regional health centre (RHC) staff and the MCCs. Interviews were also held with the Staff of the National Malaria Control Centre and UNICEF. A total of 59, members of 17 Malaria Control Committees (MCCs) (4 in Mansa, 3 in Mwense, 5 in Nchelenge and 5 in Samfya) were interviewed, along with several households in two districts (Mansa and Mwense). Programme documentation from UNICEF, NMCC, the DHMTs and the MCCs was reviewed. This documentation included the mid-term review, the evaluation, action plans, proposals and reports.

Key Findings and Conclusions

Essential findings are that the programme has succeeded in developing significant capacity at community level, such that some communities have been empowered to make important, innovative decisions regarding management of the project. The programme has even succeeded in serving as a pathfinder for district-based management structures within the reforms in projects where the relationship between the community (community agents, Neighbourhood Health Committees and Malaria Control Committees), staff at rural health centres and at district level (DHMTs) is working well.

In a few communities where the RHC supervisor was working closely with the MCC, there was evidence of empowerment of the community. Two communities in particular, Mambilima in Mwense and Kambuali in Nchelenge, were working with their supervisors and showing real evidence of thinking around the messages they were giving to the community. They were coming up with innovative ideas to improve what they saw as problems with the programme. There were others, but these were the communities that shone. In Kambuali, the motivating relationship was between the RHC and the MCC; in Mambilima, it was between the RHC and the DHMT.

In recent years, the national malaria control programme has suffered from several changes in line management, moving from the Central Board of Health to TDRC-Ndola in 1996 and back to CBoH again in 1999, leading to lack of continuity of programme management and programme interventions. This has been accompanied by considerable staff turnover in the Programme Manager's position, further weakening the programme.

Since 1998, UNICEF has paid the salary of the current Programme Manager, a highly motivated individual, and the programme has been considerably strengthened as a result. UNICEF has also provided significant technical assistance to the national programme, with particular focus on the monitoring and evaluation of the CBMPCP in Luapula Province. The UNICEF Health Project Officer has provided ongoing technical and management support to NMCC in the development of the national programme (such as the development of the malaria situation analysis and the RBM strategic plan, and participating in selected technical working groups) and in managing the community-based programme.

The CBMPCP strategy relies very heavily on staff in DHMTs, RHCs and NHCs to implement the programme. However, it is clear from the current rapid review and previous evaluations of the programme (1996, 1998) that these institutions are generally weak, with infrequent meetings and almost non-existent supervision of programme activities or follow-up. Secondly, the strategy places an additional burden on health centre staff in particular, and also at the district, at a time when they are facing considerable staff shortages and are already overloaded. Of particular concern is the capability of these staff to deliver on project activities when health facilities are struggling to meet even basic health service requirements. The success of CBMPCP in any district, therefore, currently depends on the strength of the DHMT.

In general, supervision and monitoring of the project was found to be poor at all levels. At the DHMT level, for example, not a single tool was found that could be used to monitor the progress of the programme. None of the districts visited was able to give information on the quantity of nets and insecticides received since the inception of the me in their districts. There was a total lack of instruments to monitor stock as well as progress in net selling/coverage per district. Tools are desperately needed at this level. The DHMT Mwense identified a lack of forms for reporting and the prohibitive cost of photocopying them for so many communities as a major constraint in monitoring at community level. This was also mentioned by Mambilima MCC, Mwense District.

The poor supervision and monitoring of the programme have contributed to the low rates of cost recovery due to leakage of nets and insecticide from the revolving fund. The reasons for these losses include damaged nets, stolen nets, perceived ineffectiveness of insecticide, stolen money, burning of money in house-fires, loss of money through the giving of ITNs on credit.

In the 4 districts of Mansa, Mwense, Samfya and Nchelenge, there have been a total of 96 DHMT and RHC staff trained in the CBMPCP, and 666 malaria agents. A total of 96,200 nets have been procured by the programme, 54,223 of which are reported to have been distributed to the districts, leaving 41,977 nets undistributed. Four thousand of these nets were still in FAO storage (at the time of the consultancy), leaving 37,977 (39% of the total nets) unaccounted for. Based on the number of nets sold by the CBMPCP since January 1996, it would take 415 years to achieve 100% coverage of all Zambian children under five and pregnant women using the current approach.

ITN retreatments have been negligible in the expanded programme. Of the four communities visited that should have been carrying out retreatments for the last two months, only 10 retreatments have been conducted. This gave a less than 1% retreatment rate. There are many possible reasons for this near absence of retreatment; however, the reason given by every agent interviewed was that the ICON was not working. Several communities had even stopped selling nets because they did not believe that the ICON was effective anymore. We do not know whether the ICON was actually effective or not. There has been no entomological monitoring of this programme in the 6 years it has been running. The first review of the programme in September 1996, recommended that both bioassay and insecticide resistance kits be purchased from WHO. Bioassay kits were purchased, but they were never used. Although the effective life of the ICON is accepted as 12 months, there has been no operational research to support this in any of the districts.

UNICEF is responsible for the procurement of all logistics for the programme; these logistics include mosquito nets, insecticide, bicycles, basins, jugs, gloves, stationery for MCCs and CHWs, calculators and cash boxes. There have been delays in the procurement and distribution of logistics to the districts; in some communities, the period of time between their training and receiving of logistics has been as long as 6 months.

Net sales, in general, have been very low. The malaria agents feel that this is an affordability problem. Although questions directed at willingness to pay were included in the KAP surveys in the original communities in Samfya District and in the pilot communities in Mansa, Mwense and Nchelenge Districts, no affordability studies have been carried out. There was one question in the knowledge, attitudes and practice (KAP) study on household income, but this was not a well-designed question. This places the cost of an ITN at around one third of the average monthly expenditure of nearly 70% of the population of Luapula Province.

The sustainability of the revolving fund was threatened when the barter system was in place, due to the inability of communities to convert goods into cash. However, a removal of this system from the projects may be a major factor in the poor sales of nets among the target population, ranging from 15.4% in Samfya District (the original pilot district, initiated in 1994) to 2.5% in Nchelenge. This, and an independent review of CBMPCP, reveal that most of the nets have been purchased and used by men. A small, rapid household survey is recommended in order to determine barriers to purchasing nets, net use among target populations and net retreatment. The cost recovery rate for the programme, as a whole, is poor, estimated at around 10%.

Recommendations

Recommendations have been made to strengthen all aspects of the existing framework for CBMPCP, covering technical and management issues at all levels of the programme. A number of tools have been developed and field tested to streamline the monitoring of programme activities. Finally, a number of key outcome indicators, which are essential to the evaluation of RBM in Zambia as a whole, are proposed for evaluating programme impact.

Based on the findings of the review, it is recommended that UNICEF support the piloting of three alternative strategies for ITNs, all of which aim to improve the targeting of pregnant women and children under five, before considering expansion of the programme. These can be implemented and evaluated in 2001, during which time the existing community projects are strengthened according to specific recommendations. Three different approaches to improve targeting of vulnerable groups will be explored in three 'new' districts.
- In the first, the impact of improved IEC strategies managed by the Society for Family Health will be determined, while maintaining the existing distribution mechanism for nets and insecticide, and management of the revolving fund.
- In the second, nets and insecticide will be provided to vulnerable groups through ANC and MCH clinics at highly subsidised rates or, preferably, free (free treated net plus first retreatment) so that there is no financial management involved and no cost recovery. SFH manages the IEC component.
- In the third, it is proposed that SFH will provide procurement and distribution services as well as coordinate the IEC strategy. Where possible, nets are supplied to vulnerable groups at retail outlets based on the presentation of a valid health card and/or a voucher received at clinics. Of course, this is only feasible where there are retailers, but the network of retailers is likely to expand significantly in the near future. In the meantime, clinics will be the main outlet for treated nets and net retreatment.

It is proposed that the pilot projects be evaluated after a period of one year, and the findings used to redefine the framework of CBMCP prior to expanding to 28 new districts in 2002, at the start of the new UNICEF country programme cycle.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Date:
2000

Region:
ESARO

Country:
Zambia

Type:
Evaluation

Theme:
Health - Other

Partners:

PIDB:

Follow Up:

Language:
English

Sequence Number:
2000/800

New enhanced search