Evaluation database

Evaluation report

2004 MOZ: Evaluation of the Integrated Supplementary Feeding Programme (ISFP)



Executive summary

Background:

As a response to the impact of the triple threat of drought, HIV/AIDS and poverty/weak governance, UNICEF and WFP, upon request of the Ministry of Health (MoH), implemented an integrated supplementary feeding programme from March 2003 to May 2004, in 19 'hot spot' districts of 6 provinces. It comprised blanket distribution of CSB, participatory education for health and nutrition, vitamin A distribution, de-worming, and MUAC screening. The purpose of the evaluation was to assess the impact, value and processes of implementation of this response.

Purpose/Objective:

To evaluate the criteria, appropriateness, value, impact and effectiveness of the ISFP as a response to the triple threat of drought, HIV/AIDS and poverty and use these findings to contribute to the further development of national MoH protocols and policies for ISFPs in Mozambique, as well as for UNICEF and WFP future programming.

The specific objectives were:

  • To assess the effectiveness of the process of the Supplementary Feeding Programme in terms of coverage (Vitamin A, de-worming, MUAC screening), CSB distribution, participatory education, implementation arrangements (financial aspects, human resources);
  • To assess the impact of the ISFP looking at the nutritional status of children, and improved knowledge and practices for diarrhoea prevention and treatment, and for young child feeding;
  • To identify the key constraints of the ISFP;
  •  To analyse the value and appropriateness of the ISFPs in the context of Mozambique, the criteria for the establishment and phasing out ISFP interventions, and to review the efficacy and cost-effectiveness of the ISFP intervention in comparison with other potential alternatives;
  • To document lessons learnt for emergency preparedness and for future emergency response (short-term vs long-term) in nutritional interventions, especially in relation to ISFPs, and propose a way forward for future UNICEF/WFP programming to address emergency and underlying causes of malnutrition;
  • To contribute to the further development of national MoH protocols and policies on supplementary feeding interventions;
  • To contribute to the strengthening of WFP/UNICEF collaboration for nutritional and food security, and HIV/AIDS.

Methodology:

A random sample of the target population was enrolled in a cluster survey for the purpose of measuring the coverage of several components of the programme, to assess the current nutritional status of the population, and attempt to measure the effects/impacts of the supplementation and educational activities. The household questionnaire followed as closely as possible the relevant methodology and indicators used during the second (2002) and third (2003) multisectoral assessments, since those provided, generally speaking, reference/baseline information for this evaluation. Three other structured questionnaires were also developed for the village chiefs, community workers ("activistas", four per district), as well as district/provincial level authorities (district administrator, district and provincial health directors). These aimed at collecting information on their knowledge and involvement in the ISFP, and to note their comments and recommendations regarding its implementation. Another structured questionnaire assisted in gathering information on the nutrition units of the Health Centre mainly to assess their functioning, the availability of equipment and products, as well as get information on the number of cases of malnutrition and respective mortality. Finally, one structured questionnaire was elaborated for the participating NGOs, covering in detail the various components of the ISFP.

The plan was initially to cover 60 clusters in 18 districts of six provinces (Maputo, Gaza, Inhambane, Sofala, Manica and Tete). The study population was composed of 683 individuals: 56 community leaders, 40 community workers (18 women and 22 men) and 587 women (as focus groups). It was intended to interview more community workers but they were sometime absent at the time of the survey, or were involved with other communities not covered by the programme. Overall, 59 focus groups were organized with 587 women between 21 and 30 years of age. The majority were married with a mean of 3.5 children each, 5.2% zero to 6 months old, and 51% 7 months to 5 years of age. Each focus group had between 6 and 12 participants.

Findings and Conclusions:

A total of 133,992 children 6-59m and 69,660 pregnant/lactating women were registered into the programme. Statistics reported by the implementing NGOs point to a mean monthly coverage of registered beneficiaries with CSB of 54% over the period March 2003 to May 2004, with a mean ration of 111gr per covered beneficiary per day (children and women). For vitamin A, a mean coverage at six month intervals of about 30% was reportedly achieved over the same period (39% for the last six months), and 12% for mebendazole (14% for the last six months). The information provided permits to estimate the mean monthly coverage of MUAC screening of registered children 12-59m at 42%, while participatory education activities reportedly reached a mean of 16% of the registered adult beneficiaries per month (a mean of 81 participants per session.

The qualitative survey found that the majority of the over 800 community workers involved had a good knowledge of the programme and its components. The training of these community workers had both theoretical and practical content and included: child nutrition, preparation of the soya mix, household and children hygiene, hygiene of water and food conservation, measurement of MUAC and nutritional surveillance, as well as prevention and treatment of diarrhoea. Their tasks included the distribution of CSB, the listing of beneficiaries, home visits to the families and education of the mothers, MUAC measurement and, in some cases, vitamin A and mebendazole distribution.

There appears to have been a lack of clarity among community workers about participatory education, its instruments and methods. They mostly referred to the 'demonstration poster' as a 'content', a fact which suggests that they had not correctly understood that it is an 'instrument' for participatory education. They, nonetheless, feel that the use of such posters made comprehension easier. Community workers generally considered positively the involvement of the communities, which they attributed in great part to the good work of the community leaders. However, less than half of them considered that the mobilization of the beneficiaries had been good. They, nonetheless, felt that the programme had been well implemented and of benefit to the population. Most mothers also considered that their relationship with the community workers had been good. On a less positive note, over a third of the community workers complained that they had not been paid regularly by the NGOs or given the full amount that had been agreed upon, resulting in lack of motivation. Stated insufficiencies in transportation means for the products and the staff were also mentioned not only by the community workers but by the community leaders as well.

Community leaders were generally informed about the ISFP and had a good knowledge of the target groups. They were often involved in selecting community workers and in elaborating the lists of beneficiaries. About half of the community leaders considered good their relationship with the community workers and implementing agencies. Problems were sometime caused by the lack of clarity about who had decision power and the lack of involvement of the leaders during implementation. Nonetheless, community leaders generally regard the programme as beneficial to the population in terms of acquired knowledge and improved health conditions of mothers and children.

Measured programme activities:

An estimated 85% of the target population was reached in one way or another by the ISFP. Given the extent of the geographical area to be covered, this can be considered as a good, or even excellent, gross coverage. The Southern provinces had a significantly lower coverage than Tete (74% versus 97%). There were, however, large variations in the period and frequency of CSB distribution between areas, linked not only to the time of the signature of the agreement but also to difficulties in making CSB available on a constant basis, which placed a major constraint on implementation.

The CSB supplement was intended to be provided twice a month, or at least once a month, to the eligible families. For most of the NGOs, having taken part of the programme for a period of 8 to 11 months, a similar number of distribution rounds was necessary to provide the families with a regular supply. However, less than 15% of the families benefited from six or more distributions. Although quantities for up to two months at a time were provided, in some cases, to compensate for this irregularity, many of the registered families experienced repeated shortages over the period of the programme. While 80% of eligible families received CSB at least once, only 55% received the product three times or more (about 70% of families that were reached at least once). Once again, Tete is shown to have done better than the southern provinces. Under the qualitative study, the great majority of women who participated in the focus groups declared having received some form of food assistance from the ISFP or other programmes over the period. Given the prevailing situation, two thirds had received products under the WFP food-for-work programme. The food thus received constituted the fundamental part of the diet over that period, or was eaten in addition to the regular products.

Low levels (28%) of vitamin A supplementation in children were reached during the last six months (South 41%, Tete 14%). If we consider children without a health card as not covered, the overall coverage is reduced to 21%. Vitamin A coverage at least one time since March 2003 was 43% for children with a health card and 33% for all children. Such results indicate that vitamin A supplementation was not implemented systematically, particularly in the province of in Tete. De-worming activities had an even lower coverage.

Data from the quantitative survey show that nearly half the households were covered by participatory education activities, with a somewhat higher coverage in the southern provinces. Families with a higher number of CSB distributions had a higher likelihood of having participated in education sessions (66% for families with 3 or more distributions). Overall, 85% of the surveyed families reported having attended two participatory education sessions or less, but recall bias may be large. The qualitative study found that a large majority of the women who participated in the focus groups had attended participatory education sessions, information meetings or health/nutrition-related activities.

Gross coverage for MUAC screening (at least one measurement) was 44%, while only 5% of the surveyed children had reportedly had their MUAC measurement taken three times or more from March to June 2003. This figure is much lower than the monthly 42% coverage estimated from NGO reports. Recall bias on the part of the household informants may be large.

Estimated impact:

Taking CSB distribution as a proxy for all programme activities, households that had received the product three or more times were taken as the intervention group, and the others as comparison. The differences between the intervention and comparison groups all point towards a positive impact of the ISFP activities: 3.2% versus 5.3% for wasting, 33.4% versus 40.7% for stunting, and 18.5% versus 25.4% for underweight (statistically significant). The intervention group included about 57% of the children of the target areas. Those children benefited from more MUAC screening, from mothers more knowledgeable in health and nutrition issues having participated in interactive education activities, and from a better nutritional status of their pregnant or breastfeeding mothers. They did not, however, receive more vitamin A. It should also be noted that figures for the "comparison" group are all somewhat higher than the estimates of the VAC 2003, suggesting that the population that did not receive significant assistance from the various components of the programme, but lived in the same geographical areas, may be following a downward trend in terms of nutritional status, even though the changes are not dramatic. Additional comparative analysis of the situation of the population in the same geographical areas before the start of the ISFP – i.e. data from the 2002 VAC – and the situation at the time of this evaluation – will be undertaken at a later stage and circulated as an addendum.

The period prevalence estimates for diarrhoea or fever during the last two weeks preceding the survey are both much lower than those of the 2003 VAC and are comparable between the intervention and comparison groups. Although ISFP activities don’t appear to have impacted on the incidence of these pathologies, they may well have had a favourable impact on their outcome (malnutrition and mortality).

Mortality estimates in children under five years of age compare relatively well with those found during the 2003 VAC, although they are slightly higher (68.3/1000 person-years at risk, versus 62.6). Once again, the intervention group (39.6) fares better than the comparison (107.1) group. Although the point estimates are wide apart, the relative smallness of the sample size means that the difference is only borderline significant from the statistical point of view. These results must, therefore, be interpreted with caution. They, nonetheless, reinforce the previous findings suggesting that the supplementary feeding programme had a positive impact on the health and survival of children in areas where its activities were reasonably well implemented.

The education themes best remembered by the women who participated in the qualitative study focus groups were: preparation of the CSB, collective and household hygiene, hygiene with water and food preservation, prevention and treatment of diarrhoea, nutrition of children and the construction of latrines. The main causes of diarrhoea were stated as being the lack of hygiene, drinking non-potable water that is not treated or not boiled, eating food that is spoiled or has been kept over from the previous day, and not using a latrine, which is all very much in line with the content of the health education activities. While 70% of the quantitative study respondents whose child had diarrhoea two weeks before stated that they had given him/her an oral rehydration solution, only 29% declared having, in practice, increased fluids during the same episode. On the other hand, half the breastfeeding mothers said they had breastfed more often their child sick with diarrhoea. The mothers generally said that a child can be given food from the 2nd, 3rd or 4th month of age, depending on the needs, indicating that that particular message was not well assimilated.

Conclusions:

The ISFP was designed to avert excessive morbidity and mortality in the affected populations before they become patent, and appears to have exceeded its initial objectives in areas where it was reasonably well implemented, since it contributed to not only maintain, but improve, the nutritional status and survival of the children. As far as the targeted populations now enter a more productive agricultural cycle and are able to more fully recover during the next months and years, it can be said that the ISFP was adequate to the humanitarian crisis.

Recommendations:

  • to develop indicators and methodologies to better measure at community-level processes that indicate when communities' coping mechanisms are being stretched beyond reasonable limits, so that a timely well-targeted intervention can be put in place;
  • to better appreciate the magnitude of the task involved in such programmes and the capacity of the implementing partners;
  • to reinforce supervision; to involve more fully district and local authorities;
  • to provide more support to community workers involved in such programmes in terms of transport and regular/complete payment of their stipends; and
  • to use community workers with a sufficiently high education level so that they can better understand and implement participatory education methods.


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Date:
2004

Region:
ESARO

Country:
Mozambique

Type:
Evaluation

Theme:
Emergency – Nutrition

Partners:
WFP, Ministry of Health

PIDB:

Follow Up:

Language:
English

Sequence Number:
2004/007

New enhanced search