Evaluation database

Evaluation report

2008 Senegal: Long-term Evaluation of the Tostan Programme in Senegal: Kolda, Thies and Fatick Regions



Author: Macro International Inc.

Executive summary

Background:

In 1998-1999, Tostan initiated a capacity-building programme in the villages of Thiès/Fatick and Kolda. During that period, the Tostan programme focused on teaching the following modules: problem resolution; basic hygiene; oral rehydration and vaccination; material and financial management; leadership; feasibility study (economic projects, micro-credit); women’s health (including sexuality, child-bearing, and rights); children’s development; democracy; and sustainable management of natural resources. The outcome of the programme was to mobilize communities and have them publicly declare that they were giving up harmful practices.

Quantitative component
The quantitative component of the long-term evaluation of the Tostan programme was conducted in 2006 by the Center for Research in Human Development (CRHD) in the rural areas of the Kolda, Thiès and Fatick regions. The qualitative assessment of the Tostan programme was conducted by the Population Council staff in Dakar. In conducting the quantitative component, the CRHD utilized the technical assistance of Macro International Inc. through the international Demographic and Health Survey (DHS) programme.

Appropriate guidelines for evaluating the impact of the Tostan programme were defined in agreement with the various stakeholders. Evaluations focused on the levels and trends of early marriage and the practice of female circumcision; knowledge and behavior patterns in the area of health; knowledge of Tostan and its programme; participation in a public declaration; and participation by community members in a Tostan programme and an assessment of the benefits derived from it.

Qualitative component
In 1998-1999, Tostan initiated a capacity-building programme in the villages of Thiès/Fatick and Kolda. During that period, the Tostan programme focused on teaching the following modules: problem resolution; basic hygiene; oral rehydration and vaccination; material and financial management; leadership; feasibility study (economic projects, micro-credit); women’s health (including sexuality, child-bearing, and rights); children’s development; democracy; and sustainable management of natural resources. The outcome of the programme was to mobilize communities and have them publicly declare that they were giving up harmful practices.

The qualitative evaluation methodology is based on conversations conducted and observations made in two categories of villages: group A villages, which both applied the Tostan programme and participated in a Public Declaration (PD) to abandon circumcision, and group B villages, which only took part in a public declaration without having previously benefited from the programme; group C villages (control villages) were not visited by the qualitative component. Interviews were conducted in 12 villages. It turned out that only two group B villages could eventually be considered, as all others had been subjected to the programme at a later stage.

In total, 150 individual interviews were conducted with the following groups: women who had participated in the programme, women from either type of village who had not participated, facilitators who ensured programme implementation, and leaders and other focal points of these localities. It should be noted that collecting evidence from leaders, whether men or women, about perceptions and outcomes of the programme can elicit responses that are biased on the positive side.

The establishment of the programme in the villages followed several participatory stages: preliminary discussion, the identification and choice of participants, and the development of the programme. In the process of selecting villages, Tostan would set a certain number of conditions including the village’s commitment to take care of the facilitator, establishing lists of beneficiaries, and building a shelter for use by the class. Some informants also emphasized that the abandonment of circumcision should be set as a prerequisite. Overall, these communities significantly contributed to the establishment and the implementation of the programme in the villages.

Subsequent to the education programme, various changes occurred in the villages. Results show that the programme contributed to the improvement of knowledge, both among participating women and women who did not take part in the programme. The women emerged with greater knowledge of their rights and duties, especially regarding the place and role of women in their communities.

The organization of public declarations evolved considerably over time, even between 1996 and 2000. The information collected shows that the strategy was initially suggested by Tostan, then implemented by the women of the Malicounda class. An evolution is evident at the third declaration, which took place at Médina Chérif. The increased involvement of several parties within the community led to the organization of the PD and mobilization for the abandonment of circumcision in this village. Ultimately, the implementation of the measures announced during the PD required contributions from various social groups of the villages. Data analysis points to a collective dedication marked by a determination by communities to meet their commitments, with the involvement of leaders, committees and especially women, which would determine the scope of the declaration. However, the general population of group B villages was not really involved in the PD. A few people from these villages heard about a celebratory event that was going to take place in a neighboring village. Some people from these villages decided to go and while at this event, they received information on the PD for the abandonment of circumcision.

In villages where the programme was applied in its entirety, there was better awareness of the risks involved in circumcision, but less knowledge about the risks arising from early marriage. This translated into a mobilization of the population around a PD to give up these practices – which was perceived as the culmination of the implementation process of the Tostan programme. The data collected reveal that communities have been abandoning the practice of circumcision since the advent of the PD. However, some accounts show that there is still some resistance in the villages. Interviews revealed that early marriage is declining, but influences on this phenomenon cannot be exclusively attributed to Tostan.

Some important constraints are limiting communities in their will to capitalize on the assets provided by this programme. The formation of social groups in the villages took place before the development of the programme. The PD seems to have contributed to the strengthening of these committees as far as monitoring the measures taken and the decisions made during these declarations. However, with the passage of several years, these groups/committees have all but vanished. The lack of organized follow-up and the absence of basic infrastructures in the villages limit the full use of the new capacities that populations now have.

Methodology:

The survey focused on a sample of women who ranged in age from 15 to 49. They come from 53 rural villages in the regions of Fatick, Kolda and Thiès, where circumcisions have been common practice. Public declarations were made in these communities in 2000 or earlier. The villages were split into three categories:
• Group A villages benefited from a Tostan programme before 2000 and publicly declared that they would abandon the practice of circumcision.
• Group B villages were associated with a public declaration before 2000 but did not directly benefit from the Tostan programme. The survey team could not find group B villages in the regions of Thiès and Fatick, and subsequently selected villages in the region of Kolda, where circumcisions were being performed.
• Group C villages are the “control villages”. This group includes villages which perform circumcision but have not been directly or indirectly exposed to the Tostan programme.  A total of 600 households (200 in each village category) and about 900 women aged 15 to 49 were covered by the study.

In the Fatick and Kolda regions, about 85 per cent of women are rural. By contrast, 47 per cent of women in the Thiès region are urban. Almost half of the women in Thiès (47 per cent) have attended school, while female illiteracy is widespread in the other two regions: 68 per cent of Fatick women and 78 per cent of Kolda women never attended school. The three regions also have a very different ethnic make-up: in Fatick, 61 per cent of the population are Serers and 25 per cent Wolofs; in Kolda (not including the Department of Sédhiou), 61 per cent are Poulars and 18 per cent Mandingues; lastly, in Thiès, 49 per cent are Wolofs, and Serers account for 27 per cent.

Knowledge of Tostan and of its programme
Women who are considered to know the Tostan programme are those who declared that they knew it or heard about it. The level of knowledge about the Tostan programme is highest in group B villages (92 per cent). However, group A comes a strong second (82 per cent), while residents of group C vllages have the lowest knowledge of the programme (40 per cent).

In group A villages, the most widespread way of transmitting information is learning “through someone living in the same village,” representing 39 per cent of all cases. Other main sources of information noted by residents of group A villages are radio (cited by 25 per cent of villagers) and the presence of a Tostan officer (26 per cent). Television and other ways of transmitting information have a very low representation rate in all villages.

Maternal and child health
Indicators of Tostan’s impact on maternal and child health include the number of prenatal visits, place of delivery, vaccine coverage of the children, and the prevalence and the treatment of diarrhoea and of acute respiratory infections.

With respect to prenatal visits, group A villages do not stand out as a role model. We also observe that the percentage of women who gave birth at home (61 per cent) in group A villages is higher than in control villages (50 per cent). The percentage of immunized children is above 95 per cent in all types of villages for most vaccinations. Overall, the proportion of 12- to 23-months-old children who had all their shots ranged from 70 per cent in group A villages, to 58 per cent in group B, to 76 per cent in group C villages.

Ultimately, the quantitative analysis did not find that the programme had an impact on the health of women and children, since immunization rates and the use of health and other public services are no better in the villages where the programme was based. The intervention may have once been effective in improving the health situation in this region, but its impact may have faded after a few years, or social conditions and the condition of the services were just not conducive to effecting measurable long-term changes.

Nuptiality
Early marriage is most prevalent in group B villages, where 35 per cent of women were married before the age of 15, versus 20 per cent in group A villages. This partly reflects the different ethnic customs of the communities: most group B villages (Kolda region) are inhabited by Poulars, known for their early marriage practice, and among whom nearly all women are married before age 20.

Most marriages (54 per cent in A and B villages, and 58 per cent in C villages) occur in the 15-to-19 age group regardless of the village type. Almost no marriages occur after the age of 20 in group B villages, while in the other two groups the percentage of first-married women over age 20 reaches or exceeds 12 per cent.

Statistics reveal that the proportion of girls married before the age of 15 has decreased over the last 10 to 15 years in all three village types. The most significant declines were observed in the intervention villages (group A) for girls aged 10- to 14- years: 23 per cent of girls were married before the age of 15 in the period 10- to14- years before the survey, 16 per cent in the period 5- to 9- years before the survey and 12 per cent in the period 0- to 4- years before the survey. In the control villages, marriages of girls under age 15 fell from 18 per cent to 13 per cent in the last 10 to 15 years. A comparison of group A and group C villages following intervention reveals that public declarations and Tostan programmes may have combined to slightly lower the prevalence rate of marriages under age 15. However, when one examines the overall marriage rate for girls under 18, the difference between the group A villages and the control villages disappears.

Knowledge and practice of female circumcision
The proportion of women who know about circumcision is higher in group C villages (99 per cent) in all age groups. Women in group A and group B villages have roughly the same level of awareness about circumcision, with 90 per cent and 92 per cent of women, respectively, professing knowledge. The proportion of women who underwent circumcision is lowest in group A villages (64 per cent), followed by group B (81 per cent) and group C villages (87 per cent).  The starting point for assessing changes in the rate of circumcision was therefore lower in Tostan intervention villages (group A).

A lower percentage of women in group A (30 per cent) than in group C villages (69 per cent) declared that at least one of their daughters underwent the procedure. Among women whose daughters did not undergo the procedure, three times as many mothers in group A than in group C villages declared that they did not intend to have it done.

Statistical results on circumcision performed on girls reveal that its prevalence has decreased in Tostan intervention villages and in villages that took part in a public declaration (groups A and B). For the 0-to-9 age group, 15 per cent of girls underwent circumcision in group A villages, versus 8 per cent in group B villages and 47 per cent in group C villages. We clearly see a result of participation in a public declaration in the seven Kolda villages (group B) even without the Tostan programme. In conclusion, circumcision still exists in all villages, but its frequency has strongly decreased in intervention villages. By contrast, in control villages, practices and opinions remain favorable to circumcision.

Purpose/Objectives:

This evaluation aimed at assessing the long-term impact of the Tostan programme in the Kolda, Thiès and Fatick regions of Senegal.

The main objective of the study’s quantitative component is to see whether it can be statistically established that the Tostan programme has had an impact on the prevalence rate of circumcision among girls and on their age at their first marriage, and to assess whether the health status of mothers and children has improved.

The main objective of the study’s qualitative component is to examine Tostan’s establishment process in the villages, to understand how villages organized their participation in public declarations, and to learn what women have to say about the impact of the Tostan programme.

This report presents the methodology and results of an evaluation aimed at assessing the long-term impact of the Tostan programme in the Kolda, Thiès and Fatick regions of Senegal. The survey was conducted in 2006 for UNICEF’s New York office under the leadership of Macro International Inc. For the implementation of this survey, the Senegal Ministry of Women, the Family and Social Development, which sponsored the research, set up a steering committee in charge of monitoring and supporting the technical team.

This evaluation is the result of a collaborative joint effort between several institutions: UNICEF, Macro International Inc., the Population Council, the Center for Research in Human Development (CRHD) and Tostan. With funding from UNICEF, Macro International Inc. conducted this evaluation with the assistance of the Population Council and CRHD. USAID, the second financial partner, provided the human resources support required for this evaluation through the Frontiers Programme of Reproductive Health implemented by the Population Council.

This study has two components, a quantitative and a qualitative one, conducted respectively by CRHD and the Population Council office in Dakar. In the implementation of the quantitative component, the CRHD received technical assistance from Macro International Inc., through the Demographic and Health Survey (DHS) programme. In the implementation of the qualitative component, the Population Council office in Dakar received technical assistance from Macro International Inc. to train the surveyors and prepare the conversation handbooks used during data collection.

The main objective of the study’s quantitative component is to see whether it can be statistically established that the Tostan programme has had an impact on the prevalence rate of circumcision among girls and on their age at their first marriage, and to assess whether the health status of mothers and children has improved. The main objective of the study’s qualitative component is to examine Tostan’s establishment process in the villages, to understand how villages organized their participation in public declarations, and to learn what women have to say about the impact of the Tostan programme. This report presents the broad outline and most salient conclusions of the study’s two components.

Several evaluations have been made to assess the impact of the intervention (Tostan’s actions) on its beneficiaries (participating women) in the various villages. However, there are no evaluations, particularly at the community level, that assess the impact of the long-term programme and participation in public declarations (PDs) on early marriage and the practice of circumcision.

UNICEF, one of the agencies supporting Tostan, wants the approach developed by Tostan to reduce and even get rid of early marriage, frequent pregnancies and the practice of circumcision. To refine this strategy, UNICEF Headquarters envisaged collecting relevant data on the way the programme works and on its impact through PDs and the interventions targeting men and women.

Two main objectives are being pursued:
• To evaluate the impact of the Tostan programme, through specific indicators, on daily life in villages and the impact that it has on the rates of early marriage and circumcision;
• To evaluate the impact of village participation in public declarations on early marriage and circumcision.

Findings and Conclusion:

The objective of this evaluation was to assess the impact of the Tostan programme on the daily lives of the villages and on early marriages and circumcision. The information collected shows that the programme helped the villages to develop a set of skills around how to bring about change: in knowledge, in the human and social order, and in the practical perceptions related to circumcision.

In the quantitative analysis, the focus was on the effect of the programme on a social scale, the evolution that took place in health, and the changes that occurred at individual levels. On the social scale, the programme fostered an improvement in social relations within the villages. Informants indicated that the status of women was given higher value within the villages. There was also a kind of synergy the developed, where actions that were being conducted for the programme promoted mutual assistance in the villages.

In the human and social order, significant positive change occurred in perceptions and interpersonal relations, a result of the Tostan programme promoting communication within the villages. According to leaders and participating women, the programme helped reduce conflicts between spouses and thus contributed to the improvement of marital relations.

On the health front, the Tostan programme was instrumental in encouraging people to make better use of health care services; this is one by-product of improving women’s knowledge of immunization and prenatal consultations. It is essentially in the implementation of lessons learned that the real impacts in the health sector can be assessed. However, while accounts of women’s experiences emphasized greater access to health care, the quantitative analysis did not find an effect of the programme on the health of women and children; immunization rates for children, the use of services and access to health services were no better in the villages that benefited from the programme.

In our conversations about public declarations, we heard that there were wide variations in the organization of PDs and in the quality of participation. However, the people surveyed all indicated that they took very seriously the commitment that their village made to abandon circumcision after the PD.

The establishment of social groups in the villages took place before the programme’s implementation, but the programme contributed to their enhancement. The PD seems to have been instrumental in strengthening some of these committees, especially in promoting follow-up to the decisions that were made at the declarations. The majority (71 per cent) of the women surveyed claimed that there was a committee in charge of following up on public declarations, but these groups/committees were no longer found in the villages that were visited by the qualitative team.

People still encounter difficulties when accessing social services. This and other factors limit the extent to which communities can build upon assets gained through the Tostan programme. Beyond this, there is a lack of follow-up in the field, as well as a lack of supporting structure. Tostan attempted to address these problems by creating committees, but in reality, the committees did not work as expected: people find it hard to identify the committees and define their missions, focus, actions and their achievements within the localities covered by the study.

The status of early marriage remains rather ambiguous in intervention villages because the population does not agree on its relevance. Statistics indicate that the rate of very early marriages (under 15-years-old) decreased in the last 10 to 15 years in all three village types.  The biggest drops were observed in intervention villages (group A) for girls aged 10- to 14- years-old: 23 per cent, 16 per cent and 12 per cent during the periods 10- to 14-, 5- to 9- and 0- to 4- years before the survey. In control villages, a less dramatic drop can also be observed for marriages under 15-years-old (from 18 per cent to 13 per cent). The comparison between group A and C villages therefore shows that the combination of PDs and Tostan programmes were instrumental in bringing about a small decrease in the prevalence of under-15 marriages.

There appears to be a very clear change in perceptions relating to circumcision in the villages covered by the study. On this issue, the Tostan programme contributed to the mobilization of communities that were subsequently united around the abandonment of the practice. The majority of the people surveyed declared that circumcision was no longer performed in their village.

Statistical results on the circumcision of girls show that the prevalence of circumcision decreased in Tostan intervention villages and in those that took part in a public declaration (groups A and B). The percentage of circumcised girls between the ages of 0 and 9 is 15 per cent in group A villages, 8 per cent in B villages and 47 per cent in C villages. It is notable that we see effects in the seven Kolda villages (group B villages) of participation in a public declaration that was outside the Tostan programme. It is important to note, however, that about 5 per cent of girls 0- to -9 years old will still get circumcised in the next few years. This fact
means that in all three types of villages, the reduction in prevalence is slightly lower. In the case of type A villages, for example, the prevalence would be 15.4 per cent rather than 14.7 per cent.  Circumcision still remains in all villages, but its frequency has gone down drastically in intervention villages. By contrast, in control villages, practices and opinions remain favorable to circumcision.

It is generally accepted that Tostan’s intervention was globally beneficial to all populations in the zones where the NGO was active. For the general population, these benefits are mostly seen in the drop in circumcision rates. For participating women and some others, they continue to reap the benefits of the knowledge and capacities acquired in class.  Commitments made by the population during and after public declarations also help bring down the frequency of circumcisions.

Recommendation:

An important issue is whether structural interventions (to reduce early marriage, abandon the practice of female circumcision) must come with or follow capacity-building programmes. The problem is that it is not necessarily the domain or the role of an NGO such as Tostan. But the search for partnerships with other programmes endowed with these skills appears to be a necessity.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search