“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”
The overall purpose of the research was to evaluate My Future is My Choice (MFMC), a national peer education HIV prevention life skills programme in Namibia, and to provide recommendations for programme improvement and strengthening. An official extra-curricular life skills program of the Ministry of Education at secondary and combined schools, the MFMC programme seeks to protect young people from HIV infection and sexually transmitted diseases as well as prevent unintended pregnancies.
As specified in the terms of reference, the specific objectives of the assignment were to:
• Assess the impact and influence of MFMC on young people, both learners and peer facilitators
• Assess the programme delivery mechanism of the MFMC programme, including the quality and ability of facilitators and trainers to deliver the programme
• Identify and analyse MFMC programme strengths and weaknesses based upon evaluation results
• Make realistic recommendations for improving the MFMC programme
Key questions that the evaluation sought to address include:
• Is the MFMC programme effective in meeting its goals?
• Is the programme being implemented as intended?
• What impact has the programme had on the knowledge, skills, and behaviours of MFMC graduates and MFMC peer facilitators?
• What are key obstacles to the effective implementation of the programme?
• What are the strengths and weaknesses of the programme?
• How do we revise the content and/or delivery mechanism?
• How do we improve the MFMC programme?
In order to answer the main questions proposed by the My Future is My Choice (MFMC) evaluation study, a mixed-method approach that gathered both qualitative and quantitative data was used. There was a strong emphasis on soliciting feedback not only from MFMC participants and MFMC peer facilitators, but also stakeholders involved in overseeing programme implementation as well as monitoring and evaluation.
In terms of primary data collection, the evaluation of the MFMC programme utilised the following methods:
• Questionnaires with learners who participated in the programme between 2006 and 2008
• Interviews with current MFMC peer facilitators, contact teachers, school principals, RACE Coordinators, and the Trainer of Trainers
• Focus group discussions with MFMC graduates and MFMC peer facilitators
• Participant observation of MFMC training courses, both of facilitators and learners
Data was collected in two phases between July and October 2008 across six regions of Namibia. Phase I involved collecting quantitative data (primarily) which was used to inform and shape qualitative data collection in Phase II. The quantitative data gathered in Phase I helped identify programme strengths and weaknesses which were further probed in depth through qualitative data collection in Phase II.
The MFMC programme is broadly perceived by users, implementers, and stakeholders in a positive light and as beneficial for MFMC participants as well as participating schools. Although there is room for improvement, the content and curriculum are considered useful and relevant to the lives of young people and the issues that they face. That there is a uniform call from MFMC participants, MFMC facilitators, and stakeholders alike to expand the programme to reach more young people speaks to the perceived benefits of the programme.
Commonly reported strengths identified by stakeholders, MFMC participants, and MFMC peer facilitators alike include that the MFMC programme:
• Creates awareness of HIV/AIDS, STIs, teenage pregnancy, and the dangers of risky sexual activity
• Promotes abstinence
• Teaches participants how to use condoms correctly
• Provides information that some learners may not otherwise be exposed to, especially in rural areas
• Teaches participants to care for people living with HIV/AIDS (PLWHA)
• Teaches participants important communication, decision-making and problem-solving skills
• Enhances participants’ self-esteem and self-confidence
• Teaches participants how to resist peer pressure
In addition, the programme reinforces and deepens what is learned in other classes such as Life Skills and Science. The interactive participative methods empower young people not only to absorb the learning, but also voice their opinions and speak openly about sexuality and HIV/AIDS.
The qualitative data indicates that the MFMC programme has, for many MFMC participants and MFMC peer facilitators, enabled important, positive, and health promoting changes. For example, MFMC graduates report, amongst other things, increased knowledge with regards to reproductive health and HIV/AIDS, the adoption of more protective sexual attitudes and practices, increased awareness of the effects of peer pressure as well as increased awareness of the dangers of alcohol. Similarly the qualitative data suggests that the MFMC programme has been an important, positive influence in the lives of peer facilitators. Through participation in the programme as peer facilitators, they report various changes in their lives such as becoming more knowledgeable about sex, reproduction, and HIV/AIDS, becoming more open and confident, learning how to facilitate in front of a group, becoming more accepting of HIV positive people, and engaging in less risky sexual and related behaviours. Indeed many peer facilitators articulated the importance of these changes in terms of shifting their worldview and leading them to make better decisions and choices.
There are however various obstacles which compromise the effective delivery and implementation of the programme. Key obstacles include:
• Facilitators have limited knowledge and skills
• High peer facilitator turnover compromises programme continuity
• The programme reaches only a small proportion of the target group
• The programme is not integrated into the school curriculum
• There is insufficient school buy-in, especially from principals
• There is poor communication between the MFMC programme office and the schools
• There is uneven monitoring and evaluation and insufficient quality control
In terms of the facilitator as the agent of delivery, the research indicates that although the programme has built peer facilitator capacity, gaps in their knowledge levels persist, and that the programme requires a more skilled, committed, and better compensated cadre of peer facilitators to effectively deliver the intervention. Related to the issue of facilitator capacity is high turnover among peer facilitators. Essentially volunteers who my future is my choice receive a stipend for delivering the course, the compensation peer facilitators receive is often insufficient to retain them over time. This compromises programme continuity and the cultivation of a skilled cadre of peer facilitators over time.
In addition, the marginalisation of the programme within the school as an add-on rather than integral component of the school curriculum compromises its effectiveness. As an after school activity, the MFMC programme is easily sidelined by school staff (principals and teachers) and made secondary to academic demands. Moreover, due to the timing of the programme as an after school activity, access is restricted to those who are able to stay after school and who do not have other commitments or responsibilities. The programme thus does not necessarily reach those who need it most.
Lastly, another significant obstacle to the effective delivery of the MFMC programme is uneven monitoring and evaluation and insufficient quality control. Monitoring and evaluation often occur haphazardly such that it is difficult to determine what is actually happening in the classroom, or ascertain whether the peer facilitator is delivering the intervention as intended.
The quality and success of the MFMC programme depends on a skilled peer facilitator delivering the programme in a supportive, enabling school environment with clear accountability structures and a strong functioning monitoring and evaluation system. As the programme is currently being implemented, many of these critical ingredients are missing.
The recommendations address many of the obstacles discussed to effective implementation and also call for the expansion and institutionalisation of the MFMC programme. However, all issues of programme quality must be resolved (i.e. training and retention of peer facilitators, monitoring and evaluation, revision of curriculum, etc) before efforts to expand and institutionalise the programme are undertaken.
PDF files require Acrobat Reader.