2018 Eswatini: Impact Evaluation of the Teen Club Program for Adolescents living with HIV in Eswatini
Author: Leonard Maveneka
With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.
Young people aged 10–24 years are one of the most vulnerable, yet overlooked, populations affected by the human immunodeficiency virus (HIV) epidemic. Worldwide, an estimated 2,100,000 people aged 10–19 years were living with HIV in 2016, 80 per cent of whom were residing in Sub-Saharan Africa. With high numbers of estimated new infections among older adolescents (15–19) and many of the 920,000 children receiving antiretroviral therapy (ART) surviving into adolescence, adolescents have become a critical age group to focus on in terms of their vulnerability. Nonetheless, adolescents continue to be underserved by current interventions across the HIV cascade. Adolescents have signiﬁcantly inferior access to and coverage of ART, higher rates of lost to follow-up (LTFU) poor adherence and increased need for psychosocial support and sexual reproductive health (SRH) services.
Despite growing awareness about effective interventions to prevent HIV transmission among young people, they still make up 45 per cent of new transmissions worldwide. Whether infected during the neonatal period or during adolescence, adolescents living with HIV (ALHIV) have unique and pressing psychosocial needs on top of the daily challenges of being adolescent. Adherence to medication; disclosure of HIV status; issues relating to sex and lack of support networks are problems faced by all ALHIV.
Access to treatment and care for ALHIV remains inadequate. Following HIV Testing and Counselling (HTC), there are poor linkages to and retention in care for most populations and Anti-Retroviral Therapy (ART) coverage rates for adolescents are lower than for other age groups. Interventions and support for sustained treatment adherence and retention in care are challenges in many settings, the inability to address these issues has led to treatment failure and the high levels of HIV-related morbidity and mortality increasingly being recognized in this group.
The purpose of the evaluation was to determine the relevance, effectiveness, impact, efficiency and sustainability of the Teen Club initiative in addressing the needs of ALHIV in Eswatini. The evaluation also assessed the extent to which the Teen Clubs for ALHIV achieved their intended objective over the last 5 five years, the period covering the evaluation, and to provide recommendations on how to scale up the initiative and make relevant adjustments in the programme.
The specific objectives of the evaluation were:
- To provide current status of the functioning of the Teen Clubs, membership (age and sex), type of activities conducted, trained teen leaders (including sex disaggregation) at each site and the management and mentoring aspects.
- To assess the quality of outputs, outcomes and results of the project relate to knowledge, attitude and practices among active and former Teen Club members in the four facilities, on issues related to their health and well-being.
- To determine the effectiveness of the Teen Club model in providing support to adolescents in disclosing their status, transitioning between pediatric and adult care and treatment services, acquiring quality HIV information and education, adhering to treatment and adopting safe sexual behaviours.
- To identify key and unique needs of adolescent boys and girls, the differentiated needs of rural and urban based adolescents and whether these were met by the programme interventions.
- To identify key good practices and key lessons learnt, identify gaps, what strategies and interventions to continue and/or discontinue, and to what extent and make recommendations for future improvement based on evaluation findings.
- To determine the extent to which the programme took a rights based approach both in its design and implementation.
- To determine the extent to which the programme factored in gender and equity issues in its design and implementation.
The evaluation used a non-experimental design which tracked outcomes and impact for the target beneficiaries only. The design enabled changes within the ALHIV to be measured. The design was also useful for obtaining information relating to service delivery, extent of reach of the intervention and progress towards objectives.
To enhance the design, the evaluation used a variety of quantitative and qualitative data collection methods and sources of information as highlighted below:
• Briefing meetings
These were held with UNICEF technical personnel, Baylor College of Medicine and Ministry of Health to provide a broad overview of the programme.
• Literature review
Pertinent programme documents and other relevant literature relating to the programme that were provided by UNICEF and Baylor, including Baylor progress and activity reports were reviewed.
• Key Informants
Key informant interviews were conducted with stakeholders from UNICEF, Baylor Eswatini and Ministry of Health. Other key stakeholders interviewed included clinic staff (doctors, nurses, counsellors and social workers) as well as volunteers.
• Focus Group Discussions
FGDs were conducted with adolescents participating in the programme and those who have graduated. FGD participants were divided into two groups by age (10–14 and 15–19 years) to make it easier to discuss age-specific issues
• Face to face interviews
The survey was administered to adolescents participating in the programme as well as those who have graduated to determine the level of knowledge and to assess the attitude and practice of adolescents in the programme. Structured questionnaires were used administered by trained Research Assistants.
Findings and conclusions:
The evaluation of the Teen Club shows that the Programme is relevant as it addresses some of the most critical challenges faced by ALHIV, including LTFU resulting in defaulting on treatment, poor adherence, self-stigmatisation and stigma in communities against the adolescents
The Programme has been highly effective in supporting full disclosure among Teen Club members, which is the first step in accepting one’s status as it has brought positive changes in the lives of the adolescents through improved adherence, which in turn has resulted in viral load suppression among most of the Teen Club members.
The evaluation noted that Teen Club Pprogramme is efficient and cost effective and is good value for money as it costs USD79 per person per year. If the intervention had not been implemented, the adolescents would be in a much worse state than they are in terms of their health status as Teen Club members are recruited from among those who would have defaulted, have low viral load suppression and who have low self-esteem.
The evaluation concluded that the Teen Club is a promising initiative which has achieved most of its intermediate outcomes – increased adherence, high viral load suppression and empowered adolescents who have overcome self-stigma and can fight stigma against them in the community.
There was a general feeling among the adolescents, which was also shared by Baylor, of the need for a more well thought out strategy to ensure a smoother transition for adolescents graduating out of the programme;
• Ensuring a smooth transition into adulthood
Both Baylor and the young adults admit that the transition from the Teen Clubs has not been smooth and has been too abrupt leaving them feeling abandoned. Baylor should design a comprehensive programme to facilitate the smooth transition of the adolescents into adulthood. Such a programme should address their bio-medical as well as their psychosocial needs.
• Addressing the social determinants of adherence
There is need for a more holistic approach to adherence , which takes cognizance and tries to address factors in the external environment that can impact on adherence. Such an approach should strengthen the coping mechanisms of the adolescents and also seek ways to mitigate the effects of the external environment on the adolescents.
• Need for more intense anti-stigma programmes
The Ministry of Health, working with development partners, should design and implement comprehensive anti-stigma programmes to be rolled out across the country as a long term strategy to address the high levels of stigma in the Eswatini society.
• Strengthening mental health services
Baylor should strengthen its mental health services to enable it to screen and provide services to more adolescents and parents/guardians who need the services.
• Designing an exit strategy
Partners should advocate for the government to take over funding of all the Teen Cclubs and for their roll out to the remaining health facilities in the country that offer HIV services, so that more ALHIV can access the services
• Broadening the range of services available to adolescents
In rolling out the Ten Clubs Programme across the country, it will be necessary to incorporate the In-Reach and the Mental Health Programme as well as the Challenge Clinic to ensure the provision of a comprehensive package of services for adolescents
• Strengthening use of U-report
UNICEF and Baylor ensure that questions asked are responded timely and appropriately responded to.
- Teen Clubs are effective as forums for ALHIV to meet share experiences and provide each other with mutual support to improve adherence, gain self-confidence and address self-stigma and fight stigmatisation against them in the wider community.
- The importance of carrying out a baseline survey and designing a theory of change at the beginning of an intervention to set clear benchmarks and indicators against which programme performance will be measured. This will also make it easier to assess the programme’s contribution to change using a control group.
- The importance of making SRH services available to adolescents, including condoms, family planning services and treatment for STIs. By creating safe spaces for adolescents within their health facilities, uptake of the services has improved as well as the health seeking behavior of the adolescents.
- The importance of Baylor coming up with a more comprehensive transition programme for young adults that will adequately prepare them for life outside the clubs. Such a programme should also be linked to interventions to address the pervasive stigma and discrimination now prevailinig in Eswatini to create an environment that is more accepting of PLHIV.
- The need for a more integrated approach to addressing adherence among adolescents that takes into consideration socio-economic factors in the adolescents’ home environment that make them default such as lack of food, lack of money for transport for ART refills. Such an approach would aim to strengthen the government social welfare system to ensure that poor and marginalised children do not fall through the cracks.
- The need for Baylor to design an M&E framework with indicators that capture the key result areas of the Teen Clubs to facilitate better monitoring of the project.
Please find the attachments labelled as follows:
- Evaluation Report - Report
- GEROS Evaluation Review - Part 2
- GEROS Feedback Summary - Part 3
- Evaluation Management Response (EMR) - Part 4
Full report in PDF
PDF files require Acrobat Reader.