Evaluation database

Evaluation report

2009 Philippines: Evaluation of the HIV Prevention Interventions for Most-at-Risk Adolescents



Executive summary

 

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report.”

A. Introduction

In line with its Sixth Country Programme for Children (CPC6), the 2005-2009, four NGOs were

contracted by UNICEF-Philippines to implement intervention measures to prevent HIV and STI

among vulnerable and most-at-risk children and young people (MARCY) in selected areas of the

country. During the period, 2005-2008, the four NGOs carried out a number of activities designed to

reduce the transmission of HIV and STI among the targeted beneficiaries at the nine project sites.

The intervention measures focused on two main thrusts, namely: (i) provision of HIV and STI

prevention, care, and support for MARCY; and (ii) promotion of a supportive and enabling

environment for HIV prevention. The provision of HIV and STI prevention services was carried out

mainly through a peer education strategy that was anchored on the recruitment, training, and

deployment of peer educators (PEs).

This evaluation study was an attempt to determine the quality, coverage, relevance, effectiveness,

efficiency, and sustainability of the pilot projects. For purposes of the evaluation, four sites were

selected (Pasay, Iloilo, Zamboanga, and Davao). In all of the four sites, qualitative and quantitative

research techniques were utilized in evaluating the peer education strategy as well as the strategy

to create and promote an enabling environment for HIV and STI prevention work. The following

research techniques were used in the conduct of the evaluation study: (i) records review; (ii) indepth

or key informant interviews (IDIs/KIIs); (iii) observations of actual peer education sessions; (iv)

focus group discussions (FGDs); and (v) a quantitative survey of selected respondents from the

different target groups, including women in entertainment establishments (WEE).

B. Results

Adequacy of Project Design. This project is the first attempt to address the needs of young mostat-

risk populations (MARPs) following UNICEF’s CPC 6, and this period can still be considered part

of the country’s ‘learning curve’ with respect to addressing the needs of high-risk and vulnerable

children and youth. The project proposals submitted by the NGOs and approved by UNICEF for

funding are major documents that are needed to assess the adequacy of the project design. The

evaluation team, however, had to rely largely on the progress and terminal reports submitted by the

NGOs to ‘reconstruct’ the project conceptual and implementation framework. However, no

conceptual or implementation framework was reflected in the said documents. Moreover, the

implementation of the peer education strategy was not adequately defined and described. The

importance of the need to collect baseline data to be able to measure project accomplishments was

not adequately emphasized, and this shortcoming made the task of project evaluation quite

challenging.

Relevance of Project Objectives and Strategies. The project is relevant because it addresses the

risks and vulnerabilities of young MARPs in a low HIV prevalence setting. Each pilot project had its

own set of objectives and activities, and a review of these showed that they are, in general, relevant

to the project goal, which is to reduce transmission of HIV and STI among vulnerable and high-risk

children, adolescents, and young adults.

Effectiveness and Achievement of Project Objectives. The evaluation showed that adequate

numbers of peer educators were trained, with almost all of the project sites exceeding their targets.

However, it appears that the training provided to the peer educators was inadequate, judging from

their level of knowledge on HIV/AIDS, and STI – the test results showed that only 54% of the PE

respondents obtained a perfect UNGASS (United Nations General Assembly Special Session)

composite knowledge score. Moreover, the examination of outcome, output, and process measures

showed that the projects were unable to show improvements in the target populations’ knowledge,

attitudes, and practices to limit transmission of HIV. Only 14% of the survey respondents

representing the target population obtained a perfect UNGASS composite knowledge score.

Condom use based on recall during last sexual intercourse was low, with only 32% of respondents

reporting such practice. Despite the initiatives of the pilot projects to improve the knowledge of the

target population and reduce the risk of HIV/STI in this young population, they remain vulnerable to

HIV and other STIs, and these vulnerabilities put them at a high risk of exposure to the known risk

factors of HIV and STI infection. The peer education strategy was constrained by a number of

factors, such as the lack of standardized training approaches and materials, high turnover of trained

PEs, lack of good IEC materials and condom supplies, failure to set up an adequate reporting and

monitoring system, and a weak PE oversight, monitoring, and feedback mechanism .

Efficiency. Based on the total project cost and the actual number of peers reached (24,868), the

cost to reach one peer was estimated at P1,327, on the average, although there was a considerable

variation in cost across the four pilot sites – from a low of P491 in Davao to a high of P3,172 in

Pasay. In comparison, the Fourth Philippine AIDS Medium Term Plan (AMTP IV) estimated the cost

of similar prevention interventions for high-risk and vulnerable populations at P94 for out-of-school

youth (OSY) and P409 for street children.

Other Benefits. The evaluation study revealed that aside from the large number of peer educators

trained under the pilot projects, other benefits could be attributed to the projects. For one, the PEs’

personal satisfaction with their roles as PEs has made an impact on how they addressed their own

HIV risks. Secondly, at the community level, the events organized or participated in by the NGOs

and PEs had the effect of bringing health issues to the surface for the information of the general

public, and not only the target populations. Thirdly, the local government units (LGUs) have

institutionalized Local AIDS Councils (LACs) and Barangay AIDS Councils (BACs) and supported

the Barangay Child Protection Council (BCPC). Budgets have been allocated for HIV prevention

programs and activities, and LGUs have come under heavier pressure to enforce ordinances as a

result of the monitoring done by project implementers and other stakeholders. At the NGO level, the

pilot projects provided the opportunity to continue what they started and renewed their commitment

to the cause of HIV prevention.

Sustainability. The above benefits are likely to be sustained as a result of the increased awareness

and knowledge of PEs and their peers on HIV and STI. Also, since some of the pilot projects were

implemented in cooperation with the Sangguniang Kabataan (SK) (Youth Council) and BCPC, there

is some assurance that the welfare of MARCY will not be sidelined, and their concerns will be

mainstreamed through SK and BCPC programs. The involvement of SK may be effective in scaling

up the peer education strategy. The support of LGUs and employers of women in entertainment

establishments (WEE) is also a major factor for project sustainability as they have the resources to

support HIV/STI prevention activities. The NGOs have also tried different ways to sustain their

operations after project completion, including initiating revenue generating activities such as

condom social marketing.

C. Lessons Learned

First, projects need to have a clear conceptual/operational framework. These not only facilitate

evaluation but also help implementers track and monitor the progress of project implementation.

Second, projects also need to have clear disengagement and sustainability plans to mainstream

interventions into local development plans and programs and ensure that benefits or gains from the

projects are sustained beyond Project life.

Third, the engagement of NGOs with considerable experience in the program helps reduce cost

and increase efficiency. However, even experienced NGOs need technical guidance and support.

Fourth, support from communities helps, but there is no substitute for LGU commitment to ensure

project sustainability at the local level.

Fifth, experiences and resources from earlier but similar projects can still be relevant and prove

useful for future projects. Strategies which were found effective in earlier projects could be

replicated.

Sixth, program interventions that work in other countries need to be refined, adjusted, and adapted

to be locally effective.

Seventh, the passage of local policies and ordinances does not guarantee automatic program

support from local government units.

D. Recommendations

Ensure availability of pre-tested IEC materials and a continuous supply of condoms. As it has

been established that IEC materials play a major role in any project, they should be pre-tested to

assess the appropriateness and effectiveness of their content and style before their use in the

project. Participation of the target groups is needed to produce culturally and gender-sensitive and

age-appropriate IEC materials. The use of the peer education workbook developed by PATH, which

the PEs found useful, is highly recommended.

Behavior change among peers is needed for a shift in mindset from a reliance on free condoms to

accessing socially marketed condoms which have to be purchased at a minimal fee. NGOs need to

establish a condom distribution system tailored to the prevailing policies and practices at the LGU

level. The allocation and delivery of condoms should be carefully planned, supported by available

budgets, and synchronized with the requirements of PEs in the field. Adequate quantities should be

produced for the number of peers targeted.

Improve the recruitment, retention, and follow-up of PEs. NGOs should devise a system of

retention for PEs through promotions, trainings, and incentives. The provision of reasonable

incentives and a clear system of monitoring and follow-through will go a long way towards

motivating the PEs to higher levels of performance.

Standardize the conduct of training courses for PEs. NGOs need to provide PEs with

opportunities to improve their knowledge and skills. They should be given training not only at the

start of the project, but throughout its life and even beyond, if possible. A stepladder approach to the

introduction of a package of topics to the PEs can be adopted so as not to overload them with a lot

of information in a very short period of time. The quality of training for PEs must be maintained, and

training should be interesting, engaging, and interactive. Training materials developed by the NGOs

should be produced using high-quality material (to lengthen their life span) and made available to

the public. Moreover, since conflicts may arise from the differences in group behaviors, training

efficiency and effectiveness may be enhanced if the groups are targeted separately. Such

management of training courses may prevent the increase in risky behaviors, which may result from

social networking of the vulnerable groups.

Improve the design of peer education sessions. NGOs should enhance the design of peer

education sessions, focusing on the weak but key components of the program, and should follow

the general standards prescribed by international organizations such as UNFPA. PEs should be

engaged at all levels of program planning, development, implementation, and evaluation. Also, peer

education activities should be well-coordinated with the LGU, DOH, NGOs, and private companies

to avoid duplication of services and address gaps in service delivery.

AN EVALUATION OF HIV AND STI PREVENTION INTERVENTIONS FOR VULNERABLE

AND MOST-AT-RISK ADOLESCENTS AND YOUNG PEOPLE,

PHILIPPINES

EXECUTIVE SUMMARY Page ix

Advocate for more LGU and community support. NGOs need to work more intensively with the

LACs and community leaders to obtain more support from the local governments and the

community. LGUs should be more fully and actively involved in the implementation and monitoring

of peer education activities. They can incorporate HIV prevention activities into their existing

programs, and youth groups, such as the SK, can be mobilized to integrate peer education activities

into their health and youth programs. The BCPC and LAC, which are said to be the most effective

venues for sustaining community-based peer education systems, should mainstream these behavior

change programs and promote their ownership by the community to sustain their benefits.

E. New Strategies

To sustain the gains made by the Project, the following innovative strategies are recommended:

• Promote HIV prevention and peer education programs and ‘sell’ them to other

organizations such as private establishments, NGOs, and donor and government

agencies.

• Incorporate the Child’s Rights Framework in addressing the multifaceted and

multidimensional nature of the HIV and AIDS problem. The structural and systematic

factors contributing to the target population’s vulnerability should form part of the

campaign against the spread of the diseases.

• Improve the quality of services provided by public and private health facilities especially

for the children and youth.

• Tap colleges and universities to engage student interns in implementing interventions for

the children and youth.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search