Life skills

Monitoring, Evaluation and Assessment

Sample indicators for monitoring and evaluating skills-based health education programmes


"Process evaluation" documents what has been done and with whom.  Documenting such activities helps you know what services have been delivered, to whom and when, with what level of quality. "Outcome evaluation" documents whether what has been done has made a difference. It is conducted to determine any changes that have occurred and to demonstrate that the changes identified are the result of the intervention itself, not some other factors.

Both process indicators and outcome indicators are necessary for a thorough evaluation, and should be considered at different levels (i.e. for participants, for the facilitators or teachers, and for the overall programme).

Immediate

For programme level:

  • Was the programme implemented as intended? (Process) 
  • Did students' knowledge, attidudes and skills change as intended? (Outcome)

For facilitators / teachers :

  • Was the teaching methodology enhanced? If so, how?  (Process)
  • Did teacher confidence and satisfaction change as intended?  (Process)
  • Did teachers' knowledge, attitudes and skills change as intended?  (Outcome)

For participants:

  • Did target audience actively participate in design, delivery and evaluation of programme?  (Process)
  • Was target audience satisfied?  (Process)
  • Did learning outcomes improve as a result of the programme?  (Outcome)

Medium to Long-term

For programme level:

  • Was the programme implemented as intended?  (Process)
  • Did students' behaviour change as intended?  (Outcome)

For facilitators / teachers:

  • Was the teaching methodology enhanced?  If so, how?  (Process)
  • Did teacher confidence & satisfaction change as intended?  (Process)
  • Did teachers' knowledge, attitudes and skills change as intended?  (Outcome)
  • Were teachers effective?  (Outcome; assessed via student outcomes and satisfaction) 

For participants: 

  • Did target audience actively participate in design, delivery and evaluation of programme?  (Process)
  • Was target audience satisfied?  (Process)
  • Did risk behaviour change as intended?  (Outcome)
  • Did attitudes and values change as intended?  (Outcome)
  • Did health outcomes change as intended?  (Outcome)

Process indicators for the programme level

Process indicators for the programme level focus on questions such as:

  • "Did the programme reach the intended audience?"
  • "Was the programme acceptable to the audience?"
  • "Was the programme implemented in the intended way?"

In addition to process indicators about acceptability of the programme or client satisfaction, there is a need to consider whether the programme actually reached the intended audience, and whether the program elements were ever implemented at all, or were implemented in the intended way. Coverage and quality of the programme are two key domains of inquiry for programme level process evaluation. Some ideas for quality standards are provided below.

Coverage: Is the intended audience being reached? Who is not reached?

Is the programme being offered in all intended settings?  (i.e. In all schools?)

  • % of schools offering programmes, formal and non-formal

Is the programme reaching the intended audience of facilitators/teachers?

  • % of all teachers/facilitators trained

Is the programme reaching the intended audience of children and young people?

  • % of girls/boys (rural/urban; ethnic groups, other...) 

Quality: Are facilitators/teachers implementing the programme according to quality standards?

Possible Programme Quality Standards

  • Is behaviour change part of programme objectives?
  • Is there a balance of knowledge, attitudes & skills?
  • Are participatory teaching & learning methods central?
  • Is the programme based on student needs? (relevant)
  • Is it gender-sensitive?
  • Are policies in place to support the programme? (teacher preparation, in-service, ongoing support)
  • Are related support services accessible to the audience/participants?
  • Are stakeholders consulted? Involved?
  • Are facilitators/teachers trained for this purpose?
  • Are facilitators/teachers supported in implementation phase?
  • Is the programme of sufficient duration to achieve the desired objectives?
  • Are relevant educational materials utilized? (accurate, gender sensitive, age appropriate, accessible, appropriate language, durable...)
  • Is the programme based on relevant, current, accurate information and methods?
  • Is programme impact and process evaluation in place?

Outcome Indicators

Outcome indicators focus on questions such as:

  • "How did the audience or issue change as a result of the program?"
  • "To what extent were the objectives achieved?"
  • "To what extent was the ultimate goal reached?"

Outcome evaluation is possible at a number of levels, for example, the immediate outcome of a lesson/session or set of lessons/sessions in terms of what people learned is quite different the outcomes for an overall program, e.g., how people change their behaviour over a period of time (probably as a result of many strategies beyond education alone) or whether they are actually better off in the long term. The level at which evaluation is done, and how it is done, should depend on the purpose.

Examples of indicators at three levels of Outcome Evaluation
 
LEVEL 1.  Session or classroom level

Immediate outcomes:  Knowledge, Attitudes and Skills.  Assessed by the facilitator/teacher at the time of, or very soon after the educational activities are completed.

Knowledge

a.  Transmission & non-transmission

  • Do participants feel confident they know how to reduce their risk of HIV/AIDS and STIs?
  • What (behaviour/attitudes/ knowledge or lack thereof) transmits HIV/AIDS/STDs?
  • What does NOT transmit HIV/AIDS/STDs?
  • Do participants know how to use a condom?
  • What & how social justice/rights, gender, culture, norms, discrimination.. affect HIV/AIDS/STDs risk and those affected/infected by HIV/AIDS, e.g., orphans, employment.

b.  Various types of research - prevalence, impact of HIV, etc.

  • How widespread are HIV/AIDS/STD/s? Where? Who? What is my risk?
  • What are symptoms of HIV/AIDS/STDs?

c.  Care & support

  • What care & support is available?

d.  Reproductive health, general health

  • What (behaviour, attitudes, knowledge) keeps people healthy?
  • What are the consequences of various risk behaviours/early pregnancy/ other...? 

Attitudes

The term "attitudes" is used here to encompass a wide range of concepts including: intentions, beliefs, feelings about self (confidence) and others (discrimination), values, thoughts, social, religious and cultural tenets, morals and ethics. 

  • Do participants "intend" to use a condom if they have sex?
  • Do participants intend to "wait" to have sex until: older? marriage?
  • Do participants feel "connected" to peers, family, school?
  • How do participants feel towards those affected/infected by HIV/AIDS?
  • Do participants feel confident they know how to reduce their risk of HIV/AIDS and STIs? 

Skills

The term "skills" is used here to refer to life skills: psychosocial and interpersonal skills that can be applied to AIDS prevention and related issues. These skills are important because they can facilitate and may lead to behaviour change, when supported in comprehensive ways.

  • Can participants apply the life skills to hypothetical or practice situations related to HIV/AIDS/STI risk and discrimination? (e.g., Through unfinished sentences, scenarios, short answers, story telling, ranking, role play, etc.)
  • Do participants feel confident they can apply the skills in real life situations?  (e.g.  To: communicate well with peers, teachers, parents, others; refuse undesired sex; resist pressure to use drugs; refuse unprotected sex; insist on/negotiating protected sex; identify personal risk level; act on human rights issues, such as acting against discrimination; identify consequences of decisions and actions; weigh up pros and cons of decisions about early pregnancy or other risk situations; demonstrate correct condom use in hypothetical situation; seek trusted person for help; identify and utilize health services)

LEVEL 2.  Behavioural level

Short term behavioural outcomes.  Assessed a short time after intervention.  It is assumed that achievement of the outcomes of Level 1 will lead to achievements at this level.

Behaviour

  • Was a condom used at last sex?
  • Has the number of sex partners reduced?
  • Is age at first sex increasing? (Is the partner low risk? What age? Older men?) 
  • Is intra-venous drug use decreasing?
  • Are more intra-venous drug users cleaning needles?
  • Are less intra-venous drug users sharing needles? 
  • Are participants (and others) affected by HIV/AIDS treated as well as others? 
  • Are participants seeking help for health issues? (trusted adult, professional) 

LEVEL 3.  Social Health Epidemiology level

Long term health and social outcomes.

Health and Social Outcomes

  • Are STIs decreasing? (Is the average duration of STI decreasing? Are health services accessed more or earlier)?
  • Is age of first pregnancy increasing?
  • Is age of first marriage increasing?
  • Is HIV decreasing?
  • Are those affected by HIV/AIDS healthier? Living longer (than before)?
  • Is drug addiction decreasing? 
  • Is mental health improved? e.g., self esteem, self confidence, outlook, connectedness/sense of community? 
  • Are more children affected by HIV/AIDS staying at school?
Documents and Links

Assessment Strategies for Skills-based Health Education with a focus on HIV prevention and related issues. UNICEF, 2003.
[Word]

Assessing Learning Achievement. UNICEF, 2003.
[Word]

Toolkit for Assessing and Promoting Equity in the Classroom. EIC Project.
[PDF]

Handbook for Evaluating HIV Education; US Centers for Disease Control.
This CDC handbook includes evaluation designs and measurement tools necessary to collect data on the basic programme components of policy development, curriculum design, teacher training, and student outcomes.

HIV/AIDS Survey Indicator Database
The database is overseen by a technical advisory committee that includes representatives from USAID, UNICEF, CDC, UNAIDS, WHO, US Census Bureau, Family Health International, MEASURE Evaluation, The Synergy Project, and MEASURE DHS+

Learning to Live: Monitoring and evaluating HIV/AIDS programmes for young people
This is a practical guide to developing, monitoring and evaluating practice in HIV/AIDS-related programmes for young people, based on the experience of projects around the world. It focuses on recent learning from work with young people in peer education, school-based education, clinic-based service delivery reaching especially vulnerable children, and working with children affected by HIV/AIDS.  The full text can be ordered from Save the Children UK.

Measures and Indicators for Evaluating Life Skills-Based Education Programmes. Draft - UNICEF, 2002
[Word]

Monitoring and Evaluation in UNAIDS
Monitoring core indicators for the Declaration of Commitment on HIV/AIDS from the United Nations General Assembly Special Session on HIV/AIDS.

Monitoring and Evaluating Adolescent Reproductive Health Programs. Family Health International.
YouthNet conducts rigorous research, monitoring and evaluation of its programmes and interventions on adolescent health.



Documents and Links

Assessment Strategies for Skills-based Health Education with a focus on HIV prevention and related issues. UNICEF, 2003.
[Word]

Assessing Learning Achievement. UNICEF, 2003.
[Word]

Toolkit for Assessing and Promoting Equity in the Classroom. EIC Project.
[PDF]

Handbook for Evaluating HIV Education; US Centers for Disease Control.
This CDC handbook includes evaluation designs and measurement tools necessary to collect data on the basic programme components of policy development, curriculum design, teacher training, and student outcomes.

HIV/AIDS Survey Indicator Database
The database is overseen by a technical advisory committee that includes representatives from USAID, UNICEF, CDC, UNAIDS, WHO, US Census Bureau, Family Health International, MEASURE Evaluation, The Synergy Project, and MEASURE DHS+

Learning to Live: Monitoring and evaluating HIV/AIDS programmes for young people
This is a practical guide to developing, monitoring and evaluating practice in HIV/AIDS-related programmes for young people, based on the experience of projects around the world. It focuses on recent learning from work with young people in peer education, school-based education, clinic-based service delivery reaching especially vulnerable children, and working with children affected by HIV/AIDS.  The full text can be ordered from Save the Children UK, and an abbreviated text can be downloaded here.

Measures and Indicators for Evaluating Life Skills-Based Education Programmes. Draft - UNICEF, 2002
[Word]

Monitoring and Evaluation in UNAIDS
Monitoring core indicators for the Declaration of Commitment on HIV/AIDS from the United Nations General Assembly Special Session on HIV/AIDS.

Monitoring and Evaluating Adolescent Reproductive Health Programs. Family Health International.
YouthNet conducts rigorous research, monitoring and evaluation of its programmes and interventions on adolescent health.



 

 

New enhanced search