|Pakistan: Consulting Community Information Board, member of Parents Teachers Association (PTA) discuss school and community issues.|
Evaluation involves the collection, synthesis and analysis of inputs, outputs, outcomes and impact of C4D initiatives. Evaluation takes place at specific points in the programme process and measures behavioural, organizational and social change outcomes, including social norms. These evaluations are carried out to ensure and improve the efficacy of the programme, as well as to determine the contribution of C4D to the development outcomes.
Data is collected through a variety of methods such as surveillance reports, usage figures, field visits and observations to manage and communicate performance levels and programme achievement. This stage of a C4D initiative addresses the “so what” question by providing evidence of what the initiative has achieved and how it positively affects the lives of children. It is imperative to collect both quantitative and qualitative data. Data generated from monitoring and evaluation feeds into the next round of strategy development and revisions.
In 2006 and 2007, as part of the UN system response, UNICEF intensified work in the prevention and control of Avian Influenza with special focus on communities and households. Good monitoring demonstrated that participant groups — especially families engaged in backyard poultry production — became more aware of the risks posed by the H5N1 virus and more inclined to practice protective and preventive behaviours. Behaviours related to handwashing and thorough cooking of poultry increased.
For instance, UNICEF Egypt conducted a communication campaign to improve the knowledge and behavioural practices surrounding avian influenza. Baseline research facilitated the production of clear communication objections as well as geographic and demographic targets. Nearly 11,000 community organizers were then trained, and a wide range of materials were produced in a participatory process. The campaign then rolled out 2753 Street Banners, 420 hours of radio and television programming, 13,000 posters and 150,000 fliers. Most significantly, nearly 4 million households received four visits from community organizers. Rigorous monitoring of behaviour throughout the campaign allowed the programme to measure a 56% increase in handwashing practices and a 36% increase across the 14 promoted practices.
In 2012, close monitoring of local beliefs about polio in the Democratic Republic of Congo (DRC) allowed the polio programme to quickly and accurately address various communities concerns. Traditional leaders remain very influential in the DRC and some of the traditional groups were opposed to immunization.
Understanding the reasons why these communities refused immunization lead to an effective analysis of how to engage them. The programme reached out to traditional leaders and other key influencers in the resistant communities and within months, three of the four resistant groups in Bas-Congo province welcomed the immunization teams.
The use of “indicators” is essential in data collection and analysis. An indicator is an observed fact, action, or statement taken as a sign that some process is or is not occurring, or that the programme has or has not achieved its desired progress or outcomes. Indicators reveal whether progress is being made. Although many indicators can usually be established for each result, it is important to select a limited number of key indicators that will best demonstrate the desired changes.
There is no standard list. Good indicators limit and focus data collection. Some indicators (e.g., mortality related ones) may not be feasible to track in the time period available or if resources are limited.
The most useful indicators are identified using SMART criteria:
- Specific: Is the indicator specific enough to measure progress towards the results?
- Measurable: Is the indicator a reliable and clear measure of results?
- Attainable: Are the results in which the indicator seeks to chart progress realistic?
- Relevant: Is the indicator relevant to the intended outputs and outcomes?
- Time bound: Are data available at a reasonable cost and effort?
Participatory Action Research