We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2017 Bangladesh: Programme Evaluation of UNICEF Bangladesh Communication for Development (C4D) Programme from 2012 to 2016



Author: Tristi Nichols and Fuad Pasha

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

The 2014 UNICEF Communication for Development (C4D) vision paper highlights that the approach is a cross-cutting programming strategy based on human-rights and amplifies the voices of communities and marginalised groups to demand their rights, connecting them to upstream policy and legislation.  The approach is guided by the Social Ecological Model (SEM) which is a “theory-based framework used for understanding the multifaceted and interactive effects of personal and environmental factors that determine behaviours, and for identifying behavioural and organizational leverage points.” 

From 2012 to 2016, the C4D Section of UNICEF, in collaboration with the Ministry of Information (MoI) and Ministry of Women and Children’s Affairs (MoWCA), seven partner non-governmental organizations (NGOs) worked towards engaging communities in bringing about social and behavioural change through the Engaging Communities for Social and Behavioural Change programme.   The evidence from the United Nations Development Assistance Framework (UNDAF) geographical districts, among other rich analytical tools, led UNICEF to select seven hard-to-reach districts where the C4D interventions could be targeted.  Further, based on specific criterion (e.g., natural disaster vulnerability, hard-to-reach locations, and poor performance in achieving the MDGs), seven districts and three Upazillas within the districts (totalling 21 Upazillas).  

The programme’s targeted communities, a total of 1,013,489 households or/and approximately four million people, were exposed to the programme since January 2012.   Based on the information provided, the total budget for this four-year project was US$ 12,649,309 with 83 per cent of the budget funded through Regular Resources (RR).  This Evaluation Report describes the scope of and activities for the evaluation of the UNICEF Bangladesh “Engaging Communities for Social & Behavioural Change” programme.

Purpose/Objective:

This evaluation focuses on helping to amend any unmet assumptions, and validate, fine-tune details regarding the following areas:

  1. What priorities ought to be pursued specifically in Bangladesh;
  2. How best to translate its overall vision of commitment to children’s rights and equity into action; 
  3. What specific opportunities for scaling up proven C4D interventions exist which take into account:
    • strategies and interventions that contribute to behaviour change and the participation of  communities/families; and
    • reliable data gathering that is considered useful and that is used to improve programme delivery.

Objectives and Intended Audience
According to the Terms of Reference, the evaluation covers the period from July 2012 to July 2016 and is designed to support future C4D interventions, has the following objectives:

  • Assess the effectiveness, efficiency, relevance, and sustainability of C4D programme;
  • Assess the potential scale-up in areas for selected components of the C4D programme;
  • Identify enablers, bottlenecks and barriers to the achievement of results;
  • Assess the appropriateness of results monitoring system put into place; and
  • Provide recommendations for the design, management, coordination, implementation and measurement of progress of future C4D-related programmatic interventions for the new country programme 2017-2020.

The intended audience of this evaluation is UNICEF, the Government of Bangladesh, the Implementing partners, including NGOs and BRAC University, and other practitioners interested in understanding how the C4D approach was applied in Bangladesh.

Methodology:

To adequately address the evaluation questions, this evaluation drew upon a mixed-methods design, with multiple information sources, including: (1) a literature review; (2) a survey with a random sample of household community members; (3) key informant interviews (KIIs) and focus group discussions (FGDs) with UNICEF in Dhaka and the field offices, GoB at national and subnational levels, the IPs, the media, BRAC University, and the community;  (4) data collected by BRAC University, or the Knowledge Management Outreach Sites (KMOS) system, and a baseline study followed by a midline survey; and (5) organisational data, including UNICEF Country Programme for 2012-2016 and 2017-2020, budget information, AWPs from each Section, training manuals (in Bangla language only), IP quarterly reports, and UNICEF monitoring data.

Generally, a total of 773 household surveys, 19 FGDs (n=150 persons), and 75 Key Informant Interviews (KIIs) were completed by Manitou, Inc. and Mitra and Associates.  To gather information which corresponds to the needs of a programme evaluation, where different stakeholder groups are expected to contribute to the evaluation, it was necessary to involve all partners at all stages.  A gender equality lens was applied when creating the sample parameters. A specific number of surveys and KIIs for women, men, adolescents, and older persons was planned, thereby ensuring that a representative number for each stakeholder group would present in the quantitative sample.  There were three phases of the evaluation: Desk Phase, the Field Phase, and the Production Phase, and the total duration of the evaluation was from January through April 2017. 

Findings and Conclusions:

  1. Relevance
    At the programmatic level, the top three messages deemed most relevant by rights holders are Child Marriage, Handwashing, and Birth Registration. 
  2. Effectiveness
    Based on the data presented in this report, while most of those surveyed demonstrate that they know about the eight life-saving care and protective behaviors, the groups within the overall sample who could still benefit from further efforts offered by this programme, are: (1) mothers with children under six months; (2) adult males; and (3) adolescents.  Specifically, the first Intermediate Result (IR) has been partially achieved.
  3. Efficiency
    The instutionalised monitoring and evaluation (M&E) systems, collecting data from 21 Upazillas, were effectively used by the IPs and BRAC. However, an unintended, or unplanned, result was that BRAC University used the evaluative evidence to inform some of their current research efforts. 
  4. Scalability
    The specific stakeholder groups which showed a level of knowledge and practice that did not meet the lowest target, were: (1) mothers with children under six months; (2) adult males; and (3) adolescents.  The protective behaviours which ought to receive more dedicated resources are: (1) ARI; (2) antenatal care; (3) birth registration; (4) exclusive breast feeding; (5) HIV; and (6) handwashing. 
  5. Sustainability
    Pervading social norms are still present necessitating the continuation of C4D.  All stakeholders at every level voiced the importance of programme extension.  While the programme design included a systematic promotion of national ownership, where subnational GoB stakeholders received training and had an instructive experience in C4D implementation, management, and coordination, due to capacity limitations, however, the prospect of GoB structures sustaining this programme in the future is very limited. 

Recommendations:

  1. Programme Strategy for Expansion [Future Programme Designs] Since behaviour change takes time, it is recommended that the C4D programme is continued.
  2. Institutional Strengthening for UNICEF [Future Programme Designs/Management] Given the cross-cutting nature of C4D and the level in which it was integrated into the 2012-2016 Country Programme, it is recommended to increase capacity strengthening efforts in C4D.
  3. Programme Strategy for Modifying Messages (and Activities) to Be More Inclusive [Future Programme Designs/Implementation]
  4. Assess Strategies for Being More Inclusive with Fathers, Men, and Adolescent boys. [Future Programme Designs/Implementation] UNICEF should consider other programming strategies to increase the participation of men. 
  5. Assess Different Media [Future Programme Implementation]
  6. Capacity Strengthening Strategy in C4D for GoB [Future Programme Designs/Implementation] Given that UNICEF’s Strategic Plan emphasises the importance and need to strengthen national capacity, the identification of GoB (national and subnational) capacity needs in C4D programming should be undertaken followed by the development of a longer-term capacity strengthening strategy.
  7. Strengthening Evaluation Systems to Use Data to Improve Programme Delivery [Programme Measurement/Implementation] To promote the systematic use of evidence-based decision-making, UNICEF should ensure that capacity strengthening efforts in this area are provided to IPs in future C4D programmes.
  8. Formulate a Problem Analysis and Theory of Change [Programme Design/Measurement] UNICEF should formulate a ToC for its future C4D programming.  Completing this framework would provide the C4D Section with a clearer understanding of the change that could be expected from different interventions and an established link between the outcomes and the specific responsibilities of the C4D Section.

Lessons Learned:

Lesson #1:  One of the clearest and most valuable lessons to draw from this programme evaluation is that when it comes to behaviour change, the two-way conversation among family and community is very effective. 

Lesson #2: Another valuable lesson learned is the critical importance of rights-holders having the opportunity to reflect on changing their ways and demanding their rights, which was an experience provided through this programme. 

Lesson #3: This programme evaluation has also evidenced that focusing on communities and families located in hard-to-reach areas and women, as an approach to promote life-saving, preventative behaviours and to reduce harmful practices against children’s rights, has resulted in noteworthy outcomes. 

Lesson #4: Given that it was not possible to recreate a ToC, as this would have been beyond the scope of this evaluation, it is important to learn that such an exercise is very important.

Lesson #5: As evidenced from three different stakeholder groups, the IPs experienced payments that were not time-bound, and in turn reduced the momentum of implementing programme activities.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2017

Country:
Bangladesh

Region:
ROSA

Type:
Evaluation

Theme:
Advocacy and Communication

Partners:
Ministry of Information (MoI) and Ministry of Women and Children’s Affairs (MoWCA)

Language:
English

Sequence:
2017/001

New enhanced search