We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2016 Belize: Impact Evaluation of Belize's Cash Transfer Programme



Author: Troy Thomas

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

In response to a worsening poverty situation as reflected in the 2009 Country Poverty Assessment report (GoB & CDB, 2010), the GoB implemented several social policy reforms among which was the flagship safety net scheme referred to as Building Opportunities for Our Social Transformation (BOOST). The BOOST programme came into existence in 2010 and it had two main aims: a) to achieve immediate poverty reduction through a small, but regular cash payment; and b) break the family’s cycle of poverty through human development of children. These aims were to be achieved through small but steady cash transfers to poor households that qualified for the programme.

Purpose/Objective:

The specific question to be answered is “How effective has the BOOST CCT programme been in achieving poverty alleviation and breaking the family’s cycle of poverty through human development of children?”
This evaluation is also expected to generate recommendations to enhance the impact of the programme, sharpen its equity focus and maximise the use of available resources.

Methodology:

The evaluation employed a combination of quantitative and qualitative analysis to capitalise on the unique but valuable contributions of the two approaches. It utilised, focus group discussions and interviews of beneficiaries, discussions and interviews of staff and officials of the MHDSTPA and interviews with other stakeholders including principals, credit unions and international organisations. The quantitative data were obtained largely through a face-to-face survey of beneficiaries and non-beneficiaries and some data were also obtained from available records and the BOOST database.
A multi-stage, probabilistic sampling of beneficiaries was done and the non-beneficiaries were sampled to mirror the selections of the beneficiaries. The steps included stratification by district, systematic selection of villages and cluster sampling of households in the selected villages. The target was 344 for each group but samples of 223 and 159 beneficiaries and non-beneficiaries were realised. Normalised sample weights.
were applied in the analysis to correct for geographic and group non-response.
In the analysis of impact, propensity score correct for section bias given the no-experimental design of the programme. The propensity score is used as a control variable when regression models are employed. A combination of parametric and non-parametric techniques are employed based on the merits of the data.
Confidentiality of the information collected from respondents/participants in the evaluation was made a priority in the evaluation in addition to respecting their right to full disclosure regarding the purpose of the study and how the data will be handled. Respondents/participants were also free to refuse to participate, to stop at any point or to refuse to answer questions with which they were uncomfortable.

Findings and Conclusions:

In spite of an effective system of transferring funds from the Ministry to the CUs, the beneficiaries and the CUs seem unaware of a set timetable (though one exists) for such payments with the result that beneficiaries often show up at CUs and are disappointed. . In addition, the waiting time for payments whenever corrections are necessary was described as long and the CUs expressed a need for clarity on where queries about payments should be addressed.
The targeting mechanism including the referral system and application of the PMT are progressive and confirm to standards applied internationally.
Impact is separated into core and spinoff issues. In the first five years of its existence, the BOOST programme was able to achieve some positive impacts in relation to its core foci. These are:

  1. Improved hope of the children exiting poverty which is used in this evaluation as a proxy measure for transitioning out of poverty.
  2. Improved well-being (measured by consumption).

There is widespread belief that the programme will continue among officials of the MHDSTPA based on the recognition that it has received in the political arena in spite of the absence of a legal framework.

Recommendations:

The recommendations of the 2012 process evaluation should be fully implemented.
The timetable for transfer of funds to the CUs should be communicated to the CUs and the beneficiaries.
Address the process of updating beneficiary data.
The process involved in transition from primary to secondary school should be simplified.
The process followed when corrections have to be made to payments should be revised.
There should be planned refresher engagements every three years to update the external stakeholders on the BOOST processes and expectations.
The state of knowledge about the programme among the masses should be addressed to reduce exclusions errors if a scale-up is attempted, but also to improve current attitudes towards and perceptions about the programme.
The PMT should be continuously updated as more data become available.
An audit of the list of beneficiaries and recertification of beneficiaries every three to five years.
Produce information included in country level studies that facilitate identification of BOOST beneficiaries to facilitate impact studies.
A policy about the combinations of programs that a family can access subject to multidimensionality of poverty considerations.
The size of the benefits should be reviewed in connection with the cost of compliance and inflation.
The penalty in the form of reduced benefits for noncompliance system should be maintained but it should be accompanied by efficient compliance checking and case management.
The ability to determine compliance with health requirements and impact on health of the beneficiaries in general through official channels would be a major achievement of the programme and its institutional arrangements.
There should be a further sensitisation programme for principals and teachers that  targets discrimination of beneficiary students in the schools.
The BOOST programme should be combined with workshops for parents in for example parenting and financial management.

Lessons Learned:

The BOOST programme was well designed based on sound principles. There are many alternative approaches to the design of CCTs and the particular combination of parameters implemented in the design of the BOOST make for a rigorous programme. This encompasses everything from targeting and selection to payment of beneficiaries.
Execution of at CCT requires much effort and institutions and individuals that are dedicated to the purpose. This is evident in the execution of the BOOST programme. The team is dynamic and composed of individual with a wide range of expertise including computing, data management and analysis and the social aspect of engaging people. Judging from what it has achieved over the first five years including the extent of coordination with external stakeholders, the BOOST management has been effective and dedicated to the task at hand. It has also benefited from a set of supportive stakeholders. Apart from some slippages especially in relation to beneficiary data update and compliance checking and perhaps in communication as time wore on, the programme has been administered well and in the interest of the people being served.
This combination of good design and strong implementation has made the programme effective. In addition to this there are signs of early impacts including improved well-being of the beneficiaries and spin-off impacts such as beneficiaries accessing other services through the CUs. The areas in which the execution of the programme can be improved point to the overarching issue of it having moved on to the stage where dedicated staff for some functions might be necessary. This is a stage where it appears best that the programme becomes more institutionalised that viewed as a project.
To a large extent, the recommendations advanced in this evaluation focus on enhancing what is already there. The BOOST programme is one of which the MHDSTPA should be proud.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2016

Country:
Belize

Region:
LACR

Type:
Evaluation

Theme:
Social Policy (Cross-cutting)

Partners:
Ministry of Human Development Social Transformation and Poverty Alleviation; Ministry of Education Youth Sports and Culture, Ministry of Health; UNDP

Language:
English

Sequence #:
2016/001

New enhanced search