We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2015 Belize: Process Evaluation Final Report - School Community Liaison & Security Programme



Author: John D. Flowers, M.A.

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

The School Community Liaison and Security Programme forms an integral part of the Ministry of Education, Youth and Sports’ service mix.  This Programme started in September of 1998 in Belize City, as the Truancy Initiative to address non- and irregular school attendance of children 5 – 14 years in Belize. Around June 1999 the Scholl Security Component/Scheme was added to the School Attendance and Participation (Truancy Management) Component/Scheme as a measure to address the serious problems the country was facing with the abduction and murder of children from late 1997 through to 1998 with the principal objective of ensuring the safety and security of children at schools (MOEYS, 2014).

Notwithstanding more than fifteen years of operation, Management and the National Committee for Families and Children found it necessary to commission a Process Evaluation. This formative evaluation offers a double dividend for the social sector. First and foremost, it offers an opportunity to reflect soberly on the SCLSP in terms of WHAT is being done and HOW what is being done is being done with the purpose of enhancing the SCLSP through strengthening of its overall design, operational processes and institutional arrangements based on good practices.

Furthermore, the Process Evaluation may be instructive to the wider social sector, as it examines ways through which to improve on service delivery from an evidence-based approach.  Perhaps more than often the question of programme effectiveness has arisen in the development debate, i.e. notwithstanding the huge budget outlays invested, results are not commensurate and social conditions continue seemingly on a downward spiral with occasional moments of reprieve.  To this end, it is imperative that major focus be placed on the technical and administrative processes (HOWs) necessary and sufficient to effectively transform resources into meaningful results.

Purpose/Objective:

The Evaluation of the School Community Liaison and Security Programme is undertaken within the policy and legal framework of the National Plan of Action for Children and Adolescents (NPA), MOEYS Education Sector Strategy, GOB-UNICEF CPAP, the Education and Training Act and the Families and Children’s Act taking into consideration, other relevant social sector reform initiatives under the MOEYS, e.g. School Feeding and Quality Child-friendly Schools Initiative; those coordinated by the Ministry of Human Development, inter alia, BOOST, Public Assistance, SISB and FAMCare, Counselling and Diversion and those from other relevant organization, e.g. RESTORE Belize(EWS).

The objective is to conduct a process evaluation of the MOEYS School Community Liaison and Security Programme(SCLSP), inclusive of an institutional analysis to inform recommendations for enhancing its overall management, organizational structure and operations.

Methodology:

The Evaluation Study is characterized as an interim process evaluation utilizing a mixed-method approach and participatory methodologies; its use is intended as formative for programme enhancement. Principal methods employed in achieving the specified objectives of the Study included, but were not limited to: desk review, literature search, key informant interviews, focus group discussions, rapid survey techniques, and gap and SWOT analyses.
Evaluation Design and Framework
In tandem with the objectives and the overall approach to the Study, the Process Evaluation considered the technical processes in place to transform inputs into desired results as the principal focus; complementary criteria also focussed on the institutional arrangements in place to implement the program; human resources responsible for implementing the procedures; and review of allocated financial inputs in relation to desired results

Additionally, it was agreed that the OECD-DAC criteria, would be most useful to inform a prospective analysis of the SCLSP and to strengthen its overall design, implementation processes and institutional arrangements. Given the nature of the Evaluation, emphasis was placed on the criteria of Relevance and to a lesser degree, Effectiveness and Sustainability; criteria related to efficiency and impact were not broached. The overall framework, as adapted from Europe Aid.

Findings and Conclusions:

Programme Relevance
When reviewed and analysed, the defined goals and functions are found to be consistent with the overall vision of theSCLSP and in tandem with established policy commitments, inter alia: Education Sector Strategy 2011-2016, Policy Objectives 1, 2 and 3:National Plan of Action for Children and Adolescents, 2004-2015;National Poverty Elimination Strategy and Action Plan, 2009-2013;Millennium Development Goals.

Beneficiary Perception
Beneficiaries unanimously agreed that the Programme was an important one, but hastened to say that they would have appreciate more pointed support from the SCLO in addressing the underlying and root issues which gave rise to the ‘truancy/absenteeism’ presenting issue.
In similar manner, school principals where the School Security scheme was in place, were quick to recognize the good intent of the Scheme and noted in many instances very good working relationships with the wardens and their managements. 

Approved Estimates and Expenditure
Programme expenditure for the FY 2013/14 totalled BZ$1.2 M with projections for 2015/16-17 at over BZ$1.0 M. Quick analysis reveals a tripling of the financial resources available to the Programme relative to FY 1999/00.  Additionally, breakdown of the financial resources indicates the lion’s share going towards personal emoluments with very little allocated to training and grants to individuals, the line from which immediate programmed education/public assistance may be drawn from to assist identified families.   

Recommendations:

Adoption of Model II, or variation thereof; develop SOP Manual and train 100% of officers.
Initiate policy dialogue with general- and local-level school management.
Adopt/strengthen/implement tiered case management. Adoption of simple e-tools to facilitate documentation, follow-up and reporting. Integration with MHD case management system (FAMCare) and full data exchangewith OpenEMIS.
Explore toll-free hotline for public to report matters of truancy accompanied by a strong communication campaign on the importance of community involvement and the links between truancy/absenteeism and associated issues.
Advocate for matters related to truancy be heard before the Belize Family Court as opposed to the magistrate courts for greater sensitivity; in addition, revise the existing piece of legislation to offer options beyond a fine of BZ$100.
Aligned with core competencies and performance evaluation gaps, develop and implement a human resource development and training plan with core funding from recurrent budget.
Give consideration to staff re-deployment to level off the demand for services across the districts; increasing by three SCLOs the SCLSP. Formalize the SCLO coordinator post given the need for urgent field oversight and added work load/responsibilities as it relates to coordination and management of cases and files.
An annual programme planning and review mechanism in tandem with Programme Based Budgeting be immediately instituted.
Introduce field supervision by the central management team/DECs for improved oversight and quality control of processes and outputs.
Institute transparent recruitment process based on clear criteria.
Institute merit-based recruitment and hiring procedures, to include child protection screening of candidates.
Institute a basic management structure for supervision of wardens by zones within district/
An interim management support to the SCLSP Management Unit as it transitions.

Lessons Learned:

Data for the most recent five years indicates that on the average, 513 children drop out of school annually.
A total of 17 SCLOs and 191 school wardens provide services under the SCLSP distributed across the 6 administrative districts.
Two basic operational models emerged via which cases with presenting issues of absenteeism are being addressed to some degree by the various actors on the ground.
These emergent models showed varying degrees of interplay among and between schools-parents-state actors(MOEYS-SCLS Unit; MNS-ZBLO Unit; MHD-FSS Unit; and MLLGINNEM-DOL) as well as a wide variance in applicability across the six districts. While none of the models represent a holistic or complete management model, a number of ‘good/innovative practices’ have been identified which are instructive for systematizing the SCLSP.

Model I: SCLO Centered Model
This Model is extremely resource intensive being heavily reliant on SCLOs to visit all primary schools to identify cases of absenteeism/truancy and effectively manage such cases. Additionally, the Model pre-supposes access to readily available transportation services given the number and distribution of primary schools across the geographic spread in the six administrative districts.

Model I Process Flow and Identified Good Practices
Given the direct involvement of school principals and teachers in the identification of presenting issues associated with absenteeism/truancy not only serves to empower schools to take charge, act and reinforce the school-parent-Ministry relationship, but also addresses the limitations identified in Model I with respect to opportunity costs and trade-offs associated with SCLOs case identification efforts.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2015

Country:
Belize

Region:
LACRO

Theme:
Education

Type:
Evaluation

Partners/Institution:
realsurveyworks - consultancy firm

Language:
English

New enhanced search