Evaluation database

Evaluation report

2010 Egypt: Reaching the Unreached Forward looking assessment of the 'Community Schools Project' in Egypt



Author: Ms. Eszter Szucs; Dr. Habiba Hassan-Wassef

Executive summary

Background:
In 1992, the Ministry of Education (MOE) signed an agreement with UNICEF for developing a community based educational model for providing quality primary schooling to children living in remote underserved areas, giving priority to increasing enrolment of girls. UNICEF was to support the development of a technically sound replicable model that can reach the hard to reach, thus improving the primary school enrolment rates and narrowing gender disparities. The objective being to accelerate progress towards achievement of Education for All and the MDGs. Remote communities in Upper Egypt were targeted, with a focus on girls, thereby addressing core causes of disparity: geography and gender. Underpinned by the Convention on the Rights of the Child, emphasis of this Community Schools Project (CSP) is on quality education using child-centered techniques.
The community schools project is implemented through a partnership between the Ministry of Education (MOE), non-governmental organizations (NGO), local communities and UNICEF. The partnership was extended to include the Canadian International Development Agency (CIDA) in 1994 and World Food Programme (WFP) in 2006. The MOE provides facilitator salaries, curriculum and textbooks, technical supervision and auxiliary services. UNICEF, with support from CIDA provides school furniture, supplies and is responsible for day to day management and capacity-building of the facilitators, supervisors and project partners. WFP provides daily snacks and take home rations. Communities make available the classroom locale, participate in management of the schools and ensure their local relevance through education committees (EC). NGOs, contracted by UNICEF, provide field presence for technical supervision, management and administration of community schools (CS).
In October 2008, UNICEF commissioned an assessment of the phase III (2003 – 2009) aiming to "enhancing Egypt's national capacity to deliver quality basic education for all, focusing on girls, by consolidating and expanding a community schools model for diffusion to Egypt's mainstream educational institutions". The assessment was to be comprehensive, acknowledging achievements of the CSP, identifying best practices and recommending future requirements for capacity building and quality assurance, once funding from CIDA comes to an end and management of schools is handed over to MOE and NGOs. Since the structure for the CSP handover has already been identified, the assessment is mainly concerned with ways to preserve and optimize project accomplishments (above all in terms of access and quality). Because the project was last evaluated in 2001, generating new reliable information that can provide an up-to-date image was an explicit objective.

Purpose/ Objective:
The purpose of the evaluation is to provide a forward looking assessment based on the lessons learned from the final or sustainability phase (2003 – 2009) of the community schools project. The main focus of the evaluation will be to develop a project sustainability strategy. It is expected that the evaluation will yield evidence on the impact of community schools on increasing access, improving quality, and achieving learning outcomes (within the framework of the criteria of child rights in education as indicated in the Child Rights Convention) as well as the impact on gender issues. Furthermore, it will contribute to the Ministry of Education and other education stakeholders' strategy with regards to Community Based Education.
The main objective of the evaluation was to undertake a comprehensive and final assessment of the community schools project that provides scientific based evidence to substantiate the evolution, progress and achievements of the final phase of the CSP. In particular, the evaluation was designed to:
1. Provide substantiated evidence as to the achievements of the Project principal goals and objectives as well as broader outcomes (Appendix 10.3, MOE-UNICEF 1992 Agreement refers).
2. Provide the basis for the forward looking recommendations and identifies the strengths and weaknesses of the CSP.
3. Analyze the reasons underlying the factors that challenged the achievement of the sustainability Phase through a review of the management model and of the financial aspects of the CSP.
4. Studying the programme relevance, effectiveness, efficiency, impact (qualitative and quantitative), sustainability, and costing of community schools

Methodology:
In addition to surveys and secondary data sheets intended to measure access, learning conditions and community relations, standardized tests – Critical Thinking, Achievement and Problem Solving (CAPS) in Arabic, mathematics and science; and pedagogical skills – were administered to Grade 4 students and teaching staff of the CSP after having been adapted to suit the learning outcomes. Moreover, students sat a life skills test that followed UNESCO and UNICEF guidelines in its design. In formulation and administration of all tools, respect for the characteristics of the study population as well as confidentiality were emphasized. The primary purpose of the qualitative component was to allow insight into profound CSP outcomes and management processes. In-depth and group interviews, focus groups and spontaneous exchanges were conducted. Although an attempt was made to discuss with all stakeholders, sample size, exception for management investigations - was restricted. For project history and cost analysis, a review of CSP and related records and documents was performed.
Tools and techniques were pre-tested in two CS, randomly selected from Assiut and Qena (understood to represent divergent quality). Using census, stratified random and cluster sampling, the principal assessment involved 70 CS, with maximum 36 NGO supervisors (technical and field), 138 facilitators and 1,896 students. Since the main aim of the assessment is to provide information on the CSP as a whole, sampling criteria were determined for the entire population (rather than for sub-units). The data processing, analysis was conducted with the aid of SPSS software.
Given the interest in sustainability, the information was collected by governorate to represent different structures of management. To shed light on inequities, additional desegregation of target groups was conducted. Principal themes in qualitative research were synthesized in line with the assessment framework. The details of the framework as well as the analysis were designed to allow for continuous refinement based on insights emerging from the field. Validation relied upon triangulation, with background research, quantitative and qualitative components. The study respected UNICEF’s evaluation policy for facilitation of evidence-based decision-making and its practical implications throughout the research process, as well as the national policy and that of the other partners. Recommendations are aligned with the organization’s shifting priorities, in light of Egypt’s lower middle income status.

Findings and Conclusions:
• In terms of access and learning outcomes, the community schools approaches have been overwhelmingly effective if compared to other educational enhancement and poverty reduction programmes across the country. In line with the above requirements, best practices from CS (mainly in terms of curricula and teaching methods) have been extended to MS, OCS, girl-friendly schools and other education projects. Most recently, the community schools model has been recognized as a basis for all CBE initiatives in the Strategic plan. However, beyond extending the reach of the project, effectiveness in terms of outcomes needs further attention. Weaker access and learning achievement on the part of MS and OCS point to an urgent need for measures to improve them through a fresh look at these systems.

• Age and gender disparities in enrolment, poor attendance and attrition have been largely eradicated, and in a number of communities most of the children eligible for education have been covered. The objectives of school completion for CSP students and increasing the popularity of education have been successfully attained.

• Completion is around 90 percent and of those who completed CS, progression to preparatory school is nearly 100 percent. Early marriage and child labour have to a large extent diminished in communities served by CSP schools.

• CS students perform better in core subjects than their counterparts in mainstream and in one classroom schools (MS and OCS) both in lower and in upper-cycle grades. On the whole, Grade 4 children scored two thirds of questions correct across tests: 50 percent in Arabic, 63 percent in mathematics, 67 percent in science and 79 percent in life skills. Highest scores were in Assiut, and Qena did better than Sohag, except in life skills. There were no significant gender differences for achievement and for other socio-economic risk factors, which were, for the most part, found to be equal.

• Unlike technical supervisors, CS children and facilitators were found to perform weakest on critical thinking (compared to other cognitive domains), pointing to a need to improve training and transfer of these skills. The Strategic plan’s emphasis on staffing and quality assurance of CBE, along with the lower percentage of facilitator salaries in the budget of CS versus other school systems, provides a unique opportunity for policy dialogue towards securing capacity-building, oversight and staffing, as required in quantity and quality to sustain community schools. Particularly in light of the weak relationship between qualifications and performance, the possibility for facilitators to have secondary education in case of need should be promoted (all the while ensuring that required conditions for performance are in place).

• For the present, extensive supervision and training - though representing a much higher percentage of expenses in CS than in MS and OCS - are of questionable relevance and efficiency, given the considerable experience and qualifications of facilitators along with concomitant gaps in skills in dealing with contemporary requirements such as children at risk and information technology. Recurrent cost per child is higher in CS than for MS. While the difference in costs may be reduced when calculated against effectiveness, it is not necessarily eliminated.

• At a time when the development of Upper Egypt and its people has become a national priority, more could be done towards enhancing cooperation between CS, different schools, service providers and communities to rationalize the harnessing of resources for CS and other education systems.

• Some deficiencies were revealed to exist in the supply of textbooks, supplementary materials, furniture, supplies and in infra-structure services, and were encountered in as much as a quarter or a third of schools. Facilitators in a number of schools took the initiative to use local resources to produce their own teaching and learning aids. At times, supplies and school meals were excessive while at other times limited, with public and privately provided school feeding programmes sometimes overlapping. Supply and demand were not aligned as there was no precise and effective school mapping which made possible an equitable distribution of CS.

• Good practice examples were encountered in Assiut, whose community school students had the highest and most equal scores in spite of the greatest percentages of disadvantaged students. The successful community partnerships and favourable material quality of schools that complete the profile for Assiut were apparently not undermined by the high percentage of disadvantaged EC members. As for Sohag, despite having highest vulnerability indicators, the CS clientele of Sohag is not considered the most disadvantaged. However, even with greatest proportions of socio-economically advantaged EC members, the volume of donations was lowest. This indicates the need to enhance the efficiency and relevance of the role of the EC.

• While prolific in identification and promotion of good practice, the findings in this report show that in changing the scope of the CSP to prioritize mainstreaming, the original objective of reaching remote and disadvantaged communities with quality education has been undermined. Coverage of target populations – the hard-to-reach, especially girls - is a fraction of the expected. Not more than 54 percent of CS children are girls (versus the anticipated 75 percent), barely over 10 percent are of poor economic status, and 7 percent of students in Sohag and 27 percent of facilitators overall are from towns (37 percent have post-secondary qualifications). Interventions to increase local relevance of community schools such as community education and creation of income generation activities were minimal and short lived and are underscored by the less intense student, facilitator and EC engagement with community members (and vice versa).

• A further consequence of the change in scope of the CSP that was revealed by the study is the lessened attention to sustainability. The protective management structure, which was strongly governed by UNICEF, may have been called for in the early phases of setting up a pioneering initiative and making it a corner stone of education planning and practice. However, under such a management structure, sustainability could not be established. The shift in strategy to sustainability has brought to the surface limitations in management capacity thus compromising – the outcome of the training received. This was evident in cases where new actors had to be brought in as with NGOs. The capacity-building training that had been provided by UNICEF to project staff, NGOs and EC as well as the MOE that focused on management and financial skills succeeded in reducing the reliance on UNICEF for day-to-day project management. However, self reliance could not be guaranteed. The Rationalization of project costs has started but more needs to be done, including rigorous targeting and creation of modalities for cost sharing.

• Shortcomings of the CSP are due mainly to one factor, which has interfered with full realization of the innovation and improvement potential not only of CS but also of other school systems, namely, ineffective planning, monitoring and evaluation, especially at higher levels. In line with demands of the final phase, if not a formal database, relatively systematic information on students and facilitators is being gathered. However, monitoring and examination are not the same as evaluation. In the latter especially, insufficient attention to all relevant aspects in detail (desegregation of cognitive domains, competence of staff by rank, properties by school and governorate), along with inability to detect evolving conditions in and around the CSP, has served to compromise equity, quality, costing and efficiency all the way from project strategy down to the classroom. Any initiative based on partnership needs to have the capacity not only in terms of research but also for planning and implementation at all management levels to effectively gather, digest, communicate and react to information.

Recommendations:
- With physical access no longer an issue in many CS locations, novel concerns to do with increasingly acute forms of exclusion and progression in education and broader life gain in importance. While the Strategic plan shifts attention as a result of improved access to quality considerations, the possibility of educational evolution in terms of access needs to be integrated into planning not only of community-based education (CBE) but also general education, given the growing significance of linkages between the two systems.

- With regards to progression, community schools, their students and staff need to be better connected to mainstream opportunities, to be secured through an improved interface with national systems (including effective liaison between CS and MS at different administrative levels).

- On reaching the most excluded, the characteristics of excluded communities and individuals (both children and youth) need to be precisely determined and new ways devised to effectively meet their needs. More active and targeted outreach is required for promoting establishment of CBE in remote localities and enrolment of the hardest to reach children.

- Emphasis on access (in particular of girls), retention and achievement in education is necessary but represents only the first step. There is a need to balance improving gender equity in school outcomes with promoting equality through women’s empowerment.

- Continuity in support to less conventional subjects such as life skills, arts and sports, positive affiliation with CS on the part of students and staff, equity in access and learning outcomes, and enhanced community interactions will require to be guaranteed following the CSP handover.

- Mainstreaming good practices from CS would benefit from targeted attention to broader child-friendly techniques, suited to the conditions of different school systems. Weaker access and learning achievement on the part of MS and OCS point to an urgent need for measures to improve them through a fresh look at these systems.

- Educational quality is not complete without material conditions. Opportunities provided by the Strategic plan’s emphasis on building and equipment, buttressed by the lower percentage of capital expenditure in CS versus other budgets , should be taken advantage of in policy dialogue to improve the CSP’s learning conditions. Good practice examples, such as ways for replicating Assiut’s most effective educational achievements, favourable material quality of the schools, and successful community partnerships need to be further investigated and promoted, especially where they are most relevant. Further investigations may also shed more information on the good practices in Sohag where, despite highest vulnerability indicators, the CS clientele of Sohag is not the most disadvantaged.

- While prolific in identification and promotion of good practice, this report argues that with the change in scope of the CSP to prioritizing mainstreaming, the original objective of reaching remote and disadvantaged communities with quality education has been undermined. The achieved coverage of target populations – the hard-to-reach, especially girls - is less than expected. There is a need to revitalize local relevance of community schools through a number of interventions, from teaching inputs, through community education, and income generation.

- Handing over CS to the MOE and NGOs as was already done in Qena and negotiated for all schools in line with the final phase stipulations, offers a unique opportunity to optimize policy and societal relevance, along with providing the opportunities for scaling-up through which the Strategic plan envisions creation of a CBE system. The handover process should be closely monitored and lessons clearly extracted in order to identify and maximize the application of best CBE practices. In this way, a valuable contribution can be made towards taking forward this relevant niche (of CBE).

- The limitations in managerial capacity and financial stewardship brought to the surface as a result of the shift in strategy to sustainability of the project necessitate adapting the training in capacity building to bridge the identified gaps and to introduce rational use of resources and modalities for cost sharing.

- Although communications and networking between partners have been promoted, breadth and depth of collaboration along with types of participants need to be extended, complete with line ministries, private and cooperative sectors, service providers in and out of communities, past and present CS students and families. There has to be a relevant and clear vision adhered to by all, with everyone fulfilling their precisely defined and matched role, having sufficient competence and empowerment to effectively act at their level. In addition, continued facilitation of networks of cooperation as well as of trust can be furthered by different types of incentives.

- Measures are needed to compensate for the insufficient attention accorded to details available through an appropriate management information system (desegregation of cognitive domains, competence of staff by rank, properties by school and governorate), as well as for the inability to detect evolving conditions in and around the CSP. The above shortcomings have served to compromise equity, quality, costing and efficiency all the way from project strategy down to the classroom. Therefore, any initiative based on partnership needs to have the capacity not only in terms of research but also for planning and implementation at all management levels to effectively gather, digest, communicate and react to information. This requires continuous and refined research that is flexible enough to detect and adapt to circumstances that depend on the stage of country’s development and the educational evolution of the communities in question. The refined research is expected to be respectful of local preferences and open to relevant national and international best practices.

Lessons Learned (Optional):
 All stakeholders are accurately aware of CS (CBE) properties and based on them have a precise plan of action, grounded in needs assessment with monitored objectives.

 Roles and responsibilities of all stakeholders are clearly defined and arrangements are made for them to be effectively accomplished (leaving time for those no longer involved to find alternative engagements).

 Project personnel are all officially informed and prepare for handover (the more they can be motivated the better), with key information passed on to partners and charges (e.g. harmonizing with inspectors logistics).

 Outstanding questions that may provide good practice examples (such as ways for Assiut’s most relevant coverage and learning outcomes) are answered. Additional good practices, for instance Qena’s sustainable education budget and ways to ensure access and learning achievement, along with Assiut’s most effective local committees, should be shared.

Limitations
The limitations and unexpected extended duration of the study made it difficult to answer the following questions which required further indepth studies:
• Prepare capacity building plan for each stakeholder mentioned above with clear guidelines and also the assessment techniques/tools of the capacity building exercise.
• Examine the existing monitoring and evaluation processes that are in place for the community schools project and suggest ways of improving them.
• To what extent have the linkages/coordination between the decision-makers/community leaders and the project at its various management levels been successful?



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search