We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2018 South Sudan: Evaluation of Global Partnership for Education Programme in South Sudan



Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 3’ of the report, and the executive feedback summary labelled as ‘Part 4’.

Background

The GPE programme (GPEP) in South Sudan was implemented from December 2013 to November 2017 and extended at no-cost until May 2018. The programme was funded both by GPE and USAID. The overall funding envelope was USD 66.1m of which USD 30.1m was the USAID contribution from December 2013 to December 2016. UNICEF is the GPEP Grant Agent and worked closely with the Ministry of General Education and Instruction (MoGEI) on GPEP implementation.1
The GPEP was designed in 2012 by government and development partners to support implementation of the General Education Strategic Plan (GESP) 2012-2017.
The GPEP had three main objectives:
1. Strengthen national systems that are fundamental to providing equitable access to quality education; 2. Improve school performance, and in the process generate model approaches for improving quality; 3. Attract additional support to the education sector in South Sudan by demonstrating sustainable successes.
GPEP was conceptualized in a context of peace, but implemented in the context of civil war, economic crisis, and other emergencies. The most notable political, economic and security events during the GPEP implementation period (2013-2017) were
1) the outbreak of violence in Juba in December 2013, which led to protracted conflict in the Greater Upper Nile states;
2) an economic crisis and hyperinflation of the SSP against the USD since mid-2015;
3) Presidential decrees creating 28, and now 32 states out of the previous 10;
4) the outbreak of violence in Juba in July 2016 followed by intensifying conflict in Greater Equatorial and other previously more stable areas. The conflict has caused mass population displacement and food insecurity across South Sudan.
These developments not only created an extremely challenging operating context, but also affected education sector planning and coordination as most donor funding shifted to humanitarian relief.

Purpose/Objective

GPEP was designed as a development programme and did not shift focus to Education in Emergencies over the course of implementation. GPEP continued to work with the Ministry of General Education and Instruction on the implementation of activities that were aligned to the General Education Strategic Plan (GESP, 2012-2017), albeit with some adaptations to the original plans. As the importance of longer-term planning for education in emergencies and protracted crises is increasingly evidenced, evaluating the GPEP experience in South Sudan presents an opportunity to learn from the implementation of an education development programme in a fragile context.

Methodology

The endline evaluation was carried out over a period of three months by a team of five consultants. The team worked remotely during a two-week inception phase in November to refine the workplan, finalise the evaluation framework and start up survey design work. The team agreed with UNICEF that the inception phase and the evaluation design phase for GPEP component B would run concurrently. Evaluating component B required data collection at school level and there was only a very short window to collect this before the end of the 2017 Academic Year in mid-December. These time constraints limited the sampling approach and sample size to what could be collected within the window.

The endline evaluation administered a total of seven surveys at school and county level.
In administering the final evaluation, the conditions and tools used in the baseline were replicated as closely as possible, while streamlining the baseline material by asking those questions that are relevant to the evaluation questions.

2.3 Data collection approaches

Given the limited timeframe for the assignment before the end of the academic year 2017, and logistical and security challenges related to short-term international consultants’ travel out of Juba, Forcier Consulting was contracted to collect data from ten selected schools in five (former) states.
Data collection in Juba
Data collected through surveys at sub-national levels was complemented with interviews with key informants in Juba, that largely focused on Components A and C. 
2.4 Outline of data analysis procedure
Research instruments were aligned to the evaluation framework to allow analysis of the results and to answer each evaluation question set out in the RFP.

Findings and conclusions

Nearly all stakeholders interviewed said the lack of qualified teachers was the biggest challenge facing the South Sudan education sector. GPEP could have addressed teacher training beyond the leadership training that has benefitted head teachers and selected teachers. Some respondents cited studies that showed teacher training has a bigger impact on learning outcomes than school construction. Without improving the quality of teaching, the impact of school construction on the quality of education remains limited. This is confirmed through the Action Research and endline research results which show limited or no gains in learning outcomes.
However, it was noted by education sector partners that a large-scale multi-donor programme targeting teachers was being planned in 2014-2015. In the expectation that this programme would start imminently, GPEP would not have prioritized teacher training at the time of design. The programme was conceptualized but did not materialize due to political developments.
Other issues that were mentioned included:
● Secondary school construction: GPEP focused on constructing only primary schools, whereas there is a shortage of secondary schools as well. In some counties there is no secondary school at all.

● Pastoralists Education: Several stakeholders mentioned there should be greater targeted support for children from pastoralist communities. The PEP Strategic Framework was validated on 30 August 2017 with participation of senior MoGEI officials, UNICEF GPE Manager and UNESCO. This suggests that the implications of the framework are yet to be seen at school and community level and the next phase of GPEP could consider how to support its implementation.

Recommendations

Recommendation 1: Maintain long-term development focus
GPEP should maintain a long-term development focus. Shifting focus to short-term emergency programming will not benefit the education sector. It creates fragmentation and lacks sustained impact. EiE appears to be sufficiently funded at the moment.

Recommendation 2: Focus on fewer core components
The design of GPE2 needs more focus. Fewer core components will make the programme more coherent. This will help to achieve greater visibility and understanding of the programme objectives and develop a strategy for government ownership and sustainability. Despite having a long-term development focus, the GPEP still appeared fragmented due to many different (sub) components. Partners contracted to deliver these covered only a set of specific activities for a relatively short period of time.

Recommendation 3: Support the MoGEI to lead sector coordination
GPE2 should be more visible and more proactive in sharing information with sector partners, and realise the potential of its role to support MoGEI to lead sector coordination. The roles and responsibilities of CA and GA need to be clarified for more efficient management and to avoid unnecessary delays.

Recommendation 4: Implementation model should be considered at design stage & Recommendation 5: Implementation model should strengthen education structures to deliver results in schools
GPE2 needs to achieve more coherence between national-level activities and roll out to schools. GPEP design seems not have thought through a workable model to reach schools at scale. As a result, contracting seems to have been ad hoc, timeframes too short and rushedwith no time to follow up after training was completed. Materials’ distribution to schools was delayed or not fully completed.

Lessons Learned

GPEP maintained its relevance as an education development programme supporting the implementation of the GESP, but the GPEP approach to school construction lost relevance in the context of conflict and economic crisis. GPEP could have reflected more frequently on the continued relevance of the design’s core components in particular school construction. The programme would also have been more coherent if component A (systems strengthening) and B (community and school-based education delivery) were more closely joined.

Teacher training should be expanded in terms of numbers of teachers trained, duration of the training, and amount of follow-up, refresher courses, and mentorship. Ensure that learning materials are distributed to schools, provided in local language of instruction, and teachers receive adequate training on their use. Evaluation findings suggest that improving quality of education and learning outcomes, requires prioritising support to learning materials and teacher training over school construction, as head teachers, teachers and other key informants perceive these to have a greater impact. It should be noted that due to limited sample size and selfselection, responses from teachers may not represent the opinions of all teachers in the school or of the GPEP programme.

School construction remains important to increasing access, but the need for schools is so great that building 25 schools has not made a big impact. GPEP should consider lower-cost initiatives like renovating existing schools and continuing support for community-based schools. As the conflict spread, the school construction component and model school approach lost their relevance due to rising costs and risks of damage and destruction of the school buildings.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year: 2018

Office/Country: South Sudan

Region: ESAR

Type: Evaluation

Theme: Education; Education sectoral or partnership; EFA

Language: English

Sequence #: 2018/001 

New enhanced search