We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2015 South Africa: Evaluation report of the Technogirl programme in South Africa



Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

The Techno Girl programme was conceptualised in 2006 and  piloted in three provinces (KwaZulu-Natal, Eastern Cape and Limpopo). The overall goal of the programme is to enhance and increase the uptake of careers in science, technology, engineering and mathematics (STEM) by disadvantaged girls, thereby increasing the number of women working in careers critical to the economic growth of South Africa. The beneficiaries of the programme are selected on the basis of academic merit. The main criterion for selection is that they must be doing well in science and mathematics. In collaboration with Uweso Consulting, the Department of Basic Education (DBE) identifies eligible participants, mainly from schools in quintiles 1–3; i.e. the ‘poorest’ 60% of all schools in South Africa and are no-fee schools. To be selected, the beneficiary schools should be located within the proximity of the job-shadowing sites offered by the local implementing partner companies
The achievement of the programme objectives were made possible by the public–private-partnership-driven model.
Uweso Consulting formed partnerships with host organisations through Memorandums of Agreement (MoAs) to provide workplace exposure to STEM-related career fields through job shadowing for girls from disadvantaged communities. The main assumption underpinning the programme was by participating in a ‘structured job-shadowing process’, girls would improve their knowledge and understanding of careers that are in demand in the job market. Job shadowing would expose girls to job experience and role models in STEM careers, which would subsequently motivate the girls to aspire to following STEM careers and improve the quality of their learning outcomes in STEM subjects sufficiently to take up studies in these fields.
By 2012, the programme had expanded its job-shadowing programme to all the nine provinces of the country. It had also created an alumni association and introduced a new teacher development component.

Purpose/Objective:

This evaluation examined how the Techno Girl programme operates, and the determinants for its efficiency, effectiveness, relevance, impact, and sustainability in its furtherance. It assessed what has been achieved, and established whether the goals and objectives of the programme remain consistent with the needs and priorities of the beneficiaries. The evaluation aimed to gather evidence to show the partners what the beneficiaries are gaining and whether the programme contributes to improving the quality of learning in STEM subjects and enhancing the uptake of careers in STEM fields.

The evaluation covered the entire period of the programme, from its inception in 2006 to June 2014.

The findings of this evaluation are expected to facilitate dialogue between the programme’s stakeholders to strengthen plans for its future development. It identified lessons learnt and formulated recommendations to reconstruct the programme’s theory of change that will, inter alia, facilitate future assessment of the impact of the programme.

Objectives of the evaluation
The overarching objectives of the evaluation were fivefold:

  1. Effectiveness: To establish what the Techno Girl programme has achieved with regard to its stated objectives and expected outcomes;
  2. Quality and relevance of design: To assess the continued relevance of the design and the validity of objectives;
  3. Efficiency: To determine if resources are being used economically to deliver the programme;
  4. Impact: To examine the contribution of the programme to the overall goal of increasing and enhancing uptake, by marginalised girls, of STEM-based careers critical to economic growth of South Africa;
  5. Sustainability: To assess the potential for sustainability of the programme in its furtherance.

Methodology:

A multi stage sampling technique was used to select respondents at province, district and school levels. Qualitative methods-desk reviews, interviews and discussions were used to establish the activities of the programme and their effects on the beneficiaries, and then to compare findings across provinces to identify key equity issues where possible.

Findings and Conclusions:

The findings of the evaluation provided evidence that the TechnoGirl programme has a clear and compelling case that women, like men, flourish in STEM when they are given the opportunity, support and a cognitively stimulating environment for science education. It is this opportunity, offered under the program, through job shadowing, provided under a unique pubic private partnership arrangement that was identified as one of the most significant achievements of the program over the last seven years.
The evaluation recognizes that providing girls the opportunity to practice their newly acquired knowledge, skills, values and attitudes in educationally safe environments was an impact theme that was the most successful in the programme. Results reported on this theme were central in the success of the programme as a whole.
The TechnoGirl programme has evolved over time. This imposed limitations on methodology, in particular, the use of contra-factual analysis. For this reason, the evaluation was based on contribution analysis, in which qualitative methods - desk reviews, interviews and discussions were used to understand differences in perception to make comparability of findings across provinces possible to mitigate the conformation bias.
The lack of reviews since inception of the program also meant lack of a learning system from which the evaluation could determine what and how much the program has learnt from experiences and how these may have been used to improve the quality of results overtime.
Several indicators that were prioritized could not be verified due to lack of data.  The absence of basic monitoring data meant the evaluation had to rely on interviews and discussions to compare, validate and triangulate data and findings. With the exception of the job-shadowing programme, there was little data available from the monitoring system to track progress on the following key result areas of the programme.

Recommendations:

Strengthen the mentorship framework to ensure better coordination and delivery of the mentorship content and integration of the job-shadowing programme into partner company specific routines and standards.

Develop a more comprehensive narrative on what the mentorship programme entails to reduce variations in the mentorship approach across companies and create the synergy needed to address the challenges to the scale and translate the success of the programme at individual, school, company or provincial level into a national scale.

Develop a strategic narrative on how the programme would seek to grow and expand its intake equitably within schools, companies, districts and provinces both in the mid to long term.

Adopt a more clustered approach to programming as the nature of growth; expansion and the transition of project activities affect how benefits are delivered and how beneficiaries interact with one another and with their mentors.

Ensure a stronger alignment between job shadowing and career interest of beneficiaries.
Strengthen the selection criteria. Uweso needs to find ways to select beneficiaries for whom the programme makes the biggest difference.

Develop a narrative that addresses all seven objectives of the programme. The Techno Girl Initiative also needs to elaborate a comprehensive narrative that links implementation to all seven strategies in order to impact fully the goal of the programme.

Develop strategies to successfully combine or link job shadowing with remedial support for content in line with the goal to increase and enhance the uptake of STEM careers. Develop an explicit narrative on what the programme intends to do and how it would accomplish this objective.

Consider expanding the partnership model to organisations that have the competencies and the expertise to address the content matter of school based STEM subjects



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2015

Country:
South Africa

Region:
ESARO

Theme:
Education - Girls

Type:
Evaluation

Partners:
Department of women, Uweso consulting

Language:
English

New enhanced search