We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2013 South Africa: Formative evaluation of textbooks and workbooks in South Africa



Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding, Best Practice”, “Highly Satisfactory”, “Mostly Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.

Background:

Textbooks and workbooks have long been considered a critical challenge within the South African education system. The Textbook Development Institute notes that: concern about the quality and effectiveness of the textbooks used by educational institutions is the most neglected and underrated factor impacting on the quality of education in South Africa. Teachers are often blamed for the poor standard of education in the country. Little attention is however given to the poor standard or quality of the resources, particularly textbooks, which teachers are required to work with.
These issues have been identified as a major concern by the Presidency, and the Department of Basic Education was required by a Presidential injunction to develop and provide resources (and specifically workbooks) that would assist learners to improve performance in the critical areas of literacy and numeracy.2 The DBE has undertaken a major initiative to provide Mathematics and Language workbooks to learners in order to accelerate progress towards Education for All, in terms of access to and quality of education. In addition to the development of workbooks, the DBE mandated that English as a first additional language (FAL) be introduced from Grade 1 in 2012.
The first delivery of workbooks to schools took place in 2011. The second delivery took place in 2012. The workbooks for Grades 1 to 9 were aligned with the CAPS. The optimal use of the workbooks, the Annual National Assessment (ANA), and the CAPs are seen to be the three priorities that will drive improvements in the Basic Education sector in South Africa.
Supplementary textbooks in Mathematics and Physical Science (developed by the Shuttleworth Foundation) for Grades 10-12 were also delivered. Grade 10 textbooks have also been aligned to the CAPS. The provision of textbooks in Grades 10-12 is intended to provide adequate resources for students and to reinforce language and literacy skills. 

Purpose/Objective:

The evaluation of the Workbook project is formative. It aims to:
1. Contribute towards the improvement of both the effectiveness (including quality) and utilisation of workbooks and textbooks in schools in South Africa;
2. Provide feedback on the performance of language and mathematics workbooks, to guide adjustments to future editions

Methodology:

In order to address the research questions, the study was separated into three parts; (a) a large-scale study drawing on a representative sample of schools, (b) three school case studies and (c) a desk review of workbooks and textbooks.
1. For the large scale study component of the evaluation, a nationally representative sample of schools was selected. This part of the study was designed to give a global sense of how the intervention is working.
2. For the case studies, three schools were selected. This part of the study was designed to provide insights into the perceptions held by stakeholders in schools about the workbooks, to refine understanding of the performance and utilisation of workbooks, and to guide adjustments for future editions. 
3. The desk review evaluated the textbooks and workbooks using criteria, that were developed drawing on information from a literature review on the characteristics of quality workbooks and textbooks.

All three components of the study focused on Grades 3, 6 and 9 for the workbook evaluation. Data related to the Grade 10 textbooks came from classroom observations and the desk review. A total of 327 schools were selected to participate in the evaluation.  They were represented in these proportions in the sample. Data were collected using an inventory/observation tool, Head of Department questionnaire, student questionnaire, teacher questionnaire, school governing body representative questionnaire, semi-structure School Leader interview, semi-structured grade 9 student focus group, semi-structured parent focus group and semi-structured teacher interview. 
A total of 969 teachers responded to the teacher component of the survey.  The Learners component of the survey had 455 respondents. There were 337 respondent records in the HOD data file with a similar number of respondents from each province.  There were 269 representatives who responded to the survey. 

Findings and Conclusions:

There were 290 schools from which data were available for this question. Of these, 63.5 per cent had all teachers reporting that learners were using workbooks, 22.1 per cent had no teachers reporting the use of workbooks, and in the remaining 15.5 per cent of schools, some teachers did and other teachers did not report the use of workbooks. Thus, in just under 80 per cent of schools, at least some teachers were using the workbooks.  Generally, at all case study schools, teachers and school leaders saw the workbooks as an effective tool in teaching and learning. 

Data on use of workbooks in class indicate teachers reported using workbooks more often in the last week than textbooks, teacher made worksheets and materials and other worksheets and materials. Teachers indicated that the workbooks cater to a wide range of learner abilities. Teachers expressed some concern about the rate at which students are expected to progress through the books. The vast majority of learners take their workbooks home most days.

Barriers to full utilisation of the workbooks can be summarised as late or no workbook delivery, lack of communication with schools and parents regarding the aims and objectives of the workbooks, lack of formal assessment and monitoring , and errors and perceived errors in the workbooks. Errors were typographical only. Perceived errors were due to the use of academic language rather than ‘playground’ language.
A literature review was undertaken in order to identify the characteristics of quality textbooks/workbooks. Evaluation tools developed across international and national settings were also reviewed. A quality rubric was developed on the basis of the national and international reviews and each of the focus workbooks and textbooks were reviewed against the criteria.  Assessors found that the workbooks have all of the characteristics of quality workbooks to a moderate or to a major extent.

Recommendations:

On the basis of the available evidence, the following recommendations are made in order to contribute towards the improvement of both the effectiveness and utilisation of workbooks and textbooks in South Africa and provide feedback to guide adjustments to future editions:

1. In terms of what schools can influence, the most important factor is the quality of the teacher in front of the classroom.8 It is recommended that professional training for teachers related to using the workbooks be undertaken. A professional training pilot program might incorporate and compare teacher perceptions of those completing training and those who had not in order to compare pedagogic differences.

2. Provide training DVDs for teachers showcasing the ways in which the workbooks can be used creatively within classrooms.

3. Develop a communication strategy to inform School Management, teachers, parents and learners about the workbook project, including the aims of the project. It is crucial for parents to understand the aim of the workbooks and have an understanding of how it might help their child. Developing a communication strategy for parents, specifically, is likely to increase utilisation in the home. As part of the communication strategy, it is recommended that the aims and objectives of the workbook project be made explicit to schools and parents. For teachers, this might include explanations regarding the approach to language within the workbooks in order to reduce the perception of errors.

4. Consider the addition of an information sheet for parents with each workbook.

5. Consider making learning goals explicit within the workbooks.

6. Provide templates to monitor learner progress in workbooks.

7. Ensure all workbooks undergo further editing.

8. Provide more space within workbooks for learner responses.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2013

Country:
South Africa

Region:
ESARO

Theme:
Education

Type:
Evaluation

Language:
English

New enhanced search