2011 South Africa: UNICEF OVC Programme Component Evaluation
Author: Patricia Martin, Vuyiswa Mathambo and Linda Richter
“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”
The HSRC has been commissioned to evaluate UNICEF‟s OVC programme component of the country programme. The objectives of this evaluation are:
To evaluate the contribution of project activities to the key results identified in the country programme action plan using the standard evaluation criteria of effectiveness, efficiency, relevance, impact and sustainability
To recommend strategic priorities for the next 4 years – specifically, how UNICEF should position itself in order to better influence the care and support of vulnerable children in South Africa.
Challenges in improving the care, support and protection of children affected by HIV/AIDS in South Africa
Children are affected by HIV and AIDS in a number of possible ways. Poverty significantly aggravates the vulnerability of affected children; poverty is both a cause and a consequence of heightened vulnerability for children affected by HIV and AIDS.
HIV increases children‟s economic vulnerability. Infection leaves affected families poorer when adults fall ill or divert their income into caregiving or health care costs. The livelihood of children is placed at risk when their parents die, especially their fathers as many face uncertain inheritance prospects. It further impacts on their well being as income poverty prevents the realisation of children‟s basic needs and rights such as health, protection, welfare and education.
There is agreement that the complexity of the impact of HIV and AIDS on the most vulnerable children requires the provision of a complex range of services and support to ensure their care, protection and wellbeing. The South African government has responded positively and developed a number of integrated policies and plans of action which seek to provide children affected by HIV and AIDS with a range of material and psycho-social support. In reality however, there have been a number of deep seated challenges to realisation of these commitments, including poor coordination, poor access to services in vulnerable communities and households; insufficient data about vulnerable communities; inadequate planning and budgeting; insufficient social workers; and various policy gaps resulting in a number of inadequacies in the services available for vulnerable children. 6
Summary of UNICEF South Africa’s OVC Programme
UNICEF South Africa‟s programme component entitled "Social protection for orphans and vulnerable children" (OVC) has two pillars: (1) to provide technical assistance to strengthen government and civil society capacities to implement and monitor quality services for OVC, and (2) to develop capacities to establish community-based support structures that support OVC to access essential services. Two key results were anticipated from this programme component: (1) an increase in coverage of community- and home-based care, support and protection to over 70 per cent of OVC, and (2) the Department of Social Development would be better enabled to coordinate and monitor OVC services.
Evaluation of programme 2007-2010
UNICEF‟s activities and key results as set out in the Country Programme Action Plan (CPAP) and Annual Work Plan are shaped by and responsive to two central priority development pressures that impact negatively on children in South Africa, namely poverty and HIV/AIDS. The shape and focus of the CPAP‟s responses has been moulded by UNICEF South Africa‟s country programme document and its Medium Term Strategic Plan ("MTSP") which are both located within a broader international response to the negative impact of poverty and HIV and AIDS on rights in South Africa and the region more broadly.
The "Social Protection for OVCs" component ("the OVC Programme) of the CPAP under review seeks to overcome vulnerability caused by poverty and orphanhood by supporting the South African government and other role players in civil society and academia in strengthening the delivery of social protection services to orphans and other vulnerable children within the framework of the National Action Plan for Orphans and Children Made Vulnerable by HIV and AIDS ("the NAP").
The overarching outcome is an increase in the percentage of OVCs that receive home-based care, support and protection. The OVC Programme has taken its direction from the MTSP and sought to design interventions that have a greater strategic focus for sustained results. In consequence, the 7
OVC Programme overtly works with government within the boundaries of national OVC policies, laws and programmes; emphasises upstream engagement and building of national and provincial partnerships so as to improve national coordination; uses the UNICEF brand; supports replicable community-based models of support; fills data gaps to improve planning, budgeting and implementation of services; and seeks to systemise best practices through training and the development of national guidelines for services.
UNICEF has successfully used its global brand, visibility and normative role to leverage funds, prestige, awareness of and national conformity with practices and principles promotive of the best interests of the child in South Africa.
UNICEF‟s financial and technical resources were optimally used in the development of partnerships and collaborations to ensure there is no duplication and redundancy within the OVC sector, and this has resulted in a greater complimentarity amongst actors.
In a similar vein, it has used it resources to effectively identify and fill knowledge gaps. Specifically with a view to providing evidence to move implementation forward through appropriate and informed planning.
The Country Programme Action Plan (CPAP) sought to realise two key results:
- Coverage of community-and home-based care, support and protection increased to 70% of OVCs;
- Department of Social Development is able to better coordinate and monitor OVC services. (UNICEF, 2007: pages 6-7)
Measuring the extent to which result a) has been realised is difficult because of a number of factors. The first is the lack of a baseline in relation to OVC. There is a lack of clarity about who qualifies as an OVC; how many OVC were and are receiving care and support; and where they are located. Furthermore, the scattered bird-shot approach to setting up and supporting community-based models of care has resulted in a dispersed collection of community-based sites of varying complexity and 8
size across various municipalities, districts and provinces. This makes it very difficult to assess the extent to which these sites have grown on a national scale. The evidence that is available tends to show a lack of scale-up necessary to realise the 70% target.
Whilst the intended scaling up of models has not found effective traction for effective national scale-up, the development of training modules and guidelines for future use in the design and implementation of projects by government and civil society has proven effective in improving the coverage and quality of home and community-based care and support.
With regards to objective b), the information is indicative of an improvement in the ability of the Department of Social Development to better coordinate, but not necessarily to monitor OVC services. UNICEF‟s support to NACCA resulted in it operating more effectively and this in turn improved the effectiveness of NACCA as a coordination forum. The monitoring ability of NACCA would have been substantially improved if the support for improved information and data collection and management at local, provincial, national and at NACCA level had yielded more positive results. Apart from the national maternal orphan count project, these systems have stalled through lack of maintenance and ownership by the relevant government structures.
Main constraints faced by the project team
There were a number of constraints impeding the effectiveness of the programme interventions. One of these was historically inappropriate pilot sites. Programme staff agreed that the municipal sites they inherited were not in fact chosen on the bases of their being most strategically placed to found long-term scaled up initiatives. The insufficiently strategic foundation was compounded by a lack of national leadership to drive scaling up.
The lack of national leadership is mirrored in the correlating lack of traction between
UNICEF and Government‟s respective "cogs"; there has not been a sufficiently strong symbiosis between their respective interests. 9
Whilst the relationship between UNICEF and the Department of Social Development ("DSD) is long standing and strong, inherent limitations within this relationship have inhibited realisation of UNICEF‟s mandate. Limited leadership buy-in within the Department of Social Development, outside of the HIV/AIDS‟ Directorate and NACCA, and the limited ability of NACCA and the DSD‟s HIV/AIDS‟ directorate (the main locii of UNICEF activity) to compel action provincially, locally and cross-departmentally has frustrated the implementation of a number of OVC related activities supported by UNICEF.
UNICEF has, through its longstanding relationship with the DSD and NACCA provided critical support in the development of the OVC Policy Framework and the NAP, as well as in the strengthening of NACCA.
UNICEF has sought to strengthen information management systems at a local, provincial and a national level, specifically to facilitate the identification of OVC and tracking support provided to them. This initiative was particularly successful in relation to the maternal orphan count project,
The documentation and sharing of information by UNICEF through the NACCA structure has created opportunities for self-reflection against international standards and practices, resulting in opportunities for improving national local projects.
At a project level there has been an increase in the number of community-based sites of support for OVC in various provinces, districts and municipalities in South Africa. Not only are there more sites due to support from UNICEF of the Isibindi model and CCFs; the sites have also been strengthened through training and capacity building supported by UNICEF.
In addition, UNICEF has supported the DSD to implement the Children‟s Act by documenting and costing various service delivery and care models; which documentation and costing has provided a platform for implementation of various provisions of the Act which are of particular relevance to care and support for OVC. 10
Various informants and UNICEF staff members shared concerns about the low levels of appropriation by government and incorporation into government planning, implementation and monitoring processes of the community-based support and information management projects supported by UNICEF.
This mainstreaming of key UNICEF endorsed approaches and solutions into two of South Africa‟s core national policy and programmatic responses to OVC, namely the OVC Policy Framework and NAP has resulted in the mainstreaming of international standards and practices into programmes and processes developed in terms of these two documents.
Orientations for future programming
This evaluation has adopted a utilization-focused approach. It is concerned, not just with documenting what was achieved and what progress was made towards outcomes, but also to identify the challenges, the detours, and how the programme has evolved. Therefore the question of UNICEF‟s positioning, comparative advantage and role was examined with Programme stakeholders with a view to developing recommendations as to how best to move forward to address these challenges. In moving forward, the following recommendations were made for UNICEF to maximise its positioning, comparative advantage and role.
- Review the relevance of the OVC orientation of the Programme. There appears to be a need to review the relevance of the current programme focus on OVC as opposed to adopting a more holistic approach drawing in all vulnerable children, subject to promoting sensitivity within this broader context of the needs of OVC.
- Rationalize geographical targeting of support to respond effectively to the needs of vulnerable provinces and communities.
- Align the Programme more closely with the Millennium Development Goals.
- UNICEF‟s OVC Programme must clarify and publicize its role in relation to the realisation of the rights of vulnerable children, especially in relation to the role to be played by other stakeholders.
- Focus on addressing systemic blockages affecting service delivery generally, before honing in on specific services.
- UNICEF should pursue a more upstream focussed advocacy strategy to ensure systematized and sustainable solutions at a community level.
- UNICEF should practice more of what it seeks to advance; namely more internal coordination and collaboration across UNICEF‟s country programme components.
- UNICEF should adopt a more holistic approach to building relationships with government departments and structures outside of the Department of Social Development.
- The UNICEF agenda should be clarified and the scope delineated through clear definitions of key concepts such as "child protection" and "vulnerable child" and should then be disseminated widely.
Proposed strategic priorities for the next four years
- Support all vulnerable children. Support the strengthening of all vulnerable children‟s social protection policies and programmes as a whole so as to ensure that they are OVC sensitive, rather than OVC targeted.
- Ensure the ongoing realisation of material and psycho-social support. Whilst the poverty alleviation development goals are critically important, it is equally important that the psychosocial needs of vulnerable children are not neglected.
- Continue to support and advocate for effective prevention and early intervention strategies.
- Engage with the full spectrum of coordinating structures set up to coordinate services for vulnerable children.
- Utilise the Children‟s Act to ensure government accountability to support and scale up community and family-based models of care and support.
- Redefine "orphanhood "to make it more Afro centric and define vulnerability in line with international and regional developments.
- Plan and budget for preparatory advocacy, especially those aiming to realise scale-up of programmes.
- Develop appropriate monitoring and evaluation frameworks which focus on advocacy outcomes, the development of appropriate indicators and quantifiable base-line figures.
Partnerships and expertise required to address these priorities
- Foster a nation-wide reach. A national country-wide response to reaching the most marginalised vulnerable children requires nation-wide reach in terms of government stakeholders.
- In moving a national coordinated response to vulnerable children forward, it is essential that government responses are monitored and held accountable to a credible, legitimate and overarching coordinating structure. The Ministry for Women, Children and People with Disabilities has been created for precisely this purpose and it ought to be supported to realise this central role.
- UNICEF is urged to build stronger partnerships with international and national donors to ensure synergy of effort towards a common vulnerable child agenda.
- Partnerships ought to be developed so as to synergistically tackle long-standing service delivery fault lines.
- Strengthen internal partnerships within UNICEF so that the programmes do not operate in silos so as to realise UNICEF Country Programme‟s vision of a coordinated national response to vulnerable children.
- Increase the UNICEF OVC Programme team size and capacity. This does not necessarily mean huge increases in staff size, but a more strategic use of cross-programmatic staff.
Full report in PDF
PDF files require Acrobat Reader.