Evaluation database

Evaluation report

2007 UGD: Evaluation of the Joint United Nations Emergency Health, Nutrition and HIV/AIDS Programme in North and North Eastern Uganda



Author: Kathy Attawell, DFID.

Executive summary

Background
The joint UN emergency health, nutrition and HIV/AIDS programme in North and North Eastern Uganda commenced January 2006. DFID’s total commitment for 2006 and 2007 is £11,305,676 (£6,837,699 ($11,829,220) in 2006 and £4,467,976 ($8,489,156) in 2007 (see Annex 1).

The programme Goal is: Reduced morbidity and mortality among internally displaced persons, returning and returned populations and those affected by cattle rustling in the Acholi, Lango and Karamoja sub-regions and in the two districts of Amuria and Katakwi in the Teso sub-region. The programme Purpose is: Sustained delivery of an integrated health services system including strengthening the health management information system (HMIS), epidemic preparedness and response in internally displaced people (IDP) camps, transitional camps and return populations. Programme Outputs focus on strengthening human resources for health, health infrastructure, HMIS and epidemic preparedness and response (EPR), and coordination of the health sector response; and on delivery of child health, malaria control, reproductive health, and HIV/AIDS interventions.

In July-August 2007, DFID commissioned two consultants to evaluate the programme.

Purpose
To: 1) identify impact, outcomes and outputs resulting from the programme;
2) clarify short-term humanitarian and health, nutrition and HIV/AIDS needs in Northern Uganda and recommend areas for joint UN programme support; and
3) identify key challenges for the health sector during the transition from the current humanitarian situation to recovery and development.

Methodology
1. Literature review of relevant documents including the performance reports, minutes of relevant meetings, survey reports etc (List of relevant documents attached in annex 1)

2.Key informant interviews with the UN agencies, GoU, other donors including DFID and SIDA at both country and head- quarters level, Local governments collaborating NGOs, and civil society. List of suggested initial key informants will be provided prior to the field work.

3. Field consultations with the district authorities, NGOs and community groups including focal group discussions with IDPs.

Findings and Conclusions
1. The joint programme is the main source of external support to the health sector in Northern Uganda and has played an important role in ensuring that the population receives basic health care. However, the programme is ambitious in scope and coverage relative to its timeframe and resources, and there is a risk that some interventions will achieve limited impact. In practice, activities have largely been concentrated in districts in Acholi and Lango. Teso and Karamoja have received limited attention.

2. A mortality survey conducted in 2005 showed that crude mortality rates (CMR) in Northern Uganda were significantly above emergency thresholds. A follow-up mortality survey, planned for 2007, has not yet been conducted, so it is not possible to comment on the impact of the programme at Goal level. Achievement of the programme Purpose is likely to be difficult in view of the health sector challenges in Northern and North Eastern Uganda. At Output level, the joint programme has performed better in some areas than others.

3. The programme has provided funds to the Ministry of Health (MOH) to finance payment of a one-time allowance to health workers to address recruitment and retention problems in Northern Uganda. It is too early to judge whether or not this will be sufficient to attract health workers to the region or to reach the target of 55% of posts filled. Feedback from district health teams (DHTs) indicates that recruitment of doctors will continue to be a challenge and that task shifting may be required to address this. Human resource shortages are exacerbated by weaknesses in human resource management.

4. Progress with improvements in health infrastructure has been relatively slow. Construction and renovation of health facilities seen by the evaluation team was incomplete. None had water supply, power and sanitation in place or adequate housing relative to staffing norms. More consideration could have been given to phasing of activities, specifically completion of housing before advertising staff posts, given the impact of lack of accommodation on staff recruitment and retention.

5. Provision of training and equipment has increased the timeliness and completion of HMIS reporting. Further improvements in HMIS are constrained by the limitations of the system itself. The programme has piloted community-based disease surveillance (CBDS) and completeness of reporting is over 95%. Sustainability is a concern, since the system requires multiple forms to be completed by community volunteers and the MOH has no budget for CBDS. The joint programme has also strengthened district EPR capacity and outbreaks have been well investigated, managed and contained.

6.The programme has increased childhood immunisation, vitamin A supplementation and de-worming coverage. Maintaining high coverage will be a challenge when populations move from IDP camps and without external funding. Progress in improving TT coverage has been more limited. The programme has trained health workers and Community-Owned Resource Persons (CORPS) to improve management of common childhood illness at facility and community levels. The extent to which health facilities manage sick children according to IMCI standards is difficult to assess.

7. There is a lack of clarity concerning the roles and responsibilities of CORPS and community volunteers who comprise the village health team (VHT). There is no consistent training for VHT to enable them to provide a basic standard set of prevention and care interventions. Training for VHT and CORPS appears to vary in quality and has not been planned or delivered in a coherent or efficient way. Supervision and monitoring of VHT and CORPS by health facilities is also a concern.

8. Programme nutrition activities have concentrated on surveys and management of malnutrition in young children. Survey findings indicate that global acute malnutrition rates have fallen below the 10% emergency threshold but have increased in return areas in some districts. Community feeding centres are being phased out but support is continuing for NGOs to provide treatment for children with severe malnutrition at health facilities. Underlying illness, responsible for most severe malnutrition, does not always appear to be well addressed. The programme provided additional rations to women attending maternal and child health (MCH) services. Districts will not be able to sustain this approach and there could be an adverse effect on uptake of services when food incentives are withdrawn.

9. The introduction of community coartem has been spearheaded by the programme. VHT have been trained to manage fever and available data indicates improvement in community malaria management. One round of indoor residual spraying (IRS) has been completed in Kitgum and Pader and is planned in Gulu. IRS has taken place without an adequate budget for a second round of spraying or for appraisal of effect. Available funds are insufficient to conduct the planned vector control feasibility study in Karamoja. The evaluation also highlighted some concerns about retention and use of ITNs.

10. There has been significant improvement in access to prevention of mother to child transmission (PMTCT) services. However, the evaluation found limited evidence of scale up of adolescent health, emergency obstetric care (EmOC) or sexual and gender based violence (SGBV) services. No data are available on the proportion of HC III currently providing basic EmOC. No HC IV is providing comprehensive EmOC. Adolescent friendly health services (AFHS) training commenced recently, so provision of AFHS through the programme has not yet begun. It is too early to judge the impact of family planning training. Community networks for management and referral of survivors of SGBV have been established in some IDP camps and the programme reports an increase in referrals to health facilities.

11. The HIV/AIDS component of the joint programme focuses on HIV counselling and testing (CT), quality of care, and increasing awareness among youth. Comprehensive data are not available on the proportion of the population with access to HIV counselling and testing (CT), outcomes for patients on ART or the extent of comprehensive knowledge among young people.

12. Health services availability mapping (SAM) has been conducted in Acholi, Lango and Karamoja regions. The programme has provided support for cluster functioning, and feedback indicates that the cluster approach has significantly improved coordination and information sharing at national and district levels. In some districts, the cluster has helped to reduce duplication of effort, identify gaps, improve coverage, increase joint planning and monitoring of activities, and promote harmonised reporting formats and adherence to national standards. In principle, MOH participates in the national cluster and district clusters are co-chaired by WHO and the District Health Officer (DHO). In practice, MOH and DHO involvement is limited, due partly to a perception that the cluster approach is a UN initiative and partly to limited capacity.

13. The evaluation highlighted a range of benefits and challenges associated with joint UN programming. The joint programme has helped to clarify UN agency mandates, determine roles based on comparative advantage, improve communication and strengthen joint planning. Good working relations have been established between technical staff at central and district levels. Joint programming has also helped take forward the cluster approach and reduced transaction costs for the MOH and DFID. However, joint programming increases transaction costs for the UN agencies, since considerable time is required for joint planning and proposal development. Differences in planning cycles, operational procedures and reporting systems are also challenges to joint programming.

14. In practice, individual agencies implement programme activities separately and the joint programme appears to have contributed to an expansion of UN staffing at district level. The evaluation also raised wider questions about joint programming, including the need for criteria to determine when joint programming is the most appropriate approach and indicators for measuring the efficiency and effectiveness of joint programme processes and outcomes.

15. Districts and NGO partners are positive about their engagement with joint programme UN agencies and the technical support they receive. Challenges cited relate to delays in disbursements and differences in UN agency reporting requirements. The evaluation also identified the need for a common UN agency approach to selection of implementing partners, monitoring performance or providing feedback on performance.

Recommendations
Recommendations for the remainder of 2007 and for future programming, which assumes continuation of a joint health, nutrition and HIV/AIDS programme implemented by the UN, are based on the evaluation findings and challenges for the health sector

During the remainder of 2007 the programme should:

• Together with MOH and local government, develop a system whereby the one-time allowance is paid on the basis of performance and after staff have been in post for a specified length of time; and monitor the impact of the one-time allowance on recruitment and retention of different cadres of health workers as well as progress towards the overall target of 55% of posts filled. UN agencies and DFID should continue dialogue with the MOH on task shifting as an option for addressing critical human resource shortages.

• Take immediate steps to complete planned renovation and construction of health facilities and staff accommodation, and provision of equipment and furniture, and ensure that work is well planned and carried out is to a sufficiently high standard. Completion of staff housing should take priority, to promote retention of existing and newly recruited staff.

• Start to plan for handover of responsibility for HMIS and EPR to the districts in Northern Uganda, and explore with MOH options for simplifying the HMIS. The programme should also start planning for reduction of support for interventions that districts are responsible for delivering as part of the UMHCP. Dialogue with central MOH and districts will be critical to ensure adequate resources are allocated for these interventions. Future support should give greater emphasis to community mobilisation, to create and sustain demand for these services

• Evaluate the accuracy and effectiveness of the CBDS pilots before further roll out. If CBDS proves worthwhile, the programme needs to consider ways in which it can be sustained by the districts and, more specifically, how reporting requirements for VHT can be streamlined and VHT can receive feedback.

• Take steps to follow up training, to assess the extent to which new knowledge and skills are being put into practice and the impact on quality of care provided.

• Together with MOH, conduct a review of VHT selection criteria, roles, training, supervision, monitoring and incentives, and agree common standards and approaches to ensure that VHT can provide a consistent set of community prevention and care interventions. The programme should also assess coverage with trained VHT as the population moves out of camps to transitional settlements or return areas, to determine future selection and training of VHT. The future role of CORPS as the population moves out of camps should also be clarified.

• Ensure that children treated for severe malnutrition are also assessed for other illnesses and receive appropriate treatment and care, in particular strengthening links with paediatric HIV/AIDS diagnosis, treatment and care.

• Monitor the impact of provision of additional food rations to HIV and TB patients and phase out provision of extra rations through MCH services, focusing instead on raising community awareness of the importance of these services.

• Collaborate with other actors, e.g. PMI and the Malaria Consortium, to provide evidence of effectiveness and specific impact of IRS on malaria mortality and morbidity in the endemic context of Northern Uganda.

• Take steps to strengthen and monitor provision of EmOC.

• Continue support for the health, nutrition and HIV/AIDS cluster approach, and explore ways in which dependence on external support could be reduced. The programme should also take action to increase MOH, DHT and HSD ownership and to support their active participation in the clusters; convening meetings at the MOH and at DHT offices and encouraging DHOs to chair meetings should be a first step.

• Consider how district clusters can ensure that comprehensive data on provision of CT is captured. Quarterly QoC supervisory site visits should include meetings with HIV/AIDS patients and review of factors that influence treatment adherence.

• Develop a common set of criteria for selection of NGO partners, a common approach to monitoring their performance and feedback systems. The UN agencies should also streamline and harmonise financial and narrative reporting requirements and take steps to improve the efficiency of disbursements to implementing partners.

• Develop a joint programme plan that sets out what data needs to be collected to measure impact and a coherent approach to data collection and analysis, which uses existing systems as far as possible.

• Together with DFID establish an efficient mechanism to provide oversight to, and monitor the performance of, the joint programme.

DFID should fund a further phase of health, nutrition and HIV/AIDS programming in Northern Uganda during the period January 2008-March 2009, focusing on phase out and transitional support in line with the JMC transition approach; and a separate programme of support tailored to the specific needs and context in Karamoja.

The next phase of programming should:

• Reflect the 2008 CAP and be developed in consultation with the districts. Updated SAM, information on support provided by other development partners and data on population movement should be used to inform ongoing planning of future programming and prioritisation of interventions. Flexibility should be built in to programme design and contractual arrangements to enable the programme to respond to an evolving context and changing needs.

• Hand over responsibilities for delivering UMHCP minimum standards and for HMIS and EPR to districts; provide strategic support for strengthening health infrastructure in return areas and district management and supervisory capacity; identify and implement innovative approaches to address shortages of human resources for health; strengthen EmOC and FP services; and strengthen community level preventive and basic health care, including greater emphasis on hygiene, nutrition, FP and HIV education.

• If managed by the UN, identify one programme manager; deliver the UN contribution in line with ‘One UN’, with joint UN offices and teams at district level; and determine which UN agencies are involved on basis of district priorities and agency comparative advantage.

• Fund NGO implementing partners that have demonstrated good performance and focus on fewer stronger partners.

• Ensure baseline information is collected and an effective log frame and M&E framework, including SMART OVI, are in place.

More specifically, depending on the priorities identified in the CAP and during district consultations, a future programme should:

• Develop and implement additional innovative approaches to address shortages of specific cadres of personnel, for example, Medical Officers and laboratory technicians, in the worst affected districts, drawing on experience in other countries in the region and including performance-related incentives.

• Provide strategic support to strengthen district and sub-county human resource management and supervisory systems and capacity.

• Provide strategic support to strengthen health infrastructure in return areas in districts with the greatest need. This support should include 1) ongoing rehabilitation and construction of facilities (based on district priorities, an updated SAM and the parish approach) and of accommodation (for cadres of staff that are most difficult to attract and retain) and 2) building the capacity of districts to manage and supervise contractors.

• Continue to monitor nutrition status in return areas, including exploring ways in which nutrition indicators can be captured by existing systems. There should be increased emphasis on community nutrition education to address the potential risk of worsening nutritional status in these areas, and specific interventions to address high rates of malnutrition in Karamoja.

• Give greater emphasis to community education on net use.

• Develop a strategy to support districts to strengthen EmOC. This should include, at a minimum, ensuring that all HC III can provide basic EmOC and at least two HC IV in each district have functional theatres, ensuring that a referral system is in place and that ambulances are functional, and that data on provision of EmOC is collected.

• Consolidate efforts to improve FP provision and give higher priority to raising community awareness and creating demand for FP services.

• Work with district clusters to identify gaps in HIV/AIDS prevention, treatment and care and support efforts to achieve comprehensive coverage and service provision. The potential of district clusters to strengthen coordinated planning and implementation and, specifically, to improve coordination of HIV/AIDS activities, including those funded through the US Government, should be exploited fully.

• Increase support for HIV prevention and awareness, including community education and availability of condoms. Condoms should be made available through VHT and included in family kits given to households when they return to their original homes.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Date:
2007

Region:
ESARO

Country:
Uganda

Type:
Evaluation

Theme:
Emergency-Public Health

Partners:
WHO, WFP & UNFPA

PIDB:

Follow Up:

Language:
English

Sequence Number:
2007/016

New enhanced search