Evaluation database

Evaluation report

2012 Zimbabwe: Evaluation of the Use of Point of Care PIMA CD4 Cell Count Machines for HIV Positive Women and their Families in Maternal, Newborn and Child Health (MNCH) Settings in Seven Districts in Zimbabwe



Author: Dr Sekesai Mtapuri-Zinyowera, Edward .T. Chiyaka

Executive summary

"With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is "Outstanding, Best Practice", "Highly Satisfactory", "Mostly Satisfactory" or "Unsatisfactory" before using it. You will find the link to the quality rating below, labeled as 'Part 2' of the report."

Background:

The HIV pandemic remains the most serious infectious disease challenge to global public health. There are nearly 900 new infections per day in children <15 years of age in 2011, compared to 1,500 per day in 2006, of which 97% of them occurred in the low and middle income countries; over 95% of which are associated with mother to child transmission (MTCT). In Zimbabwe, HIV remains one of the leading causes of mortality (26%), among pregnant women with an estimated Antenatal Clinic (ANC) HIV prevalence of 16% reported in 2009. This high prevalence in women of child bearing age results in a high number of children infected through mother to child transmission.

In response to the MoHCW’s request to support districts reported as performing poorly in terms of providing PMTCT services to those in need, starting October 2009, UNICEF supported scale up of comprehensive PMTCT/Paediatric HIV interventions integrated in MNCH services in seven districts in the country. In October 2010, UNICEF procured 45 point of care (POC) PIMA CD4 count machines, and distributed them to 35 high volume ANC sites whilst the other 10 supported the Zvitambo project. The country plans to scale up use of these machines in health facilities nationwide.

UNICEF and MoHCW commissioned an evaluation to generate evidence in order to improve understanding of the effectiveness of the PIMA POC CD4 count machines in maternal and new-born child health (MNCH) settings in country; document best practices, lessons learnt, challenges and recommendations related to scale up of this new technology.

Purpose/Objective:

The purpose of the evaluation was to generate evidence that will lead to an improved understanding of the effectiveness of PIMA POC CD4 machines in MNCH settings; document best practices, lessons learnt, challenges and recommendations related to the scaling up of this new technology.

1. To document the processes involved and any challenges experienced in the procurement, distribution, and maintenance of the POC CD4 machines, the health worker training, the EQA.

2. To assess utilization of the PIMA POC CD4 count machines for CD4 cell count, the usefulness of the job aids; and document best practices and challenges faced.

Specific objectives:

3. To determine the effectiveness of the PIMA POC CD4 count machines in improving assessment of ART eligibility within the MNCH settings for pregnant women and their families; and the access to ART for HIV positive pregnant women for their own health in a timely manner during ANC.

4. To derive lessons learnt to inform future scale-up of use of POC CD4 machines across different MNCH settings.

Methodology:

A mixed methodology approach was used in this evaluation. By using different methods at various points in the evaluation process, the evaluation team intended to build on the strength of each type of data collection and minimize the weaknesses of any single approach. Also, the mixed method approach to evaluation intended to increase both the validity and reliability of evaluation data. This approach sought to accomplish the following:
- Make the best possible estimate of the extent to which the project/program has produced its intended outcomes.
- Identify the factors that positively or negatively influence the magnitude and the direction of the outcomes.
- Facilitate an assessment of the change that the PIMA POC CD4 machine may have made to improving access to CD4 cell count testing and improved initiation to ART by pregnant women and their families, in comparison to sites without the POC CD4 machines.

Data collection methods employed included; review of literature, consultative meetings with UNICEF staff and key stakeholders, indepth interviews with stakeholders at national, district and local levels, among them trained users of the PIMA POC CD4 count machines at health facilities and other health facility staff members such as MNCH staff, counsellors, laboratory staff and ART personnel. Additionally, a data abstraction tool was used to collect data from patient records from the 35 sites with the PIMA POC CD4 count machines and 10 other comparative sites without the PIMA POC CD4 count machines. Field work was conducted from the 2nd of May 2012 and ended on the 21st of June 2012. Content analysis of qualitative data was carried out and for quantitative data frequencies, descriptive statistics, and inferential statistics were used. Triangulation of the complementary methods of data collection was also done to ensure that findings of the evaluation are reliable, valid, comprehensive and informative.

Findings and Conclusions:

- UNICEF Zimbabwe adopted an evidence based approach to inform the PIMA POC CD4 count testing programme implementation. This included the utilization of evidence from the country’s appraisal report of the PIMA POC CD4 machine in 2009, conducting a situational analysis of pregnant mothers, and consultations with the Ministry of Health and Child Welfare and other key stakeholders. Roles and responsibilities were clearly defined from the outset of the programme.
- The study revealed that the machines can be easily used and maintained at the sites. The battery makes it a more appropriate technology for areas with inconsistent power supply and remote areas.
- It was widely acknowledged that the PIMA POC CD4 count machines improved patient management since results were given on the same day, which was not the situation before their introduction.
-The program has been successful in improving ART eligibility and ART initiation at sites that were already ART initiating sites.
- At sites with no PIMA POC CD4 cell count machine the average period to commence ART after getting a CD4 cell count result was 32.7days. For sites with PIMA POC CD4 cell count machines but with no initiation of ART the average number of days clients with CD4 cell results waited to be initiated on ART was 30 days, whilst for the 18 sites with the PIMA POC CD4 cell count machines and where ART initiation occurred, the average waiting period was 15 days.
-Users trained on the job by the formally trained users expressed confidence in using the machine.
The introduction and use of the PIMA POC CD4 count machines in selected health facilities’MNCH settings, with support from UNICEF Zimbabwe; has contributed to the improvement of service delivery at site level through provision of a timely CD4 cell count to pregnant mothers in order to assess eligibility to ART for their own health and prevention of mother to child transmission of HIV. 

Recommendations:

The following are the key recommendations of the evaluation:
- Provide training for supervisors and for more staff members at sites such that in the event of staff movement, CD4 testing will continue.
- Ensure that there is a floating machine to cater for health facilities in hard to reach places in case there are machine breakdowns or the battery runs down so that services are not disrupted.
- There is need to add more POC PIMA machines at hospitals, specifically in the ANC, OI and outpatient departments to cater for an increased patient load.
- There is need to provide health facilities with power back-ups such as solar power.
-Ensure that PIMA POC CD4 cell count sites are also ART initiating sites.
- Engage the hospital equipment maintenance units especially on maintenance and repairing of the machines to allow for sustainability.
- Establish necessary structures to ensure that all sites that do the CD4 testing are able to initiate patients on ART. This may also include training of nurses from all sites with the PIMA POC CD4 count machines to be able to initiate patients on ART and being able to do Hb measurement.
- Engage the Government of Zimbabwe to unfreeze posts for health cadres and also to increase the health facility establishments to ease workload on personnel.
- Provide health facilities with power back-ups such as solar power.
- An EQA program has to be implemented to check the quality of results.
- Purchase more conventional machines for District laboratories as through put is higher compared to the POC CD4 cell count machine.
- Enforcement of daily or routine internal quality control procedures.

Lessons Learned:

There is need for a clear training strategy that takes into account the need for on-the job training, training of supervisors, training of trainers 100% reliance on those formally trained users.
- Training of users should pay equal attention to both practicals and theory to enhance its effectiveness. It should include more emphasis on error codes, software, data storage and retrieval.
- Supervisors of the formally trained users were not included in the training workshops.
- No clear communication regarding MoHCW’s position regarding skills transfer through on the job training of other staff by those formally trained.
- Adequate funds should be made available for monitoring activities.
- EQA is important for ensuring that quality results are being produced by users. Ideally, it should be factored in from programme inception.

Additionally, a best practice that have the potential of being replicated is establishing One-Stop-Centres; providing HIV counselling and testing, CD4 testing using the PIMA POC CD4 count machine, and ART initiation at the same health facility; providing HIV counselling and testing, CD4 testing, and ART initiation. This model was observed at some of the sites like Mtora Mission Hospital. However, for noting is that such a model would benefit from attention to issues of confidentiality and staff workload. Clients interviewed at sites that had this model perceived the services received as effective in addressing their needs.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search