We’re building a new UNICEF.org.
As we swap out old for new, pages will be in transition. Thanks for your patience – please keep coming back to see the improvements.

Evaluation database

Evaluation report

2016 PD: DFATD: Improving Treatment of Childhood Diarrhoea and Pneumonia: A proposal to scale up coverage of ORS, Zinc and Amoxicillin External Evaluation Report



Author: Ipsos Kenya

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System (GEROS)". Within this system, an external independent company reviews and rates all evaluation reports. The quality rating scale for evaluation reports is as follows: “Highly Satisfactory”, “Satisfactory”, “Fair” or “Unsatisfactory”. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report, and the executive feedback summary labelled as ‘Part 3’.

Background:

UNICEF’s Diarrhoea and Pneumonia Treatment Initiative in four countries- Kenya, Tanzania, Niger and Ethiopia, was a two-and-a-half-year project supported by DFATD-Canada (now Global Affairs Canada) from April 2013 to December 2015. The overall goal of the project was to contribute to a reduction in child morbidity and mortality through increased coverage of effective treatment for diarrhoea and pneumonia. The ultimate purpose of the project was embedded in the renewed commitment to child survival through A Promise Renewed (APR) - to support scale-up efforts in the countries that accounted for the majority of under-five mortality. UNICEF planned to use this project to accelerate effort to address two of the major child killers – diarrhoea and pneumonia – through building capacity of health systems and reaching out to non-traditional partners.

Purpose/Objective:

The objectives of this initiative were to:

  1. increase public awareness and generate demand for appropriate diarrhoea and pneumonia care-seeking among providers and caregivers,
  2. increase availability of essential medicines used in management of diarrhoea and pneumonia through strengthening procurement and supply chain management,
  3. strengthen public-private partnerships and make better use of private sector channels for appropriate diarrhoea and pneumonia treatment and (iv) expand access to effective integrated treatment services for diarrhoea and pneumonia at community and front-line health facility level.

Methodology:

The study employed a mixed method approach, which comprised of both desk research and primary research methods. The desk research phase comprised of review of various project related documents including project baseline report, project monitoring data e.g. training reports, supervision reports, and other related materials such as UNICEF procurement and supply project data, relevant national policies, workshop/ meeting minutes as well as administrative reports among other documents. Further, the study collected primary data by carrying out key informant interviews targeting key project stakeholders, relevant national government officials, diarrhoea and pneumonia coordination group leads as well as regional persons in charge of health in the different areas. Additionally, the study made use of face-to-face interviews using a semi-structured instrument with caregivers of children under the age of five years.

Findings and Conclusions:

In this chapter, the evaluation findings are presented in detail. The evaluation report is structured according to the OECD/DAC criteria of relevance, effectiveness, efficiency, sustainability and impact as they are listed in the ToR. The evaluation questions and the corresponding results also need to be attributed to the OECD/DAC criteria. Results referring to the crosscutting issues (poverty, gender, and environment) also need to be considered under the OECD/DAC criteria or the evaluation questions, but additionally need to be described separately. Statements and conclusions must be comprehensible and be supported by data. Wherever it seems relevant, data must be presented and interpreted in a sex-disaggregated manner. Hypotheses must be verified and falsified.

In this section, we present findings of the external evaluation for Kenya, Niger, Ethiopia and Tanzania in different sections of this chapter but structuring the findings for each country around the OECD criteria of relevance, efficiency, effectiveness and sustainability. 

Recommendations:

The project was able to make a number of achievements. For instance, it was able to increase the level of awareness for childhood diarrhoea and pneumonia as well as contribute towards the increase of the demand generated for appropriate diarrhoea and pneumonia treatments among providers and caregivers. Further, it was able to increase the number of children under five treated for diarrhoea and pneumonia and this could have been attributed to the improvements made in service delivery and to the improvements in the capacity of frontline workers to provide integrated case management. However, there were challenges with the quality of supervision of these frontline health workers. While structures and procedures had been laid down for their supervision, this did not happen in most countries and this needs to be managed in future programs. Additionally, there was an increase in public private partnerships for diarrhoea and pneumonia treatment, and these ranged from partnerships for demand creation, partnerships for improving service delivery to partnerships for improving supplies management. These partnerships were broad in their inclusion and brought together government players at both national level and local level, as well as other originations with programs similar to the implemented program. There were co-packaging initiatives that UNICEF laid the ground for and the approaches used differed from country to country. In some, the policy and regulatory framework was laid down to support these, in others, private sector players were engaged to support these bundling efforts, while still in others, the central drug stores that are in charge of procurement were engaged to support these co-packaging efforts. However, challenges were observed in the attempt to improve access to appropriate diarrhoea and pneumonia treatment through public and private sector channels.

Lessons Learned:

A number of challenges were faced during the study’s implementation processes. These included the following:

  • Desk research materials- at the onset of the study implementation, a list of desk research materials needed for review were requested for from the various UNICEF project implementation teams. There was however a delay in the collation and provision of these materials. This challenge was overcome through liaison with the lead coordination teams who assisted in following up with the individual teams responsible. In addition, appointments and physical visits were made to the offices where materials were still missing for review of these at the local offices.
  • Ethical clearance- for implementation of any research work in each country of interest, Ipsos and its partners liaised with the relevant authorities for the acquisition of required research permits to allow for the smooth implementation of research studies. Where the nature of the research study is deemed intrusive, further ethical clearance is sought on a project by project basis. For the implementation of the primary research methods in the evaluation work carried out in this study, the acquired in-country research permits sufficed since the nature of the data collection was not deemed intrusive to warrant for the seeking of further ethical clearance. In Tanzania however, it was found that though Ipsos had the required permit to carry out research work in the country, and though the nature of the data collection being carried out was not intrusive to warrant for the seeking of ethical clearance certification, this was still required. Ipsos therefore pursued the acquisition of this certification from NIMR (National Institute for Medical Research), a process which delayed the primary research phase of data collection in the country for a period of 3 months.


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Year:
2016

Office:
Programme Division

Region:
Headquarters - Global

Type:
Evaluation

Theme:
Emergencies

Language:
English

Sequence #:
2016/111

New enhanced search