Evaluation database

Evaluation report

2011 Moldova: Evaluation of Integrated Management of Childhood Illnesses Initiative in the Republic of Moldova, Years 2000-2010



Author: Institution: Center for Health Policies and Analysis in Health; Partners: Ministry of Health

Executive summary

With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labeled as ‘Part 3’ of the report.

Background:
The Republic of Moldova was among the first countries in the WHO European Region to implement the Integrated Management of Childhood Illness (IMCI) Initiative starting in 1998 as the most cost-efficient strategies of improvement of mother and child care.
The IMCI initiative in the Republic of Moldova was implemented in three phases.
Phase 1: Program adaptation and introduction (1998-2000)
Phase 2: Program piloting (2000-2002)
Phase 3: Program scale-up (2003-2010)
End-of-program evaluation of IMCI implementation was needed to see if IMCI had an impact, and especially what remaining barriers need to be addressed in order to ensure its sustainability, acceptance and integration.

Purpose/Objective:
The main objectives of the evaluation were:
a) To assess the relevance of Project outcomes, the effectiveness and efficiency by which IMCI project outcomes are being achieved, their sustainability and contribution to overall policy framework in the context of Mother and Child Health Care system

b) To conduct impact evaluation on child health, taking into consideration initial situation (baseline), the timing of the interventions and how long it might take from intervention e.g. training of medical staff or other stakeholders to see the effect on child health and other beneficiaries with particular attention to gender issues and reaching the most vulnerable ones when appropriate.

Methodology:
The evaluation revolves around the evaluation criteria: (i) relevance, (ii) efficiency, (iii) effectiveness, (iv) impact, (v) equity and (vi) sustainability. The initial design was a formal summative evaluation to rely on analysis and synthesis of existing data sources and complement existing quantitative data with qualitative research. However, in the process of refining the evaluation methodology and during the desk review phase we found limitations in data availability and quality and additionally included quantitative survey of health providers and care givers, outcomes in order to measure knowledge and behaviors of these categories.

The data sources for this evaluation were: (1) national routine statistics for data on infant mortality rates and under-5 child mortality rates, immunization coverage, anemia and malnutrition rates in children under five years; (2) national IMCI M&E reports for program output and process indicators; (3) survey data of primary health care providers; (4) survey data of health seeking caregivers of children of 0-5 years; (5) structured interviews with city and rayon health managers responsible for IMCI in their administrative unit; (6) structured interviews with key informants - national stakeholders involved in IMCI implementation and coordination.

A representative sample of 10 administrative units was included in the evaluation. The target populations were (1) health managers of raion or municipal Family Medicine Centers; (2) primary health care providers from urban and rural family medicine or health centers; (3) main care givers of children of 0 to 5 years (mothers, fathers and grandmothers); (4) key informants.

For data collection a face-to-face interview was used for the survey of PHC physicians and caregivers and structured in-depth interviews and a focus group discussion were used for the qualitative data collection form health managers, key informants, PHC providers and caregivers. SPSS 13 software was used for data entry and analysis. After entering all data, the database was checked for accuracy based on filter questions, transition questions and the internal logic of the questionnaire. Data analysis included frequency reporting. The structured interviews were recorded, transcribed and coded.

Findings and Conclusions:
The overarching conclusion is that IMCI has worked well in the Republic of Moldova. The IMCI strategy has had impact on decreasing under-5 child mortality and infant mortality rates, although difficult to evaluate the extent of contribution to this significant reduction because of parallel interventions. Yet, the IMCI strategy has probably had an impact on changing the child mortality structure and on significant decrease in U5MR and IMR due to acute respiratory diseases and acute diarrheal diseases and decrease in postneonatal IMR. The area where IMCI had less significant and sustained effect is the percentage of at-home deaths and within 24 hours of hospitalization in children and mortality due to unintentional injury and poisoning.

1. Impact: the under-five mortality rate has seen a significant reduction from 23.2 per 1,000 live births in 2000 to 13.6 in 2010, with steepest reductions registered in the years 2000-2006, when most IMCI training and other activities have been implemented in country districts. The IMR has also seen a gradual and stable reduction from 18.3 in 2000 to 11.7 in 2010, even with the adoption of the WHO definition of lifebirth in 2008 that has not shown a large increase in the following year. It is difficult to evaluate the extent of IMCI contribution to reductions in IMR and U5MR given several other major processes that have contributed to decreasing mortality rates.
2. Equity: Mother and Child Health Equity Analysis conducted in 2010 concluded that whereas overall child mortality rate decreases, the remaining rates disproportionately affect vulnerable children from rural areas, lower SE quintiles and Southern region. The same analysis reported a moderate level of inequity in nutritional status and anemia rates in children.
3. Relevance of IMCI strategy is perceived differently by health providers. PHC physicians from rural areas, PHC who have been retrained from internal medicine and nurses and younger health providers find it highly useful, while PHC nearing the retirement age, former pediatricians and those living in Chisinau, Balti and some district centers perceived the program as too basic and underestimating medical practice.
4. Barriers: Only half of physicians are satisfied with IMCI implementation, the main impeding factors being an overburden of paperwork and reporting and the list containing drugs with low acceptance or restrictions in the choice of antibiotics.
5. The national M&E system has been successfully instituted nationwide, but currently needs major revisions in decreasing redundancy of the list of indicators and better definitions, as well as decreasing the burden on health providers.
6. Institutionalization and sustainability have been a contionuous focus from design and throughout implementation. The coordinating function of whole program is implemented by a national coordinator under the Mother and Child Department of the Ministry of Health; the national monitoring and evaluation of the IMCI activities are performed by the M&E unit of Mother and Child Health Institute; the IMCI trainings are credited and accounted in the Continuous Medical Education system; the trainees are certified and deployed within the system by Ministry orders; the venues are offered by the system; the IMCI is included in curriculum for medical students (both physicians and nurses); the IMCI materials (Mothers Agenda and Child Medical Records) are official medical forms, institutionalized by the system.

Recommendations:
1. To strengthen IMCI activities in the areas where changes have not yet produced a sustained effect:
a. For effecting at-home mortality, to continue and increase focus on danger signs awareness, particularly in the most vulnerable families. The initiative of the local IMCI coordinator in Cimislia to print danger signs as a poster and give it to providers to be posted in their homes could be scaled-up to areas with higher mortality rates than the country average. The second initiative of a health manager, to develop a list of children at higher risk for morbidity and mortality and setting responsible family physicians and social assistants could also be considered as a model for scale-up.

b. An additional in-depth study on anemia control in children might be required, to better understand the increasing trend in anemia rates in children. Based on the qualitative findings of this study, we would recommend to NHIF to look into possibilities to include a more acceptable drug in the basic package, and for health providers to intensify advice on the need of sufficient protein and vegetable intake and supervise the quality of screening and iron-supplement prescription by PHC workers. Flour fortification with iron could be considered as an option as well.

c. For strengthening counseling for development, to revise the current module to see if it is effective, devise a standard algorithm for counseling for development and provide local refresher courses to PHC workers. In order to increase time available for counseling, decrease the reporting burden from PHC workers.

2. To make a revision to IMCI provisions based on its relevance to national protocols and DCM classification, revise the current list of drugs and revise the current IMCI assessment form to make it more useful and user-friendly for family physicians and more integrated with the other requirements.

3. To revise the current M&E reporting requirements, namely to revise the overall list of indicators, improve and standardize their definitions, revise data collection forms and reduce the reporting burden at the health provider level, by decreasing the periodicity of and complexity of reporting.

4. Explore a more sustainable model for institutionalization of printing costs of informational materials for care givers, as the model where local health authorities are responsible for its printing has not worked thus far.

5. To continue in-service training for PHC nurses through standard 96-hour training and maintain high quality of IMCI training in medical. Additional actions to enhance effectiveness of nurses might include revision of their job description, performance-based payments, additional training on helath promotion and other as found appropriate by national stakeholders.

6. To provide IMCI orientation training to emergency room health staff



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search