Evaluation database

Evaluation report

2010 Myanmar: Evaluation of Malaria Risk Micro-stratification Strategy



Author: Department of Health & UNICEF

Executive summary

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”

Background:

A malaria risk micro-stratification was undertaken in order to identify high-risk malaria villages and communities so that malaria control interventions can be highly targeted. The United Nations Children’s Fund (UNICEF) has supported micro-stratification in 80 townships for three years and it is time for re-stratification. This evaluation documents the results thus far from adopting a malaria micro-stratification strategy in UNICEF supported townships.

Purpose/Objective:

The main objective was to evaluate the malaria risk micro-stratification strategy in terms of the process and its impact on the effectiveness of malaria control interventions. Both qualitative and quantitative methods were employed.

The specific objectives were to: 1) Examine the epidemiological impact, i.e. malaria morbidity and mortality, of the 80 UNICEF supported townships after implementation of the micro-stratification strategy; 2) Assess the understanding of the concept and purpose of the micro-stratification strategy among various staff categories; and 3) Explore the changes among staff in level of awareness, confidence and willingness related to malaria control activities at township and health centre level

Methodology:

Quantitative analysis
The quantitative component will examine primarily the impact of LLIN distribution on malaria morbidity and mortality. Since the introduction of the micro-stratification strategy, distribution of LLIN has been focused on vulnerable communities where the risk of malaria transmission is highest. In theory, this highly targeted approach should bring about a visible impact on malaria morbidity and mortality. This component of the study will quantify the impact of the interventions, using epidemiological data available in health centers and hospitals. The analysis will focus on epidemiological changes taking place after introduction of the micro-stratification strategy.

Design
Intervention groups were selected assuming a “wedged” introduction of the interventions. Townships were categorized into five intervention groups in order to assess the effect of various interventions including UNICEF micro-stratification. Yearly data sets from health facilities provided by central VBDC were used to detect epidemiological changes.

Intervention groups
In 2007, 28 UNICEF supported townships were provided with LLINs (phase-one). Of these, three townships are overlapping with WHO/Three Diseases Fund (3DF) supported townships. Therefore, in the intervention group “UNICEF supported townships” only 25 townships have been included. In the remaining three townships it is not possible to attribute any observed changes to the intervention carried out by UNICEF as any observed epidemiological change might also result from the intervention done by WHO/3DF.

Out of 28 UNICEF townships overlapping with WHO/3DF townships, only eight townships received LLINs from UNICEF in 2007 which is why two groups are formed; overlapping townships with and without UNICEF distributed LLINs in 2007.

The two other intervention groups included in the analysis are “WHO/3DF supported townships” and “Non-UNICEF / WHO-3DF supported townships.”

Data collection and analysis
Malaria morbidity and mortality reports from health centers and hospitals were the primary sources of data for the study. Normally, the reports are prepared by local BHS and submitted from health facilities to the township level. The data is aggregated at the township level and submitted to the VBDC project staff at State/Division level. It is further forwarded to the central VBDC where annual datasets are compiled and distributed to organizations like UNICEF and WHO. Analysis yearly data from health facilities will make it possible to detect any epidemiological change that might have taken place within the township.

Pre- and post intervention analysis was conducted for each UNICEF supported township (historical comparison of single townships) and data was aggregated for all intervention townships to assess the overall impact (historical comparison of multiple intervention townships). The same historical analysis of multiple intervention townships was carried out for non-UNICEF supported townships in order to compare the overall disease trend between townships with different interventions (comparison between different intervention groups).

Figure 3 80 Figure 3 UNICEF supported townships (in blue)

Qualitative analysis
The qualitative component will focus on the process. The micro-stratification strategy was adopted by UNICEF not only for improving the effectiveness of the interventions but also for increasing the awareness of responsibilities among Department of Health (DOH) staff involved in malaria control activities at all levels. This component of the study will explore the understanding, perception, empowering and motivating aspects of the micro-stratification strategy among the NMCP staff, TMOs and local BHS. People involved in implementing the micro-stratification strategy were included in the qualitative analysis.

Design
Qualitative data were collected by key informant in-depth interviews. Annex 2 lists people interviewed and places visited.


Sample size and sampling procedure
Initially it was planned to include 20 key informants in the study. However, the number of key informants increased to 30. The sampling methodology was purposive sampling.

Selection of study subjects
All key informants were health staff at either township or State/Division level from one of the three groups listed below. They all worked in townships where micro-stratification had been undertaken.

1. Vector Borne Diseases Control project staff
According to the proposal, at least one regional officer (RO) and four team leaders (TLs) had to be selected. However, five ROs and three TLs were included because of convenience and availability.

2. Township Medical Officers
Although the targeted number of TMOs for the interview was five, only two were eligible as the remaining three were transferred to other places. Instead, one Health Assistant (HA) and two Township Health Nurses (THNs) were interviewed.

3. Basic Health Staff
Four Health Assistants and 12 Midwives were interviewed.

Data collection and analysis
In-depth interview questionnaire guidelines were developed for all VBDC project staff, TMOs and BHS in consultation with the Program, Monitoring & Evaluation (PME) section of UNICEF Myanmar. The core questionnaire guidelines for in-depth interviews are presented in Annex 1.

Due to the nature of the study, formal anthropological analysis was not performed. The report summarizes the findings from in-depth interviews on the understanding, perception, empowering and motivating effects of the micro-stratification strategy at different levels (VBDC project staff, TMOs and BHS). It also includes interpretation of results as well as discussion of strengths and weaknesses in program implementation, leading to recommendations for future program direction.

Findings and Conclusions:

Out of 80 townships UNICEF supported for micro-stratification and related activities, only 25 townships, which were not overlapping with other partners’ malaria programs, had completed distribution of long lasting insecticide nets (LLINs) in 2007. For all 25 UNICEF supported townships combined, the malaria morbidity rate (MBR) was higher in 2008 than it was in the preceding four years. Nevertheless, the 2008 MBR remains slightly below the 1999-2006 average. The malaria mortality rate (MTR) continues to decrease as has been the case since 2005 and remains well below the 1999-2006 average.

It was found that 60% (n=15) of UNICEF townships experienced a reduction in MBR in 2008 compared to the average rates of the pre-intervention period 1999-2006, whereas a reduction in MTR was observed in 96% (n=24) of townships. Reduction of both MBR and MTR was observed in 60% (n=15) of the townships while only 4% (n=1) of townships saw an increase in both MBR and MTR.

In UNICEF townships, the MBR has been reduced by 5% in 2008 compared to the average of 1999-2006, while MTR was reduced by 68%. The MBR reduction in UNICEF townships represents the lowest reduction observed among the intervention groups. The magnitude of MTR reduction is similar to findings in other intervention groups. Population migration and seasonal migrant workers remain the most likely explanation to the limited reduction in MBR with many townships actually seeing an increased caseload in recent years. Increased awareness and proper treatment seeking behavior have contributed to a significant reduction of MTR after introducing the micro-stratification strategy.

Most health staff had a clear understanding of the micro-stratification strategy and activities involved in its implementation. However, some less experienced basic health staff (BHS) did encounter difficulties in the stratification process and were not clear about some of the key terms and definitions which are important for determining the correct stratum for each village. Most staff mentioned that the introduction of the strategy did not adversely affect their ability to carry out other tasks. The participatory process fostered empowerment and a sense of ownership in the community and also led to motivation of implementing health staff.

Recommendations:

Ways and means of improvement of the micro-stratification strategy as suggested by key informants as well as other recommendations generated from in-depth interviews are presented below:

• Micro-stratification guidelines should be modified and simplified. For this, a micro-stratification modification workshop should be arranged in collaboration with VBDC, WHO, JICA and other stakeholders working on malaria prevention and control. At this workshop, retired malariologists, VBDC ROs and senior TLs should also be invited as they have already acquired experience from the present micro-stratification exercise.

• VBDC ROs and TLs recommended that scientific methods should be included in the guidelines but should be feasible for the BHS (e.g. spleen rate).

• BHS suggested that operational definitions should be made clearer for the terms “local transmission,” “indigenous case,” “imported case,” “forest,” and “vector breeding sites.”

• Entomological information should be provided to know the vectors and their breeding sources up to the village level if possible. This information is to be provided by the state and division VBDC teams.

• Hard-to-reach villages assigned to stratum 1a should be reviewed again as not all hard-to-reach villages are malarious by nature.

• TMOs, ROs and TLs pointed out that problems of migrant populations should be considered separately in the micro-stratification guidelines. The criteria used for malarious villages like “distance from forest,” “distance from health centre,” “distance from breeding source” should be substituted with other criteria.

• Re-stratification should take place every three years.

• One of the ROs pointed out that micro-stratification is village level based and if there is an unexpected emergence of development projects in a previous bare land area, nobody are taking responsibility for stratification of this area. With this example he pointed out that stratification of such areas should be considered in the guidelines.

• Refresher training on micro-stratification is requested by BHS. As learning capacity may vary, BHS requested longer duration of the training.


The following are the activity-related recommendations:

• After micro-stratification, support for related activities like LLIN distribution should follow as soon as possible.

• The LLIN distribution policy of one net per household should be reconsidered, i.e. to consider two LLINs per household.

• Transport charges of LLINs from township to villages are not adequate. For this, VBDC ROs suggested to consult them for the local transport charges.

• All townships suggested supplying RDT/ACT with good expiry date.

• More volunteers should be recruited to cover the 1a stratum villages which are hard-to-reach.

• LLINs should be supported in two forms – single and double nets.


Recommendations related to evaluation:

• Qualitative evaluation was done in only five townships of two states and two divisions. The findings of the in-depth interviews in the five townships cannot be generalized to the whole country. This evaluation is only small scale and findings and results are preliminary ones. Therefore, the evaluation should be extended to include at least one township per remaining 10 states and divisions in order to get a more complete view.

• The evaluation mainly focused on the health provider aspect. For future evaluations, in-depth interviews should include local administrative bodies, beneficiaries and stakeholders.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search