Evaluation database

Evaluation report

2000 IDS: External Assessment of Canadian CIDA-supported Safe Motherhood Programme



Author: Agriteam Canada Consulting Ltd.; Luz de Luna Internacional

Executive summary

Background

In October 1995, CIDA and UNICEF-Indonesia signed a five-year, CAD $14.5 million agreement to support the Government of Indonesia (GOI) Repelita VI Safe Motherhood Programme-related activities in three provinces (South Sulawesi, East Java and Central Java). The agreement outlines CIDA support to the three particular areas of the 1995-2000 UNICEF-GOI Master Plan of Operation: Community Capacity (CCB), Maternal Health Service Delivery (MH), and Access to Water and Sanitation. The main project goal was to contribute to the reduction of maternal and neonatal mortality and morbidity in the three provinces. This contract is due to end March 31, 2000. A team of two Canadian consultants, supported by two Indonesian consultants, carried out an assessment of the Community Capacity Building and Maternal Health Service Delivery activities from January 15 to February 9, 2000.

Purpose / Objective

The general objective of this mission was to "review and assess the implementation progress vis-a-vis stated objectives of the Maternal Health and Community Capacity building components of the Safe Motherhood Programme in the CIDA-assisted provinces of South Sulawesi, Central Java and East Java". The stated objectives against which progress was to be measured were taken from the Contribution Agreement and the revised objectives as outlined in the May 1999, Third Annual Progress Report for CIDA on the Safe Motherhood Programme Initiative.

Methodology

Qualitative and quantitative data were collected via group and individual interviews, focus group sessions, field visits, and review of all pertinent documentation. Over 70 group and individual interviews involved over 250 individuals, along with 15 focus group sessions.

Key Findings and Conclusions

The situation in Indonesia during the life of the Safe Motherhood project has created a challenging working environment. The monetary crisis, a dramatic increase in the proportion of the population living below the poverty line, and repeated outbursts of violence across the country have all caused increased personal and social insecurity.

Within the dramatically changing landscape of Indonesia in the last five years, UNICEF-Indonesia also faced challenging transitions. Foremost among these, and unforeseen by the Indonesia office, was the slashing of its general resources budget (UN contribution) from a USD $12 million annual ceiling in 1995 to $5 million in 1999. The second major change was a shift from a "project" to a "programmed" approach. Before 1995, the Country Program consisted of 28 to 30 different projects with a focus on child survival. The 1995-2000 CP saw these projects organized as fourteen components under three main components: Service Delivery, Capacity Building and Advocacy and Planning. The third major shift was the focus on developing an urgent response to Indonesia's multiple crises (environmental, economic, and political) in 1997-98. Approximately one-fifth of UNICEF's total budget now is allocated to Crisis Response. The decision to decentralize UNICEF operations was the fourth major change. Within the last five years, field offices opened in Central Java, Moluku and Irian Jaya, and the East Java office was moved from Mataram to Surabaya.

These transitions placed serious pressures on UNICEF capacity. UNICEF experienced gaps in staffing at both the headquarters and field office levels. When key technical advisory positions were vacant in Jakarta, other Jakarta personnel had to provide the needed assistance, sometimes outside their area of expertise.

The original UNICEF-CIDA project design had a large investment in CCB programming but did not sufficiently address the complexity of social change required for programming success on institutional and community levels. A more comprehensive, iterative, and less ambitious design would have incorporated specific and detailed attention to: the existing patterns of authority that control information flow, decision-making and resource allocation; the cultural value conflicts between the principles of CCB and the societal /governmental status quo (respect for and submission to authority); the societal tensions incurred with movement toward democratic civil society with decentralization and reform; the actual institutional capacity of UNICEF to implement change programs in the face of expansion of field offices and few staff available with grounded experience to provide guidance; and the need for a clearly-defined methodology and understanding of the skills required, with accompanying definition of UNICEF's role (e.g. liaison, monitoring, facilitation, or executor/implementer). Inadequate attention to these factors, particularly when coupled with the major restructuring of the Jakarta office that began in 1996 (with significant cuts to staff position), all contributed to making the programming time-consuming and difficult. The limitations of the project design and concurrent staff cuts placed UNICEF on a steep learning curve throughout the five-year period.

UNICEF does well at country situational analysis of well-being of women and children. Equally important, however, is a more thorough and critical analysis of UNICEF's own institutional capacity in real terms. While the 2001 MPO has attempted to scale down programming, the concurrent use of "family empowerment" and "community empowerment" as core concepts and goals still implies longer, rather than shorter, and larger, rather than smaller, investments of time, staff and energy. UNICEF, very likely, is still overextending its staff and capacities in the next CP/MPO.

The adoption of log frames is a positive development within UNICEF. UNICEF should continue and extend the work using log frames (and/or other appropriate planning/monitoring tools), recognizing the current limited input from the field offices and striving to expand their role beyond activity identification. UNICEF should work with CIDA to determine the best means to address RBM reporting requirements so that, as much as possible, reporting to different donors can be simplified. Ensure that a qualified gender specialist (in-house or external) assists with the review of log frames and MPO in order to incorporate a gender perspectives, paying special attention to inclusion of rights and empowerment of women.

The MH component of this project had three clear objectives: to increase the percentage of women, specifically from low-income groups, having antenatal care, deliveries assisted by trained health personnel, and having at least two post-partum visits. The assessment team's examination of maternal health service delivery data from the three project provinces revealed that each field office should be able to report small to moderate increases in coverage indicators (K4, PN and KM), over time, in project villages and sub-districts. Unfortunately, up until now, this has not been clearly reported to CIDA. This sub-optimal reporting is partly due to the constraints of the GOI monitoring system but is also due to poor presentation of data (e.g., using a mix of absolute numbers and percentages, not defining terms or denominators) on the part of UNICEF.

Besides improved maternal health service coverage in project villages, a number of other clear benefits of the UNICEF-GOI MH component were identified. The assessment team witnessed excellent collegial relationships and communication between UNICEF Field Officers and the various levels of government working on the Safe Motherhood Programme. The joint planning of Maternal and Child Survival, Development and Protection programmers initiated by UNICEF has resulted in a broader and deeper understanding of Safe Motherhood issues, particularly at the provincial level.

The standardization of village emergency transport plans and village ambulances were in evidence, as was the presence of written Standard Operating Procedures (PROTAP) for the treatment of obstetric emergencies at every level of health service delivery. In South Sulawesi, where few project districts have hospitals with a blood bank, the blood donor pairing programme may be of vital importance although the assessment team suggests that there are some ethical considerations that need to be reviewed.

The future of Safe Motherhood activities, in general, in Indonesia, continue to rely on the village midwife or Bidan di Desa (BDD), who occupies the central role of linking communities to community health services. The future of the BDD is uncertain and many analysts suggest that within the next few years, there will be a mass exodus of BDD from village posts. The situation of the BDD demands urgent attention. The GOI and its development partners must carefully examine the issues and the GOI needs to declare a clear policy on how this issue will be addressed.

Recommendations

UNICEF's obvious strengths lie in the advocacy of child rights and well-being, creation of promotional resources and situational reports, and collaboration with GOI to build civil society. Given the current human and fiscal resources, this recommendation suggests that the organization build on strength and revise the expectation that UNICEF should demonstrate on-the-ground work to empower communities. Instead, UNICEF should collaborate with agencies carrying out community-oriented/based work, gather information about these activities and their results, and consider acting as a clearinghouse for this information. This information would feed into the situational analysis reports as well.

UNICEF needs to be more explicit, both internally and with donors, about actual programming activities, progress and problems. This will become critical as UNICEF increasingly communicates across distance with provincial staff, with multiple levels of government personnel and systems, and with private and international funders. Greater clarity and frequency in communication between the field offices and Jakarta, and between CIDA and Jakarta would strengthen the corporate (institutional) culture and improve programming. This should involve a critical assessment of UNICEF human resources in terms of personnel, time, cost, responsibilities and activities in order to realistically review upcoming plans for family and community self-help programming. Invest central and provincial time to grapple with community program "methodology" -- definition of terms, identification of principles, objectives, responsibilities, processes, monitoring. Avoid jumping to identification of indicators prior to clarification of these aspects.

UNICEF wishes to promote community empowerment through collaboration with NGOs. Caution is needed here because of the paucity of NGOs and existing NGOs' inexperience with community developmental initiatives. UNICEF states that internal monitoring has been weak. Monitoring (contracted) NGO partners will not be any easier. A careful analysis of the range of potential NGO partners for any program should be conducted in order to adequately assess pros and cons prior to entering into any agreements. This includes working with religious NGOs.



Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

Date:
2002

Region:
EAPRO

Country:
Indonesia

Type:
Evaluation

Theme:
Health - Pre/Post Natal Care

Partners:

PIDB:

Follow Up:

Language:
English

Sequence Number:
2000/003

New enhanced search