Evaluation database

Evaluation report

2011 Timor-Leste: Joint Program Promoting Sustainable Food and Nutrition Security in Timor-Leste Mid-Term Evaluation



Executive summary

“With the aim to continuously improve transparency and use of evaluation, UNICEF Evaluation Office manages the "Global Evaluation Reports Oversight System". Within this system, an external independent company reviews and rates all evaluation reports. Please ensure that you check the quality of this evaluation report, whether it is “Outstanding”, “Good”, “Almost Satisfactory” or “Unsatisfactory” before using it. You will find the link to the quality rating below, labelled as ‘Part 2’ of the report.”

 

Background:

Timor-Leste became independent in 2002 after much destruction had taken place due to a campaign of violence by militia groups. After a period of crisis in 2006 and the re-establishment of security GOTL engaged in a process to revive the economy and to tackle the country’s social problems. One of those is the high level of malnutrition of children under five and the low nutritional status of women of reproductive age. In addition to aspects of food security, the causes of malnutrition include inadequate child feeding practices, high incidence of illness and inadequate access to health and nutrition services.

The Joint Programme in Timor-Leste is meant to address the malnutrition of children under 5 years of age and the low nutritional status of women of reproductive age, which issues reach the level of a severe public health challenge. It is implemented in four districts of Timor-Leste and builds on support that the individual UN agencies concerned, i.e. FAO, UNICEF, WFP and WHO, have been providing before. The overall goal of the programme is meant to be achieved through three outcome areas: improved health and nutritional status of under-five children and pregnant and lactating women; school feeding programme; and establishment of a food security and nutrition surveillance system.

Purpose/Objective:

The present mid-term evaluation is formative in nature and seeks to generate knowledge, identify best practices and lessons learned, and improve implementation of the joint programme in Timor-Leste for the remaining period of implementation.

Methodology:

The evaluation makes use of four of the five DAC Evaluation criteria i.e. relevance, effectiveness, efficiency and sustainability. It combines a desk review of materials and a country visit in which key stakeholders were consulted making use of semi-structured interviews, focus group discussions and a validation meeting.

Findings and Conclusions:

The relevance of the programme is relatively high with the programme goal and objectives responding to the severe conditions of children under five and those of pregnant and lactating mothers. The programme appears in line with the policy of GOTL which has an enhanced focus on achieving the MDGs and an increased ownership of initiatives to enhance the nutritional conditions of children and women. The goals and objectives are also in line with the UNDAF outcomes.

The programme has been relatively effective in the parts in which activities could be started. In particular this has been the case for strengthening the health system’s capacity to deliver quality nutritional services and increasing demands for such services by families and communities. Moreover, activities were started in enhancing the availability of micro-nutrient rich foods. In particular in the programme components on food supplementation, salt iodization, vitamin A supplementation and production of blended food, results are being achieved. Promotion of vegetable gardens, small livestock rearing and aquaculture activities were delayed substantially. Moreover, programme outcome areas on school gardens and establishment of a food security and nutrition surveillance system have been delayed. This has meant that results in these parts of the programme have lagged behind expectations.

Given the delays in support to small scale food production, linkages between nutritional and food security components of the programme could not yet be realized. While the startup and initial implementation of the programme has required a high level of coordination amongst parties concerned, the synergy between the various components of the programme have so far been limited, meaning that the ‘costs’ of joint programming have not yet been followed by the benefits in terms of enhancing the results at the local level.

While there appears a balanced approach between supply and demand side issues in the support to the development of health systems, there is less of a balance between supply and demand side issues in the support to agriculture and gardening, where there is room to further work with groups of farmers to improve farming practices. Moreover, in areas with market access, the drive for vegetable growing appears cash oriented which differs from the design of the programme in which a subsistence approach to vegetable production is assumed. This needs to be incorporated in the implementation framework of the programme. The concerns on quality and quality control and on efficiency of production as practiced in the set-up of a supplementary food plant need to be more rigorously applied to the salt iodization component of the programme in order to enhance its implementation.

Socio-cultural aspects play a substantial role in the way in which issues of food security, nutrition and malnutrition are explained and dealt with at the local level. There is a need for the programme to incorporate this diversity of socio-cultural aspects of the local context in Timor-Leste and to fine tune programme implementation accordingly, ensuring the inclusion of vulnerable and difficult to reach populations and groups.

The activities of the various parties of the MDG-Fund are part of a wider range of programmes and policies of the stakeholders concerned. Therefore the changes at the level of programme outcomes and goal cannot necessarily be attributed to the joint programme alone. The extent as well as the way in which the programme contributes to outcome- and goal-level changes can be established by inclusion of sufficient intermediate level indicators that address capacity changes and other intermediate changes needed to reach broader objectives and goals.

With the high level of incidence of parasites found in young children, enhanced attention to de-worming appears required in order to improve nutritional conditions.

Monitoring of the programme is relatively weak with the programme results framework not containing enough indications of progress along the results chains in the various components of the programme. This means that though the framework might provide an overall impression of programme progress, the data and information provided is not sufficiently detailed to inform programme management. There is a need to further enhance the framework in this respect in order to support results-based management in the second part of programme implementation. This could also reinforce programme reporting, which has been weak, in particular during the first year of the programme. Parties should take the opportunity to regularly analyse monitoring data and in this way enhance learning and inform programme management decision-making based on progress made so far and constraints met.

The governance set-up of the programme has been combined with the Gender Equity and Women’s Rights Joint Programme, making use of the same National Steering Committee. For management purposes the programme has used an existing working group, which has enhanced efficiency of governance and management arrangements of the programme and reduced transaction costs.

Many of the delays in the programme have been related to staff turn-over and recruitment. Partly this relates to the specific situation of Timor-Leste, in which it appears difficult for most organizations to recruit technically qualified national and international staff. Moreover, UN recruitment procedures prove to be time consuming and there appeared no HR format for a joint programme coordinator within the UN system.

The systemic focus of the programme, building in particular capacities at the organizational level, enhances the likeliness of the results to sustain beyond the life time of the programme. Additional attention to the process of capacity development, enhancing local knowledge development and experience through a ‘learning by doing’ approach could further increase the options for sustainability.

Relationships with NGOs are often driven by annual plans rather than by longer-term partnership arrangements and concerns, which at times jeopardizes opportunities to build longer-term civil society capacities, which is of particular importance in Timor-Leste.

Private sector involvement in the blended food plant can be expected to sustain results and it will be useful to document the experiences gained so far so that lessons learned can be used for engagement with the private sector in other contexts.

The MDG-F Joint programme has enhanced the focus of GOTL and UN agencies on nutrition and food security and has shown how these issues can be addressed. There is a high expectation of UN agencies and other donors regarding GOTL allocation of resources to address the issues concerned. With the severity of the present conditions there will need to be a longer term commitment and investment of the GOTL and the donor community in order to sustainably address the issues concerned.

Recommendations:

  1. Further enhance the development and consolidation of a shared understanding of the reasons for malnutrition in Timor-Leste, amongst GOTL Ministries and Departments and UN agencies, - developing a framework in which immediate as well as underlying causes of malnutrition and undernutrition are identified, paying particular attention to the relationships between nutritional aspects and issues of food security; - jointly advocate for the use of such a framework in the development of initiatives to address nutritional conditions as well as in the reporting of the Food and Nutrition Security Task Force; 
  2. Enhance relationships between the various components of the programme in particular between food security and nutrition aspects and enhance the synergy that is meant to occur through the combination of programme components at the local level;
  3. Enhance selected capacity development aspects of the programme;
  4. Reinforce the targeting of the programme and the extent to which it is adapted to socio-cultural specific contexts and requirements of vulnerable groups through further focusing the participatory learning and action approach of the programme in selected districts and communities;
  5. To enhance results-based management and maximizing program results through improved program monitoring, adapting the programme results framework and conducting focused joint monitoring initiatives for key aspects of the programme;
  6. Enhance reporting of the programme by improving the analysis of monitoring data involving the various GOTL and UN agencies concerned.


Full report in PDF

PDF files require Acrobat Reader.


 

 

Report information

New enhanced search